Loading...
A Randomized Experiment Assessing the Accuracy of Microsoft’s “Bing It On” Challenge
Ayres, Ian Atiq, Emad Li, Sheng Lu, Michelle Maher, Tom Tsang, Christine
Ayres, Ian
Atiq, Emad
Li, Sheng
Lu, Michelle
Maher, Tom
Tsang, Christine
Files
Citations
Altmetric:
Abstract
In advertisements associated with its "Bing It On" campaign, Microsoft claimed that "people preferred Bing web search results nearly 2:1 over Google in blind comparison tests." We tested Microsoft's claims by way of a randomized experiment involving U.S.-based Amazon's. Mechanical Turk ("MTurk") subjects and conducted on Microsoft's own www.bingiton.com website. We found that (i) a statistically significant majority of participants preferred Google search results to Bing search results (53% to 41 %); and (ii) participants were significantly less likely to prefer Bing results when randomly assigned to use popular search terms or self-selected search terms instead of the search terms Microsoft recommends test-takers employ on its website. Our findings suggest that some of the claims implicit in Microsoft's advertisements warrant legal scrutiny.