uTest “Battle of the Search Engines” Uncovers 600+ Bugs in Google, Bing & Yahoo!

Crowdsourced Software Testing Leader Holds Competition for Software Professionals to Compare Search Engines Based on Accuracy, Speed, Real-Time Relevance and Usability

 uTest, the world’s largest software testing marketplace, today announced the results of its Search Engine Bug Battle. More than 1,100 uTesters from 50+ countries around the world competed in the quarterly competition to report bugs in Google, Bing and Yahoo, with the late addition of Google Caffeine. The uTest community also ranked the engines for search results accuracy, page load speed, real-time relevance and usability. Google emerged as the clear leader, followed by Bing, then Yahoo.

Over the course of one week in August, a total of 606 bugs were reported across four search engines (see full Bug Battle report for details). The uTest community classified 13 percent of these bugs as “show stoppers,” which are defined as defects in need of immediate attention. Nearly $4000 in prize money was awarded to those who provided the best bugs and feedback.

Top-Line Findings:


  • Overall, Bing had the most reported bugs; however, Bing scored high in usability and design.
  • Nearly 90 percent of all survey respondents said that Google was their favorite search engine; however, 10 percent said after testing all three, they would now make Bing their default engine.
  • More than 30 percent noted that Bing surprised them favorably.
  • An overwhelming 71 percent identified search results accuracy as their most important criteria when choosing a search engine.
  • In terms of accuracy, Google led the way with a top two box score (rated as “excellent” or “good”) of 90 percent. Google Caffeine followed closely with 83 percent. Yahoo and Bing trailed with 53 and 42 percent, respectively.

Final Bug Battle Results:



Google

  • 130 bugs reported
  • 8 percent classified as “showstoppers”
  • 28 percent were Technical, 46 percent were Functional, and 26 percent were GUI

Bing

  • 321 bugs reported
  • 14 percent classified as “showstoppers”
  • 29 percent were Technical, 46 percent were Functional, and 25 percent were GUI

Yahoo

  • 70 bugs reported
  • 10 percent classified as “showstoppers” and 50 percent as “high severity”
  • 28 percent were Technical, 57 percent were Functional, and 15 percent were GUI

Detailed results of this study are available at http://www.utest.com/bugbattle/q309/results.

"It’s clear Google’s leadership position remains strong with 67% of the search market. However, the recent Microsoft/Yahoo deal, combined with Bing’s innovative multimedia search and sheer marketing muscle, may represent a viable future challenge,” said uTest VP of Marketing and Community, Matt Johnston. “With this Bug Battle, uTest has clearly demonstrated how quickly our community can mobilize to provide global testing coverage and usability feedback for any application.”

The top tester of the Bug Battle was Joseph Ours of Columbus, Ohio, who reported the most valuable bugs and provided the best testing feedback. Other winners included noted testing blogger Pradeep Soundararajan (top bug) as well as top novice Max Sterneck from Austria (though he now lives in Japan). A complete list of the Bug Battle winners can be found at http://www.utest.com/bugbattle/q309/results.