There are many different ways to test the accuracy and relevancy of a search engine, but has anyone ever considered using a trivia-based game as barometer? If they have, they haven’t told anyone, at least until now.
As pointed out by Danny Sullivan at the SearchEngineWatch Blog, Michael Liedtke, an AP business writer, decided to play a game of Trivial Pursuit using a number of search engines including Google, Yahoo, MSN Search, Ask.com, and Answers.com.
In order to gauge whether or not the corresponding engine got a correct answer, Liedtke conducted a query based on a question and the engine that had the right response as the first link on the page was awarded a point.
What he discovered was by using his criteria; Ask and Answers were slightly ahead of the big three, scoring 10 out of 20 (50%), with Google being next in line, scoring an 8 out of 20 for a 40% rate. Yahoo was third with 5 out of 20 and MSN brought up the rear, scoring 2 out of 20, giving each a 40 and 20 percent effectiveness rate, respectively.
Liedtke then expanded his parameters to where he awarded a point for having a link on the first page, instead of just the first link on the page. When the questions were asked using these stipulations, Google, Ask.com, and Answers got the correct response 17 out of 20 times, giving them an 85% success rate. Yahoo and MSN Search missed the answer 6 times out of 20, which gives them 30 percent failure rate.
While this may not be the most scientific approach (something Liedtke acknowledges), it still provides an entertaining look at how search engines fare when it comes to searching for answers. Liedtke’s test does, however, give an insight into one thing: semantic search is still a little ways off.
Chris Richardson is a search engine writer and editor for Murdok. Visit Murdok for the latest search news.
