Researchers at the University of Illinois at Urbana-Champaign studied Yahoo’s claims of a 20 billion item index and found them lacking.
Neither Google nor Yahoo makes direct verification of their index claims available. But an Associate Director at the National Center for Supercomputing Applications (NCSA) and two researchers conducted a brief test to compare the two search engines themselves.
In a report released by the researchers, they described a two-part methodology to their test. One part assumes both engines return all the relevant results of a search, and only filter duplicates. The second part contends that if Yahoo’s index is twice the size of Google’s, then it should return twice as many results.
The testers soon found they had to eliminate any keyword search that returned more than 1,000 results. They focused on those results that returned less than 1,000 results on both Yahoo and Google. By randomly selecting keywords, the test ended up with a sample of 10,012 searches conducted over an 18-hour period.
In purposely looking for “obscure documents,” the researchers hypothesized a broader index should return more results. That didn’t happen in the test. “(T)here are a number of cases in which Google returns dozens of results while Yahoo! only returns one or two results, or none at all,” the researchers note in the study.
The average search results, excluding duplicates, was 38 to 14 in favor of Google. Including duplicates changed that average to 64 to 22. Out of the 10,012 tests, 9,676 had Google returning more results, 307 showed Yahoo did better, and 29 found both engines returning the same number.
Yahoo has disputed the results of the admittedly limited test.
“The study cannot be used to determine how many documents are in the index,” Yahoo spokesman Aaron Ferstman said in a TechWeb.com article. “It can simply be used to see how well the index returns documents. What really matters ultimately is relevance.”
David Utter is a staff writer for Murdok covering technology and business. Email him here.
