Danny Sullivan again says Google and Yahoo need to dispense with the whole index size issue.
In a post on his web site, Mr. Sullivan urges the top two search engines to quit playing the “size matters” game and start reporting on relevance instead.
MSN Search has claimed an index of some 5 billion items. Google says it’s got 8 billion. Yahoo says 20 billion. But all the pages in the world don’t mean much to a user who can’t find the information he needs to locate.
Counting files in an index is a trivial task, even if it’s a big index. Counting relevancy, as Mr. Sullivan would like to see done, poses a much different problem. Relevance for a given search keyword or phrase will vary depending on who makes the query.
Consider the old riddle Isaac Asimov used in one of his Union Club mysteries, where Griswold gains the confidence of a young murder witness by demonstrating he’s at least as intelligent as the boy. The boy asked Griswold to identify the six letter word he could write down, and Griswold wouldn’t know how to pronounce it.
Of course, Griswold knew the word, the boy recounted the murder, and the story tied up as neatly as always. But a search engine would have trouble with the term. Relevance would be much different to a shoe care professional than a researcher of Eastern European history.
Mr. Sullivan acknowledges the relevance challenges in his post:
“I want to see the major search engines come together to develop a unified, accepted way to measure relevancy in various ways: web search, local search, advanced queries, whatever. Establish a research center, a consortium or something and a methodology that all will agree upon. Then test every four to six months and pledge you’ll accept the results publicly.”
David Utter is a staff writer for Murdok covering technology and business. Email him here.
