We often take for granted the results we get for any given web search. When we search, we expect to find what we are looking for. That’s the way it should be. The average user doesn’t normally consider what it takes for a search engine to deliver those results, but there are so many factors at play, working behind the scenes and coming together to (hopefully) deliver the user the information they seek.
“Spiffy features are great, but if they’re wrong or don’t trigger in some reasonable way that your mind can predict, the failure is worse somehow,” says Google’s Cutts on his blog. “The same holds true with the organic search results: a catastrophic search failure can stick in your mind much more than the 200 searches that worked well. Search quality evaluation is tricky because you need to take that factor plus hundreds more into account. It’s taken years for Google to really evaluate our quality well, and we still continue to learn important new things.”
Are you more likely to remember searches that worked well or ones that didn’t? Comment here.
Searchers expect greatness. It’s not even something they consider. They just expect to get what they want. It’s only when they don’t get it that they really notice. But what does it take for a search engine to keep a user from noticing a flaw? What goes into providing quality results?
The search quality team for the most widely used search engine in the world has given a tremendous amount of insight in a series of lengthy interviews with the publication BusinessWeek.
Cutts says Google’s strategy for search quality is a balance of the analytical and serendipity. This comes from a cross between a lot of evaluation metrics and a lot of feedback from both the inside and the outside. Google gets tons of feedback from users, but the search quality team also gets tons of feedback right from within the company. Cutts says they have a mailing list comprised of about 20,000 Google employees who are always complaining and leaving feedback.
The human element steps in in a variety of ways. Cutts says that when he is just out and about on his own time, he will come across places, things, and sites that he will look up to find out what kind of results Google delivers for them, and looks for ways to improve this. He says that Google has gotten better at things like spelling, morphology, synonyms, stemming (“where somebody types in ‘runners’ and maybe they meant ‘runner,’ or ‘running'”), etc.
“We ran over 5,000 experiments last year,” Google’s Udi Mamber told BusinessWeek. “Probably 10 experiments for every successful launch. We launch on the order of 100 to 120 a quarter. We have dozens of people working just on the measurement part. We have statisticians who know how to analyze data, we have engineers to build the tools. We have at least five or 10 tools where I can go and see here are five bad things that happened.”
Google may rely on the human element to some extent, but don’t take that to mean that more emphasis is placed on this than the machine element. Cutts says Google certainly relies much more on computers and algorithms than any other major search engine (at least historically). He does reiterate a point he has made in the past, however, and that is that Google has become more willing to listen to feedback.
Google’s strategy for improving search results consists of a mixture of humans, formulas, and experiments. These are the elements that it takes to deliver what Cutts says are fresher and more comprehensive results than those from other search engines.
Do you agree with Cutts that Google delivers “fresher and more comprehensive” results? Share your thoughts.
