Google employees are hinting (strongly) that the search engine is being more discriminatory about website search results appearing in Google’s search results. That cross-mojonation, if you will, isn’t what searchers want.
And while that seems simple on the surface – a search result leading to another search result in a vicious cycle is pretty frustrating for most users – it leaves a lot to think about from the webmaster side.
You’ve probably seen what they’re talking about. Danny Sullivan provides screen-caps and a lengthy examination at SearchEngineLand.
The discussion begins with Google’s Matt Cutts, who addresses the situation on his blog. “In general,” he writes, “we’ve seen that users usually don’t want to see search results (or copies of websites via proxies) in their search results… Google does take action to reduce the impact of those pages in our index.”
Cutts recommends adding a “Disallow: /results” to the robots.txt file to prevent the Googlebot from indexing search results. Google updated its technical (e.g., not its quality) guidelines with the following:
Use robots.txt to prevent crawling of search results pages or other auto-generated pages that don’t add much value for users coming from search engines.
Though it wasn’t a clear directive with any clearly outlined consequences, Sullivan argues that SERPs in the SERPs are not high on Google’s list of likeable things.
“[T]he only way they get to a page of search results,” he writes, “is if there’s a link that will generate them.”
“Practically no one links to our search results. But now thanks to the new Google guidelines, out of the blue, I have to go block off the /fastsearch area or potentially be seen as spamming Google. What a pain.”
There are instances when search results count as legitimate content, and it will be interesting to see how Google separates one SERP from another. Sullivan goes on to call the new policy a “can of worms” for shopping comparison sites especially that pop up in a search for, say, DVD players.
While there’s no official policy yet, webmasters wouldn’t be stretching too far to think that it soon will be.
