Question: I would like to know what is the importance of submitting all of the web pages to search engines instead of only the main page?
Shari’s Answer:
In an ideal situation, a web site owner should only have to submit a single page from his site (usually the home page). Then, the spider-based search engines (Google, FAST Search, AltaVista, Inktomi) should be able to spider the rest of the site based on that single submission.
But the web is not an ideal place, and web site designers often do not consider the search engines when they create a site layout. Below are some issues to consider that might prevent your site from being “naturally” crawled by the search engines.
Splash Pages
One of the worst home page designs is a splash page. A splash page consists of either (a) a large graphic image and a link instructing visitors to “Enter” a web site, or (b) a Flash animation, a link to skip the Flash animation (“Skip Intro”), and a redirect to a new page after the animation is complete.
The redirect is particularly problematic because most redirects are considered to be spam. Therefore, the search engines want to go directly to the second page (the one that users are directed to) because that is clearly what the web site owner intends. If you’re redirecting a page, then the first page is useless to the search engines, isn’t it?
When you only have a single link from your home page to the rest of your site, you are communicating to the search engines (and site visitors) that you only consider a single page on your site to be important. One of the simplest solutions is to have a search-engine friendly link to a site map on your home page. That way, a web site owner communicates that the entire site is important, not just a single page.
A site map also communicates that you are willing to help your visitors. In the event that your visitors get confused, they should be able to click on a link to a site map to help them find what they are searching for.
Navigation Scheme
Another responsibility that site owners have is to create a search-engine friendly navigation scheme. Since search engines have problems with navigation buttons surrounded by JavaScript, image maps, menus, and Flash, the text links compensate for those problems.
Remember, the search engines’ attitude is, “If you link to it, you must think it’s important.” Make sure you cross-link related pages.
Don’t cross-link just to make the search engines happy. In all likelihood, many pages on your site should naturally link to each other. For example, if you have an FAQs (frequently asked questions) section, some of those questions should have direct links to your Products or Services section.
Directory structure
Another search engine attitude is, “If it’s at the top, you must think it’s important.” Therefore, pages at that are placed at the top levels of your directory are considered more important than pages placed at lower levels. Search engines generally will crawl the top 2 levels. For example:
http://www.companyname.com/page.html is on the top (first level).
http://www.companyname.com/directory1/page.html is on the second level.
http://www.companyname.com/directory1/directory2/page.html is on the third level.
To encourage deep crawling (i.e. beyond the second level), make sure you cross-link related pages. Also, if you can do a link building campaign for pages deep into the directory structure, that will also encourage search engine spiders to crawl more pages on your site.
If time or budget permits, consider flattening your site’s directory structure.
URL Structure
The most search-engine friendly URLs have little or no parameters in them. The fewer parameters, the more search-engine friendly the URL is. For example:
http://www.companyname.com/page.asp has no parameters.
http://www.companyname.com/page.asp?product_no=25 has a single parameter.
http://www.companyname.com/page.asp?product_no=25&product_sortord=asc has a two parameters.
Conclusion
To encourage optimal search engine spidering, web site owners should do the following:
(1) Create at least one spider-friendly navigation scheme
(2) Make a site map
(3) Link to the site map from the home page
(4) Cross-link related pages
(5) Encourage other sites to link to pages on your site
The search engines are not perfect. Pages are removed from the search engine indices periodically, even though a site owner hasn’t changed anything on his site. When this happens, all you need to do is resubmit the dropped pages, and they should appear in the next indexing cycle.
Shari Thurow is Marketing Director at Grantastic Designs, Inc., a full-service search engine marketing, web and graphic design firm. This article is excerpted from her book, Search Engine Visibility (http://www.searchenginesbook.com) published in January 2003 by New Riders Publishing Co. Shari can be reached at shari@grantasticdesigns.com.
Shari Thurow Answers SEO Questions: Click Here For Free Answers
