Do you remember a time when search engine submission strategies were fairly easy and worked almost across the board? Sure, we recommended manual submissions, and yes, we recommended daily submission limits, but the strategies were uncomplicated.
No more.
These days, knowing when, and if, to resubmit your pages is one of the most confusing areas of search engine positioning.
For example, some engines seem to place more relevancy on pages they find themselves. So, it’s to your benefit to let the engines find your pages, rather than to submit them individually.
Do we need to resubmit our pages on a regular basis? And what if we submit a page and it doesn’t get in the index? Do we need to submit the page over and over and over again until it finally gets in?
What about AltaVista, where we’re told not to resubmit pages that haven’t been changed or have been changed very little? How much is “very little”?
To write this article, I interviewed Jim Stob with PositionPro http://www.positionpro.com. Obviously, anyone who operates a sophisticated search engine submission service with as much success as Jim Stob knows the ropes on search engine submission policies.
First, let’s look at PositionPro to see why the service is such a good role model in how to submit pages.
PositionPro has thousands of domains carrying several million URLs in their system, and every one of their submissions comes from the same IP address.
What does that say to you? Because PositionPro submits every page from the same IP address, if they get in trouble one time, their entire operation is in jeopardy. “They can track us without any problem and shut us down,” says Stob.
Jim has working relationships with many of the engines, which is why he is able to learn what they want to see in search engine submissions and can walk the fine line to stay out of trouble. With his impressive list of clients, including Canada.com, it’s obvious he’s doing something right.
Using AltaVista as an Example
We can learn a lot by tailoring our search engine submissions along the same lines as PositionPro. Let’s look at AltaVista as an example.
In the past, we were told to submit just one a page a day to AltaVista. But with AV’s new submission procedure, we can submit more. How many can we safely submit? Let’s look at how does PositionPro handles AltaVista.
“We submit five pages per domain per day, and we don’t have any problems. Could we submit ten? Probably. Could we do it over a time period? No,” explains Jim.
By the way, it might interest you to know that PositionPro doesn’t accept cookies. As you probably know, we’ve advocated cookie control when working with AV. Is this why they’re able to submit five pages a day? Probably not. “I think if you talk to AV, they would be fine with five,” Stob says.
What about pages that have had no changes made to them? Does PositionPro resubmit those pages?
“We only do a submission one time to start out. So, with a site that contains 50 pages, it will be fully submitted within 10 days,” says Jim. Then, once the pages make it into the index, the service doesn’t resubmit them again, at least as far as AltaVista is concerned.
After that, the spider will pick up any changes you make to the pages on its next spider run. But, let’s say that you’ve made significant changes to your pages, and you want those changed pages picked up before the next spider run. At PositionPro, you can tell the service to manually submit your page. However, it won’t allow you to resubmit the same page any sooner than 21 days since the last search engine submission.
What if you submit a page and it never makes it into the index? Should you submit over and over again until the page gets in?
“No, that’s inappropriate,” explains Stob. “If I submit to AltaVista, and if it’s not accepted, there’s a reason that the page wasn’t accepted. It wasn’t because of the search engine submission. The engine went in there, and their algorithm judged it not to qualify for their index.”
So, what are your options? For one thing, keep in mind that the engines operate on a timetable – their timetable! Be patient and wait to see if the page makes it into the index. If it doesn’t, before rushing out to resubmit, look at your page carefully and consider what additional optimization strategies you can employ to boost your chances at getting in the index. Remember: content is what the engines are after.
Note: AltaVista now has a pay inclusion program, where you can pay to have your site indexed by AV.
Safe Search Engine Submission Guidelines
Based on the guidelines that have kept PositionPro out of trouble, let’s create a list of “safe submission” guidelines that we can follow for our own sites.
We’ll get into engine-specific search engine submission guidelines later in this article, but for now, let’s look at some general guidelines.
Submit new pages to the engines, and then wait to see if the pages make it into the engines’ indexes. Remember that it can literally take weeks (months?) for pages to get into an index.
If the pages don’t make it into the index, don’t resubmit them, but look them over carefully to see if you need to employ additional optimization strategies before resubmitting.
Submit all of your important pages to the search engines that allow you to submit more than just the main page of your site.
Stob explains, “None of the spiders crawl as well as they say they do. Sometimes they do an excellent job, but in most cases, they don’t.”
After a page has made it into the index, unless you make significant changes to the page or the page has dropped from the index, don’t resubmit it.
If a page is dropped from the index, wait for two weeks to see if the page is picked back up, and then resubmit it.
If you’ve made significant changes to a page, resubmit it, but no sooner than 21 days since your last search engine submission. It might benefit you to play it safe!
Consider adding visible links to all of your important pages on the main page of your site. Give the engines something to spider!
What about using hidden links, which is a very popular strategy these days? Stob says to begin to approach this practice with caution. “I believe AV is beginning to look for hidden links, so try to stay away from them. I’m not saying that you will have trouble based upon their spider. However, they do have people reviewing, and I believe they are looking at this closer.”
Submit your pages manually, or use a search engine submission service or software that mimics a manual submission. By that, we mean that the software or service waits for verification before submitting another URL. If you use a search engine submission software program, make sure that you’re able to choose the schedule for your submissions. If you can’t, you may find yourself in trouble with the engines for over-submitting.
Keep in mind that if it takes too long for an engine to access your site, it probably won’t get indexed.
Engine-Specific Guidelines and Tips
AltaVista
Submission URL: http://add-url.altavista.com/cgi-bin/newurl?
How long does it take to index a page? a few days with Express Inclusion, their pay inclusion program; 6 weeks through their free Add URL page.
Submission Limit per Day: through free Add URL: 5 pages or more with their new procedure
With AltaVista, rely on the use of links to get the pages in your site indexed. Unless you’ve made significant changes to your pages, don’t resubmit them. Instead, let AV’s spider find those changes during its next spider run.
Try submitting your pages using “www” and not using it in the URLs.
AV won’t index a site submitted as an IP address. A rep from AltaVista explains, “The spider is doing reverse DNS lookups when an IP site is submitted. You must be registered with Internic and have DNS running to be indexed by the spider.”
Does AltaVista have a limit to the number of pages from a particular domain that can be in its index? AV answers, “In reality, the whole site could be indexed with no restriction on depth, except that we have size limitations and stop the spider once the index of the site has a certain size.” In other words, they aren’t saying. (Is this surprising?)
To check to see if your pages are in the index, enter your URL in the search box like this:
url:yourdomain.com (reveals every page under the root domain)
url:members.prodigy.com/yourwebsite/ (narrows the search to pages within your site, if you’re under someone else’s domain)
url:members.aol.com/yourwebsite/yourwebpage.htm (finds a specific page at your site)
Submission URL: http://www.google.com/addurl.html
How long does it take to index a page? 1-2 months
Submission Limit per Day: 10 pages
With Google’s supplying Yahoo!’s supplemental index, submitting to Google has become even more important.
Submit all of your important pages to Google. However, with Google’s unique way of indexing, your links should be picked up by this engine with no problems. But, why take a chance – submit your important pages.
You can submit up to 10 URLs to Google a day and be safe. “Google doesn’t have a number per se,” says Stob. “When I first approached them, they asked me to keep it low so we started at 2 per day. They were in the middle of building a new index. Since then we have brought the count to 10 per day.”
Stob continues, “Could I submit 50 a day? Probably. Would I consider doing it on a large site? Probably.”
With Google, be sure to have other sites link to your Google pages, or those pages may not be picked up in the index.
To check to see if your pages are in the index, search for your domain name like this: www.yourdomainname.com
Then, click on “Find web pages that contain the term.” From there, you can choose “Search within results” to narrow down the search to particular pages.
HotBot / Inktomi Engines
Note: Your best bet for submitting to the Inktomi engines is to go through their pay inclusion program. For more information, visit: http://www.positiontech.com/. If you don’t want to pay to submit, your next best bet is to submit to Inktomi through Anzwers.com.
Submission URL: http://www.anzwers.com.au/cgi-bin/print_addurl.pl?
How long does it take to index a page? 2 weeks if not through their pay inclusion program; 48 hours if through pay inclusion
Submission Limit per Day: 50 pages
As you know, Inktomi supplies results to HotBot, GoTo, MSN Search, and many more search services. To get your site into the Inktomi engine faster, try submitting to Anzwers.com instead of through HotBot, or again, through their pay inclusion program.
Be sure to submit all important pages of your site to HotBot. Try submitting the same page using different variations of the page. For example:
http://www.HereIAm.com
http://HereIAm.com
http://www.HereIAm.com/index.html
You may find that your sites using the “www” are ranked slightly higher than those without the “www.”
When you submit a site to Inktomi, have you noticed that it may be appear in the index for a month or two, then be dropped? Jim explains what might be happening. “Inktomi will analyze a page for inclusion in their index. It may show up for a month or two and then be dropped. It is dropped because Inktomi has not seen it come up in a search and has determined that it doesn’t have any value.”
So, if the page doesn’t get accessed through a search engine, it will be dropped from the Inktomi index?
Stob answers, “Yes for Inktomi, no for AltaVista. Inktomi has two indexes as you probably know.”
Have you also noticed that your rankings fluctuate in HotBot or other engines? Stob explains, “Rankings will alter by the hour with some engines. If you lose the spot for more than two weeks, you should be concerned.”
To check to see if your pages are in the index, use their Check URL form: http://hotbot.lycos.com/help/checkurl.asp
Or, visit HotBot’s main page and choose the “Advanced Search” button, which is on the left-hand side of the page. The SuperSearch page will appear.
Find the Location/Domain box toward the middle of the page, and enter your domain without the “http://www” prefix in the domain box. You can choose other variables, such as the page depth and the number of search results you’ll be shown. Click on Search.
Lycos / FAST
Submission URL — Lycos: http://www.lycos.com/addasite.html
Submission URL — FAST: http://alltheweb.com/add_url.php3
How long does it take to index a page? 20-40 days (in FAST)
Submission Limit per Day: 50 pages
With both Lycos and FAST, you can safety submit all of your important pages, even though both engines are “deep crawlers” and should find the links on their own.
Try omitting the “www” when submitting your pages to Lycos.
To check to see if your pages are in the index, search for your root domain, such as: thisismysite.com
Some of your pages will usually be listed. If not, search again, but leave off any suffixes, such as “.com” or “.edu” like this: thisismysite
In Conclusion
When submitting to the search engines, play it safe. Don’t submit over and over again, and look at each engine’s submission guidelines carefully. If you’re going to err, err on the side of being conservative.
As Jim Stob says, “The moral of the story is, don’t trick them. Give them content.”
Robin Nobles conducts live SEO workshops
(http://www.searchengineworkshops.com) in locations across North
America. She also teaches online SEO training
(http://www.onlinewebtraining.com). Localized SEO training is now
being offered through the Search Engine Academy.
(http://www.searchengineacademy.com) Sign up for SEO tips of the
day at mailto:seo-tip@aweber.com.
