Sitemaps Affect on Website Crawlers

You created a Sitemap and submitted it to the search engines. Did you ever started thinking if this really helps the site out and what’s the effect when you submit a Sitemap on the site. Why we need to use sitemaps, Using sitemaps on your websites really going to affect on the Rankings of your website. Many of us are unaware what are the Advantages of using the Sitemaps on our websites.

I was going through this Excellent post written by chenry where he discussed some interesting points pertaining to Sitemaps with good display of stats. To find out the uses of Sitemap chenry he wrote a PHP script and installed in on his client website to track every time a bot accessed the Sitemap, every time the Sitemap was submitted, and every page it crawled on the website. The script was designed to store this information in a MySQL database along with a timestamp, IP address, and the user agent. He also modified the Sitemap generator to insert a timestamp every time the sitemap was submitted to the search engines.

The Primary aim of his experiment was to find out if submitting a Sitemap to Google and Yahoo would show any effect on the time it took Google to crawl and index the page. To his surprise the results were quite amazing!!. When a Sitemap was submitted the average time it took for the bot to visit the new post was 14 minutes for Google and 245 minutes for Yahoo. When no Sitemap was submitted and the bot had to crawl to the post, it took 1375 minutes for Google and 1773 for Yahoo. The averages were calculated on 12 different posts, 6 with Sitemaps being submitted, and 6 with the Sitemaps not being submitted.


Crawltime with no-Sitemap



Crawltime with Sitemap



For further reading visit chenry
Add Post To: |Digg| Twitthis| Technorati| del.icio.us| Stumbleupon| Sphinn| Reddit| BlinkList| Furl| Spurl| Yahoo| Simpy|