Twitter
Email
RSSWhat Is a Sitemap? Do I Need It?
This post is part of the weekly Q&A section. Just use the contact form if you want to submit a question.
Rajasekharan asks:
I couldn’t find an XML sitemap in your blog. Do you have one? If no, why are you reluctant to include an XML sitemap?
First things first, what is a sitemap? According to Sitemaps.org:
Sitemaps are an easy way for webmasters to inform search engines about pages on their sites that are available for crawling. In its simplest form, a Sitemap is an XML file that lists URLs for a site along with additional metadata about each URL (when it was last updated, how often it usually changes, and how important it is, relative to other URLs in the site) so that search engines can more intelligently crawl the site.
Web crawlers usually discover pages from links within the site and from other sites. Sitemaps supplement this data to allow crawlers that support Sitemaps to pick up all URLs in the Sitemap and learn about those URLs using the associated metadata. Using the Sitemap protocol does not guarantee that web pages are included in search engines, but provides hints for web crawlers to do a better job of crawling your site.
The key message is in the second paragraph: “Web crawlers usually discover pages from links within the site and from other sites. Sitemaps supplement this data.”
Google and the other search engines rely mainly on their bots and web crawlers to discover new pages around the web. Sitemaps help them in some situations, but they are not compulsory nor necessarily beneficial.
Here is what Google itself says about Sitemaps:
Sitemaps are particularly helpful if:
- Your site has pages that aren’t easily discovered by Googlebot during the crawl process – for example, pages featuring rich AJAX or Flash.
- Your site is new and has few links to it. (Googlebot crawls the web by following links from one page to another, so if your site isn’t well linked, it may be hard for us to discover it.)
- Your site has a large archive of content pages that are not well linked to each other, or are not linked at all.
Unless you fall into one of those cases, therefore, having a Sitemap will not be “particularly helpful” for your site.
In fact, some people claim that it might even be the opposite. There are cases where the addition of a Sitemap actually reduced the crawl rate of the website, possibly because Google no longer need to crawl the site completely to discover its internal pages.
On most of my sites I don’t use Sitemaps because I always try to have an efficient internal link structure in place. For example on Daily Blog tips every single page of the site is accessible within 2 clicks of the homepage.
If your website is having indexation problems, has a poor internal link structure, or falls in one of the three situations described by Google, then using a Sitemap would probably be a good idea.
Got Your Free eBook?
- Subscribe to the Daily Blog Tips newsletter and you will be able to download the "Make Money Blogging" eBook for free.
- You will also receive tips to improve your blog, strategies to make money and useful resources from around the web.
Related Articles
Subscribe
21 Responses to “What Is a Sitemap? Do I Need It?”
Got something to say?
Sponsors
Recent Articles
- Bloggers These Days…
- What Is Smart Pricing on Google AdWords?
- Find Brandable Domain Names with Wordoid.com
- Blogging Lessons From TechCrunch and Mashable
- 22 Ways To Find Advertisers For Your Website
- The Marketplace for Websites is Heating Up
- How To Increase Your Blog Traffic With Article Marketing
- Got Blogging Related Questions? Ask Away!
- Business Idea: Copyright Protection For Web Publishers
- Blogging Dynamite: The Power of Connections
- Ever Considered Launching a Membership Site?
- 5 Tips To Get The Most Out of Your 99Designs.com Contest
- Do I Need to Register A Business To Make Money With My Website?
- Because 10,000 People Can’t Be Wrong
- Download Google’s SEO Report Card
I still feel is you have a relatively new website or one that just doesn’t have many back links you should be using a site map.
Another one of the best uses for it is during a redesign. making sure that one is in place before and after so the bots can see exactly what is happening with your website.
I use a sitemap for second reason listed by Google. And it certainly has helped. However, from personal experience I also think it can be useful to offer a sitemap formatted more for human readers too (although I still haven’t had time to make one for my site).
It is no harm to include a sitemap. Just in case Google couldn’t crawl your content due to some issue, sitemap will help.
And we can another kind of HTML page sitemap which is useful to human being.
Thanks dude for picking that question up.
I removed my sitemap also just for the reasons you said and after noticing that you don’t use XML sitemaps.
However, I recently heard tweets from Jason Calacanis that soon after he “resubmitted” sitemap of Mahalo to Google, he had heavy traffic to knock his server down couple of times! (Read it here: http://tinyurl.com/bfgyqj) Strange coincidence that you picked this question soon after that.
Note the word “resubmitted.” Strange! Mysterious! Or any other hidden reason?
@Rajesekharan, we would to know what was happening on the site when that re-submission was carried out. For example, they could have been having some crawling issues.
Excellent post Daniel you definitely brought to my attention what could be a negative point of having a sitemap. Its tempting to actually get rid of my sitemap now however I think I shall keep it for now.
Having a sitemap has one positive side effect. Actually the sitemap should be submitted to Google Webmaster Tool.
Here is a post from John Mueller (Googler) about hard-to-detect website exploits that silently redirect Googlebot to third-party sites.
http://johnmu.com/pagerank-hac.....omi-co-in/
“There’s another way to spot this kind of hack with Google Webmaster Tools: When you submit a Sitemap file, Google will show warnings for URLs that redirect. By design, you should be listing the final URL in your Sitemap file, so if the URL is redirecting for our crawlers (as in this case), we’ll show a warning in your account.”
And when I talked with a webmaster of one of the hacked sites (who lost his PR and search engine positions:
http://oldtenspeedgallery.com/.....g-hacked/), it appeared that he actually saw the warnings but didn’t know how to interpret them.
I got that it is important to make sure that you create your links as you go so that you create an internal map.
I had not even started to look at what a site map is. Now I know it should be one of the important first steps for a new blog.
Thank you
Sheila
Oh thanks for the info, as long as I don’t have to do something like this, I am happy
But looking at the way search engines have brought traffic, I am amazed at the terms that they catch. I guess they go well into the post (much beyond the tags) and crawl the entire content and bookmark them to recommend to the readers. There is one more thing I have noticed, If I write a story and immediately write something on two mobile phones, still the search engines know how to differentiate between the two posts and whom to refer what. That’s amazing!
Destination Infinity
Well, I’d like to say that the XML Sitemap is really very useful to a blog, I have used it for nearly one year, the most pages of my first blog are indexed by Google, all thanks by Google XML Sitemap plugin for WordPress.
@Clarky, I am not sure if your pages are indexed by Google thanks for the Sitemap or thanks for the backlinks that your site has. Probably the latter
.
I am considering that site map is very important for our site, to make web crawlers easily crawl our site.
Using Google sitemaps is vastly underrated but it is still the quickest way to get your site indexed by Google. One tip…if your update is set to monthly for a sitemap update, you can also resubmit manually every time you write a new post/article.
I always make sitemap for my sites. I’t realy help crawlers with indexing my webpages.
I don’t have a sitemap myself, but I should get to work. Is a sitemap very important? Or just a little difference?
yes sitemap very important for search engine and also for visitor….I always use frr sitemap generator
I will always start a site with a sitemap now, but I have seen the negative effect to adding one with an older site.
That’s a great point about the sitemap possibly being a negative. I think there are three big benefits to a sitemap that I didn’t see mentioned here.
1) The last modified date should encourage the googlebot to crawl the page – even for small pages – to at least see what changed.
2) The sitemap priority to reduce the importance placed on less important pages (like a contact page) and also influence the links google will display in your authority listing (I think they call them sitelinks).
3) Having a sitemap allows you to ping Google, Yahoo, and MSN when you’ve updated your site. They may not crawl your site immediately, but it should get them back crawling your site sooner than if you had waited for the bots to come on their own.
Most good content management systems will take care of all three of these for you.
Building a good site map is also critical for web pages which use JavaScript menus, because most search engines cannot follow JavaScript links a Site Map will help those web pages to be indexed by the search engines.
Well, I’d like to say that the XML Sitemap is really very useful to a blog, I have used it for nearly one year, the most pages of my first blog are indexed by Google, all thanks by Google XML Sitemap plugin for WordPress.
I don’t have a sitemap myself, but I should get to work. Is a sitemap very important? Or just a little difference?