The Blogger’s Guide to Google Webmaster Tools

Ever wonder how Google sees your site? Ever wish you could click a button and correct all of your site’s search engine woes? As it turns out, you can, at least to a certain extent. Webmaster Tools is a free product from Google that allows you to interact with the world’s most (in)famous search engine spider and indicate how your site should be indexed.

Getting Started

Head over to Google Webmaster Tools (you’ll need a Google account to log in) and type your URL into the text input box. Make sure that you precede it with “www” if you intend to set that as your preference later on, because Google won’t allow you to upload a sitemap if your domain here doesn’t match your preferred domain later. Click the “Add Site” button once you’re sure.

That’s enough to get you in. Before you can really use the console, though, you’ll need to verify your site. This is a very simple process if you have FTP access. Click the “Verify your site” link. On the next page, select “Upload an HTML file” from the “Choose verification method…” drop-down (you can use a Meta tag if you want, but I find the file method to be less of a hassle). You’ll be given a file name like “google#.html”, where the hash mark is a string of alphanumeric characters. Just open your favorite text editor, save a blank file by that name, and upload it to your site’s base directory. Click the “Verify” button and you should be ready to dig in.

Overview

Once you’re in, you’ll immediately see when your site was last crawled and whether or not it is included in the index. Like a lot of Google’s Webmaster Tools, these are good indicators of possible problems. Obviously, a recent crawl date and proper inclusion are preferable. If you’re not included, Google will give you clues as to why.

Diagnostics

Here you’ll see any problems Googlebot has had trying to crawl your site. This includes pages that couldn’t be accessed for one reason or another, why they were inaccessible, and what sort of crawl (web or mobile) was attempted. Ideally, you don’t want anything to show up here, so be sure to review and, if possible, resolve every error that you see.

Statistics

Everything under Statistics is potentially interesting but not necessarily vital. You can see the top queries your site shows up for, which ones bring the clicks, what words are used in links to your site, what your PageRank distribution looks like, and how many subscribers you have. As Darren Rowse has pointed out, the subscription counts only include Google feed readers, and the numbers don’t often match up with FeedBurner, so be ready for some disparity. If you’re interested in more of this sort of data, I suggest checking out Google Analytics.

Links

The Links tab shows you where your pages are linked, both externally and internally. It’s a far cry from Google’s “link:” operator, providing a breakdown of almost every page that links to your site. The important thing here is not to get overly excited about a large link count. For starters, link juice is about quality, not quantity. Secondly, Webmaster Tools doesn’t distinguish between followed and nofollowed links. So, by all means, scope out the pages that are linking to yours; just don’t expect all of them to be valuable.

Sitemaps

Obviously, you won’t see much here until after you’ve added a sitemap. If you’ve got a WordPress blog, the sitemap generator plugin is a quick way to automate the process. Once you’ve added your sitemap, you can see when it was submitted, when it was downloaded, and whether or not there were any errors reading it. Don’t forget to add a sitemaps autodiscovery line to your robots.txt to make sure other search engines can find it also.

Tools

The Tools section deserves its own special attention because it provides most of the features you can use to communicate with Google.

Analyze robots.txt. This tool will show you the contents of your robots.txt file and allow you to make sure it isn’t blocking anything it shouldn’t.
Manage site verification. If you followed the steps above, you’ll never need to use this. Just be sure the file you uploaded stays right where it is to keep your site verified.
Set crawl rate. If Googlebot seems to visit too often for your liking, you can tell it not to come around as frequently. If your site is large and important enough, you may also be offered the option of faster crawling. Of course, a faster crawl rate doesn’t influence your rankings any; it just lets Google index your newest content that much faster.
Set preferred domain. I mentioned earlier that the domain you entered into the Google Webmaster Tools dashboard and your preferred domain had to match. This is where to make that selection. As above, be sure they’re the same or you won’t be able to submit a sitemap.
Enable enhanced image search. Opt in and Google will more thoroughly index the images on your site. Honestly, I can’t think of a good reason why anybody would say no to better search relevancy. Tick the check box, hit “OK”, and move on.
Remove URLs. Believe it or not, there are times when you want a page removed from the index. It’s rare, especially for bloggers, but it does happen. When the need arises, this is the tool you use to get the offending page delisted.

Dashboard Tools

You may have thought that was all there was, and you’d be partly correct. That’s all Google gives you on a site-specific level. If you click back out to the dashboard, however, you’ll notice several other menu options to the right of your site list.

Message Center. This is a new feature Google added to contact webmasters directly. If they’ve got anything to say about sites you’ve verified, it will show up here.
Download data for all sites. Most of us only have a handful of sites, but some power users may manage dozens or even hundreds. This tool allows you to generate aggregate reports for all the websites linked to your account.
Report spam in our index. The infamous spam report lets anyone report spammy websites in the Google index. Note that you don’t need to be a Webmaster Tools user to report spam to Google. Exactly what effect it has on the reported website is anyone’s guess, but it might be useful if you’re dealing with a scraper.
Report paid links. I doubt that Google considers spam and paid links to be separate issues. However, they do offer separate forms for each. Considering how many bloggers use paid links as a source of revenue, using this tool may amount to bad netiquette.
Request reconsideration. If you’ve been bad and gotten your site thrown out of index, here’s where you beg and plead for Google to let you back in. Be sure to read the fine print and make sure your site adheres to Google’s webmaster guidelines, or else your chances of reinclusion are slim to nil.

More to Come…

Google’s way ahead of the game with Webmaster Tools, giving us the unprecedented ability to monitor how Googlebot interacts with our site. More importantly, they seem to come out with new Webmaster Tools on a regular basis. Learn it, love it, and check out the Google Webmaster Central Blog for regular updates.

Stephen on October 9, 2007 Subscribe to our RSS Feed

Bookmark

41 Responses to “The Blogger’s Guide to Google Webmaster Tools”

Mike Pedersen Golf on October 9th, 2007 4:18 am

It’s so tough to always hear about google. I wish there would be a better, more relevant search engine to battle them. Google has buried many good online businesses for no reason. I realize putting all your eggs in the google basket is not wise business, but I’ve heard of some horror stories of legit sites be obliterated by them.
Daniel on October 9th, 2007 4:28 am

Mike, you can’t argue with the fact that they provide excellent stuff though.

The search engine is the most accurate, AdSense-AdWords is the best online advertising model ever, and so on.

Yet I agree with you that it is not good for anyone except for Google if they solidify their monopolistic position.
Stephen on October 9th, 2007 5:05 am

Mike, you’re correct that it’s inadvisable to “put all your eggs in one basket” by optimizing solely for Google. However, it’s worth noting that good communication with search engines is pretty universal. If you use the feedback from Google Webmaster Tools, it can only help to improve your rankings overall, not just in Google.

As for me, I can’t help but feel like Google’s just about the best thing since sliced bread. Gmail, Google Talk, Google Docs, Google Webmaster Tools, Google Analytics, Google Reader, Google AdSense, Google AdWords, and, of course, Google Search are all a part of my everyday life. As I like to say, I’ll be at the head of the line the day Google starts indexing brains.
Dexter on October 9th, 2007 5:09 am

Nice artticle. I bve been using their service and it help me to find thise dead links at my blog.
Daniel on October 9th, 2007 5:09 am

Haha Stephen, that was funny.
Caribbean Web Development on October 9th, 2007 6:15 am

nice article, I did a similar one a few weeks ago as well

http://www.caribwebdev.com/goo.....-features/
HairySwede on October 9th, 2007 6:30 am

I believe it was Carnegie that said “Put all of your eggs in one basket and watch that basket”

He did ok for himself by watching that basket.
Bull3t on October 9th, 2007 6:39 am

Brilliant article! This will be added to my Weekly Roundup links this week - thanks for sharing this with us. I use Google Webmaster Tools and many other webmaster tools that are offered by other companies. This should help me make the most of Google Webmaster Tools.
Keith on October 9th, 2007 7:40 am

I have been using these tools for some time now. However, for some reason Google shows that the last time GoogleBot accessed my homepage was almost a month ago. However, I can see via the Bot Tracker Wordpress plugin that GoogleBot has still been accessing my homepage on a regular basis (at least a couple times a day).

Should I be concerned about this? I don’t see any obvious errors in the Google Webmaster tools that would lead me to believe that I have a site design problem.

Thanks in advance! - Keith
Stephen on October 9th, 2007 8:10 am

Keith, I wouldn’t be too concerned. If Googlebot is visiting your site, this is most likely a reporting issue rather than a crawling issue. To be sure, check to see if any of your most recent posts have made it into the Google index (query site:yoursite.com). If they have, you shouldn’t have anything to worry about.
Paula Mooney on October 9th, 2007 8:43 am

Thanks, Daniel.

After reading your post I realized I had my robots.txt file in the wrong place on one site, and was missing it totally for 3 other sites!

I hope to God this improves my traffic.

Great post. Good insight. Nice work.

Paula
Stephen on October 9th, 2007 8:55 am

Hi Paula. I hate to be the bearer of bad news, but a robots.txt file will do nothing for your search engine rankings. Its only function is to tell search engines which pages they shouldn’t index. If your goal is higher rankings, I suggest checking out SEOmoz’s list of search engine ranking factors and/or listening to Matt Cutts.
Daniel on October 9th, 2007 9:04 am

Stephen, I will have to disagree with you here .

Consider a Wordpress blog that has full posts all over the places, like single post pages, yearly archives, categories and search results.

Each single post page could be receiving a penalty due to the duplicate content issue, and with a robots.txt file you could remove the categories and archives pages from indexation, improving the ranking from the single post page.

I agree robots.txt wont do miracles, but they might help to fix indexation problems and increase rankings in some extreme cases.
Sergio on October 9th, 2007 9:13 am

great tools, i had a few ive no used, thanks for the tips!
Stephen on October 9th, 2007 9:14 am

There is something to be said for using robots exclusion to combat duplicate content issues. However, this is much better handled by good site architecture. I would never recommend removing your archive and category pages from the index. Rather, I’d say you’re better off using snippets or just links to minimize the problem while keeping those pages indexed.

As for duplication problems on blogs in general, my experience inclines me to believe that Google understands blogs. Unless your site has poor enough architecture that the same huge block of text is repeated more than a handful times across the site, I doubt Googlebot will take much notice.
Jermayn Parker on October 9th, 2007 8:15 pm

Thanks for the tutorial, useful and helps explain what needs to be done. Thanks again
Keith on October 10th, 2007 7:14 am

Thanks for the advice Stephen. I looked into this and only a small portion of my new content is showing up.

It looks like my old Drupal based site is still fully indexed but my new Wordpress site has not been indexed properly. I have added a “Disallow: /drupal/” field to my robots.txt file to eliminate the old drupal website links but it doesn’t appear to be working correctly.

Any ideas on what may be causing this?
Michael on October 19th, 2007 9:30 am

That’s a really nice tool. I’m itching to get my site index so I can play with all the features!
dadi on November 7th, 2007 10:46 am

I’d just like to thank you for taking the time to create this internet website. It has been extremely helpful
GooHackle on November 8th, 2007 10:13 am

There’s an error in Google Webmaster Tools, it isn’t accurate in the “pages that link to yours” option…

In this article explains it: Error in Google Webmaster Tools

Also there is a online tool to count and view the real amount of pages that links to any site: Who links to me
bluefrogx on November 11th, 2007 5:38 pm

That was pretty useful, thanks. I didn’t even know such tools existed.
Affiliates On Fire on January 6th, 2008 9:55 am

Excellent article - however a warning about changing the http://www. prefix - see my article here:
http://www.affiliatesonfire.co.....ww-prefix/

Also, I’ve written a small plugin which allows you to verify easily with Google, Yahoo and MSN webmaster tools in a simple Wordpress options form:
http://www.affiliatesonfire.co.....ss-plugin/
Jones on May 7th, 2008 12:03 pm

Nice job!
sushant kumar viswas on May 12th, 2008 3:46 pm

plz explane me every details,how can i paste google search box in my blog.my site is complete but not there googe search box.without google search box i do not creating extra incom.

thank u
Kirt on June 10th, 2008 3:47 am

Thanks for that brief guide with Google’s Webmaster Tools, it helps alot especially for those online marketers and those who have PPC Campaigns
Robert Orlich on July 1st, 2008 5:36 pm

Hello y’all, I just wanted to say that the spam reporting tool in the google webmasters tools is showing some signs of improvement. I work in SEO in the pharmaceutical business, which y’all know it’s very competitive and full of spam/scams all over.

I’ve used the tool to report competitors spam on forums and blogs where all they do is put repeated and hidden keyword text, with lots of alt stuffed keywords and crosslinks. To some extent I’ve managed, or the people at google that is, to remove a lot of spam pages from the results, altought another one pops up every 2 to 3 days, and I’m talking about the first 10 SERP’s for various keywords.

It’s a little pain in the derrier to sumbit 30 and 40 sites a day, but hey someone’s gotta do it, and if it bounces my sites from #6 to #3 or 2, GREAT.

The whole point is to encourage users to report spam and abuse to Google and help clean the net.

Now some shameless quick promotion
Best priced online viagra at http://www.xlpharmacy.com/viagra-online/

Greetings and good reporting,