When and How Will Google Index My Website?

Daniel Scocco

questions and answersThis post is part of the weekly Q&A section. Just use the contact form if you want to submit a question.

Rick Regan asks:

I have a two-part question about how Google indexes sites:

1) Is Google supposed to index an entire site at once or does it do so incrementally? My blog (on my own domain) appears to be getting indexed incrementally, to the point where it’s taken a month to index all 30+ of my pages. Most of those pages were present before I manually submitted my URL to Google. I’m wondering if I’m doing something wrong or not doing something I should be doing. Does this have anything to do with my blog being new, or having no external links pointing to it?

2) Does Google eventually drop noindex and 404 pages? I have archive pages that got indexed but I have since added to them the “noindex” tag. I also deleted an empty category that Google now gets a 404 on. Will Google eventually remove those pages as it re-crawls my site?

I will answer to each question individually.

1) Most of the times Google will index a new website gradually, yes. That is at least what I have observed with most of my websites. The speed at which Google will index all your internal pages will depend on different factors though.

If you get some very trusted and relevant backlinks, and on top of that you also have a very efficient internal link structure, all your internal pages will get indexed fast. If, on the other hand, you have very few backlinks and a poor link structure, it might take a while before you get to see all your pages indexed.

Keep in mind that using the manual URL submission to Google will have a small impact upon the speed and breadth of your indexation. In fact many people recommend that if you want to get a site indexed fast you should NOT use that feature, and rather focus on getting some trusted backlinks to your site.

2) Yes, eventually Google will fix those issues. New sites don’t get crawled very often, so that is certainly the reason. As soon as Google finds out about the “noindex” tag, for example, it will remove those pages from their index.

Start Your Free Trial Today!


Browse all articles on the Q & A category or check the recommended articles for you below:

124 Responses to “When and How Will Google Index My Website?”

  • Dave

    I called my hosting service and everything looked correct. I have several websites hosted with them and they all have indexed with no problems.

    I am now using adwords for securing traffic. I have back links from high traffic sites and regular websites as well. We will have to wait it out and see what happens.

    I still think it has something to do with having it for sale and information on the website listing it for sale.

  • Ed

    @Dave that seems to be the problem. I would call upi the hosting service to dig into this more.

  • Dave

    I do not pay to have a website indexed. I have had many websites indexed over the years, but this one will not index. I paid more than a normal reg. fee for the name and it has some history going back 6 or 7 years. This is one of three that will not index on Google.

    I did receive a report that the Nameservers for the geoname.com that may be poisoning DNS caches. Replies from these nameservers contain bogus Referrals to other nameservers for top-level Zones. If the poisoning succeeds, future queries for names in the Zone Poisoned would go to one of the Referral nameservers instead of its truly authoritative server.

    I do not know if this is the problem or what it all means.

    As far as a site map, Google has to look at it first and I have a tracker and google analytics and they have not been on the site yet.

  • Ed

    What were for sale? You do not need to pay anything to get indexed. Have you submitted a sitemap to Google yet?

  • Dave

    For the first time, I am have a hard time getting indexed on Google.

    I usually get indexed within just a few days. Both KalamazooRealEstate.com and KalamazooHomesForSale.com will not index.

    They both were for sale and I did pay more than the standard rate.

    Is this one of the reasons why they will not index?

  • Ed

    They do that as there might be something in the URL or the page it connects to of interest to them and worth indexing.

  • Jokomodo

    Does anyone else have a problem, that Google tend to index irrelevant pages like tags and categories (if we are talking about blogs) instead of actual pages?

  • Ed

    @Tom, getting quality backlinks like @Andrew mentioned is a good way too. It makes you more popular and so you might get more Google visits.

  • Ed

    @Tom when you build your XML Sitemap, change the frequency of pages you update daily to “daily”. This way Google knows of it and will come to it on a daily basis. No guarantees though.

  • Andrew Reynolds

    That is probably the best way to check. If your site is new Google does not visit it so often.

    Get some good quality links and that should help. If you keep adding content Google will return more often.

  • Tom

    I want Google to index my blog more frequently. My site has been indexed and I post new content every day. However the last time Google visited the site was 5 days ago.

    I use the site:URL command to check in Google if my new content has been indexed. Not sure if there is another method.

  • Ed

    @JohnMenskil I know how to do that. I was not the one looking for that information anyway. I use Google Webmaster tools for that.

  • C. John Menskil

    @Edward I’m guessing you already figured this out, but there are a few ways to learn how many of your pages are indexed by Google. I’ll mention a couple:

    1) In Google’s search page type in “site:www.domain.com” The results will show you how many pages have been indexed by Google.

    2) Use websites such as Alexa and Web Grader to find out how many times Googles has indexed your website’s pages.

  • Ed

    If you use wordpress, the headers on the pages can be used as meta descriptions too.

  • Paul G

    That is called your meta description. You can change that within your website controls. If you don’t google will just take part of your page. You have about 150 characters and its also a good idea to use your keyword in there.

  • palahubog

    how do I get google to update the cached information on my website? I am talking about the description of my website found in the search results of google.

  • Ed

    Only Google can answer that question for you Paul G. They index pages that are of interest to them. Depends on Keywords you use.

  • Paul G

    Some great ideas here.

    Googlebot visits one of my sites everyday but does not cache it very often. It seems that even though the bot is crawling the site they are not indexing the pages. Is this only done on the days that they cache the site?

    Is this correct?

    If anyone can help I would appreciate it.

  • Ed

    Indexing can take a while as Google has billions of web pages to crawl. But submitting an XML sitemap does help as I have seen them download it in a few minutes after submitting it to them.

  • strewais

    good thoughts but I still have problem with Indexing my my website on google 🙁

  • Ed

    It can take several days or weeks for Google to index your site. Have you submitted it to Google using an XML sitemap also?
    Thanks.

  • Jamie

    I’m having the most frustrating time trying to get my site indexed! GRRrrrrr!

    I have a 30 page website built on x site pro 2 full of unique content, i have at least 200 backlinks, I have my articles and videos popping up on the front page of google from sites like hub pages, squidoo, scribd, youtube, metacafe, dailymotion, veoh, go articles, article city, digg, reddit, stumbleupon…. and many more; however its been 3 days since i finished my site and published it to the web but its still not indexed! Grrrrrr!

    I know its not indexed because when i use google page rank tracker it says its not indexed!! I have always used this tool and I know it works, so why so long this time google???

    Does anyone have any idea why its taking so long??

    You can check for yourself, pregnancy-central.com (make sure you include the hyphen!)

  • Ed

    Always better to not put it under maintenance for a while as you never know when the bots will arrive. Use a web directory submission software to get more backlinks.

  • morphusis

    My website markmystep.com got listed on the Google fairly quickly. But when I started developing the website and put it under maintenance, it somehow got lost and is not showing up on Google search.

    I have put it online now and still it is not showing up on search results. As per your suggestions, I am now working of creating some backlinks. Hope this works out well.

    Thanks for your post. If you could provide any other suggestion, which would speed up the process, that would be helpful.

  • Ed

    Another way is to try to submit to as many web directories as possible.

  • Tooey

    I have set up several different sites and is often an annoyance to get them listed up quickly I agree with everyone else though the fastest way is backlinks usually asking a couple of friends to link to me in there blogs and such gets your page up a lot quicker especially if they are a regular blogger.
    I’ve found submit xml sitemaps helps but I experimented with one of my domains and the backlinks had a much quicker effect.

  • GV Marketing

    The idea of going with tools other than direct Google submission seems to make a lot of sense.
    We are just starting out and will look into backlinking and possibly creating sitemaps for our sites.

    GVMarketing

  • Edward

    How do I find a list of my indexed URLs by Google?
    Thanks.

  • Edward

    Backlinks means high traffic sites having a higher page rank than you linking to your site. Simply means they have your URL somewhere on their site pointing to you.

  • aman

    Yes, Make sure you dont have major errors in your code, validate your html, link building will also help.

Comments are closed.