When and How Will Google Index My Website?

By Daniel Scocco

questions and answersThis post is part of the weekly Q&A section. Just use the contact form if you want to submit a question.

Rick Regan asks:

I have a two-part question about how Google indexes sites:

1) Is Google supposed to index an entire site at once or does it do so incrementally? My blog (on my own domain) appears to be getting indexed incrementally, to the point where it’s taken a month to index all 30+ of my pages. Most of those pages were present before I manually submitted my URL to Google. I’m wondering if I’m doing something wrong or not doing something I should be doing. Does this have anything to do with my blog being new, or having no external links pointing to it?

2) Does Google eventually drop noindex and 404 pages? I have archive pages that got indexed but I have since added to them the “noindex” tag. I also deleted an empty category that Google now gets a 404 on. Will Google eventually remove those pages as it re-crawls my site?

I will answer to each question individually.

1) Most of the times Google will index a new website gradually, yes. That is at least what I have observed with most of my websites. The speed at which Google will index all your internal pages will depend on different factors though.

If you get some very trusted and relevant backlinks, and on top of that you also have a very efficient internal link structure, all your internal pages will get indexed fast. If, on the other hand, you have very few backlinks and a poor link structure, it might take a while before you get to see all your pages indexed.

Keep in mind that using the manual URL submission to Google will have a small impact upon the speed and breadth of your indexation. In fact many people recommend that if you want to get a site indexed fast you should NOT use that feature, and rather focus on getting some trusted backlinks to your site.

2) Yes, eventually Google will fix those issues. New sites don’t get crawled very often, so that is certainly the reason. As soon as Google finds out about the “noindex” tag, for example, it will remove those pages from their index.



Share

124 Responses to “When and How Will Google Index My Website?”

  • Bryan Alvarez

    Wow good to hear that Ed πŸ™‚ but the problem is google did not visited my site again until now. And also what is the difference between typing (link:mysite.com w/o space recommended by google) which does gave me “0” results and typing (link: mysite.com w/ space) which shows some of the links that ive already made. I have made a couple of backlinks already, does this means that the first command (w/o space) w/c is suggested by google is incorrect?

  • Ed

    They should eventually. Google stores a page cache until it changes.

  • Bryan Alvarez

    Thank you Ed for your question, you kinda remind me of my big mistake here. My site was indeed not ready yet when google managed to indexed it. When google tried to crawl my page it was still under maintenance, so that’s maybe the reason why the cached version always looked like what it is “Preview Coming Soon”..(I found this by performing cache:mysite.com). But my site was already go on live now, should the snapshots will change to the actual page?

  • Ed

    Was the page ready when indexed by Google?

  • Bryan Alvarez

    When my i check my indexed pages from google, it will return a snapshots showing “Preview coming soon”. Clicking the link will return the correct page but i wonder why google was able to display the correct snapshot of my page that was indexed and instead it only display a blank picture with text “Preview Coming soon”. Can we do something about this?

    You can check this for yourself:
    site:portconnectintl.com

  • Ed

    There is no way you can control the Google Bot.

  • smartlife

    I feel our site are in google sandbox, I want to know how to get out of the effect quick, Is there tips for deal with this trouble?

  • Umesh

    I noticed weird thing. I think it depends on which google site I am querying currently. When I do site:mysite from different location like Australia, UK, India google gave me different result.

    Hmm pretty interesting…..

  • Ed

    That might happen when Google re-visits the site. There is no way of knowing what the bot might index.

  • Umesh

    I noticed google has indexed 22 of my pages but now again it came back to 4 pages. Do you have any idea why this will happen??

    Thanks in advance.

  • Ed

    It really depends on what Google would like to index. That is out of your control.

  • Michael P

    I just switched my domain on a pre-existing site over last week. I have about 500 pages that need indexing. Currently only 14 pages have been indexed, and already submitted a site map.

    I’m expecting all my current pages to be updated within 2 months. I really hope I’m wrong on this estimation. But so far it seems like it’s been adding 2-4 pages a day.

    …………
    PS: sorry if this is duplicate post…not sure if it submitted the 1st time.

  • Ed

    Have you tried Facebook? Not sure if it works as well as Digg.

    • justinthyme

      I would like for Google to index my site more frequently. My site is already indexed and I post new several times a week. However the last time Google visited the site was 13 days ago…any suggestions on how to increase frequency? I have posted links to twitter, digg, wordpress, FB etc with no luck.

  • Sell My Business

    Digg is a nice way to get indexed, but you should also look at other Web 2.0 sites, such as Twitter, Hubpages, etc.

    Digg has been around for a few years, so it’s not as valued as it once was.

  • Blackout Blinds

    Posting my sites to Digg always seems to short-circuit this process. I’m a little worried about the sandbox issue one poster alluded to. Should I be worried about publishing or indexing my sites too soon?

  • Ed

    You will have to wait till Google’s next visit. You cannot tell them anything about how to do their job. They will index any or all pages on your site depending on the appropriate content/keywords they deem fit to use. A title can help with the indexing process, but it is not written in stone.
    Thank you.

  • Umesh

    Hi Ed,
    Thanks for the reply. Now it seems my site is indexing. But the thing is I changed some settings little bit like the way title appears and something. Google indexed my page according to my previous setting which was blog name – title.

    IS there a way to tell Google re-index all my pages according to new page structure?

    Many Thanks.

    • Ed

      There a lot of things Google looks at to determine if any of your pages warrants indexing by their engine. What site are you referring to?
      Good Luck.

  • Ed

    It really depends on what and how Google decides to index.

  • Umesh

    I just build up my website and I am wondering how to increase page rank. Even if I searched using keyboards that I tagged, Google doesn’t show up my page for next 10 pages. πŸ™

    Any suggestion besides wait and see.

  • aaslin

    Hi,initially my website posts was indexed very quickly within hours,but now it takes many days for my post to be indexed.This all happens after i got pagerank 2

  • TriCorePillow

    Hai Daniel,

    Nice info. Digg always become my choice for fast google index!

  • Ed

    It takes time. But Google does find everyone eventually.

  • MrsD

    Hi Juanita, don’t give up. If you already register yourself to Google webmaster tools, and google analytics, it will need a while. I also almost gave up, because i havent found any datas of my blog, but tonight, Google finally found me πŸ™‚

  • Juanita

    Hi. Il ike your advice. I have my first blog up and running. But don’t know when google will index it. I hope it will be soon. Please visit and leave some comments and advice. Thanks

  • Ed

    They normally do index it pretty quick. Only if there is a problem, it might take longer.

  • vivek

    One Question i want to ask you that when i post a new post in my blog and when i search my blog with blog name i shows only few old and some time aonly formal pages it never shows latest post on search engine can u suggest me what the matter i am as begnnier wnt some tips from u r side,

  • inupinu

    Many times I observed that Google indexes pages almost instantly.
    I am exploring what processes are involved in fastest google indexing

  • Ed

    Where is your site hosted? What is the name? It can take anywhere from a day to a week or even a month for Google to index your site.

  • Shaishav

    I have my website hosted a week back. i have added the google code to my website. But when i search the google i cant find my website. I have added keywords also on my site. What else do i need to do to top it up on Google. Its a corporate website hence cant host it on other websites or cross link it. Please help.

  • Ed

    @Dave having it for sale should not matter with it being indexed. They are two different issues.

Comments are closed.