When and How Will Google Index My Website?

By Daniel Scocco

questions and answersThis post is part of the weekly Q&A section. Just use the contact form if you want to submit a question.

Rick Regan asks:

I have a two-part question about how Google indexes sites:

1) Is Google supposed to index an entire site at once or does it do so incrementally? My blog (on my own domain) appears to be getting indexed incrementally, to the point where it’s taken a month to index all 30+ of my pages. Most of those pages were present before I manually submitted my URL to Google. I’m wondering if I’m doing something wrong or not doing something I should be doing. Does this have anything to do with my blog being new, or having no external links pointing to it?

2) Does Google eventually drop noindex and 404 pages? I have archive pages that got indexed but I have since added to them the “noindex” tag. I also deleted an empty category that Google now gets a 404 on. Will Google eventually remove those pages as it re-crawls my site?

I will answer to each question individually.

1) Most of the times Google will index a new website gradually, yes. That is at least what I have observed with most of my websites. The speed at which Google will index all your internal pages will depend on different factors though.

If you get some very trusted and relevant backlinks, and on top of that you also have a very efficient internal link structure, all your internal pages will get indexed fast. If, on the other hand, you have very few backlinks and a poor link structure, it might take a while before you get to see all your pages indexed.

Keep in mind that using the manual URL submission to Google will have a small impact upon the speed and breadth of your indexation. In fact many people recommend that if you want to get a site indexed fast you should NOT use that feature, and rather focus on getting some trusted backlinks to your site.

2) Yes, eventually Google will fix those issues. New sites don’t get crawled very often, so that is certainly the reason. As soon as Google finds out about the “noindex” tag, for example, it will remove those pages from their index.



Related Articles

Please install the YARPP plugin

Share

124 Responses to “When and How Will Google Index My Website?”

  • James Robert

    Also make your site relevant to the user’s search terms and thereby Google – get lots of back links and if you’re feeling behind the competition check them out. I use back link checker – just Google it. Scopes out where my competition is getting their back links and I have found it a good way to find relevent directories to sign up for – and thereby get links from.

    Posting on blogs doesn’t hurt either 🙂

  • Lester

    I am really new to this internet marketing game I have to say I really like this site and I am using it to try and get higher rankings with my word press blog right now it seems as though I can’t even get indexed hoping to get that moving with uh what do you call it link juice lol

    thanks for everyones comments

  • David

    Getting a good ranking on google takes along time. I know how easy it is to get discourages when huge websites or even incompetent sites rank better and easier than yours. Just keep adding a reasonable amount of backlinks everyday and make sure you add to the site everyday with new, interesting content.

    goodluck

  • Chris

    Thanks for your answers. I have been having the same questions recently as my blog was getting listed but some pages of my website were not. The information that I got from everyone really helped me to understand just exactly how search engine works.

    Thanks again

    Chris

  • Sean the Pay Raise Guy

    @Jacklin – thanks, I may well do, that’s a very kind offer!

    My website is about how to negotiate a salary increase, but it’s appearing very difficult to rank highly over some of the articles from “About” etc.

    I’ve been working on backlinks and now have used a social bookmarking service to get me sorted, though that has only just finished so I’d imagine it will take google a few weeks to recognize the backlinks.

    I have read that .gov and .edu backlinks are extremely valuable also but most seem to be nofollow so I’m not getting recognition from them.

    Yahoo also seems to have about 90 backlinks recognized for me now, whereas google have none…..

  • Natasha Weston

    I have been waiting 2 weeks for my meta description to be updated, today I have added 3 monre pages to my site and I am about to submit my first sitemap to google.

    I am interested to know if Google will visit my site really fast as it’s my first site map submission?

    This is a very interesting post and I am off to work on some more backlinks.

    I did notice that no-one has mentions .edu backlinks as I was under the impression that Google likes these as they are from an authoritative domain extension. I have added about 50 in the past few weeks and I am hoping they have some impact too!

  • Rahul Malik

    Thanks for the info.
    My site was getting indexed within minutes after i had made a post, but for a week my server was down and i was not in town. when i came back, and started my site again. the google just takes days to index any new posts and i have lost all my traffic.

  • Nick

    I got index in google in 1 week after i added sitemap & robot txt, but how i am suppose to know on which page my website is showing in search result on my keyword? any suggestion?

  • SFIhomebizz

    I am using google webmaster tools, sitemap.xml, bookmarked on social web sites but I found that my sitemap.xml is not listing all blog posts hence not getting indexed in google too

  • Becky

    Help, I launched my new site and submitted it to google. Its in wordpress and I have used the settings to let everyone see it. For some reason google wont index it, am I doing something wrong?

  • Jacklin Octave

    Hey Sean,

    If you want to get in touch with me, please contact me on facebook buddy.

    Just make a search for “Jacklin Octave”

    It’s easy to understand social bookmarking sites.

    @Mike, you don’t have to pay for indexing tho. Just let google and other search engines do it for you, but you have to register in their webmaster tools.

    Please, free to contact the owner of that blog, or should me an email for more details.

    Take care!

    Jack

  • Sean

    Interesting read – can anyone give me some more insight into social bookmarking sites? I’ve heard delicious and digg mentioned in posts but I’m not aware of how they work or why I would use them.

    Thanks in advance,

  • Mike Desilva

    You have to watch out for companies that will overcharge your for “indexing service”. Some companies will take advantage of consumers/businesses and charge up to $300 dollars for this service.

    starcomsystems.net is a prime example.

  • Jacklin

    There are a plethora of ways to get indexed. You can actually use social bookmarking sites, google webmaster tools, or the quickest way is to submit your website to all the ping sites.

    Think about it, every time you make a change or a post, just ping it and you will see how gradually your traffic grows.

    And the secret is by doing what i’m doing right now….LOL

    Find out!!!

    Take care

    Jacklin

  • InkRaider

    Very nice blog.

    I am also optimizing a site. I use ezinearticles, digg, mixx, forums, press releases, etc to bring my site up in search. My site is improving on daily bases.

    I hope one day it gets on first page of the site.

  • Ed

    @Bryan, you are frustrated with just one web site. Imagine how frustrated the Google Bot can get, he has billions of site to crawl. He cannot run, so be patient.

  • Ed

    There is really no fast way to do this. If you go too fast then Google will think you just bought all those links and might even ban your site. Slow organic link building is the best way to go.

    • Bryan Alvarez

      thanks Ed, it really sounds frustrating though because now the WMT again shows no brandnew links for my site for 3 days. My blogs and directory submissions didnt give any favorable results and so were my blogs and forums participations. So these does not automatically give us links or google just missed those links during certain crawls? I want some clarifications regarding when a link will be found or listed in WM or in any link checker tools? thnks

  • Bryan Alvarez

    Hello,

    In Google’s point of view, which one it valued the most those links from directory listings, blogs, articles, or comments…Id try to do each of these but it seems directory listings works for me most of the time if i checked in webmastertools. I know making blogs/articles is a long process but im learning it now and hoping to post a better one someday. I just want to know what works the most so that i can concentrate on that to help boost my site’s ranking. My site is about a month old now but it’s still has “0” ranking and I don’t gaining enough links everyday, do you know some ways to gain links really fast? I mean in legal way possible because its bad to called spammers and it wont be good for our site. thanks

  • Ed

    Definitely you need to wait. Google has to crawl billions of pages daily. Keep that in mind.

  • Bryan Alvarez

    Ok thank you Ed…I will wait for Google to update the snapshots then. I just think it has something to do on how the webpage was designed or coded but according to you it was Google’s behavior on taking snapshots only sparingly..:D

  • Ed

    It takes time for Google to do updates on an individual basis. It looks for links more often than it takes snapshots. So be patient and it will happen.

  • Bryan Alvarez

    hey guys i really found this one disturbing. Google had crawled my site again just yesterday and took the snapshot on my current page. It managed to retain the cached version of the page that was indexed. But the problem is the image shown in the search results still the (Blank) one or the (“Preview Coming Soon”) that i stated on my previous post. I really dont know why they cant show me the correct image of the page in the search results. Can anyone explain this to me?…i need your patience guys…:(

  • Ed

    Who is SEO Quake? What queries do you run once a day?
    Thanks.

  • Buta

    I usually use SEO Quake because when you do a site command of that website you can sort by PR. I also set my Google to 100 links per page. I also use livepr.info but you can only run queries once an hour 🙁

    Hope this helps!
    Charles

  • Ed

    You can set the visit frequency of the Google Bot in your WM tools.

  • Bryan Alvarez

    Yeah, i already submitted my site to Google using (webmaster tools). By the way do we have a tool that will monitor the Googlebot’s activity (e.g crawl/index schedules for our site) aside from WM tools?..thank you

  • Ed

    Have you submitted your site to Google?

  • Bryan Alvarez

    Google still did not revisit my site up to now. They were mentioning some techniques above which i already do to no avail. Is there really an effective way to let google revisit our site and re-indexed it?…sorry if this question was already answered on previous comments but i want to be more precise and brief this time…:(

  • Ed

    You will have to wait for Google to re-visit the site in this case.

  • Ed

    The space is an extra character that is interpreted differently by browsers. I would use the one without the space to avoid confusion.

    • Bryan Alvarez

      Im also using the one without space, though im getting “0” results in it. Does this mean that google didn’t found my links at all or had my links became invalid?…when i perform (link:www.mysite.com) on google, ive got “0” links found..

Comments are closed.