Google’s John Mueller Reveals How Often a Site is Re-indexed in Search
In Google’s latest Q&A video, John Mueller answers a question from a site owner concerned about how often a site is re-indexed and that their site not being crawled often enough.
Since Googlebot can only crawl a certain amount of pages, it gives priority to pages that should be crawled more frequently. The most “important” pages, according to Google, are the home page and high-level category pages. Those pages will be crawled most often, so changes will be reflected in Google search faster than, say, an old blog post.
John Mueller’s full response is quoted below.
“Looking at the whole website all at once, or even within a short period of time, can cause a significant load on a website. Googlebot tries to be polite and is limited to a certain number of pages every day. This number is automatically adjusted as we better recognize the limits of a website. Looking at portions of a website means that we have to prioritize how we crawl.
Our Insights: Digital marketers are always looking for ways to grab more attention from Google in order to drive more traffic from the search engine.
Crawl rate and indexing can become a serious problem if you feel as if your website is not being crawled as often as you think it should, and this can potentially affect your rankings.
It seems that John Mueller made a pretty clear response on how digital marketers can win over Googlebot: You simply have to focus more of your attention towards the pages that are considered important to Google.
In other words, you would want to spend more time optimizing your site's home page and high-level category pages better than an old blog post that you've published over a year ago.
Besides, in the world of Search Engine Optimization – it's all about hitting the right notes.
Image from searchenginejournal.com