What Is Crawl Budget?
Crawl budget is the number of pages on your site that a search engine will crawl within a given period of time. It’s not unlimited — search engines allocate more or less crawling capacity to a site based on its size, authority, and how efficiently it serves pages.
What Affects Crawl Budget
Two main factors determine how much crawl budget a site gets: crawl rate limit (how much load Googlebot decides your server can handle without causing problems) and crawl demand (how much Google actually wants to crawl your site, based on its perceived importance and how often content changes).
Sites with slow server response times, large volumes of low-value pages (thin content, duplicate parameter URLs, orphan pages), or poor internal linking tend to burn crawl budget on pages that don’t matter, leaving less left over for the pages that do.
How to Optimise It
Crawl budget is mainly a concern for large sites — thousands of pages or more — where wasted crawling can genuinely mean important pages go unindexed or get recrawled infrequently. Fixes include blocking low-value URLs via robots.txt, fixing broken links and redirect chains, improving server speed, and keeping XML sitemaps clean and current so search engines know exactly which pages matter.
