Google Search works in three stages: crawling (finding and downloading pages), indexing (analysing and storing them) and serving (choosing which pages to show for a search). Not every page makes it through all three, and Google says it does not guarantee that any page will be crawled, indexed or shown.
Stage 1: Crawling
Automated programs called crawlers, mainly Googlebot, discover URLs and download their content. Discovery happens through links from known pages, sitemaps and earlier visits. Googlebot decides which sites to crawl, how often and how many pages to fetch, and it slows down if your server returns errors. It also renders pages using a recent version of Chrome so it can run JavaScript.
What can block crawling: server problems, network issues, robots.txt rules, and pages behind logins.
Stage 2: Indexing
Google analyses the page's text, key tags such as titles and alt text, images and videos, and stores what it learns in the Google index. Near-duplicate pages are grouped, and one is chosen as the canonical, the version that may appear in results.
What can block indexing: low-quality content, a noindex rule, or a site design that makes indexing difficult.
Stage 3: Serving
When someone searches, Google returns pages it judges most relevant and of highest quality for that query. Relevance depends on hundreds of factors, including the person's location, language and device. Ranking is automated, and the features shown vary by query.
An indexed page may still not appear for a query if it is irrelevant, low quality or limited by robots rules.
What this means for site owners
| Stage | What you control |
|---|---|
| Crawling | Links, a sitemap, a healthy server, not blocking Googlebot |
| Indexing | Original, useful content, clear canonicals, no accidental noindex |
| Serving | Relevance and quality for the searches you want, speed and usability |
Some practical points from Google's documentation:
- Google does not accept payment to crawl more often or rank higher.
- Meta keywords are not used, and keyword stuffing is a spam violation.
- There is no ideal word count.
- Duplicate content is inefficient but not a penalty unless it is copied from others.
Where Search Console fits
Search Console lets you see each stage: crawl and index status in the Page Indexing report, a single page in the URL Inspection tool, and results in the Performance report.
Next, read the technical SEO checklist.