03 / Indexing
An indexing test checks whether search engines can discover, crawl, render, and index a page after a change. It answers a simple question with evidence: did the change make pages eligible for search, and did search engines actually pick them up?
This lesson covers when to run indexing tests, how to use Search Console, sitemaps, logs, and crawls to measure indexing, and how to test changes such as noindex removal, canonical fixes, and new templates.
1. When to run an indexing test
- After launching a new site, section, or template
- After removing noindex or robots.txt blocks
- After canonical or duplicate content fixes
- After moving to server-side rendering
- After a migration or URL change
- When pages are stuck as Discovered or Crawled, currently not indexed
2. Test eligibility first
Before waiting for search engines, confirm the pages can be indexed at all.
| Check | Pass condition |
|---|---|
| Status code | 200 |
| robots.txt | Not disallowed for Googlebot |
| Meta robots and X-Robots-Tag | No noindex |
| Canonical | Self-referencing or pointing to the intended URL |
| Rendered content | Main content present after rendering |
| Internal links | At least one crawlable internal link |
| XML sitemap | Listed with an accurate lastmod |
A crawl confirms eligibility at scale. See the Indexing lesson and the issues for noindex and noindex URLs in sitemaps.
3. Measuring indexing
| Method | What it shows | Limits |
|---|---|---|
| URL Inspection | Index status, selected canonical, last crawl, rendered HTML | One URL at a time, with daily quotas |
| Page indexing report | Indexed and not indexed counts with reasons | Delayed by days |
| Sitemap-filtered report | Indexing status for URLs in one sitemap | Requires separate sitemaps per test group |
| Server logs | When Googlebot requested each URL | Shows crawling, not indexing |
| site: and exact phrase search | Spot check of indexed content | Incomplete and not a reliable count |
Put the test pages in their own XML sitemap and submit it in Search Console. The Page indexing report can then be filtered to that sitemap, giving a clean indexed versus not indexed count for the test group.
4. Running an indexing test
- Record the baselineIndexed count for the test group, crawl dates, and status reasons.
- Ship the changeRecord the date and exact change.
- Verify eligibilityCrawl the test pages and confirm every check passes.
- Prompt discoverySubmit the sitemap, and request indexing for a small sample of key URLs.
- MonitorTrack logs, URL Inspection samples, and the sitemap-filtered report weekly.
- CompareReport the change in indexed share against the baseline.
An indexing test result
Indexed: 410Template noindex on variant pages.
Indexed: 1,080Noindex removed and canonicals corrected.
Report the indexed share, not only individual URL checks.
5. Pages that stay unindexed
If eligible pages remain unindexed, the cause is often quality or importance rather than a technical block. Check for thin or duplicate content, weak internal linking, deep click depth, and whether the pages add unique value. See thin content and duplicate content.
6. Testing removals
Indexing tests also work in reverse. When adding noindex to low-value pages, confirm the pages remain crawlable so search engines can see the noindex, and do not block them in robots.txt at the same time. Remove them from the sitemap once they have dropped out of the index.
7. Practical exercise: Test one group of pages
Indexing test checklist
0 of 8 tasks completed
Key takeaways
- Indexing tests confirm that changes make pages eligible and indexed.
- Check eligibility with a crawl before waiting for search engines.
- A dedicated sitemap per test group gives clean indexing counts.
- URL Inspection, Page indexing, and logs each show different parts of the process.
- Eligible pages that stay unindexed often have quality or linking problems.
- For removals, keep pages crawlable so noindex can be seen.
Knowledge check
Next, test how titles affect clicks in the Title tests lesson.