01 / Experiments
An SEO experiment tests whether a change actually improves search performance. Instead of shipping a change sitewide and hoping, you apply it to a group of pages, keep a similar group unchanged, and compare the results. This turns SEO opinions into evidence.
This lesson explains how SEO testing differs from conversion testing, how to write a hypothesis, how to build test and control groups, how long to run a test, and how to read results without fooling yourself.
1. Why test SEO changes
Search performance changes for many reasons at once: algorithm updates, seasonality, competitors, and other releases. Without a control group, it is impossible to know whether a change helped, hurt, or did nothing. Testing also limits risk, because a harmful change affects only part of the site.
2. SEO tests versus A/B tests
| Aspect | Conversion A/B test | SEO split test |
|---|---|---|
| What is split | Users | Pages |
| Who sees the variant | Half of visitors | Every visitor and search engine on the variant pages |
| Main metric | Conversion rate | Organic clicks or impressions |
| Why the difference | Users can be randomized | Googlebot sees one version of each URL |
Never show search engines a different version of a page than users see. SEO tests change the page itself for everyone, on a subset of pages.
3. Writing a hypothesis
A testable hypothesis names the change, the pages, the expected effect, and the metric.
If we add the product's price range to the title tag
on category pages,
then organic click-through rate will increase,
because searchers compare prices before clicking.
Primary metric: organic clicks from Search Console.4. Types of SEO test
| Test type | How it works | Best for |
|---|---|---|
| Split test | Randomly split similar pages into variant and control groups, change only the variant | Large sites with many pages on one template |
| Time-based test | Change pages and compare before and after, ideally against unchanged pages | Smaller sites and one-off changes |
| Staged rollout | Release to one section first, then expand if results hold | Risky template or technical changes |
| Holdout | Keep a small group unchanged after a sitewide rollout | Confirming long-term effects |
5. Building test and control groups
- Choose one templateUse pages of the same type, such as product or category pages.
- Ensure enough trafficEach group needs enough clicks to show changes above normal noise.
- Split randomly or by stratified samplingBalance traffic levels and seasonality between groups.
- Check the groups matchBefore the test, both groups should trend similarly over several weeks.
- Change only the variantKeep everything else identical during the test.
6. How long to run a test
Search engines must recrawl and reprocess changed pages before results appear. Most SEO tests run for at least two to six weeks, depending on crawl frequency and traffic. Record the date the change went live, request recrawls for key URLs, and avoid ending a test early because of a few good days.
7. Reading results
Compare against control
The difference between groups matters, not the raw change in the variant.
Account for noise
Daily clicks vary. Look for a sustained difference over time.
Watch for external events
Algorithm updates during a test can affect groups unevenly.
Check secondary metrics
Impressions, CTR, and position explain why clicks changed.
Document everything
Record hypothesis, dates, groups, results, and decision.
Statistical methods such as causal impact models or confidence intervals help separate real effects from noise on larger sites.
8. Practical exercise: Plan one SEO test
SEO experiment checklist
0 of 9 tasks completed
Key takeaways
- SEO experiments test changes on groups of pages, not groups of users.
- A control group separates the effect of a change from outside factors.
- Write a hypothesis with change, pages, expected effect, and metric.
- Test pages should share one template and have enough traffic.
- Allow time for recrawling and run tests for several weeks.
- Never show search engines different content from users.
Knowledge check
Next, validate one of the riskiest SEO changes in the Redirects lesson.