Cloudflare Workers let you change HTML, headers, and redirects at the edge without touching your origin. That makes them powerful for SEO fixes and risky for SEO at the same time, because the changes never appear in your CMS. This guide shows how to audit Cloudflare Workers for SEO issues by checking what the edge actually serves.
This guide is part of our Cloudflare for SEO series.
What Workers are used for in SEO
Teams typically use Workers and HTMLRewriter to:
- Inject or fix hreflang, canonical, or structured data tags
- Rewrite titles and meta descriptions without a CMS release
- Handle redirects and URL normalization
- Run A/B tests on page templates
- Serve prerendered HTML to bots for JavaScript-heavy sites
Each of these is valid. Each also creates a gap between what your developers see in the codebase and what Googlebot receives.
Step 1: List every Worker route
In the Cloudflare dashboard, open Workers Routes for your domain and list each route pattern and the Worker attached. Also check Transform Rules and Snippets, which can change headers and HTML too. Patterns with wildcards often cover more URLs than intended, so note the real scope.
Step 2: Compare edge HTML with origin HTML
For one URL per route, fetch the page through Cloudflare and directly from the origin, then compare the SEO-critical elements:
| Element | What to check |
|---|---|
| Title and meta description | Present once, matches intent |
| Canonical | One tag, absolute URL, self-referencing where expected |
| Meta robots | No unexpected noindex or nofollow |
| Hreflang | Complete, reciprocal, valid language codes |
| Structured data | Valid JSON-LD, not duplicated |
| Internal links | Not rewritten to redirecting or broken URLs |
A common bug is duplication: the origin adds a canonical and the Worker adds another, so the page ends up with two conflicting tags.
Step 3: Check response headers
Workers can add, change, or drop headers. Use curl -sI https://example.com/page and check:
- X-Robots-Tag: no noindex on pages that should rank
- Link: canonical or hreflang headers consistent with the HTML
- Cache-Control: not caching personalized or test variants
- Vary: correct if HTML differs by device or language
Step 4: Test redirects and URL handling
Check how the Worker treats URL variants: trailing slash and no trailing slash, uppercase letters, query parameters, and HTTP versus HTTPS. Each variant should resolve in one hop to a single canonical URL. Inconsistent handling creates duplicate URLs that compete with each other.
Watch for loops. A Worker that adds a trailing slash, combined with an origin that removes it, produces an infinite redirect. Browsers show an error, but cached copies may hide the problem from your team for days.
Step 5: Compare bot and user responses
If the Worker branches on user agent, fetch the page as a browser and as Googlebot and compare. Serving prerendered HTML to bots is acceptable when content is equivalent. Serving different content, links, or text to bots risks being treated as cloaking.
curl -s -A "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" https://example.com/page
Then confirm with the URL Inspection live test, which shows the HTML Google actually rendered.
Step 6: Handle Worker errors safely
When a Worker throws an exception or hits its limits, visitors and crawlers can get a 5xx error. Check the Worker's error rate in its metrics, and make sure the code falls back to passing the origin response through instead of failing.
Workers SEO audit checklist
- All Worker routes, Transform Rules, and Snippets listed with real scope
- Edge and origin HTML compared for each route
- One canonical and no unexpected noindex per page
- X-Robots-Tag and Link headers checked
- URL variants resolve in one hop
- Bot and user HTML equivalent
- Worker error rate reviewed and fallback in place
- Full crawl compared before and after the last Worker deploy
The last item matters most. A comparison crawl catches regressions across every URL, not just the ones you spot check. See SEO regression testing after Cloudflare changes.
Frequently asked questions
- Do Cloudflare Workers help or hurt SEO?
- Both are possible. Workers can speed up fixes and improve performance, but untested edge changes can break canonicals, redirects, and indexability.
- Does Google see changes made by a Worker?
- Yes. Googlebot receives the response after the Worker runs, so edge changes count exactly like origin changes.
- Is serving prerendered HTML to Googlebot cloaking?
- Not if the content is equivalent to what users see. Different content or links for bots is the risk.
Key takeaway
Audit what the edge serves, not what the codebase says. Every Worker route should be checked for HTML, headers, and redirects, and every Worker deploy should be followed by a comparison crawl.
Related: Cloudflare for SEO · Diagnosing crawling problems on Cloudflare · Cloudflare logs vs origin logs