02 / Crawl
The technical SEO audit covers crawlability and indexability: HTTP status codes, redirects, canonical tags, robots directives, XML sitemaps and URL structure. It carries the most weight in the Site Health score.
A technical SEO audit answers one question first: can search engines fetch, render and index the pages you want to rank? SiteAuditLint checks every crawled URL for status code errors, redirect problems, indexing directives, sitemap health and URL hygiene, then explains why each issue matters and how to fix it.
1. HTTP status codes and fetch errors
Every URL is grouped by response: healthy 200 pages, redirects, broken 4xx and 5xx responses, blocked URLs and pages that could not be fetched at all because of timeouts, DNS failures or SSL errors. The Site Health view shows the split as a status code chart.
CriticalPage could not be fetched
The server returned no usable response (timeout, DNS, SSL or connection failure). Search engines cannot index what they cannot fetch.
How to fix: Check server uptime, SSL certificate validity, DNS and firewall or bot protection rules for this URL.
CriticalClient error (4xx)
The URL returns an error such as 404 Not Found. Visitors and crawlers reach a dead end and link equity is lost.
How to fix: Restore the page, 301 redirect it to the most relevant live URL, or remove the links that point to it.
CriticalServer error (5xx)
The server failed while generating the page. Repeated 5xx responses can cause pages to drop out of the index.
How to fix: Check server and application logs for this URL and fix the underlying error.
2. Redirects, redirect chains and redirect loops
Each redirect is followed to its final destination. Internal links that point at redirecting URLs waste crawl budget and add latency, and long redirect chains can stop crawlers before they reach the final URL.
InfoRedirecting URL
This internal URL redirects. Each redirect adds latency and spends crawl budget.
How to fix: Update internal links so they point directly to the final destination.
WarningRedirect chain
The redirect target redirects again. Chains slow users down and can stop crawlers before the final URL.
How to fix: Make the first redirect point straight to the final URL.
CriticalRedirect loop
The redirect never reaches a final page because it points back to a URL already in the chain. Browsers show an error and search engines give up.
How to fix: Point the redirect at the correct live page and remove the circular rule.
3. Indexability: noindex, canonicals and robots.txt
A page can return 200 and still be kept out of the index. The audit reads the meta robots tag, the X-Robots-Tag header, the rel=canonical tag and your robots.txt rules to decide whether each URL is indexable.
WarningNoindex directive
The page tells search engines not to include it in results, through a meta robots tag or X-Robots-Tag header.
How to fix: Remove the noindex directive if this page should appear in search results.
WarningBlocked by robots.txt
Crawlers are disallowed from fetching this URL, so its content cannot be evaluated or ranked on its merits.
How to fix: If this page should rank, remove or narrow the matching Disallow rule in robots.txt.
OpportunityMissing canonical tag
Without a canonical tag, search engines pick the preferred version of duplicate or parameter URLs on their own.
How to fix: Add a self-referencing <link rel="canonical" href="..."> in the head.
InfoCanonical points to another URL
This URL asks search engines to index a different URL instead of itself.
How to fix: Confirm this is intentional, and link internally to the canonical URL rather than this one.
OpportunityNo robots.txt file
robots.txt returned an error. Crawlers fall back to crawling everything and you lose a place to declare sitemaps.
How to fix: Publish a robots.txt at the site root with a Sitemap: line.
4. XML sitemap validation
The sitemap should list only live, indexable, canonical URLs. SiteAuditLint discovers sitemaps from robots.txt and common locations, compares every listed URL with what the crawl found, and flags conflicting signals.
WarningNo XML sitemap found
No sitemap was found in robots.txt or at /sitemap.xml. Sitemaps help search engines discover and prioritise URLs.
How to fix: Generate an XML sitemap, reference it in robots.txt and submit it in Google Search Console and Bing Webmaster Tools.
WarningNon-200 URL in XML sitemap
Sitemaps should only list live, indexable URLs. Errors and redirects in the sitemap reduce trust in it.
How to fix: Remove this URL from the sitemap or replace it with its final destination.
WarningNon-indexable URL in XML sitemap
The sitemap lists a URL that is noindexed or canonicalised elsewhere, which sends conflicting signals.
How to fix: Remove non-indexable URLs from the sitemap.
5. URL structure problems
Messy URLs create duplicate versions of the same page and are harder to share. The URL Structure view lists every URL with one of these patterns.
OpportunityURL contains uppercase letters
URLs are case sensitive, so mixed case can create duplicate URLs and broken links when typed or shared.
How to fix: Use lowercase URLs and 301 redirect uppercase versions to them.
OpportunityURL contains underscores
Search engines treat hyphens as word separators but not underscores, so words run together.
How to fix: Use hyphens instead of underscores in new URLs.
InfoURL contains parameters
Parameters can create many URLs for the same content and waste crawl budget.
How to fix: Link to clean URLs where possible and use canonical tags on parameter variations.
OpportunityURL over 115 characters
Very long URLs are harder to read, share and remember, and are often truncated in search results.
How to fix: Shorten the URL to the words that describe the page.
OpportunityURL contains non-ASCII characters
Special characters must be encoded in URLs, which can break links in some tools and emails.
How to fix: Use plain ASCII letters, numbers and hyphens in URLs.
OpportunityURL contains a double slash
Double slashes in the path usually come from template bugs and can create duplicate URLs.
How to fix: Fix the link or template that generates the double slash.
6. Technical audit workflow
- Crawl the siteStart from the homepage with Respect robots.txt on, the same way search engines see it.
- Fix critical issues firstStart with fetch errors, 4xx and 5xx pages and redirect loops in Critical Issues.
- Clean up redirectsPoint internal links straight to final URLs and collapse chains into single hops.
- Align indexing signalsMake sure sitemaps, canonicals and noindex directives agree with each other.
- Audit again and compareRun a new audit and check the Compare view for fixed and new issues.
Redirecting URLs, canonicalised URLs and URL parameters are informational. They are shown so you can confirm intent, and they never reduce the Technical SEO score.
Frequently asked questions
What is the difference between a redirect chain and a redirect loop?
A chain is a redirect whose target redirects again before reaching a live page. A loop never reaches a final page because the redirects point back at each other. SiteAuditLint follows every redirect to its end, reports the hop count for chains, and marks loops as critical.
Does a canonical pointing elsewhere lower my score?
No. Canonicalised URLs are reported as info so you can confirm they are intentional. Info issues never lower the score. A missing canonical tag is an opportunity with a small weight.
Why is Technical SEO weighted more than other categories?
If crawlers cannot fetch or index a page, nothing else on it can rank. Technical SEO has a weight of 1.5 in the overall score, the highest of any category.
Want the background first? Browse the SiteAuditLint Academy, or see all features.