06 / Crawl
After the crawl, SiteAuditLint checks every unique outbound link as well as internal links, and separates truly broken links from sites that simply block automated checks.
Broken outbound links hurt user trust and signal a poorly maintained page. SiteAuditLint collects every unique external link during the crawl and checks them all at the end, so one dead link used on hundreds of pages is requested once.
1. How external links are checked
- A HEAD request first. A HEAD 404 or 410 is trusted as broken.
- Other errors are confirmed with a GET request before being reported.
- Its own pool of 24 connections, at most 4 at a time per external site, so one slow host does not hold up the rest.
- An 8 second timeout per link and a 3 minute limit for the whole check.
2. Broken, could not verify, or not checked
| Result | Responses | Meaning |
|---|---|---|
| Broken | 404, 410, 5xx, unreachable host | The link should be fixed or removed |
| Could not verify | 401, 403, 429, 999, social media links | The site blocks automated checks; test in a browser |
| Not checked: time limit | Links not reached in 3 minutes | Run again or check manually |
WarningLinks to broken external pages
This page links to external URLs that return errors or cannot be reached, which frustrates visitors.
How to fix: Update the links to working pages or remove them.
CriticalLinks to broken internal pages
This page links to internal URLs that return errors, wasting link equity and frustrating users.
How to fix: Update or remove the broken links listed in the detail column.
3. Social media links
Links to Facebook, Instagram, LinkedIn, X, TikTok, Pinterest, Threads, WhatsApp, Telegram and Messenger are marked could not verify without a request, because those platforms block crawlers and would otherwise produce false broken links.
4. Progress you can see
The status line shows the link check as it runs, for example Checking external links 240 / 600, then running site-wide checks, then Loading results. On a 3,000 page test site with slow external links, the post-crawl phase dropped from 118.7 seconds to 39.5 seconds in version 0.6.1.
If other websites link to a page that is now broken, you lose that link equity. Import a backlink export to find these pages: see backlink audit.
Frequently asked questions
Why are some links marked could not verify?
Many sites block automated requests with 401, 403, 429 or 999 responses. Those links may work fine in a browser, so they are not reported as broken. Links to social platforms are marked could not verify without sending a request.
How many external links are checked?
The first 5,000 unique outbound links. The whole check stops after 3 minutes; links not reached are marked Not checked: time limit and the log says how many.
Can I turn external link checking off?
Yes, in Settings. Internal broken links are still found during the crawl.
Want the background first? Browse the SiteAuditLint Academy, or see all features.