SiteAud!tLint Support

Warning Technical SEO

Blocked by robots.txt

Severity
Warning
Impact
Medium
Effort to fix
Low
Issue code
blocked_robots

Why it matters

Crawlers are disallowed from fetching this URL, so its content cannot be evaluated or ranked on its merits.

How to fix it

If this page should rank, remove or narrow the matching Disallow rule in robots.txt.

How it affects your score

Finding this issue on any page costs the Technical SEO score 3 points, plus up to 12 more depending on the share of crawled pages affected. Weakens rankings, crawling or measurement. Fix after critical issues.

Find the affected URLs

Open Crawl > Issues and pick Blocked by robots.txt. Each affected URL is listed with its evidence, and Copy URLs or Export URLs hands the list to a developer. The issue also appears in the Action Plan, ranked by impact (medium) and effort (low), and in the issues CSV under the code blocked_robots. After fixing, run a new audit and use Compare to confirm the URLs moved to fixed.

Guides that cover this issue