Warning AEO / GEO
AI crawlers blocked in robots.txt
- Severity
- Warning
- Impact
- Medium
- Effort to fix
- Low
- Issue code
ai_crawlers_blocked
Why it matters
Some AI and answer-engine crawlers are disallowed, so the site may not be cited in AI search results.
How to fix it
Decide which AI crawlers you want to allow and update robots.txt accordingly.
How it affects your score
Finding this issue on any page costs the AEO / GEO score 3 points, plus up to 12 more depending on the share of crawled pages affected. Weakens rankings, crawling or measurement. Fix after critical issues.
Find the affected URLs
Open Crawl > Issues and pick AI crawlers blocked in robots.txt. Each affected URL is listed with its evidence, and Copy URLs or Export URLs hands the list to a developer. The issue also appears in the Action Plan, ranked by impact (medium) and effort (low), and in the issues CSV under the code ai_crawlers_blocked. After fixing, run a new audit and use Compare to confirm the URLs moved to fixed.