Opportunity Technical SEO
No robots.txt file
- Severity
- Opportunity
- Impact
- Low
- Effort to fix
- Low
- Issue code
robots_missing
Why it matters
robots.txt returned an error. Crawlers fall back to crawling everything and you lose a place to declare sitemaps.
How to fix it
Publish a robots.txt at the site root with a Sitemap: line.
How it affects your score
Finding this issue on any page costs the Technical SEO score 0.5 points, plus up to 4 more depending on the share of crawled pages affected. A smaller improvement with a modest effect on its own.
Find the affected URLs
Open Crawl > Issues and pick No robots.txt file. Each affected URL is listed with its evidence, and Copy URLs or Export URLs hands the list to a developer. The issue also appears in the Action Plan, ranked by impact (low) and effort (low), and in the issues CSV under the code robots_missing. After fixing, run a new audit and use Compare to confirm the URLs moved to fixed.