About SiteAuditLintBot

The crawler behind SiteAud!tLint: what it is, who runs it, and how to control it with robots.txt.

SiteAuditLintBot is the web crawler inside SiteAud!tLint, a desktop SEO audit tool. It is not a search engine and does not build a public index.

Who is crawling my site?

A SiteAud!tLint user running an audit from their own computer. Each audit is started by that person, usually the site owner, their agency or developer, and the requests come from their IP address, not from a central server. We do not operate the crawl and cannot see which sites users audit.

The user agent is:

SiteAuditLintBot/1.0 (+https://example.com/bot)

Users can change the user agent in their settings, so not every audit identifies itself this way.

How it behaves

• It respects robots.txt by default.
• It fetches HTML pages, robots.txt and XML sitemaps. It checks outbound links with a HEAD request.
• It slows down automatically when a server answers HTTP 429, honours Retry-After, and retries at most 4 times.
• It never submits forms, logs in, or clicks buttons.

Block or limit it

To block it, add this to robots.txt:

User-agent: SiteAuditLintBot
Disallow: /

Report a problem

If an audit caused load on your server, email hello@siteauditlint.com with the time, your domain and the IP address from your logs.