Every check / ai-training-opt-out
Which AI training crawlers does your robots.txt turn away?
We read your robots.txt for rules that turn away the crawlers that gather training data, such as GPTBot, ClaudeBot and CCBot, and list the ones it blocks.
How it decides
A note with no price on it: blocking a training crawler costs no visibility. It stays quiet when none is blocked.
Run this check on any website
Free, with no account. It sends the site nothing an ordinary visitor would not.
Why it matters
Whether a company may train on your pages is your choice, and this says what you have chosen, so you can see it was deliberate.
In more detail
Many hosts and firewalls add these rules on their own; an owner who wants to be quoted by assistants still is, because the crawlers that answer questions are a different set, which another check reads.
Who fixes it, and where it runs
- Who usually fixes it
- Whoever built your site
- Which plan runs it
- The free scan, on any website
- Proof you own the site
- Not needed. It sends your site nothing an ordinary visitor would not.
More about this check
- Who it is about
- A search engine
- Listed under
- Being found
- The standard
- Its name in a report
ai-training-opt-out