Every check / ai-policy-unstated
Does your robots.txt say what AI crawlers may do with your site?
We read your robots.txt for a rule naming any AI crawler, or a Content-Signal line saying what AI may do with the pages.
How it decides
The check speaks up when neither is there, as a note with no price on it.
Run this check on any website
Free, with no account. It sends the site nothing an ordinary visitor would not.
Why it matters
Saying nothing leaves every AI crawler to do as it likes, and leaves you unable to say later that you did not agree.
In more detail
Writing the policy down is three lines: which crawlers may read the site to answer questions, and whether any may train on it. Either answer is fine; the gap is having none.
Who fixes it, and where it runs
- Who usually fixes it
- Whoever built your site
- Which plan runs it
- The free scan, on any website
- Proof you own the site
- Not needed. It sends your site nothing an ordinary visitor would not.
More about this check
- Who it is about
- A search engine
- Listed under
- Being found
- Its name in a report
ai-policy-unstated