Every check / no-robots-txt

Does your site have a robots.txt file for crawlers to read?

We ask for /robots.txt, the first file every crawler requests from a site.

How it decides

The check speaks up when it is missing, as a note with no price on it.

Run this check on any website

Free, with no account. It sends the site nothing an ordinary visitor would not.

Why it matters

Its absence does no harm on its own.

In more detail

It is the place to point search engines at your sitemap and to decide which AI assistants may read your content, so a site without one has made neither choice.

Who fixes it, and where it runs

Who usually fixes it
Whoever built your site
Which plan runs it
The free scan, on any website
Proof you own the site
Not needed. It sends your site nothing an ordinary visitor would not.
More about this check
Who it is about
A search engine
Listed under
Being found
Its name in a report
no-robots-txt