Every check / blocking-search-engines

Is your website telling search engines not to list it?

We read your homepage's robots tags, the one for Google alone included, and the X-Robots-Tag header sent with the page, for noindex.

How it decides

We also read your robots.txt for a rule that turns every crawler away from the whole site. The check speaks up when any of them is there. robots.txt is read the way search engines read it, group by group, so a rule for one AI crawler is not mistaken for one against Google, and a header aimed at another crawler is left alone.

Run this check on any website

Free, with no account. It sends the site nothing an ordinary visitor would not.

Why it matters

Any one of them keeps you out of search results entirely, and until it goes, nobody finds you by searching.

In more detail

It is usually left over from when the site was being built, and nobody took it off at launch. The header is the easiest to miss, because nothing on the page shows it.

Who fixes it, and where it runs

Who usually fixes it
Whoever built your site
Which plan runs it
The free scan, on any website
Proof you own the site
Not needed. It sends your site nothing an ordinary visitor would not.
More about this check
Who it is about
A search engine
Listed under
Being found
Its name in a report
blocking-search-engines