Is your site blocked from AI crawlers? Check in 5 minutes — with proof.
One blocked line in robots.txt can quietly keep GPTBot, ClaudeBot, or PerplexityBot from ever reading your pages — often by accident. Run the free check, see exactly which of 9 crawler identities you allow, and fix the gaps with the exact rule in front of you.
Readiness evidence. Not rankings, traffic, or AI citation tracking.
What the check tells you
For each of 9 crawler identities, you get one of three honest answers — each tied to the rule that produced it:
Allowed
The robots.txt directives that apply to this crawler permit the page path. Real evidence was found for this conclusion.
Blocked
A Disallow rule in the crawler's applicable group covers the path. Whether that is intentional is a decision only you can make.
Not evaluated
No robots.txt evidence could be read, so access is unknown — never assumed allowed. Absence of a file proves nothing.
What this check does not claim
The result is a reading of directives, not a measurement of outcomes:
- — AI rankings or placement in answers. robots.txt says nothing about ranking.
- — Citation eligibility in ChatGPT, Perplexity, or any assistant.
- — Successful indexing. A directive is not an index report.
- — Guaranteed crawling. robots.txt is a request; visits depend on each operator.
- — Firewall behavior. A WAF or CDN rule can block a crawler that robots.txt allows.
Access is the first of four signals
The full visibility audit pairs this machine-access view with SEO, GEO, and AEO Beta diagnostics for the same URL — one report, four signals, Fix First priorities included.
Frequently asked questions
How do I check if AI crawlers are blocked from my site?
Read your robots.txt file and evaluate it per crawler: find the most specific user-agent group for each bot, then the longest matching Allow or Disallow rule for the page path. Our free checker does this for 9 crawler identities at once and shows the rule behind each answer. A free account is required; no credit card is needed.
If a crawler shows Allowed, will ChatGPT or Perplexity cite my site?
No. robots.txt only controls whether a crawler may request your pages. Whether an AI system retrieves, ranks, or cites your content depends on many factors outside robots.txt, and no tool can measure those from a robots file. Allowed means the door is open, not that anyone walked through it.
Does blocking GPTBot hurt my Google rankings?
No. GPTBot is OpenAI's training crawler and is completely separate from Googlebot. Blocking it has no effect on Google search rankings. But note the split: OAI-SearchBot (not GPTBot) is what feeds ChatGPT search results — decide each bot separately.
Why does everything say Not evaluated for my site?
The checker could not read a robots.txt file for that origin — it was missing, unreachable, or returned unreadable text. In that case every crawler is reported as not evaluated instead of assumed allowed. No evidence, no confirmation.
Can any tool guarantee my site will appear in AI answers?
No. Access is one technical prerequisite among many. Content quality, structure, freshness, and each provider's own selection all matter, and no tool can promise a citation. Anyone guaranteeing placement is selling certainty that does not exist.
Last updated: September 10, 2026