Crawler · GPTBot · OpenAI

GPTBot: how to block or allow OpenAI's training crawler

By Grady Coleman (Founder) · Last updated: September 30, 2026

GPTBot is OpenAI's training crawler: it fetches content that may be used to train models. It is a separate control from OAI-SearchBot, the crawler that feeds ChatGPT search. Here are the exact robots.txt lines for each decision — and what blocking GPTBot does not do.

Check whether GPTBot reaches your domain

Key takeaways

— GPTBot is the training opt-out token. OAI-SearchBot is the ChatGPT search token. Decide them separately.
— The token must match exactly: GPT-Bot, ChatGPT, and OpenAI match nothing.
— An empty Disallow means allow — that is not a typo.
— Blocking GPTBot has no effect on ChatGPT search visibility, and allowing it never guarantees appearance in answers.

What GPTBot actually does

According to OpenAI's crawler documentation, GPTBot crawls content that may be used to train OpenAI's models. It is not the crawler that decides whether your pages can appear in ChatGPT search — that is OAI-SearchBot's job. The two tokens exist precisely so a site can opt out of training while staying eligible for search, or the reverse. Most sites mean the first and accidentally do both.

Block GPTBot — opt out of OpenAI training

User-agent: GPTBot
Disallow: /

This blocks the whole site for GPTBot only. Every other crawler — including OAI-SearchBot — is a separate group and keeps working. To block GPTBot from specific folders instead of the whole site, set the path:Disallow: /private/.

Allow GPTBot explicitly — after a wildcard block

User-agent: GPTBot
Disallow:

If your file blocks everything with User-agent: *, a group for the exact token overrides it for that crawler. The empty Disallow: means allow. This is the line most sites are missing when a checker reports GPTBot as blocked and they did not mean it.

The combined recipe: training off, ChatGPT search on

# ChatGPT search: allowed
User-agent: OAI-SearchBot
Disallow:

# OpenAI training: blocked
User-agent: GPTBot
Disallow: /

This is the decision most sites actually want: opt out of OpenAI training data while staying eligible for ChatGPT search. Save, upload, then open the live file at yourdomain.com/robots.txt to confirm the change survived — CMS apps and deploys can overwrite it.

GPTBot vs OAI-SearchBot: the mix-up that costs visibility

GPTBot — training. Fetches content that may be used to train OpenAI models. Blocking it is a training-data decision. It has no effect on ChatGPT search, Google, or any other system.

OAI-SearchBot — search. Crawls for ChatGPT search results. Blocking it is what affects your eligibility in ChatGPT search — and per OpenAI's documentation, retrieval permission and result inclusion remain separate decisions: access never guarantees appearance.

The common mistake. A site blocks GPTBot, believes it has left ChatGPT (it has not), or allows GPTBot believing it will now be cited in answers (it will not). The checker evaluates each of the nine crawler identities separately so a training decision never hides inside a search verdict.

What blocking GPTBot does not do

Point-in-time configuration evidence for the submitted page path only. Blocking or allowing GPTBot says nothing about visits, indexing, training runs, ranking, citation in answers, traffic, or revenue. No robots.txt rule can promise an AI answer will cite you — and a firewall or WAF can still stop a crawler that robots.txt allows.

Run the free access check Rules for all nine crawlers

Frequently asked questions

How do I block GPTBot in robots.txt?

Add a group that names the token exactly: User-agent: GPTBot followed by Disallow: / blocks the whole site for GPTBot. Only that exact token is affected — OAI-SearchBot, PerplexityBot, and every other crawler are separate groups and keep working. After saving, open yourdomain.com/robots.txt in a browser and confirm the lines are live, because CMS apps and deploys can overwrite the file.

Does blocking GPTBot remove my site from ChatGPT search?

No. Per OpenAI's crawler documentation, GPTBot crawls content that may be used for model training, while OAI-SearchBot crawls specifically for ChatGPT search results. They are independent controls: blocking GPTBot is a training opt-out only. Blocking OAI-SearchBot is what affects ChatGPT search retrieval — and blocking one never blocks the other.

Does GPTBot affect my Google rankings?

No. GPTBot is OpenAI's crawler and has no role in Google Search. Googlebot handles Google Search crawling, and Google-Extended is Google's separate control token for Gemini training use. The three are unrelated decisions.