ChatGPT Search vs GPTBot vs OAI-SearchBot

OpenAI operates three distinct tokens with different access scopes. Most teams are still managing them as one policy.

What changed

OpenAI launched ChatGPT Search in October 2024 with a dedicated search crawler, OAI-SearchBot, independent of the training crawler GPTBot. A third token, ChatGPT-User, handles live user-triggered fetches — OpenAI’s bot documentation is explicit that it is not an automatic crawler and that robots.txt rules may not apply to it, because a user initiates each request. Each token controls a distinct product decision and each can be independently allowed or blocked. Before search-specific access entered the picture, most teams only thought about one OpenAI crawler decision — now there are three distinct access questions.

Why it matters

Operators who block GPTBot believing they’ve blocked ChatGPT Search have not. Operators who allow GPTBot believing they’ve opened ChatGPT Search have not done that either. The confusion is bidirectional and extremely common — teams are making high-confidence infrastructure decisions with a one-bot mental model that has been obsolete since late 2024.

What’s still true

• Training access and search access are fully independent decisions — one token controls neither both nor the other.
• ChatGPT-User is triggered by live user actions, not scheduled crawls — robots.txt applicability to this token is uncertain.
• Infrastructure controls — WAF rules, CDN rewrites, rate-limiting — override robots.txt before any bot reads the directive.
• Allowing the right crawler is necessary but not sufficient — crawlability, content quality, and trust signals still determine whether a page is cited.
• OpenAI publishes verified IP ranges — use these to validate actual access against network rules.

What to do now

Audit the three tokens separately

• Review robots.txt for OAI-SearchBot, GPTBot, and ChatGPT-User as three independent decisions.
• Confirm which decision is intentional and which was inherited or copy-pasted from an older policy.
• Document the intended state for each token before any change — see Enable AI Search Access for the full workflow.

Check infrastructure controls

• Verify CDN and WAF rules are not blocking OAI-SearchBot even when robots.txt allows it.
• Rate-limiting rules that affect crawlers can silently produce incomplete indexes.
• Cross-reference OpenAI’s published IP ranges (see Sources) against server access logs.

Separate search policy from training policy

• Training opt-out (GPTBot block) and search inclusion (OAI-SearchBot allow) are both legitimate and independent configurations.
• Decide each based on actual business intent, not assumed defaults.
• Treat ChatGPT-User as a separate category — don’t rely on robots.txt to manage it.