Tool
Can AI engines read your site?
Paste your robots.txt. The checker shows which AI retrieval bots are allowed, partially blocked or blocked, and which of those verdicts cost citations.
What this checks
Whether each AI-related user agent can fetch pages under the rules you pasted, following the most-specific-group rule. It does not test CDN bot management or security plugins, which block bots before robots.txt is read; see the crawler guide for those.
| User agent | Role | Verdict | Affects |
|---|---|---|---|
| GooglebotGoogle | search | — | Google search, AI Overviews and AI Mode |
| Google-ExtendedGoogle | training | — | Gemini training and grounding |
| OAI-SearchBotOpenAI | retrieval | — | ChatGPT search results and answers |
| ChatGPT-UserOpenAI | retrieval | — | ChatGPT fetching a page to answer a question |
| GPTBotOpenAI | training | — | OpenAI model training |
| PerplexityBotPerplexity | retrieval | — | Perplexity index and answers |
| Perplexity-UserPerplexity | retrieval | — | Perplexity fetching a page on request |
| Claude-SearchBotAnthropic | retrieval | — | Claude search results |
| Claude-UserAnthropic | retrieval | — | Claude fetching a page on request |
| ClaudeBotAnthropic | training | — | Anthropic model training |
| BingbotMicrosoft | search | — | Bing, Copilot, and part of ChatGPT's index |
| ApplebotApple | search | — | Siri and Spotlight answers |
| Applebot-ExtendedApple | training | — | Apple model training |
| DuckAssistBotDuckDuckGo | retrieval | — | DuckAssist answers |
| meta-externalagentMeta | training | — | Meta AI training and answers |
FAQ
About the checker.
The retrieval bots: OAI-SearchBot and ChatGPT-User (OpenAI), PerplexityBot and Perplexity-User, Claude-SearchBot and Claude-User, plus Googlebot, Bingbot and Applebot. They fetch pages to answer live questions. Training bots such as GPTBot and ClaudeBot do not affect citations.
robots.txt is one of four layers. CDN bot management, security plugins and JavaScript-only rendering each block retrieval bots independently. The checker only reads the file; the crawler guide covers the other three.
Yes. Block GPTBot, ClaudeBot, Applebot-Extended and meta-externalagent, and leave the retrieval bots open. The checker shows the two groups separately.
No. Parsing happens in your browser. Nothing is uploaded or stored.
Still have questions? Talk to us →
Free citation audit
Find out who AI recommends instead of you.
Written report plus a thirty-minute walkthrough. No pitch until you ask for one.