Which AI bots can read your site? Check all 16 at once.
Type your domain. See whether GPTBot, ClaudeBot, PerplexityBot, Google-Extended and twelve more are allowed or blocked, grouped by company and purpose, with the exact robots.txt line behind every answer.
Reads only your public robots.txt. Results for a domain are cached for ten minutes.
Allowed is step one. Being found is step two.
Your Search Console already shows which searches find you. Bing Webmaster shows the rest.
This checker covers one site at a time. Murkuz connects Search Console, Analytics and Bing Webmaster to Claude or ChatGPT for every site you run. Ask which long, question-style queries reach you, which pages they land on, and how Bing compares with Google, all from the same chat.
Search_AnalyticsBing_Search_PerformanceAll_Sites_OverviewThree kinds of AI crawler, three different decisions
Most AI companies now run more than one bot, and each one does a different job. Blocking the wrong one is the most common mistake we see: a site blocks every OpenAI bot to stop training and quietly drops out of ChatGPT search too.
Training
GPTBot, ClaudeBot, Google-Extended, Applebot-Extended, CCBot, Bytespider, Meta-ExternalAgent, cohere-ai. They collect pages to train future models.
AI search
OAI-SearchBot, Claude-SearchBot, PerplexityBot, Amazonbot. They build the index that answers are written from, with links back to you.
User fetch
ChatGPT-User, Claude-User, Perplexity-User. They load one page live when a person asks the assistant about it.
Blocking training does not stop AI citations
If you want your content kept out of model training but still want ChatGPT, Claude and Perplexity to cite you, block the training bots and leave the search and user bots allowed. If you block the search and user bots, those assistants cannot read your pages at answer time, so they cannot quote you or send you a click.
Why one bot can ignore your User-agent: * rules
A crawler follows only the most specific group that names it. If your file has a group for GPTBot, everything under User-agent: * stops applying to GPTBot, including any Disallow lines you meant for every bot. That is why this checker shows the exact matched rule for each crawler instead of a single yes or no.
Frequently asked questions
How do I check if AI bots can crawl my site?+
Type your domain above. The checker downloads your live robots.txt, works out which rule group applies to each of 16 AI crawlers, and shows whether each one is allowed or blocked with the exact line that decides it. You can also paste a robots.txt to test changes before you upload them.
Is my site blocking ChatGPT?+
Look at the OpenAI rows. GPTBot collects training data, OAI-SearchBot builds the index behind ChatGPT search, and ChatGPT-User fetches a page when a person asks about it. If OAI-SearchBot and ChatGPT-User are allowed, ChatGPT can find and cite your pages even when GPTBot is blocked.
Does blocking GPTBot remove me from ChatGPT answers?+
No. GPTBot is the training crawler. ChatGPT search results come from OAI-SearchBot, and live page reads come from ChatGPT-User. Blocking only GPTBot keeps your content out of future training while leaving you visible in ChatGPT search.
Does blocking Google-Extended hurt my Google rankings?+
No. Google-Extended is a control token, not a separate crawler. Googlebot still crawls your site for Search as before. The token only tells Google whether your content may be used to train and ground its Gemini models.
How does a crawler decide which rule applies?+
A crawler looks for the User-agent group that names it most specifically and follows only that group. If GPTBot has its own group, the rules under User-agent: * no longer apply to it. Inside the group, the longest matching path wins, and Allow wins a tie.
Is this AI crawler checker free?+
Yes. No login and no email. It reads only your public robots.txt file, the same file any crawler can see.
One site checked. Now ask about every site you run.
Murkuz connects Google Search Console, Analytics and Bing Webmaster to Claude or ChatGPT. Ask which questions bring people to each site, which pages they land on and what Bing sends, and get the rows back in the chat.
Free tool. No login. Murkuz is the connected version: your Search Console, Analytics and Bing data, answered inside Claude or ChatGPT.