Blog · Oct 6, 2026
Is my site blocking ChatGPT?
The short answer
Possibly — and you'd never know. When an AI crawler is blocked, nothing breaks visibly. Your site looks fine to visitors. But ChatGPT, Claude, and Perplexity quietly stop reading your pages, and your business starts disappearing from their answers.
There are two doors, not one
Blocking happens at two levels, and they don't always agree:
- The sign on the door: robots.txt. This small file at
yoursite.com/robots.txttells crawlers which parts of your site they may visit. A line likeDisallow: /underGPTBotmeans "ChatGPT, stay out." - The bouncer: your server. Even when robots.txt says "come in," your hosting or security setup can still turn the crawler away — bot-protection tools often treat AI crawlers as scrapers and block them silently.
This mismatch is the trap: most site owners check robots.txt, see nothing wrong, and assume they're fine — while the server is blocking every AI crawler at the door.
How to check the sign yourself
Open yoursite.com/robots.txt in your browser (replace "yoursite.com" with your domain). Search the page for GPTBot and ChatGPT-User. If you see Disallow: / next to either, ChatGPT is blocked at the robots.txt level.
But remember: a clean robots.txt is only half the answer. It says nothing about what your server actually does when the crawler knocks.
How blocks happen by accident
Almost nobody blocks AI crawlers on purpose. The usual causes:
- A
Disallow: /copied from a development template and never removed at launch. - An SEO or security plugin with "block AI bots" enabled by default.
- A firewall rule that lumps AI crawlers in with malicious scrapers.
The reliable way to know
You need both checks: what robots.txt says and what the server does — for each major AI crawler, not just ChatGPT. CrawlReady runs exactly that: a robots.txt verdict plus a live fetch for every major AI crawler, side by side, so mismatches are visible in one glance. One token, one full scan.