Skip to content
FixAEO
Free tool · No signup · Runs in your browser

Free Robots.txt Checker

Paste your robots.txt. See which AI + search crawlers you're allowing or blocking, with concrete AEO recommendations.

Paste your robots.txt

Drop the file content

Open yoursite.com/robots.txt in your browser, copy everything, paste here.

Need a starter robots.txt? Use our Robots.txt Generator.

Paste your robots.txt content on the left. We'll parse it, check each AI + search bot's status, and flag AEO issues.

What we check

Three things, in plain English — per-bot verdicts, AEO red flags, and syntax errors.

24 known crawlers

Every major AI search bot, AI training bot, and classic search engine — verdict computed against your rules + the catch-all.

AEO red flags

Blocking the index-time AI bots, blocking Googlebot, missing Sitemap directive, malformed lines — all surfaced as warnings or errors.

One-click report

Copy a clean markdown report of every bot verdict + issues. Paste straight into a Notion doc or ticket.

Frequently asked questions

Why paste instead of fetch by URL?
Browser CORS policy blocks JavaScript on this page from fetching robots.txt on third-party domains. So you paste it manually — open yoursite.com/robots.txt, copy the whole file, paste here. Nothing leaves your browser. The full FixAEO audit (server-side) does fetch robots.txt automatically for you.
What does 'Partial' verdict mean for a bot?
The bot is allowed in general, but some paths are blocked for it (or for *, which applies to it). It's not fully blocked — but specific URLs may be off-limits. Click into your file to see which Disallow paths are in effect.
Which AI bots matter most?
For AEO: OAI-SearchBot, Claude-SearchBot, PerplexityBot, ChatGPT-User, Claude-User, Perplexity-User, DuckAssistBot. These are the index-time bots that power live citations. Training bots (GPTBot, Google-Extended, Bytespider, CCBot) are a separate decision — blocking them opts you out of model training without affecting today's citations.
Will blocking AI bots hurt SEO?
Classic SEO no — Googlebot is a separate user-agent from Google-Extended. You can block AI training and keep all your Google Search rankings. AEO yes — blocking the index-time AI bots removes you from AI assistant citations entirely.
Why isn't my custom bot listed?
We report on the 24 most common crawlers. If your robots.txt has rules for less-common bots they still apply correctly — they just don't show on the per-bot list. The verdicts for our listed bots are computed correctly regardless of what other rules exist.

Need a starter file?

Our Robots.txt Generator builds an AEO-friendly one in 30 seconds — 24 crawlers covered, smart presets included.

Open Robots.txt Generator