What is SEOBolt best for?

SEOBolt is built for recurring technical SEO audits, LLM Search readiness checks, apply-ready fix plans, AI-agent remediation over MCP, scheduled monitoring, and health-score emails.

Does SEOBolt replace enterprise SEO suites?

No. SEOBolt is transparent about its data sources and is focused on audits, monitoring, and workflow. It does not have a proprietary backlink index like large enterprise SEO suites.

Does SEOBolt work with AI assistants?

Yes. SEOBolt has a built-in MCP server for Claude and a REST API. AI agents can sign up, run audits, and get results programmatically.

Is there a free plan?

Yes. The free plan includes 2 projects, 3 crawls per month, and 100 pages per crawl. You can also run a single-page audit at /free-audit without signing up. Paid plans are billed monthly; see /pricing.

Free SEO tools / AI Crawler Access Checker

robots.txt AI bot checker: is your site blocking GPTBot, ClaudeBot or PerplexityBot?

Enter a domain (and optionally a path). The tool reads robots.txt and shows, for each AI crawler, whether it may fetch / and your path. It then requests your homepage once with each crawler's published user-agent string and compares the answer with a normal browser, which exposes firewall and CDN rules that block AI crawlers even when robots.txt allows them.

Free, no sign-up, no email needed.

How it works

We request https://<domain>/robots.txt and parse it with the longest-match rules of RFC 9309.
We request https://<domain>/ eight times, once with a normal browser user agent and once with each of OpenAI's OAI-SearchBot, ChatGPT-User and GPTBot, Perplexity's PerplexityBot and Perplexity-User, Common Crawl's CCBot and Apple's Applebot, using the example user-agent strings those vendors publish. Each request has a 6 second timeout.
We flag a crawler when it gets a blocking status (401, 403, 406, 429, 451, 503) or a bot-challenge page while the browser request does not.
Every request goes through a guard that refuses private, loopback and cloud-metadata addresses, checks each redirect, and caps size and time.
The live requests come from our server, not from the vendors' published IP ranges, so we do not pretend to be the real crawler: a firewall that verifies crawlers by IP can answer differently to the real one.

Limits

Claude-SearchBot, Claude-User and ClaudeBot: Anthropic publishes the robots.txt tokens and IP ranges but no user-agent string, so only the robots.txt verdict is shown for them.
Google-Extended and Applebot-Extended are robots.txt tokens that send no requests of their own, so they have a robots.txt verdict only.
Only the homepage is fetched live; the path you enter is checked against robots.txt only.
Bytespider is not covered: its operator publishes no documentation we could read, so its token and user agent could not be verified.
3 runs every 5 minutes per IP address.

Sources: OpenAI crawlers · Anthropic crawlers · Perplexity crawlers · Common Crawl CCBot · Applebot

Frequently asked questions

Which crawler decides whether I appear in ChatGPT search?

OpenAI documents OAI-SearchBot as the crawler for ChatGPT search results; sites opted out of it are not shown in ChatGPT search answers. GPTBot is a separate crawler for model training, and ChatGPT-User fetches pages when a ChatGPT user's request needs them.

If I block GPTBot, am I removed from ChatGPT?

Per OpenAI's documentation, GPTBot and OAI-SearchBot are independent settings: disallowing GPTBot signals that content should not be used to train its models and does not opt you out of ChatGPT search.

robots.txt allows the bot, but the live test shows a block. Why?

robots.txt is a request to polite crawlers. A firewall, CDN or bot-protection rule can still return 403 or a challenge page to a user agent. Check your WAF or CDN bot settings and the vendor's published IP list.

Does a passing live test mean the real crawler gets in?

Not necessarily. We send the vendor's user-agent string from our own server. Firewalls that verify crawlers by IP address, as the vendors recommend, may treat the real crawler differently. A pass means no rule keyed on the user agent alone is blocking it.

Why is Perplexity-User or ChatGPT-User allowed even though robots.txt blocks it?

The tool reports what robots.txt says. OpenAI and Perplexity both document that user-initiated fetchers may not follow robots.txt rules, so a robots.txt block may not stop them.

Want the whole site checked, not one URL?

The free audit runs an instant check of your domain across SEO and AI-search readiness, with a fix for each issue. No sign-up.

Run the full free audit for your site

How it works

Limits

Frequently asked questions

Related

Want the whole site checked, not one URL?