Does nextjs.org block ChatGPT, Claude and Perplexity?
nextjs.org has no robots.txt, so every AI crawler is allowed by default.
Checked nextjs.org/robots.txt on (HTTP 404: no file). Category: Developer tools and SaaS. Profile: Open to every AI crawler we test.
What it means
nextjs.org publishes no robots.txt, so nothing stops the OpenAI, Anthropic, Perplexity, Google, Common Crawl or Apple crawlers from reading the homepage. Whether an assistant cites the site then depends on its content and authority, not on crawler access.
robots.txt is a request, not a lock: it does not show what a crawler does in practice, and it cannot reveal firewall rules. Blocking is often a deliberate business decision.
Crawler by crawler
| Crawler | Verdict and the rule behind it |
|---|---|
OAI-SearchBotOpenAI · search | AllowedThe site has no robots.txt, so nothing restricts this crawler. |
ChatGPT-UserOpenAI · user request | AllowedThe site has no robots.txt, so nothing restricts this crawler. |
GPTBotOpenAI · model training | AllowedThe site has no robots.txt, so nothing restricts this crawler. |
Claude-SearchBotAnthropic · search | AllowedThe site has no robots.txt, so nothing restricts this crawler. |
Claude-UserAnthropic · user request | AllowedThe site has no robots.txt, so nothing restricts this crawler. |
ClaudeBotAnthropic · model training | AllowedThe site has no robots.txt, so nothing restricts this crawler. |
PerplexityBotPerplexity · search | AllowedThe site has no robots.txt, so nothing restricts this crawler. |
Perplexity-UserPerplexity · user request | AllowedThe site has no robots.txt, so nothing restricts this crawler. |
Google-ExtendedGoogle · model training | AllowedThe site has no robots.txt, so nothing restricts this crawler. |
CCBotCommon Crawl · model training | AllowedThe site has no robots.txt, so nothing restricts this crawler. |
ApplebotApple · search | AllowedThe site has no robots.txt, so nothing restricts this crawler. |
Applebot-ExtendedApple · model training | AllowedThe site has no robots.txt, so nothing restricts this crawler. |
What each crawler does, and where its token is documented, is on the AI Crawler Access Checker.
Other signals AI tools read
- llms.txt
- Found: /llms.txt answers with plain text (14,810 bytes).
- Sitemap
- robots.txt lists no sitemap. /sitemap.xml is a valid sitemap.
- Structured data on the homepage
- No JSON-LD found.
- Open Graph and canonical
- og:title missing; canonical link missing.
Compared with developer tools and saas
Of the 102 developer tools and saas sites we read, 2 (2%) block GPTBot, 2 (2%) block ClaudeBot and 0 (0%) block PerplexityBot. nextjs.org blocks 0 of 12.
If you run a site like this
Two separate choices: whether AI assistants may find and cite your pages (search crawlers) and whether your content may be used for training. A robots.txt can answer them differently.
Stay citable in AI search
User-agent: OAI-SearchBot
Allow: /
User-agent: Claude-SearchBot
Allow: /
User-agent: PerplexityBot
Allow: /Opt out of training only
User-agent: GPTBot
Disallow: /
User-agent: ClaudeBot
Disallow: /
User-agent: Google-Extended
Disallow: /Questions
Does nextjs.org block GPTBot?
No. The site has no robots.txt, so nothing restricts this crawler. Read 19 November 2026.
Does nextjs.org block ClaudeBot?
No. The site has no robots.txt, so nothing restricts this crawler. Read 19 November 2026.
Does nextjs.org have an llms.txt file?
Found: /llms.txt answers with plain text (14,810 bytes). Read 19 November 2026.
Check your own site the same way
Free audit, no signup: homepage, robots.txt, sitemap.xml and llms.txt, with the fix for each issue.
No signup and no payment details needed. Checks the homepage, robots.txt, sitemap.xml and llms.txt.
Snapshot of 19 November 2026. Data for all sites: CSV. Something wrong, or want this page removed? Email support@seobolt.app. SEOBolt builds the audit tool this data comes from.