About this bot
ClaudeBot is a crawler associated with Anthropic and used for model development crawling.
It can be identified in HTTP traffic by the ClaudeBot identity family.
Directory status: Active. Identity classification: Verified with High confidence. Identity type: official-documented.
It is recorded as honoring robots.txt under ClaudeBot.
Match the documented ClaudeBot identity and distinguish it from Claude-User and Claude-SearchBot.
How to identify this bot
- User-agent and crawler identifiers
-
ClaudeBot - Also known as
- Anthropic training crawler
Details
- Kind
- Crawler
- Family
- Anthropic
- Purpose
- Model development crawling
- Common Use
- Automated crawling for Anthropic model development and related data collection governed by the ClaudeBot robots token.
- Respects robots.txt
- Yes
Verification
- Identity Type
- Officially documented
- Evidence Method
- Match the documented ClaudeBot identity and distinguish it from Claude-User and Claude-SearchBot.
- Detection Notes
- ClaudeBot is the automated model-development crawler, not user-triggered retrieval.
Website controls
- Robots.txt Snippet
User-agent: ClaudeBot Disallow: /