Bot intelligence record

ClaudeBot

Review first

Use the ClaudeBot identifier to separate Anthropic AI crawler, assistant, or AI search traffic from normal visitor requests in server logs.

Ai Ai Crawl Official Documented Confidence: High Verified: Yes robots.txt: Yes

User-Agent Pattern

Anthropic
ClaudeBot

Robots.txt Snippet

Copy-ready 
User-agent: ClaudeBot Disallow: /

Handling Guidance

Depends

Use this record as bot intelligence, then verify the request source and behavior before allowing, blocking, or rate limiting.

AI crawling, assistant retrieval, AI search, or model-supporting discovery depending on the user-agent.

Record Details

Structured data
Operator
Anthropic
Family
Anthropic
Type
Ai
Purpose
Ai Crawl
Identity type
Official Documented
Confidence
High
Last verified
2026-04-01
Last checked
2026-04-30
Source type
Official

Notes

ClaudeBot is listed in the Botcrawl directory as an AI crawler from Anthropic. The primary identifier for log review is ClaudeBot.

Identification

  • User-agent pattern: ClaudeBot
  • Family: Anthropic
  • Type: AI
  • Kind: Crawler

Common use

AI crawling, assistant retrieval, AI search, or model-supporting discovery depending on the user-agent.

Verification and handling

Anthropic says it does not currently publish IP ranges and may use service-provider public IPs; rely on the official bot token and robots.txt controls.

Directory guidance marks the risk level as Neutral and the blocking decision as Depends. Do not rely on the user-agent string alone because user-agent strings can be copied or spoofed.

Robots.txt handling: Yes.

Evidence and Source

  • Anthropic says it does not currently publish IP ranges and may use service-provider public IPs; rely on the official bot token and robots.txt controls.
  • AI crawling, assistant retrieval, AI search, or model-supporting discovery depending on the user-agent.

https://privacy.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler

Same family: Anthropic

Claude-User Neutral

Claude-User

Claude-User represents user-initiated Claude retrieval activity rather than broad background crawling.

Anthropic Anthropic Ai Ai Assistant
Verified robots.txt: Unknown Block: Depends Official Documented
Claude-SearchBot Safe

Claude-SearchBot

Claude-SearchBot is Anthropic’s search-focused crawler for retrieving web content.

Anthropic Anthropic Ai Ai Search
Verified robots.txt: Yes Block: Depends Official Documented

Similar purpose: Ai Crawl

Google-CloudVertexBot Safe

Google-CloudVertexBot

Google-CloudVertexBot is an AI crawler from Google used for AI-related crawling, retrieval, search, or assistant activity.

Google Google Ai Ai Crawl
Verified robots.txt: Yes Block: Depends Official Documented

WordPress Bot Protection

Bot Blocker for WordPress

Monitor bot traffic, review live activity, and control AI crawlers, scrapers, scanners, spam bots, and fake trusted bots from one clean WordPress dashboard.

Buy Now Start Free Trial