Bot intelligence record

Claude-SearchBot

Usually allow

Use the Claude-SearchBot identifier to separate Anthropic AI crawler, assistant, or AI search traffic from normal visitor requests in server logs.

Ai Ai Search Official Documented Confidence: High Verified: Yes robots.txt: Yes
Operator
Anthropic
Family
Anthropic
Type
Ai
Source type
Official
Last checked
2026-05-20

User-Agent Pattern

Anthropic
Claude-SearchBot
Verification note

User-agent strings are identification signals, not proof of identity. Confirm important allow, block, or rate-limit decisions with logs, DNS or IP evidence, request behavior, or operator documentation when available.

Robots.txt Snippet

Click snippet to copy 
User-agent: Claude-SearchBot Disallow: /

Click the snippet to copy it, or highlight the text manually.

Handling Guidance

Depends

This bot is usually safe to allow when the request source is verified and the traffic matches your site policy.

AI crawling, assistant retrieval, AI search, or model-supporting discovery depending on the user-agent.

Record Details

Structured data
Operator
Anthropic
Family
Anthropic
Type
Ai
Purpose
Ai Search
Identity type
Official Documented
Confidence
High
Last verified
2026-04-01
Last checked
2026-05-20
Source type
Official
Verification
Anthropic does not currently publish dedicated IP ranges; use the official token and robots.txt controls.
Spoofing risk
User-agent strings for Claude-SearchBot can be spoofed. Treat user-agent detection as a classification signal, then verify with published IP ranges, reverse DNS, signatures, operator documentation, or published operator documentation, IP ranges, reverse DNS, signatures, or other verified identity signals before allow-listing.

Notes

Claude-SearchBot is listed in the Botcrawl directory as an AI crawler from Anthropic. The primary identifier for log review is Claude-SearchBot.

Identification

  • User-agent pattern: Claude-SearchBot
  • Family: Anthropic
  • Type: AI
  • Kind: Crawler

Common use

AI crawling, assistant retrieval, AI search, or model-supporting discovery depending on the user-agent.

Verification and handling

Anthropic does not currently publish dedicated IP ranges; use the official token and robots.txt controls.

Directory guidance marks the risk level as Safe and the blocking decision as Depends. Do not rely on the user-agent string alone because user-agent strings can be copied or spoofed.

Robots.txt handling: Yes.

Evidence and Source

  • Anthropic does not currently publish dedicated IP ranges; use the official token and robots.txt controls.
  • Match `Claude-SearchBot` as a case-insensitive substring in HTTP user-agent logs. Review bot_aliases for alternate names or product labels. Do not treat a user-agent match alone as proof of identity for allow-listing.
  • AI crawling, assistant retrieval, AI search, or model-supporting discovery depending on the user-agent.
  • User-agent strings for Claude-SearchBot can be spoofed. Treat user-agent detection as a classification signal, then verify with published IP ranges, reverse DNS, signatures, operator documentation, or published operator documentation, IP ranges, reverse DNS, signatures, or other verified identity signals before allow-listing.

https://privacy.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler

Same family: Anthropic

ClaudeBot Neutral

ClaudeBot

ClaudeBot is Anthropic’s crawler used to discover and retrieve public web content for Claude-related systems.

Operator and family: Anthropic Type: Ai Purpose: Ai Crawl
Verified robots.txt: Yes Block: Depends Official Documented
Claude-User Neutral

Claude-User

Claude-User represents user-initiated Claude retrieval activity rather than broad background crawling.

Operator and family: Anthropic Type: Ai Purpose: Ai Assistant
Verified robots.txt: Unknown Block: Depends Official Documented

Same operator: Anthropic

Claude-Code Neutral

Claude-Code

Claimed Claude Code user-agent identifier associated with Anthropic developer-agent workflows.

Operator: Anthropic Family: Claude Type: Ai Purpose: Developer Agent Fetch
Not verified robots.txt: Unknown Block: Depends Observed

Similar purpose: Ai Search

PerplexityBot Safe

PerplexityBot

PerplexityBot is an AI crawler from Perplexity used for AI-related crawling, retrieval, search, or assistant activity.

Operator and family: Perplexity Type: Ai Purpose: Ai Search
Verified robots.txt: Unknown Block: Depends Verified Bot
OAI-SearchBot Safe

OAI-SearchBot

OAI-SearchBot is OpenAI’s search crawler used to surface and retrieve web content for search-related experiences.

Operator and family: OpenAI Type: Ai Purpose: Ai Search
Verified robots.txt: Yes Block: Depends Official Documented

Monitor This Bot In Edge

Botcrawl Edge

Use Botcrawl Edge to see matching traffic, create allow or block rules, and control this bot across connected sites.

Get Edge Now Open BotDB