Bot intelligence record

Claude-SearchBot

Usually allow

Use the Claude-SearchBot identifier to separate Anthropic AI crawler, assistant, or AI search traffic from normal visitor requests in server logs.

Ai Ai Search Official Documented Confidence: High Verified: Yes robots.txt: Yes

User-Agent Pattern

Anthropic
Claude-SearchBot

Robots.txt Snippet

Copy-ready 
User-agent: Claude-SearchBot Disallow: /

Handling Guidance

Depends

This bot is usually safe to allow when the request source is verified and the traffic matches your site policy.

AI crawling, assistant retrieval, AI search, or model-supporting discovery depending on the user-agent.

Record Details

Structured data
Operator
Anthropic
Family
Anthropic
Type
Ai
Purpose
Ai Search
Identity type
Official Documented
Confidence
High
Last verified
2026-04-01
Last checked
2026-04-30
Source type
Official

Notes

Claude-SearchBot is listed in the Botcrawl directory as an AI crawler from Anthropic. The primary identifier for log review is Claude-SearchBot.

Identification

  • User-agent pattern: Claude-SearchBot
  • Family: Anthropic
  • Type: AI
  • Kind: Crawler

Common use

AI crawling, assistant retrieval, AI search, or model-supporting discovery depending on the user-agent.

Verification and handling

Anthropic does not currently publish dedicated IP ranges; use the official token and robots.txt controls.

Directory guidance marks the risk level as Safe and the blocking decision as Depends. Do not rely on the user-agent string alone because user-agent strings can be copied or spoofed.

Robots.txt handling: Yes.

Evidence and Source

  • Anthropic does not currently publish dedicated IP ranges; use the official token and robots.txt controls.
  • AI crawling, assistant retrieval, AI search, or model-supporting discovery depending on the user-agent.

https://privacy.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler

Same family: Anthropic

ClaudeBot Neutral

ClaudeBot

ClaudeBot is Anthropic’s crawler used to discover and retrieve public web content for Claude-related systems.

Anthropic Anthropic Ai Ai Crawl
Verified robots.txt: Yes Block: Depends Official Documented
Claude-User Neutral

Claude-User

Claude-User represents user-initiated Claude retrieval activity rather than broad background crawling.

Anthropic Anthropic Ai Ai Assistant
Verified robots.txt: Unknown Block: Depends Official Documented

Similar purpose: Ai Search

PerplexityBot Safe

PerplexityBot

PerplexityBot is an AI crawler from Perplexity used for AI-related crawling, retrieval, search, or assistant activity.

Perplexity Perplexity Ai Ai Search
Verified robots.txt: Unknown Block: Depends Verified Bot
OAI-SearchBot Safe

OAI-SearchBot

OAI-SearchBot is OpenAI’s search crawler used to surface and retrieve web content for search-related experiences.

OpenAI OpenAI Ai Ai Search
Verified robots.txt: Yes Block: Depends Official Documented

WordPress Bot Protection

Bot Blocker for WordPress

Monitor bot traffic, review live activity, and control AI crawlers, scrapers, scanners, spam bots, and fake trusted bots from one clean WordPress dashboard.

Buy Now Start Free Trial