Bot intelligence record

Claude-User

Review first

Use the Claude-User identifier to separate Anthropic AI crawler, assistant, or AI search traffic from normal visitor requests in server logs.

Ai Ai Assistant Official Documented Confidence: High Verified: Yes robots.txt: Unknown

User-Agent Pattern

Anthropic
Claude-User

Robots.txt Snippet

Copy-ready 
User-agent: Claude-User Disallow: /

Handling Guidance

Depends

Use this record as bot intelligence, then verify the request source and behavior before allowing, blocking, or rate limiting.

AI crawling, assistant retrieval, AI search, or model-supporting discovery depending on the user-agent.

Record Details

Structured data
Operator
Anthropic
Family
Anthropic
Type
Ai
Purpose
Ai Assistant
Identity type
Official Documented
Confidence
High
Last verified
2026-04-01
Last checked
2026-04-30
Source type
Official

Notes

Claude-User is listed in the Botcrawl directory as an AI agent from Anthropic. The primary identifier for log review is Claude-User.

Identification

  • User-agent pattern: Claude-User
  • Family: Anthropic
  • Type: AI
  • Kind: Agent

Common use

AI crawling, assistant retrieval, AI search, or model-supporting discovery depending on the user-agent.

Verification and handling

Anthropic does not currently publish dedicated IP ranges; use the official token and robots.txt controls.

Directory guidance marks the risk level as Neutral and the blocking decision as Depends. Do not rely on the user-agent string alone because user-agent strings can be copied or spoofed.

Robots.txt handling: Unknown.

Evidence and Source

  • Anthropic does not currently publish dedicated IP ranges; use the official token and robots.txt controls.
  • AI crawling, assistant retrieval, AI search, or model-supporting discovery depending on the user-agent.

https://privacy.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler

Same family: Anthropic

ClaudeBot Neutral

ClaudeBot

ClaudeBot is Anthropic’s crawler used to discover and retrieve public web content for Claude-related systems.

Anthropic Anthropic Ai Ai Crawl
Verified robots.txt: Yes Block: Depends Official Documented
Claude-SearchBot Safe

Claude-SearchBot

Claude-SearchBot is Anthropic’s search-focused crawler for retrieving web content.

Anthropic Anthropic Ai Ai Search
Verified robots.txt: Yes Block: Depends Official Documented

Similar purpose: Ai Assistant

SpyglassesBot Neutral

Spyglasses Bot

Spyglasses Bot evaluates website content to identify issues that would affect visibility by AI assistants.

Spyglasses Spyglasses Ai Ai Assistant
Verified robots.txt: No Block: Depends Verified Bot
Nava/ Neutral

Nava Labs ASP (Dev)

Bot to assist social workers in navigating safety net benefit websites.

Nava Labs Nava Ai Ai Assistant
Verified robots.txt: No Block: Depends Verified Bot
TwinAgent Neutral

TwinAgent

TwinAgent is an AI fetcher from Twin used for AI-related crawling, retrieval, search, or assistant activity.

Twin Twin Ai Ai Assistant
Verified robots.txt: No Block: Depends Verified Bot
Perplexity-User Neutral

Perplexity-User

Perplexity-User is an AI agent from Perplexity used for AI-related crawling, retrieval, search, or assistant activity.

Perplexity Perplexity Ai Ai Assistant
Verified robots.txt: Unknown Block: Depends Verified Bot

WordPress Bot Protection

Bot Blocker for WordPress

Monitor bot traffic, review live activity, and control AI crawlers, scrapers, scanners, spam bots, and fake trusted bots from one clean WordPress dashboard.

Buy Now Start Free Trial