About this bot
Cloudflare AI Search is a crawler associated with Cloudflare and used for ai search indexing.
It can be identified in HTTP traffic by the Cloudflare-AI-Search identity family, including Cloudflare-AI-Search (https://developers.cloudflare.com/ai-search; ai-search@cloudflare.com).
Directory status: Active. Identity classification: Verified with High confidence. Identity type: http-message-signature.
It is recorded as honoring robots.txt under Cloudflare-AI-Search.
Validate Web Bot Auth / HTTP Message Signatures against the AI Search key directory; use the User-Agent as a secondary claimed identity.
How to identify this bot
- User-agent and crawler identifiers
-
Cloudflare-AI-SearchCloudflare-AI-Search (https://developers.cloudflare.com/ai-search; ai-search@cloudflare.com) - Also known as
- Cloudflare-AutoRAG; Cloudflare AutoRAG Cloudflare-AutoRAG Cloudflare AutoRAG
Details
- Kind
- Crawler
- Family
- Cloudflare AI Search
- Purpose
- AI search indexing
- Common Use
- Crawling and continuously indexing websites connected to Cloudflare AI Search instances.
- Respects robots.txt
- Yes
Verification
- Identity Type
- HTTP message signature
- Evidence Method
- Validate Web Bot Auth / HTTP Message Signatures against the AI Search key directory; use the User-Agent as a secondary claimed identity.
- Detection Notes
- Cloudflare-AI-Search is for in-account website indexing. External/cross-domain retrieval uses the separate Cloudflare-AI-Search-External identity already present in the directory.
Website controls
- Robots.txt Snippet
User-agent: Cloudflare-AI-Search Disallow: /