Overview
Cloudflare Web Scanner is a scanner operated by Cloudflare to check liveness, content types, robots.txt directives, and related web properties.
Its primary user-agent pattern is CloudflareWebScanner; related patterns include CloudflareWebScanner/1.0; Cloudflare AI Crawl Control scanner.
Cloudflare Web Scanner is Unverified at the identity-evidence level. The listed identity remains useful for detection, but this record does not currently contain authoritative evidence sufficient to authenticate the identity claim.
Robots.txt behavior is not currently confirmed.
Cloudflare Web Scanner should be reviewed against site policy, source evidence, crawl rate, and requested paths before a permanent allow or block rule is created.
Identity
- User-Agent Pattern
-
CloudflareWebScanner - Aliases
- CloudflareWebScanner/1.0; Cloudflare AI Crawl Control scanner
- HTTP Agent Examples
-
CloudflareWebScanner - Robots Token
- CloudflareWebScanner
- Identity Type
- Observed
- Evidence Method
- Treat `CloudflareWebScanner` as an identity signal only. Confirm it with current operator documentation, cryptographic verification, forward-confirmed reverse DNS, source-network ownership, or other authoritative evidence before trusting the claimed identity.
Classification
- Type
- Scanner
- Kind
- Crawler
- Family
- Cloudflare
- Purpose
- Web property scanning
Behavior and handling
- Common Use
- Checks liveness, content types, robots.txt directives, and related web properties.
- Detection Notes
- Cloudflare Web Scanner traffic is primarily detected by the `CloudflareWebScanner` user-agent pattern; related patterns include `CloudflareWebScanner/1.0; Cloudflare AI Crawl Control scanner`. Compare source IPs, reverse DNS, request paths, and crawl cadence with Cloudflare infrastructure before trusting the traffic.
- Respects robots.txt
- Unknown
- Spoofing Risk
- Cloudflare Web Scanner has medium spoofing risk because the user-agent can be copied, even when the bot has strong source or documentation support.
- Risk
- Neutral
- Recommended Handling
- Depends
Rules and controls
- Robots.txt Snippet
-
# robots.txt behavior is unconfirmed. Do not rely on this rule without verification.
Relationships
- Operator
- Cloudflare Checked 2026-08-07
Relationships without an Evidence link are normalized from the canonical directory record. They should not be interpreted as independent proof of physical presence or request origin.