Cookiebot is listed in the Botcrawl directory as a security scanner from Cybot A/S. The primary identifier for log review is Cookiebot.
Identification
- User-agent pattern:
Cookiebot - Family: Cybot A/S
- Type: Security
- Kind: Monitor
Common use
Security scanning, malware checks, abuse prevention, compliance review, or vulnerability monitoring.
Verification and handling
Confirm the user-agent against server logs and use published operator documentation, IP ranges, reverse DNS, or other trust signals when available.
Directory guidance marks the risk level as Safe and the blocking decision as No. Do not rely on the user-agent string alone because user-agent strings can be copied or spoofed.
Robots.txt handling: No.
Identification
Verification And Behavior
Common Use
Security scanning, malware checks, abuse prevention, compliance review, or vulnerability monitoring.
Detection Notes
Match `Cookiebot` as a case-insensitive substring in HTTP user-agent logs. Use bot_http_agent for full user-agent examples when the client sends a longer browser-like string. Do not treat a user-agent match alone as proof of identity for allow-listing.
Rules And Blocking Notes
User-agent: Cookiebot Disallow: /