Overview
Doubaobot is a crawler associated with ByteDance, used for automated web access associated with ByteDance’s Doubao AI services.
Its primary user-agent pattern is Doubaobot; related patterns include DoubaoBot; ByteDance Doubao crawler; a representative HTTP user-agent is Doubaobot.
Doubaobot is unverified with Medium confidence. The identity type is Observed.
Robots.txt behavior for Doubaobot is not confirmed. Do not treat a user-agent-only rule as guaranteed enforcement.
The token is observed as a Doubao-related crawler identity, but no current operator-published crawler-verification method is available.
Treat the user-agent as an identity claim. Corroborate it with operator-controlled IP ranges, reverse DNS, signed-request evidence, or current documentation when available.
Identity
- User-Agent Pattern
-
Doubaobot - Aliases
- DoubaoBot; ByteDance Doubao crawler
- HTTP Agent Examples
-
Doubaobot - Robots Token
- Doubaobot
- Identity Type
- Observed
- Evidence Method
- Match `Doubaobot` and corroborate the request with current ByteDance documentation, operator-controlled network data, reverse DNS, or signed-request evidence when available.
Classification
- Type
- AI
- Kind
- Crawler
- Family
- Doubao
- Operator
- ByteDance
- Company
- ByteDance
- Purpose
- AI indexing or training
Behavior and handling
- Common Use
- automated web access associated with ByteDance's Doubao AI services
- Detection Notes
- Detect primarily by `Doubaobot`; related patterns include `DoubaoBot; ByteDance Doubao crawler`; representative HTTP user-agent: `Doubaobot`. User-agent strings can be copied, so corroborate with stronger evidence before granting trust.
- Respects robots.txt
- Unknown
- Spoofing Risk
- Doubaobot has high spoofing risk because its user-agent can be copied and the current evidence is observation-based; do not trust it by name alone.
- Risk
- Neutral
- Recommended Handling
- Monitor
Rules and controls
- Robots.txt Snippet
-
# Robots.txt behavior is unconfirmed. Monitor first and use stronger controls when enforcement is required. - Apache Rule
-
RewriteCond %{HTTP_USER_AGENT} "Doubaobot" [NC] RewriteRule ^ - [F,L] - Nginx Rule
-
if ($http_user_agent ~* "Doubaobot") { return 403; }