About this bot
DuckDuckBot is associated with DuckDuckGo and used for search indexing.
The primary identifying User-Agent token is DuckDuckBot. Representative HTTP request identity: DuckDuckBot/1.1; (+http://duckduckgo.com/duckduckbot.html).
Directory status: Active. Identity classification: Verified with High confidence.
robots.txt behavior: yes under DuckDuckBot.
Crawls and indexes web pages for DuckDuckGo Search discovery and result freshness.
How to identify this bot
- User-agent and crawler identifiers
-
DuckDuckBotDuckDuckBot/1.1; (+http://duckduckgo.com/duckduckbot.html) - Also known as
- DuckDuckBot/1.1; DuckDuckGo Bot
Details
- Kind
- Crawler
- Family
- DuckDuckGo
- Purpose
- Search indexing
- Common Use
- Crawls and indexes web pages for DuckDuckGo Search discovery and result freshness.
- Respects robots.txt
- Yes
Verification
- Identity Type
- Officially documented
- Evidence Method
- Match `DuckDuckBot/1.1` and verify the source IP against DuckDuckGo's current published DuckDuckBot address list/JSON.
- Detection Notes
- DuckDuckBot is DuckDuckGo's search crawler. Use both the UA and published network information for trust-sensitive classification.
Website controls
- Robots.txt Snippet
User-agent: DuckDuckBot Disallow: /- Apache Rule
RewriteCond %{HTTP_USER_AGENT} "DuckDuckBot" [NC] RewriteRule ^ - [F,L]- Nginx Rule
if ($http_user_agent ~* "DuckDuckBot") { return 403; }