Overview
MojeekBot is operated by Mojeek and is used for Crawling and indexing pages for the independent Mojeek search engine.
Its documented identity is Mozilla/5.0 (compatible; MojeekBot/0.6; +https://www.mojeek.com/bot.html). The robots.txt token is MojeekBot.
It follows robots.txt rules addressed to MojeekBot.
Verify MojeekBot with forward-confirmed reverse DNS under mojeek.com, or compare the source against Mojeek's current published IP list.
Identity
- User-Agent
MojeekBot- Aliases
- MojeekBot/; Mojeek search crawler
- HTTP Agent Examples
Mozilla/5.0 (compatible; MojeekBot/0.6; +https://www.mojeek.com/bot.html)- Robots.txt Token
MojeekBot- Identity Type
- Official Documented
- Evidence Method
- Verify MojeekBot with forward-confirmed reverse DNS under mojeek.com, or compare the source against Mojeek's current published IP list.
Classification
- Type
- Search
- Kind
- Crawler
- Family
- Mojeek
- Purpose
- search-indexing
Behavior and handling
- Common Use
- Crawling and indexing pages for the independent Mojeek search engine.
- Detection Notes
- Match MojeekBot, then require forward-confirmed reverse DNS or a current official IP-list match. Mojeek documents robots.txt, noindex, nocache, and nofollow support.
- Respects robots.txt
- Yes
- Spoofing Risk
- The user-agent can be copied; forward-confirmed DNS or the current official address list is required.
- Risk
- Safe
- Recommended Handling
- Depends
Rules and controls
- Robots.txt Snippet
User-agent: MojeekBot Disallow: /- Apache Rule
RewriteCond %{HTTP_USER_AGENT} MojeekBot [NC] RewriteRule .* - [F,L]- Nginx Rule
if ($http_user_agent ~* "MojeekBot") { return 403; }