Overview
ShapBot is operated by Parallel Web Systems to build and maintain its web search index.
The documented user-agent is Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ShapBot/0.1.0, with the robots.txt token ShapBot.
ShapBot is verified with High confidence from current operator documentation. The operator also publishes a machine-readable IP list that can be used to corroborate requests.
ShapBot can be managed with robots.txt.
Identity
- User-Agent Pattern
-
ShapBot - Aliases
- ShapBot/0.1.0
- HTTP Agent Examples
-
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ShapBot/0.1.0 - Robots Token
- ShapBot
- Identity Type
- Officially documented
- Evidence Method
- Match the `ShapBot` user-agent token and corroborate the source address against the current operator-published ShapBot IP list.
Classification
- Type
- Search
- Kind
- Crawler
- Family
- Parallel
- Purpose
- AI search indexing
Behavior and handling
- Common Use
- building and maintaining Parallel's web search index
- Detection Notes
- Detect by `ShapBot`; representative HTTP user-agent: `Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ShapBot/0.1.0`. Corroborate with the current published IP list when trust matters.
- Respects robots.txt
- Yes
- Spoofing Risk
- The user-agent can be copied; the current operator-published IP list provides stronger corroboration.
- Risk
- Neutral
- Recommended Handling
- Depends
Rules and controls
- Robots.txt Snippet
-
User-agent: ShapBot Disallow: / - Apache Rule
-
RewriteCond %{HTTP_USER_AGENT} "ShapBot" [NC] RewriteRule ^ - [F,L] - Nginx Rule
-
if ($http_user_agent ~* "ShapBot") { return 403; }
Relationships
- Operator
- Parallel Web Systems Checked 2026-09-02
Relationships without an Evidence link are normalized from the canonical directory record. They should not be interpreted as independent proof of physical presence or request origin.
Changes
-
Directory evidence unverified verifiedRecorded 2026-09-03 00:45:56 UTC
-
Evidence method Treat `ShapBot` as an identity signal only. Confirm it with current operator documentation, cryptographic verification, forward-confirmed reverse DNS, source-network ownership, or other authoritative evidence before trusting the claimed identity. Match the `ShapBot` user-agent token and corroborate the source address against the current operator-published ShapBot IP list.Recorded 2026-09-03 00:45:56 UTC
-
Identity type observed official-documentedRecorded 2026-09-03 00:45:56 UTC
This timeline records selected public-field transitions after Botcrawl established the 4.28 history baseline. It is not a complete reconstruction of changes made before that baseline.