About this bot
MagnetmeBot is a crawler associated with Magnet.me and used for indexing.
The primary identifying User-Agent token is Magnet.me-web/. Representative HTTP request identity: Magnet.me-web/1.0 (+https://magnet.me/bot.html).
Directory status: Active. Identity classification: Verified with Medium confidence. Identity type: verified bot.
This identity is recorded as honoring robots.txt under MagnetmeBot. Site-specific access policy should still be evaluated before enforcement.
Risk classification: Neutral. Recommended handling is to evaluate the request context before allowing, rate-limiting, or blocking.
How to identify this bot
- User-agent and crawler identifiers
-
Magnet.me-web/Magnet.me-web/1.0 (+https://magnet.me/bot.html)MagnetmeBot
Details
- Kind
- Crawler
- Family
- Magnet.me
- Purpose
- Indexing
- Common Use
- MagnetmeBot is used for search indexing, URL discovery, page rendering, result freshness, and search-quality checks.
- Respects robots.txt
- Yes
Verification
- Identity Type
- operator-documented
- Evidence Method
- Match Magnet.me-web/ versioned traffic and corroborate against Magnet.me's published crawler network for higher-assurance attribution.
- Detection Notes
- Magnet.me publishes the Magnet.me-web/ family and crawler network data; the crawler honors robots.txt including Crawl-delay.
Website controls
- Robots.txt Snippet
User-agent: MagnetmeBot Disallow: /