About this bot
Brandwatch is associated with Brandwatch and used for brand and social media monitoring.
The primary identifying User-Agent token is magpie-crawler. Representative HTTP request identity: magpie-crawler/1.1 (U; Linux amd64; en-GB; +http://www.brandwatch.net).
Directory status: Active. Identity classification: Verified with High confidence. Identity type: official documented.
This identity is documented or strongly corroborated as honoring applicable robots.txt directives.
Verification: Match `magpie-crawler` and corroborate request behavior with public content collection for monitoring and social-listening analysis.
How to identify this bot
- User-agent and crawler identifiers
-
magpie-crawlermagpie-crawler/1.1 (U; Linux amd64; en-GB; +http://www.brandwatch.net) - Also known as
- Magpie Crawler
Details
- Kind
- Social listening crawler
- Family
- Brandwatch
- Purpose
- Brand and social media monitoring
- Common Use
- collecting and indexing public web content for brand monitoring, consumer intelligence, and social listening
- Respects robots.txt
- Yes
Verification
- Identity Type
- Officially documented
- Evidence Method
- Match `magpie-crawler` and corroborate request behavior with public content collection for monitoring and social-listening analysis.
- Detection Notes
- Brandwatch's magpie-crawler indexes public blogs, forums, news, and other web content used in social listening and brand monitoring.
Website controls
- Robots.txt Snippet
User-agent: magpie-crawler Disallow: /