Overview
SemrushBotSwa is an AI training crawler from Semrush used for AI model training, dataset discovery, and collection of public web content for model-development pipelines.
Its primary user-agent pattern is SemrushBot-SWA; related patterns include SemrushBot-SWA/; a representative HTTP user-agent is Mozilla/5.0 (compatible; SemrushBot-SWA/0.1; +http://www.semrush.com/bot.html).
SemrushBotSwa is verified with High confidence. The identity type is Verified Bot, and the evidence basis is official operator documentation.
SemrushBotSwa is marked as respecting robots.txt directives for crawler access control.
SemrushBotSwa should be handled according to the site owner’s AI crawler policy, with allow, block, or rate-limit rules applied deliberately.
Identity
- User-Agent
SemrushBot-SWA- Aliases
- SemrushBot-SWA/
- HTTP Agent Examples
Mozilla/5.0 (compatible; SemrushBot-SWA/0.1; +http://www.semrush.com/bot.html)- Robots.txt Token
SemrushBot-SWA- Identity Type
- Verified Bot
- Evidence Method
- Verify SemrushBotSwa by matching `SemrushBot-SWA` to Semrush evidence, then checking reverse DNS, source-network ownership, signed request data, or published crawler documentation when available.
Classification
- Type
- AI
- Kind
- Crawler
- Family
- Semrush
- Purpose
- ai-training
Behavior and handling
- Common Use
- SemrushBotSwa is used for AI model training, dataset discovery, and collection of public web content for model-development pipelines.
- Detection Notes
- SemrushBotSwa traffic is primarily detected by the `SemrushBot-SWA` user-agent pattern; related patterns include `SemrushBot-SWA/`; a representative HTTP user-agent is `Mozilla/5.0 (compatible; SemrushBot-SWA/0.1; +http://www.semrush.com/bot.html)`. Compare source IPs, reverse DNS, request paths, and crawl cadence with Semrush infrastructure before trusting the traffic.
- Respects robots.txt
- Yes
- Spoofing Risk
- SemrushBotSwa has medium spoofing risk because the user-agent can be copied, even when the bot has strong source or documentation support.
- Risk
- Neutral
- Recommended Handling
- Depends
Rules and controls
- Robots.txt Snippet
User-agent: SemrushBot-SWA Disallow: /