Semrushbot

Neutral Block: Depends Verified robots.txt: No

Semrushbot crawls your website to analyze it for different SEO and technical issues.

Key facts

Operator: Semrush
Family: Semrush
Purpose: Site Owner Fetch
User-Agent: SemrushBot-SA
Should you block it?: Depends
Verified: Yes
Respects robots.txt: No
Identity type: Unknown
Confidence: Medium
Last checked: 2026-04-01

Bot details

Identity

User-Agent: SemrushBot-SA
robots.txt token: SemrushBot-SA
HTTP user-agent: Mozilla/5.0 (compatible; SemrushBot-SA/0.97; +http://www.semrush.com/bot.html)

Ownership

Operator: Semrush
Family: Semrush
Type: Scraper
Purpose: Site Owner Fetch

Verification and trust

Source type: Unknown
Confidence: Medium
Last checked: 2026-04-01
Verification: Validate the identifying user-agent or signature against the operator documentation before creating hard allow rules.
Operator URL: https://www.semrush.com/bot/
Source URL: https://www.semrush.com/bot/
Spoofing risk: User-agent strings can be spoofed. For allow-listing or low-friction rules, pair the published identifier with operator documentation or cryptographic verification when available.

Blocking and detection

Robots

User-agent: SemrushBot-SA
Disallow: /

Cloudflare

(http.user_agent contains "SemrushBot-SA")

Advanced server rules

Apache

RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} SemrushBot\-SA [NC]
RewriteRule ^ - [F,L]

Nginx

if ($http_user_agent ~* "SemrushBot-SA") { return 403; }

Notes

Known user-agent patterns: SemrushBot-SA Known user-agent strings: Mozilla/5.0 (compatible; SemrushBot-SA/0.97; +http://www.semrush.com/bot.html) Robots.txt handling in the directory: no.

Operator documentation: https://www.semrush.com/bot/

Evidence and source

Validate the identifying user-agent or signature against the operator documentation before creating hard allow rules.
Semrushbot crawls your website to analyze it for different SEO and technical issues.
User-agent strings can be spoofed. For allow-listing or low-friction rules, pair the published identifier with operator documentation or cryptographic verification when available.

https://www.semrush.com/bot/