SiteAuditBot

Neutral Block: Depends Verified robots.txt: No

Check for over 130 common website issues and get special reports about your site’s crawlability, use of markups, internal linking, speed/performance, HTTPS, and international SEO.

Key facts

Operator: Semrush
Family: Semrush
Purpose: Site Owner Fetch
User-Agent: SiteAuditBot
Should you block it?: Depends
Verified: Yes
Respects robots.txt: No
Identity type: Unknown
Confidence: Medium
Last verified: 2026-04-01
Last checked: 2026-04-01

Bot details

Identity

User-Agent: SiteAuditBot
robots.txt token: SiteAuditBot
HTTP user-agent: Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25 (compatible; SiteAuditBot/0.97; +http://www.semrush.com/bot.html) Mozilla/5.0 (compatible; SiteAuditBot/0.97; +http://www.semrush.com/bot.html)
Aliases: Semrush site audit crawler

Ownership

Operator: Semrush
Family: Semrush
Type: Scraper
Purpose: Site Owner Fetch

Verification and trust

Source type: Unknown
Confidence: Medium
Last verified: 2026-04-01
Last checked: 2026-04-01
Verification: Validate the identifying user-agent or signature against the operator documentation before creating hard allow rules.
IP ranges: https://www.semrush.com/bot/
Operator URL: https://www.semrush.com/
Source URL: https://www.semrush.com/
Spoofing risk: User-agent strings can be spoofed. For allow-listing or low-friction rules, pair the published identifier with operator documentation or cryptographic verification when available.

Blocking and detection

Robots

User-agent: SiteAuditBot
Disallow: /

Cloudflare

(http.user_agent contains "SiteAuditBot")

Advanced server rules

Apache

RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} SiteAuditBot [NC]
RewriteRule ^ - [F,L]

Nginx

if ($http_user_agent ~* "SiteAuditBot") { return 403; }

Notes

Check for over 130 common website issues and get special reports about your site’s crawlability, use of markups, internal linking, speed/performance, HTTPS, and international SEO.

Known user-agent patterns: SiteAuditBot Known user-agent strings: Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5376e Safari/8536.25 (compatible; SiteAuditBot/0.97; +http://www.semrush.com/bot.html); Mozilla/5.0 (compatible; SiteAuditBot/0.97; +http://www.semrush.com/bot.html) Robots.txt handling in the directory: no.

Operator documentation: https://www.semrush.com/

Evidence and source

Validate the identifying user-agent or signature against the operator documentation before creating hard allow rules.
Check for over 130 common website issues and get special reports about your site’s crawlability, use of markups, internal linking, speed/performance, HTTPS, and international SEO.
User-agent strings can be spoofed. For allow-listing or low-friction rules, pair the published identifier with operator documentation or cryptographic verification when available.

https://www.semrush.com/