Bot Intelligence Record

SemrushBot Core

SemrushBot is used by Semrush to gather website data for backlink analytics and link building features.

SemrushBot
OperatorSemrush
FamilySemrush
Typesearch
Riskneutral
Blockdepends
Verified1
Verification Guidance

User-agent strings can be copied. Use this record for identification, logging, filtering, and review. For allow lists or low-friction rules, pair the identifier with operator documentation, reverse DNS, IP ranges, request signatures, or another verification method when available.

Identification

User-Agent: SemrushBot

HTTP Agent: Mozilla/5.0 (compatible; SemrushBot/3~bl; +http://www.semrush.com/bot.html) Mozilla/5.0 (compatible; SemrushBot; +http://www.semrush.com/bot.html)

Aliases: SemrushBot/; SemrushBotBacklinks; SemrushBotLinkBuilding; SemrushBot

Robots Token: SemrushBot

Purpose: indexing

Identity: verified-bot

Handling

Common Use: Search indexing, content discovery, rendering, or search-result freshness checks.

Detection Notes: Match `SemrushBot` as a case-insensitive substring in HTTP user-agent logs. Review bot_aliases for alternate names or product labels. Use bot_http_agent for full user-agent examples when the client sends a longer browser-like string. Do not treat a user-agent match alone as proof of identity for allow-listing.

Risk: neutral

Should Block: depends

Respects Robots: no

Rules

Block Snippet

User-agent: SemrushBot Disallow: /

Sources

Source

Operator

IP Ranges

Related Bots

ZoominfoBot safe

ZoomInfo

Zoominfobot is an indexing robot for a web search engine, similar to Google. Created by Zoom Information Inc.(www.zoominfo.com), Zoominfobot’s patented technology continually scans millions.

Operator: ZoomInfo Family: ZoomInfo Type: search Purpose: indexing
Verified: 1 Robots: no Block: depends Confidence: medium Identity: verified-bot
YouBot safe

YouBot

YouBot is a search crawler associated with You.com, used for search indexing and content discovery.

Operator: You.com Family: YouBot Type: search Purpose: indexing
Verified: 1 Robots: no Block: depends Confidence: medium Identity: verified-bot
YandexVideoParser safe

YandexVideoParser

YandexVideoParser is a search crawler from Yandex used for search indexing, page discovery, or content rendering.

Operator: Yandex Family: Yandex Type: search Purpose: indexing
Verified: 1 Robots: no Block: depends Confidence: high Identity: official-documented
YandexWebmaster safe

YandexWebmaster

YandexWebmaster is a search crawler from Yandex used for search indexing, page discovery, or content rendering.

Operator: Yandex Family: Yandex Type: search Purpose: indexing
Verified: 1 Robots: yes Block: depends Confidence: high Identity: official-documented
Yeti safe

Yeti

Yeti is the web crawler for Naver, a South Korean search engine. It indexes websites to provide search results and power other services on the Naver platform.

Operator: Naver Family: Naver Type: search Purpose: indexing
Verified: 1 Robots: no Block: depends Confidence: medium Identity: verified-bot