About this bot
AhrefsBot is a web indexing crawler associated with Ahrefs and used for link intelligence and search indexing.
The primary identifying User-Agent token is AhrefsBot. Representative HTTP request identity: Mozilla/5.0 (compatible; AhrefsBot/7.0; +http://ahrefs.com/robot/).
Directory status: Active. Identity classification: Verified with High confidence. Identity type: official documented.
This identity is recorded as honoring robots.txt under AhrefsBot.
Risk classification: Neutral. Evaluate request context and site policy before allowing, rate-limiting, or blocking.
How to identify this bot
- User-agent and crawler identifiers
-
AhrefsBotMozilla/5.0 (compatible; AhrefsBot/7.0; +http://ahrefs.com/robot/) - Also known as
- Ahrefs crawler
Details
- Kind
- Web indexing crawler
- Family
- Ahrefs
- Purpose
- Link intelligence and search indexing
- Common Use
- building Ahrefs marketing-intelligence and backlink data and contributing to Yep web search indexing
- Respects robots.txt
- Yes
Verification
- Identity Type
- Officially documented
- Evidence Method
- Match `AhrefsBot` as the claimed identity signal and corroborate it with stronger request or network evidence when authentication matters.
- Detection Notes
- Match the `AhrefsBot` User-Agent identity and consider the recorded alternate tokens.
Website controls
- Robots.txt Snippet
User-agent: AhrefsBot Disallow: /