About this bot
Algolia is a site search crawler associated with Algolia and used for site search indexing.
The primary identifying User-Agent token is Algolia Crawler. Representative HTTP request identity: Algolia Crawler/xx.xx.xx.
Directory status: Active. Identity classification: Verified with High confidence. Identity type: official documented.
This identity is recorded as honoring robots.txt under Algolia Crawler.
Risk classification: Safe. Evaluate request context and site policy before allowing, rate-limiting, or blocking.
How to identify this bot
- User-agent and crawler identifiers
-
Algolia CrawlerAlgolia Crawler/xx.xx.xx
Details
- Kind
- Site search crawler
- Family
- Algolia
- Purpose
- Site search indexing
- Common Use
- extracting website content to populate and refresh Algolia search indexes
- Respects robots.txt
- Yes
Verification
- Identity Type
- Officially documented
- Evidence Method
- Match `Algolia Crawler` as the claimed identity signal and corroborate it with stronger request or network evidence when authentication matters.
- Detection Notes
- Match the `Algolia Crawler` User-Agent identity.
Website controls
- Robots.txt Snippet
User-agent: Algolia Crawler Disallow: /