Overview
Amazon Kendra Web Crawler is a crawler associated with Amazon Web Services, used for crawling authorized websites for indexing in Amazon Kendra.
Its primary user-agent pattern is amazon-kendra; related patterns include Amazon Kendra crawler; Amazon Kendra Web Crawler; a representative HTTP user-agent is amazon-kendra.
Amazon Kendra Web Crawler is Verified from current authoritative identity documentation reviewed on 2026-08-07. The identity type is Official Documented.
Amazon Kendra Web Crawler is documented as honoring robots.txt directives.
Amazon Kendra entered maintenance mode on June 30, 2026 and stopped accepting new customers on July 30, 2026, but remains supported for existing customers.
Treat the user-agent or identity as a claim and corroborate it with published network data, reverse DNS, signed-request evidence, or current operator documentation when available.
Identity
- User-Agent Pattern
-
amazon-kendra - Aliases
- Amazon Kendra crawler; Amazon Kendra Web Crawler
- HTTP Agent Examples
-
amazon-kendra - Robots Token
- amazon-kendra
- Identity Type
- Officially documented
- Evidence Method
- Match `amazon-kendra` to the current operator documentation and corroborate the request with operator-controlled verification signals where available; User-Agent strings alone can be spoofed.
Classification
- Type
- Enterprise search
- Kind
- Crawler
- Family
- Amazon Kendra
- Purpose
- Enterprise search indexing
Behavior and handling
- Common Use
- crawling authorized websites for indexing in Amazon Kendra
- Detection Notes
- Amazon Kendra Web Crawler traffic is primarily detected by `amazon-kendra`; related patterns include `Amazon Kendra crawler; Amazon Kendra Web Crawler`; a representative HTTP user-agent is `amazon-kendra`. Treat these values as identification claims and corroborate them with source-network ownership, reverse DNS, signed request data, or current operator documentation.
- Respects robots.txt
- Yes
- Spoofing Risk
- Amazon Kendra Web Crawler can be spoofed if identification relies only on a user-agent string; corroborate the request with stronger operator-controlled evidence when available.
- Risk
- Neutral
- Recommended Handling
- Depends
Rules and controls
- Robots.txt Snippet
-
User-agent: amazon-kendra Disallow: /
Relationships
- Operator
- Amazon Web Services Checked 2026-08-07
Relationships without an Evidence link are normalized from the canonical directory record. They should not be interpreted as independent proof of physical presence or request origin.