Overview
Amazon Kendra Web Crawler (Noncanonical Pattern) is an AI crawler associated with Amazon, used for AI crawler discovery, assistant retrieval, AI search support, or model-related web access.
Its primary user-agent pattern is amazon-kendra-web-crawler; related patterns include amazon-kendra-customer-id; amazon-kendra-web-crawler-*; Amazon Kendra crawler; a representative HTTP user-agent is amazon-kendra-web-crawler.
Amazon Kendra Web Crawler (Noncanonical Pattern) is unverified with Low confidence. The identity type is Documented. This legacy or noncanonical identity is retained for historical log matching.
Robots.txt behavior for Amazon Kendra Web Crawler (Noncanonical Pattern) is not confirmed. Do not treat a user-agent-only rule as guaranteed enforcement.
AWS documents the canonical robots token as amazon-kendra. Retain this older pattern only for historical matching.
Treat the user-agent or identity as a claim and corroborate it with published network data, reverse DNS, signed-request evidence, or current operator documentation when available.
Identity
- User-Agent Pattern
-
amazon-kendra-web-crawler - Aliases
- amazon-kendra-customer-id; amazon-kendra-web-crawler-*; Amazon Kendra crawler
- HTTP Agent Examples
-
amazon-kendra-web-crawler - Robots Token
- amazon-kendra-web-crawler
- Identity Type
- Documented
- Evidence Method
- Verify Amazon Kendra Web Crawler (Noncanonical Pattern) by matching `amazon-kendra-web-crawler` to Amazon evidence, then corroborating the request with source-network ownership, reverse DNS, signed request data, or current operator documentation.
Classification
- Type
- AI crawler
- Kind
- AI crawler
- Family
- Amazon Kendra
- Purpose
- Enterprise search indexing
Behavior and handling
- Common Use
- Amazon Kendra Web Crawler is used for AI crawler discovery, assistant retrieval, AI search support, or model-related web access.
- Detection Notes
- Amazon Kendra Web Crawler (Noncanonical Pattern) traffic is primarily detected by `amazon-kendra-web-crawler`; related patterns include `amazon-kendra-customer-id; amazon-kendra-web-crawler-*; Amazon Kendra crawler`; a representative HTTP user-agent is `amazon-kendra-web-crawler`. Treat these values as identification claims and corroborate them with source-network ownership, reverse DNS, signed request data, or current operator documentation.
- Respects robots.txt
- Unknown
- Spoofing Risk
- Amazon Kendra Web Crawler (Noncanonical Pattern) can be spoofed if identification relies only on a user-agent string; corroborate the request with stronger operator-controlled evidence when available.
- Risk
- Neutral
- Recommended Handling
- Monitor
Rules and controls
- Robots.txt Snippet
-
# Legacy or noncanonical identity. Confirm the current canonical token before relying on robots.txt.
Relationships
- Operator
- Amazon Checked 2026-08-02
Relationships without an Evidence link are normalized from the canonical directory record. They should not be interpreted as independent proof of physical presence or request origin.