Overview
google-xrawler is an automated web identity associated with Google.
It is used for user-triggered retrieval of merchant product feeds for Google commerce systems.
The documented User-Agent pattern is google-xrawler. User-Agent strings can be copied, so corroborate requests with operator documentation, network ownership, reverse DNS, or other request evidence.
Identity
- User-Agent
google-xrawler- HTTP Agent Examples
google-xrawler- Identity Type
- User Agent
- Evidence Method
- Match the documented User-Agent pattern google-xrawler, then corroborate it with operator documentation, network ownership, reverse DNS, or other request evidence. User-Agent strings alone do not prove identity.
Classification
- Type
- Search
- Kind
- Crawler
- Family
- Purpose
- merchant-product-feeds
Behavior and handling
- Common Use
- user-triggered retrieval of merchant product feeds for Google commerce systems
- Detection Notes
- Detect candidate google-xrawler traffic using the documented User-Agent pattern google-xrawler. Confirm the identity with supporting evidence before enforcement.
- Respects robots.txt
- Unknown
- Spoofing Risk
- google-xrawler can be impersonated because its User-Agent pattern can be copied. Treat the identity as confirmed only when supporting evidence matches.
- Risk
- Neutral
- Recommended Handling
- Depends
Rules and controls
- Robots.txt Snippet
# robots.txt behavior is unconfirmed. Do not rely on this rule without verification.