Overview
GrokBot is an AI crawler from xAI used for AI crawler discovery, assistant retrieval, AI search support, or model-related web access.
Its primary user-agent pattern is GrokBot; related patterns include GrokBot/1.0; xAI-Grok/1.0; Grok-DeepSearch/1.0; a representative HTTP user-agent is GrokBot/1.0.
GrokBot is not independently verified with Low confidence. The identity type is Observed, and the evidence basis is observed traffic evidence with a supporting public reference.
Robots.txt behavior is not currently confirmed.
GrokBot should be handled according to the site owner’s AI crawler policy, with allow, block, or rate-limit rules applied deliberately.
GrokBot belongs to the xAI Grok bot family.
Identity
- User-Agent
GrokBot- Aliases
- GrokBot/1.0; xAI-Grok/1.0; Grok-DeepSearch/1.0
- HTTP Agent Examples
GrokBot/1.0- Robots.txt Token
GrokBot- Identity Type
- Observed
- Evidence Method
- Verify GrokBot by matching `GrokBot` to xAI evidence, then checking reverse DNS, source-network ownership, signed request data, or published crawler documentation when available.
Classification
- Type
- AI
- Kind
- Crawler
- Family
- xAI Grok
- Purpose
- AI search or agentic browsing
Behavior and handling
- Common Use
- GrokBot is used for AI crawler discovery, assistant retrieval, AI search support, or model-related web access.
- Detection Notes
- GrokBot traffic is primarily detected by the `GrokBot` user-agent pattern; related patterns include `GrokBot/1.0; xAI-Grok/1.0; Grok-DeepSearch/1.0`; a representative HTTP user-agent is `GrokBot/1.0`. Compare source IPs, reverse DNS, request paths, and crawl cadence with xAI infrastructure before trusting the traffic.
- Respects robots.txt
- Unknown
- Spoofing Risk
- GrokBot has high spoofing risk because the pattern is low-confidence or observation-based; do not trust the user-agent by itself.
- Risk
- Neutral
- Recommended Handling
- Depends
Rules and controls
- Robots.txt Snippet
# robots.txt behavior is unconfirmed. Do not rely on this rule without verification.