About this bot
xAI-SearchBot is a crawler associated with Unconfirmed and used for ai search retrieval.
Primary directory identity signal: xAI-SearchBot.
Directory status: Active. Identity classification: Unverified with Medium confidence. Identity type: observed unconfirmed.
robots.txt behavior is currently unconfirmed. Do not assume robots.txt alone will control this traffic.
Risk classification: Neutral. Recommended handling is to evaluate the request context before allowing, rate-limiting, or blocking.
How to identify this bot
- User-agent and crawler identifiers
-
xAI-SearchBot - Also known as
- xAI SearchBot
Details
- Kind
- Crawler
- Family
- xAI / Grok
- Purpose
- AI search retrieval
- Common Use
- Tentatively associated with search retrieval and AI answer-generation workflows.
- Respects robots.txt
- Unknown
Verification
- Identity Type
- observed-unconfirmed
- Evidence Method
- Treat xAI-SearchBot as a claimed token only until xAI publishes a canonical identity/verification method or equivalent operator-controlled evidence is available.
- Detection Notes
- The token is retained because the identity may materially exist, but it is not being promoted from weak evidence.
Website controls
- Robots.txt Snippet
User-agent: xAI-SearchBot Disallow: /- Apache Rule
SetEnvIfNoCase User-Agent "xAI-SearchBot" botcrawl_match=1- Nginx Rule
if ($http_user_agent ~* "xAI\-SearchBot") { set $botcrawl_match 1; }