AI

AI

GoogleAgent URLContext

Google

GoogleAgent URLContext is associated with Google and is used for observed user-triggered retrieval of URLs supplied as context to Gemini API applications; the…

AI

GPT-Actions

OpenAI

GPT-Actions is associated with OpenAI and is used for historical traffic attributed to GPT Actions integrations; current OpenAI documentation uses ChatGPT-User for user-action…

AI

GPTBot

OpenAI

GPTBot is associated with OpenAI and is used for crawling public web content that may be used to improve and train OpenAI foundation…

AI

Grok-DeepSearch

xAI

Grok-DeepSearch is associated with xAI and is used for observed traffic attributed to Grok deep-research/web-retrieval workflows; the exact token lacks current operator-published crawler…

AI

GrokAgent

xAI

GrokAgent is a xAI agent used for opening and operating websites as part of user-directed Grok agent tasks and multi-step workflows.

AI

GrokBot

xAI

GrokBot is associated with xAI and is used for observed Grok-associated automated traffic; the token should not be conflated with the separately named…

AI

Haiku-SearchBot

CLERK SAS

Haiku-SearchBot is associated with CLERK SAS and is used for direct user-request retrieval for Haiku search interactions; it is not an automatic general-web…

AI

HarkBot

Hark

HarkBot is a signed Hark agent that accesses web content for user-driven personal-intelligence workflows.

AI

Henry Shopping Agent

Henry Labs

Henry Shopping Agent is an automated identity associated with Henry Labs and used for shopping and checkout actions performed through an AI agent.

AI

HuggingFaceBot

Hugging Face

HuggingFaceBot is a Hugging Face crawler used for automated web crawling associated with Hugging Face AI and content-discovery workflows.

AI

iAskBot

iAsk

iAskBot is an AI search crawler from iAsk used for AI search indexing, answer retrieval, citation discovery.

AI

iaskspider

iAsk

iaskspider is an AI search crawler from iAsk used for AI search indexing, answer retrieval, citation discovery.

AI

imageSpider

Unknown

imageSpider is an AI crawler from Unknown used for AI crawler discovery, assistant retrieval, AI search support, or model-related web access.

AI

Inline-AI-Crawler

Inline Technologies

Inline-AI-Crawler is a Inline Technologies knowledge ingestion crawler used for ingesting publisher-owned content to ground on-site AI assistant answers.

AI

IsonAI-User

Epithre / IsonAI

IsonAI-User retrieves a specified web page when someone using IsonAI requests it.

AI

Kangaroo Bot

Kangaroo LLM

Kangaroo Bot crawls Australian web content for the VegeMighty language-model dataset and is documented to honor robots.txt and Crawl-delay.

AI

Keenable-User

Keenable.ai, Inc.

Keenable-User fetches specific web pages at the request of users or AI agents.

AI

KeenableBot

Keenable.ai, Inc.

KeenableBot indexes public web pages for search used by AI agents and applications.

AI

Kimi-SearchBot

Moonshot AI

Kimi-SearchBot is a Moonshot AI crawler used for building and refreshing Kimi search indexes.

AI

Kimi-User

Moonshot AI

Kimi-User represents user-triggered web retrieval for Kimi and should not be conflated with the separate automatic KimiBot crawler.

AI

KimiBot

Moonshot AI

KimiBot is Moonshot AI’s automatic crawler and is separately controllable from user-triggered Kimi retrieval.

AI

KlaviyoAIBot

Klaviyo

KlaviyoAIBot indexes public content from domains or URLs connected for Klaviyo AI experiences. It honors robots.txt and can provide cryptographically authenticated request signals.

AI

laion-huggingface-processor

Unattributed

The token is retained as an observed AI/dataset-processing identity, but the exact operator attribution and current canonical status remain insufficiently documented.

AI

LAIONDownloader

LAION (unconfirmed identity)

LAIONDownloader is retained as an observed dataset-download identity, but current canonical operator documentation for the exact token remains insufficient.