ai

Browse bots in this category.

47 bots
AI2Bot Neutral

AI2Bot

Allen Institute for AI crawler used for AI research and related indexing workflows.

Operator and family: Allen Institute for AI Type: Ai Purpose: AI Research Crawling
Verified robots.txt: Yes Block: Depends Official Documented
Amzn-User Neutral

Amzn-User

Amzn-User is an AI fetcher from Amazon used for AI-related crawling, retrieval, search, or assistant activity.

Operator and family: Amazon Type: Ai Purpose: Ai Assistant
Verified robots.txt: No Block: Depends Official Documented
Anchor Browser Neutral

Anchor Browser

Anchor Browser is an AI agent from Anchor used for AI-related crawling, retrieval, search, or assistant activity.

Operator and family: Anchor Type: Ai Purpose: Ai Assistant
Verified robots.txt: No Block: Depends Verified Bot
Applebot-Extended Safe

Applebot-Extended

Applebot-Extended is an AI control token from Apple used for AI-related crawling, retrieval, search, or assistant activity.

Operator and family: Apple Type: Ai Purpose: Ai Assistant
Verified robots.txt: Yes Block: Depends Official Documented
browserbase Neutral

Browserbase

Browserbase is an AI agent from Browserbase used for AI-related crawling, retrieval, search, or assistant activity.

Operator and family: Browserbase Type: Ai Purpose: Ai Assistant
Verified robots.txt: Unknown Block: Depends Verified Bot
chatgpt-operator Safe

ChatGPT-Operator

ChatGPT-Operator is an AI agent from OpenAI used for AI-related crawling, retrieval, search, or assistant activity.

Operator and family: OpenAI Type: Ai Purpose: Ai Training
Verified robots.txt: Unknown Block: Depends Verified Bot
ChatGPT-User Neutral

ChatGPT-User

ChatGPT-User represents user-initiated browsing or retrieval activity from ChatGPT.

Operator and family: OpenAI Type: Ai Purpose: Ai Assistant
Verified robots.txt: Unknown Block: Depends Official Documented
Claude-Code Neutral

Claude-Code

Claimed Claude Code user-agent identifier associated with Anthropic developer-agent workflows.

Operator: Anthropic Family: Claude Type: Ai Purpose: Developer Agent Fetch
Not verified robots.txt: Unknown Block: Depends Observed
Claude-SearchBot Safe

Claude-SearchBot

Claude-SearchBot is Anthropic’s search-focused crawler for retrieving web content.

Operator and family: Anthropic Type: Ai Purpose: Ai Search
Verified robots.txt: Yes Block: Depends Official Documented
Claude-User Neutral

Claude-User

Claude-User represents user-initiated Claude retrieval activity rather than broad background crawling.

Operator and family: Anthropic Type: Ai Purpose: Ai Assistant
Verified robots.txt: Unknown Block: Depends Official Documented
ClaudeBot Neutral

ClaudeBot

ClaudeBot is Anthropic’s crawler used to discover and retrieve public web content for Claude-related systems.

Operator and family: Anthropic Type: Ai Purpose: Ai Crawl
Verified robots.txt: Yes Block: Depends Official Documented
Cloudflare Crawler Safe

Cloudflare Browser Rendering Crawler

Cloudflare crawler used by Browser Rendering crawl workflows and identified in Cloudflare Radar.

Operator and family: Cloudflare Type: Ai Purpose: Browser Rendering Crawl
Verified robots.txt: Yes Block: No Verified Bot
DeepSeekBot Neutral

DeepSeekBot

Claimed DeepSeek crawler identifier found in third-party bot directories; no official public DeepSeek user-agent documentation was confirmed.

Operator and family: DeepSeek Type: Ai Purpose: AI Model Data Collection
Not verified robots.txt: Unknown Block: Depends Observed
Devin Neutral

Devin

Devin is a collaborative AI teammate built to help ambitious engineering teams achieve more.

Operator and family: Devin AI Type: Ai Purpose: Ai Assistant
Verified robots.txt: No Block: Depends Verified Bot
EasyScan Neutral

EasyScan

EasyScan is an AI agent from codire GmbH used for AI-related crawling, retrieval, search, or assistant activity.

Operator: codire GmbH Family: codire Type: Ai Purpose: Ai Assistant
Verified robots.txt: No Block: Depends Verified Bot
ExaBot Neutral

ExaBot

Claimed Exa crawler identifier for AI search and web data retrieval.

Operator and family: Exa Type: Ai Purpose: AI Search Data Retrieval
Not verified robots.txt: Unknown Block: Depends Observed
Google-Ads-Creatives-Assistant Neutral

Google Ads Creatives Assistant

User-triggered content retrieval for Google Ads creative generation.

Operator and family: Google Type: Ai Purpose: Ai Assistant
Verified robots.txt: Unknown Block: Depends Verified Bot
APIs-Google Safe

Google API

APIs-Google is the user agent used by Google APIs to deliver push notification messages. Application developers can request these notifications to avoid the need for continually polling.

Operator and family: Google Type: Ai Purpose: Ai Training
Verified robots.txt: No Block: No Verified Bot
Google-Agent Neutral

Google-Agent

Google-Agent is an AI fetcher from Google used for AI-related crawling, retrieval, search, or assistant activity.

Operator and family: Google Type: Ai Purpose: Ai Assistant
Verified robots.txt: No Block: Depends Official Documented
Google-CloudVertexBot Safe

Google-CloudVertexBot

Google-CloudVertexBot is an AI crawler from Google used for AI-related crawling, retrieval, search, or assistant activity.

Operator and family: Google Type: Ai Purpose: Ai Crawl
Verified robots.txt: Yes Block: Depends Official Documented
Google-Extended Safe

Google-Extended

Google-Extended is an AI control token from Google used for AI-related crawling, retrieval, search, or assistant activity.

Operator and family: Google Type: Ai Purpose: Ai Assistant
Verified robots.txt: Yes Block: Depends Official Documented
Google-NotebookLM Neutral

Google-NotebookLM

Google-NotebookLM is an AI fetcher from Google used for AI-related crawling, retrieval, search, or assistant activity.

Operator and family: Google Type: Ai Purpose: Ai Assistant
Verified robots.txt: No Block: Depends Official Documented
GoogleAgent-Mariner Neutral

GoogleAgent-Mariner

Claimed Google Project Mariner user-agent token for agentic browser activity.

Operator: Google Family: Google Mariner Type: Ai Purpose: Agentic Browsing
Not verified robots.txt: Unknown Block: Depends Observed
GoogleOther-Image Neutral

GoogleOther-Image

GoogleOther-Image is an AI crawler from Google used for AI-related crawling, retrieval, search, or assistant activity.

Operator and family: Google Type: Ai Purpose: Ai Training
Verified robots.txt: Yes Block: Depends Official Documented