ai

Browse bots in this category.

47 bots
GoogleOther-Video Neutral

GoogleOther-Video

GoogleOther-Video is an AI crawler from Google used for AI-related crawling, retrieval, search, or assistant activity.

Operator and family: Google Type: Ai Purpose: Ai Training
Verified robots.txt: Yes Block: Depends Official Documented
GPT-Actions Neutral

GPT-Actions

GPT-Actions is an AI agent from OpenAI used for AI-related crawling, retrieval, search, or assistant activity.

Operator and family: OpenAI Type: Ai Purpose: Ai Assistant
Verified robots.txt: Unknown Block: Depends Verified Bot
GPTBot Neutral

GPTBot

GPTBot is OpenAI’s web crawler used to discover publicly available content that may support generative AI systems.

Operator and family: OpenAI Type: Ai Purpose: Ai Training
Verified robots.txt: Yes Block: Depends Official Documented
Grok-DeepSearch Neutral

Grok-DeepSearch

Claimed Grok DeepSearch user-agent token for xAI search/research workflows.

Operator: xAI Family: xAI Grok Type: Ai Purpose: AI Search Or Deep Research
Not verified robots.txt: Unknown Block: Depends Observed
GrokBot Neutral

GrokBot

Claimed xAI/Grok bot identifier used in third-party AI crawler lists; sightings may be inconsistent.

Operator: xAI Family: xAI Grok Type: Ai Purpose: AI Search Or Agentic Browsing
Not verified robots.txt: Unknown Block: Depends Observed
kernel Neutral

Kernel Browsers

Kernel Browsers is an AI agent from Kernel used for AI-related crawling, retrieval, search, or assistant activity.

Operator and family: Kernel Type: Ai Purpose: Ai Assistant
Verified robots.txt: Unknown Block: Depends Verified Bot
Known Agent Neutral

Known Agent

Traffic identification and management for AI agent and bot activity.

Operator and family: Known Agents Type: Ai Purpose: Ai Assistant
Verified robots.txt: No Block: Depends Verified Bot
LinerBot Neutral

LINER Bot

The web pages crawled by Linerbot are used to find the right sources that can generate answers to your questions.

Operator and family: Liner Bot Type: Ai Purpose: Ai Training
Verified robots.txt: No Block: Depends Verified Bot
Manus Bot Neutral

Manus Bot

Manus Bot is an AI fetcher from Manus used for AI-related crawling, retrieval, search, or assistant activity.

Operator and family: Manus Type: Ai Purpose: Ai Assistant
Verified robots.txt: No Block: Depends Verified Bot
MistralAI-User Neutral

MistralAI-User

Bot for user actions in le Chat by Mistral AI, for instance when asked to open a web page.

Operator and family: Mistral AI Type: Ai Purpose: Ai Assistant
Verified robots.txt: Yes Block: Depends Official Documented
Nava/ Neutral

Nava Labs ASP (Dev)

Bot to assist social workers in navigating safety net benefit websites.

Operator: Nava Labs Family: Nava Type: Ai Purpose: Ai Assistant
Verified robots.txt: No Block: Depends Verified Bot
OAI-SearchBot Safe

OAI-SearchBot

OAI-SearchBot is OpenAI’s search crawler used to surface and retrieve web content for search-related experiences.

Operator and family: OpenAI Type: Ai Purpose: Ai Search
Verified robots.txt: Yes Block: Depends Official Documented
PanguBot Neutral

PanguBot

Claimed Huawei PanGu crawler identifier used in third-party AI-agent directories.

Operator: Huawei Family: PanGu Type: Ai Purpose: AI Model Data Collection
Not verified robots.txt: Unknown Block: Depends Observed
Perplexity-User Neutral

Perplexity-User

Perplexity-User is an AI agent from Perplexity used for AI-related crawling, retrieval, search, or assistant activity.

Operator and family: Perplexity Type: Ai Purpose: Ai Assistant
Verified robots.txt: Unknown Block: Depends Verified Bot
PerplexityBot Safe

PerplexityBot

PerplexityBot is an AI crawler from Perplexity used for AI-related crawling, retrieval, search, or assistant activity.

Operator and family: Perplexity Type: Ai Purpose: Ai Search
Verified robots.txt: Unknown Block: Depends Verified Bot
SemrushBot-OCOB Neutral

SemrushBot-OCOB

SemrushBot-OCOB is an AI crawler from Semrush used for AI-related crawling, retrieval, search, or assistant activity.

Operator and family: Semrush Type: Ai Purpose: Ai Training
Verified robots.txt: No Block: Depends Verified Bot
SemrushBot-SWA Neutral

SemrushBotSwa

Data collected by SEMrushBot is used for the SEO Writing Assistant tool to check if URL is accessible.

Operator and family: SEMrush Type: Ai Purpose: Ai Training
Verified robots.txt: No Block: Depends Verified Bot
SpyglassesBot Neutral

Spyglasses Bot

Spyglasses Bot evaluates website content to identify issues that would affect visibility by AI assistants.

Operator and family: Spyglasses Type: Ai Purpose: Ai Assistant
Verified robots.txt: No Block: Depends Verified Bot
Timpibot Neutral

Timpibot

Timpi decentralized crawler observed in logs for web indexing and AI-oriented data collection.

Operator and family: Timpi Type: Ai Purpose: AI Data Collection
Not verified robots.txt: No Block: Depends Observed
TwinAgent Neutral

TwinAgent

TwinAgent is an AI fetcher from Twin used for AI-related crawling, retrieval, search, or assistant activity.

Operator and family: Twin Type: Ai Purpose: Ai Assistant
Verified robots.txt: No Block: Depends Verified Bot
vercel-build-container Neutral

Vercel build container

Vercel build container is an AI crawler from Vercel used for AI-related crawling, retrieval, search, or assistant activity.

Operator and family: Vercel Type: Ai Purpose: Ai Training
Verified robots.txt: Unknown Block: Depends Verified Bot
WordPress Neutral

WordPress Namecheap

This is the major egress IP for our containerised WordPress platform so it is likely to be many flavours of WordPress and the potential to be any domain.

Operator and family: namecheap Type: Ai Purpose: Ai Training
Verified robots.txt: No Block: Depends Verified Bot
xAI-Grok Neutral

xAI-Grok

Claimed xAI Grok user-agent token for Grok-related web access.

Operator: xAI Family: xAI Grok Type: Ai Purpose: AI Search Or Agentic Browsing
Not verified robots.txt: Unknown Block: Depends Observed