Bot Directory

Search public Botcrawl bot records to identify AI crawlers, search bots, scrapers, scanners, monitoring bots, preview bots, and user-agent patterns.

AI Crawlers Search Bots Scrapers Scanners Monitoring Preview Bots
See these bots hitting your site in real time

The public directory explains known bots and user-agents. Botcrawl Edge uses BotDB plus datacenter intelligence for live classification, rules, and enforcement.

Try Edge Free Open Edge BotDB
998 bots
SplitSignalBot Safe

SplitSignalBot

SplitSignalBot is a search crawler from Semrush used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: Semrush Type: Search Purpose: Indexing
Verified robots.txt: Yes Block: Depends Official Documented
View Record Open in Edge BotDB
Rigor Safe

Splunk

Splunk is a monitoring bot used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: Splunk Type: Monitoring
Verified robots.txt: Unknown Block: No Verified Bot
View Record Open in Edge BotDB
TwinWaveScanner Safe

Splunk Attack Analyzer

Splunk Attack Analyzer is an AI training crawler from Splunk used for AI model training, dataset discovery.

Operator and family: Splunk Type: Feed Purpose: Ai Training
Verified robots.txt: Unknown Block: No Verified Bot
View Record Open in Edge BotDB
Splunk Synthetics Safe

Splunk Synthetics

Splunk Synthetics is a monitoring bot from Splunk used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: Splunk Type: Monitoring
Verified robots.txt: Unknown Block: No Verified Bot
View Record Open in Edge BotDB
SpyglassesBot Neutral

Spyglasses Bot

Spyglasses Bot is an AI crawler from Spyglasses used for assistant-driven browsing, page retrieval.

Operator and family: Spyglasses Type: Ai Purpose: Ai Assistant
Verified robots.txt: No Block: Depends Verified Bot
View Record Open in Edge BotDB
stably Safe

Stably

Stably is a monitoring bot used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: Stably Type: Monitoring
Verified robots.txt: Unknown Block: Depends Verified Bot
View Record Open in Edge BotDB
Stape Safe

Stape Scanner

Stape Scanner is a monitoring bot from Stape Inc used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: Stape Inc Type: Monitoring
Verified robots.txt: Unknown Block: No Verified Bot
View Record Open in Edge BotDB
Statabot Neutral

Statabot

Statabot is a search crawler from StataCorp LLC used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: StataCorp LLC Type: Search Purpose: Indexing
Verified robots.txt: No Block: Depends Verified Bot
View Record Open in Edge BotDB
StatistikAustria Neutral

StatistikAustria

StatistikAustria is a search crawler from Statistik Bot used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: Statistik Bot Type: Search Purpose: Indexing
Verified robots.txt: No Block: Depends Verified Bot
View Record Open in Edge BotDB
StatsDroneBot Neutral

StatsDroneBot

StatsDroneBot is a search crawler from StatsDrone used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: StatsDrone Type: Search Purpose: Indexing
Verified robots.txt: No Block: Depends Verified Bot
View Record Open in Edge BotDB
statuscake Safe

StatusCake

StatusCake is a monitoring bot used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: StatusCake Type: Monitoring
Verified robots.txt: Unknown Block: No Verified Bot
View Record Open in Edge BotDB
SteamChat Neutral

Steam Chat

Steam Chat is a web scraper from Valve Software used for public web data collection, page extraction, content monitoring.

Operator and family: Valve Software Type: Scraper Purpose: Scraping
Verified robots.txt: Unknown Block: Depends Verified Bot
View Record Open in Edge BotDB
Stripe/1.0 Safe

Stripe

Stripe is a webhook callback used for webhook delivery, payment notifications, service callbacks.

Operator and family: Stripe Type: Webhook
Verified robots.txt: No Block: No Verified Bot
View Record Open in Edge BotDB
Stripebot Safe

Stripebot

Stripebot is an ecommerce crawler from Stripe used for product discovery, catalog checks, marketplace crawling, price monitoring.

Operator and family: Stripe Type: Ecommerce
Verified robots.txt: No Block: No Verified Bot
View Record Open in Edge BotDB
Sucuri Safe

Sucuri

Sucuri is a security scanner used for security scanning, malware checks, vulnerability assessment, certificate review.

Operator and family: Sucuri Type: Security
Verified robots.txt: Unknown Block: No Verified Bot
View Record Open in Edge BotDB
sender-9YMgn Safe

Svix Webhooks

Svix Webhooks is a webhook callback from Svix Inc. used for webhook delivery, payment notifications, service callbacks.

Operator and family: Svix Inc. Type: Webhook
Verified robots.txt: No Block: No Verified Bot
View Record Open in Edge BotDB
SwifteqLinkChecker Safe

Swifteq Link Checker

Swifteq Link Checker is a monitoring bot from Swfiteq Ltd used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: Swfiteq Ltd Type: Monitoring
Verified robots.txt: Unknown Block: No Verified Bot
View Record Open in Edge BotDB
Datadog Safe

Synthetic Bot

Synthetic Bot is a monitoring bot from DataDog used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: DataDog Type: Monitoring
Verified robots.txt: Unknown Block: No Verified Bot
View Record Open in Edge BotDB
Taboolabot Neutral

Taboola

Taboola is a search crawler used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: Taboola Type: Search Purpose: Indexing
Verified robots.txt: No Block: Depends Verified Bot
View Record Open in Edge BotDB
TangibleeBot Neutral

TangibleeBot

TangibleeBot is a commerce Crawler from Tangiblee used for product discovery, catalog checks, marketplace crawling, price monitoring.

Operator and family: Tangiblee Type: Commerce Crawler Purpose: Product Visualization Fetching
Verified robots.txt: Unknown Block: Depends Documented
View Record Open in Edge BotDB
TARAGroup-Intelligent-Bot Neutral

TARAGroup Intelligent Bot

TARAGroup Intelligent Bot is an AI crawler from TARAGroup used for AI crawler discovery, assistant retrieval, AI search support, or model-related web access.

Operator and family: TARAGroup Type: Ai Crawler Purpose: Intelligent Crawling
Not verified robots.txt: Unknown Block: Depends Documented
View Record Open in Edge BotDB
TavilyBot Neutral

TavilyBot

TavilyBot is an AI search crawler from Tavily used for AI search indexing, answer retrieval, citation discovery.

Operator and family: Tavily Type: Ai Purpose: Ai Search
Not verified robots.txt: Unknown Block: Depends Observed
View Record Open in Edge BotDB
TelegramBot Neutral

Telegram Bot

Telegram Bot is a link preview bot from Telegram used for link unfurling, rich-card generation, social previews, messaging previews.

Operator and family: Telegram Type: Preview
Not verified robots.txt: Unknown Block: Depends Observed
View Record Open in Edge BotDB
Teoma Neutral

Teoma

Teoma is a search crawler from Ask / Teoma used for search indexing, URL discovery, page rendering, result freshness.

Operator: Ask / Teoma Family: Teoma Type: Search Engine Crawler Purpose: Search Indexing
Not verified robots.txt: Unknown Block: Depends Documented
View Record Open in Edge BotDB