Bot Directory

Search public Botcrawl bot records to identify AI crawlers, search bots, scrapers, scanners, monitoring bots, preview bots, and user-agent patterns.

AI Crawlers Search Bots Scrapers Scanners Monitoring Preview Bots
See these bots hitting your site in real time

The public directory explains known bots and user-agents. Botcrawl Edge uses BotDB plus datacenter intelligence for live classification, rules, and enforcement.

Try Edge Free Open Edge BotDB
998 bots
SkypeUriPreview Neutral

Skype

Skype is a web scraper used for public web data collection, page extraction, content monitoring.

Operator and family: Skype Type: Scraper Purpose: Scraping
Verified robots.txt: Unknown Block: Depends Verified Bot
View Record Open in Edge BotDB
Slack-ImgProxy Neutral

Slack Image Proxy

Slack Image Proxy is a web scraper from Slack used for public web data collection, page extraction, content monitoring.

Operator and family: Slack Type: Scraper Purpose: Scraping
Verified robots.txt: No Block: Depends Verified Bot
View Record Open in Edge BotDB
Slackbot Safe

Slackbot

Slackbot is a webhook callback from Slack used for webhook delivery, payment notifications, service callbacks.

Operator and family: Slack Type: Webhook
Verified robots.txt: No Block: Depends Official Documented
View Record Open in Edge BotDB
Slackbot-LinkExpanding Safe

Slackbot-LinkExpanding

Slackbot-LinkExpanding is a link preview bot from Slack used for link unfurling, rich-card generation, social previews, messaging previews.

Operator and family: Slack Type: Preview
Verified robots.txt: No Block: Depends Official Documented
View Record Open in Edge BotDB
SlickBot Neutral

Slickstream

Slickstream is a search crawler used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: Slickstream Type: Search Purpose: Indexing
Verified robots.txt: No Block: Depends Verified Bot
View Record Open in Edge BotDB
slopsearch Neutral

slopsearch

slopsearch is a search crawler from Christopher Albert used for search indexing, URL discovery, page rendering, result freshness.

Operator: Christopher Albert Family: Christopher Type: Search Purpose: Indexing
Verified robots.txt: No Block: Depends Verified Bot
View Record Open in Edge BotDB
SmarshBot Safe

SmarshBot

SmarshBot is a security scanner from https://central.smarsh.com/s/ used for security scanning, malware checks, vulnerability assessment, certificate review.

Operator and family: https://central.smarsh.com/s/ Type: Security
Verified robots.txt: Unknown Block: No Verified Bot
View Record Open in Edge BotDB
SmartologyBot Neutral

SmartologyBot

SmartologyBot is a web scraper from Smartology used for public web data collection, page extraction, content monitoring.

Operator and family: Smartology Type: Scraper Purpose: Scraping
Verified robots.txt: No Block: Depends Verified Bot
View Record Open in Edge BotDB
SMTnetPMBot Safe

SMTnet PM Bot

SMTnet PM Bot is a search crawler from SMTnet used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: SMTnet Type: Search Purpose: Indexing
Verified robots.txt: Yes Block: Depends Verified Bot
View Record Open in Edge BotDB
Snap URL Preview Service Safe

Snap URL Preview Service

Snap URL Preview Service is a link preview bot from Snap Inc. used for link unfurling, rich-card generation, social previews, messaging previews.

Operator: Snap Inc. Family: Snapchat Type: Preview
Verified robots.txt: Yes Block: Depends Official Documented
View Record Open in Edge BotDB
SnapchatAdsBot Neutral

SnapchatAdsBot

SnapchatAdsBot is an ads from Snap Inc. used for SnapchatAdsBot is used for Ad landing page crawling, policy review, campaign validation.

Operator: Snap Inc. Family: Snapchat Type: Ads Purpose: Ad Verification
Verified robots.txt: Yes Block: Depends Official Documented
View Record Open in Edge BotDB
Snipcart/1.0 Safe

Snipcart

Snipcart is an ecommerce crawler used for product discovery, catalog checks, marketplace crawling, price monitoring.

Operator and family: Snipcart Type: Ecommerce
Verified robots.txt: Unknown Block: No Verified Bot
View Record Open in Edge BotDB
SnowHaze Search Neutral

SnowHaze Search

SnowHaze Search is a search crawler from SnowHaze used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: SnowHaze Type: Search Engine Crawler Purpose: Search Indexing
Verified robots.txt: Unknown Block: Monitor Documented
View Record Open in Edge BotDB
Sogou web spider Neutral

Sogou web spider

Sogou web spider is a search crawler from Sogou used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: Sogou Type: Search Purpose: Search Crawling
Not verified robots.txt: Unknown Block: Depends Observed
View Record Open in Edge BotDB
solarwinds Safe

SolarWinds Observability

SolarWinds Observability is a monitoring bot from SolarWinds used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: SolarWinds Type: Monitoring
Verified robots.txt: Unknown Block: No Verified Bot
View Record Open in Edge BotDB
Sora POS Neutral

Sora Caisse POS

Sora Caisse POS is an ecommerce crawler from Sora Caisse used for product discovery, catalog checks, marketplace crawling, price monitoring.

Operator: Sora Caisse Family: Sora Caisse POS Type: Ecommerce
Verified robots.txt: Unknown Block: Depends Verified Bot
View Record Open in Edge BotDB
sourcedash Neutral

sourcedash

sourcedash is a search crawler from Source Dashboard used for search indexing, URL discovery, page rendering, result freshness.

Operator: Source Dashboard Family: sourcedash Type: Search Engine Crawler Purpose: Property Search Indexing
Verified robots.txt: Yes Block: Depends Documented
View Record Open in Edge BotDB
sovereign.ai Caution

Sovereign AI

Sovereign AI is an AI crawler used for AI crawler discovery, assistant retrieval, AI search support, or model-related web access.

Operator and family: Sovereign AI Type: Ai Crawler Purpose: Ai Content Discovery
Verified robots.txt: Unknown Block: Monitor Documented
View Record Open in Edge BotDB
SparkShipping Safe

Spark Shipping

Spark Shipping is an ecommerce crawler used for product discovery, catalog checks, marketplace crawling, price monitoring.

Operator and family: Spark Shipping Type: Ecommerce
Verified robots.txt: Unknown Block: No Verified Bot
View Record Open in Edge BotDB
SparkUptime Neutral

Spark Uptime

Spark Uptime is a monitoring bot used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: Spark Uptime Type: Monitoring Purpose: Uptime Monitoring
Not verified robots.txt: Unknown Block: Depends Documented
View Record Open in Edge BotDB
SparkPost Safe

SparkpostBot

SparkpostBot is a webhook callback from Bird used for webhook delivery, payment notifications, service callbacks.

Operator and family: Bird Type: Webhook
Verified robots.txt: Unknown Block: No Verified Bot
View Record Open in Edge BotDB
Spawning-AI Caution

Spawning AI

Spawning AI is an AI crawler used for AI crawler discovery, assistant retrieval, AI search support, or model-related web access.

Operator and family: Spawning AI Type: Ai Crawler Purpose: Ai Content Discovery
Verified robots.txt: Unknown Block: Monitor Documented
View Record Open in Edge BotDB
Spectate Safe

Spectate

Spectate is a monitoring bot used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: Spectate Type: Monitoring
Verified robots.txt: No Block: No Verified Bot
View Record Open in Edge BotDB
Spinn3r Caution

Spinn3r

Spinn3r is an aggregator used for RSS or Atom feed polling, syndication refresh, subscription delivery.

Operator and family: Spinn3r Type: Aggregator Purpose: Feed And Blog Crawling
Verified robots.txt: Unknown Block: Monitor Documented
View Record Open in Edge BotDB