Bot Directory

Search public Botcrawl bot records to identify AI crawlers, search bots, scrapers, scanners, monitoring bots, preview bots, and user-agent patterns.

AI Crawlers Search Bots Scrapers Scanners Monitoring Preview Bots
See these bots hitting your site in real time

The public directory explains known bots and user-agents. Botcrawl Edge uses BotDB plus datacenter intelligence for live classification, rules, and enforcement.

Try Edge Free Open Edge BotDB
998 bots
SEOkicks Caution

SEOkicks-Robot

SEOkicks-Robot is an SEO crawler from SEOkicks used for SEO analysis, crawl diagnostics, backlink discovery, technical audit work.

Operator and family: SEOkicks Type: Seo Crawler Purpose: Seo Backlink Analysis
Verified robots.txt: Unknown Block: Monitor Documented
View Record Open in Edge BotDB
SequelWP Safe

SequelWP

SequelWP is a monitoring bot used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: SequelWP Type: Monitoring
Verified robots.txt: No Block: No Verified Bot
View Record Open in Edge BotDB
SERankingBacklinksBot Neutral

SERankingBacklinksBot

SERankingBacklinksBot is a search crawler from SE Ranking used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: SE Ranking Type: Search Purpose: Indexing
Verified robots.txt: No Block: Depends Verified Bot
View Record Open in Edge BotDB
serpstatbot Neutral

SerpstatBot

SerpstatBot is a web scraper from Serpstat used for public web data collection, page extraction, content monitoring.

Operator and family: Serpstat Type: Scraper Purpose: Scraping
Verified robots.txt: No Block: Depends Verified Bot
View Record Open in Edge BotDB
Server Density Safe

Server Density

Server Density is a monitoring bot from Stackpath used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: Stackpath Type: Monitoring
Verified robots.txt: Unknown Block: No Verified Bot
View Record Open in Edge BotDB
ServerHunterSpider Neutral

ServerHunterSpider

ServerHunterSpider is a search crawler from Server Hunter used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: Server Hunter Type: Search Purpose: Indexing
Verified robots.txt: No Block: Depends Verified Bot
View Record Open in Edge BotDB
SeznamBot Safe

Seznam

Seznam is a search crawler used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: Seznam Type: Search Purpose: Indexing
Verified robots.txt: No Block: Depends Verified Bot
View Record Open in Edge BotDB
SeznamHomepageCrawler Neutral

Seznam Homepage Crawler

Seznam Homepage Crawler is a search crawler from Seznam.cz used for search indexing, URL discovery, page rendering, result freshness.

Operator: Seznam.cz Family: Seznam Type: Search Engine Crawler Purpose: Homepage Indexing
Not verified robots.txt: Unknown Block: Depends Documented
View Record Open in Edge BotDB
Shap-User Neutral

Shap User

Shap User is an AI crawler from Shap used for assistant-driven browsing, page retrieval.

Operator and family: Shap Type: Ai Purpose: Ai Assistant
Not verified robots.txt: Unknown Block: Depends Observed
View Record Open in Edge BotDB
ShapBot Neutral

ShapBot

ShapBot is a search crawler from Parallel used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: Parallel Type: Search Purpose: Indexing
Verified robots.txt: Unknown Block: Depends Verified Bot
View Record Open in Edge BotDB
Shopify-Captain-Hook Safe

Shopify-Captain-Hook

Shopify-Captain-Hook is a webhook callback from Shopify used for webhook delivery, payment notifications, service callbacks.

Operator and family: Shopify Type: Webhook
Not verified robots.txt: Unknown Block: No Observed
View Record Open in Edge BotDB
Shortwave Image Fetcher Neutral

Shortwave Image Fetcher

Shortwave Image Fetcher is a web scraper from Shortwave Communications Inc. used for public web data collection, page extraction, content monitoring.

Operator and family: Shortwave Communications Inc. Type: Scraper Purpose: Scraping
Verified robots.txt: Unknown Block: Depends Verified Bot
View Record Open in Edge BotDB
Sidetrade indexer bot Neutral

Sidetrade indexer bot

Sidetrade indexer bot is an AI crawler from Sidetrade used for AI crawler discovery, assistant retrieval, AI search support, or model-related web access.

Operator and family: Sidetrade Type: Ai Purpose: Indexing
Not verified robots.txt: Unknown Block: Depends Observed
View Record Open in Edge BotDB
Silktide Safe

SilktideBot

SilktideBot is a monitoring bot from Silktide used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: Silktide Type: Monitoring
Verified robots.txt: No Block: No Verified Bot
View Record Open in Edge BotDB
site24x7 Safe

Site24x7

Site24x7 is a monitoring bot used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: Site24x7 Type: Monitoring
Verified robots.txt: Unknown Block: No Verified Bot
View Record Open in Edge BotDB
SiteAuditBot Neutral

SiteAuditBot

SiteAuditBot is a search crawler from Semrush used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: Semrush Type: Search Purpose: Indexing
Verified robots.txt: Yes Block: Depends Verified Bot
View Record Open in Edge BotDB
Sitebulb Safe

Sitebulb

Sitebulb is a search crawler used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: Sitebulb Type: Search Purpose: Technical SEO Audit
Verified robots.txt: Yes Block: No Official Documented
View Record Open in Edge BotDB
SiteCheck-sitecrawl Neutral

Siteimprove Crawl

Siteimprove Crawl is a search crawler from Siteimprove used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: Siteimprove Type: Search Purpose: Indexing
Verified robots.txt: Unknown Block: Depends Verified Bot
View Record Open in Edge BotDB
Siteliner Safe

Siteliner

Siteliner is a search crawler used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: Siteliner Type: Search Purpose: Site Quality Audit
Verified robots.txt: Yes Block: No Official Documented
View Record Open in Edge BotDB
SiteLockSpider Safe

SiteLock

SiteLock is a monitoring bot used for security scanning, malware checks, vulnerability assessment, certificate review.

Operator and family: SiteLock Type: Security
Verified robots.txt: Unknown Block: No Verified Bot
View Record Open in Edge BotDB
SiteSearch360 Safe

SiteSearch360

SiteSearch360 is a search crawler from SEMKNOX GmbH used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: SEMKNOX GmbH Type: Search Purpose: Indexing
Verified robots.txt: No Block: Depends Verified Bot
View Record Open in Edge BotDB
SiteUptime Safe

SiteUpTimeBot

SiteUpTimeBot is a monitoring bot from SiteUpTime used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: SiteUpTime Type: Monitoring
Verified robots.txt: Unknown Block: No Verified Bot
View Record Open in Edge BotDB
Skroutz ImageBot Neutral

Skroutz ImageBot

Skroutz ImageBot is a web scraper from Skroutz S.A. used for public web data collection, page extraction, content monitoring.

Operator and family: Skroutz S.A. Type: Scraper Purpose: Scraping
Verified robots.txt: Unknown Block: Depends Verified Bot
View Record Open in Edge BotDB
SkroutzBot Safe

SkroutzBot

SkroutzBot is a feed fetcher from Skroutz S.A. used for RSS or Atom feed polling, syndication refresh, subscription delivery.

Operator and family: Skroutz S.A. Type: Feed Purpose: Feed Fetch
Verified robots.txt: Unknown Block: Depends Verified Bot
View Record Open in Edge BotDB