Bot Directory

Search public Botcrawl bot records to identify AI crawlers, search bots, scrapers, scanners, monitoring bots, preview bots, and user-agent patterns.

AI Crawlers Search Bots Scrapers Scanners Monitoring Preview Bots
See these bots hitting your site in real time

The public directory explains known bots and user-agents. Botcrawl Edge uses BotDB plus datacenter intelligence for live classification, rules, and enforcement.

ZoteroTranslationServer/WMF Neutral

WMF Zotero Translation Server

WMF Zotero Translation Server is a feed fetcher from Wikimedia Foundation used for RSS or Atom feed polling, syndication refresh, subscription delivery.

Operator and family: Wikimedia Foundation Type: Feed Purpose: Feed Fetch
Verified robots.txt: Unknown Block: Depends Verified Bot
WordCountBot Neutral

WordCountBot

WordCountBot is a web scraper from Weglot used for public web data collection, page extraction, content monitoring.

Operator and family: Weglot Type: Scraper Purpose: Scraping
Verified robots.txt: Unknown Block: Depends Verified Bot
WordPress Neutral

WordPress Namecheap

Wordpress Namecheap is an AI training crawler from namecheap used for AI model training, dataset discovery.

Operator and family: namecheap Type: Ai Purpose: Ai Training
Verified robots.txt: No Block: Depends Verified Bot
Indy Library Safe

Worldline Bot

Worldline Bot is a webhook callback from Worldline used for webhook delivery, payment notifications, service callbacks.

Operator and family: Worldline Type: Webhook
Verified robots.txt: Unknown Block: No Verified Bot
WJHRO/1.0 Safe

WorldPay

WorldPay is a webhook callback used for webhook delivery, payment notifications, service callbacks.

Operator and family: WorldPay Type: Webhook
Not verified robots.txt: Unknown Block: No Observed
WormlyBot Safe

WormlyBot

WormlyBot is a monitoring bot from Wormly used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: Wormly Type: Monitoring
Verified robots.txt: No Block: No Verified Bot
Wotbox Caution

Wotbox

Wotbox is a search crawler used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: Wotbox Type: Search Engine Crawler Purpose: Search Indexing
Verified robots.txt: Unknown Block: Monitor Documented
WovnCrawler Safe

WOVN Crawler

WOVN Crawler is a search crawler from Wovn Technologies, Inc. used for search indexing, URL discovery, page rendering, result freshness.

Operator: Wovn Technologies, Inc. Family: Inc. Type: Search Purpose: Indexing
Verified robots.txt: Unknown Block: No Verified Bot
WP Time Capsule API/ Safe

WP Time Capsule

WP Time Capsule is a monitoring bot used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: WP Time Capsule Type: Monitoring
Not verified robots.txt: No Block: No Observed
WPUmbrella Safe

WP Umbrella

WP Umbrella is a monitoring bot used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: WP Umbrella Type: Monitoring
Verified robots.txt: Unknown Block: No Verified Bot
umbrella bot Safe

WP Umbrella Bot

WP Umbrella Bot is a monitoring bot from WP Umbrella used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: WP Umbrella Type: Monitoring
Verified robots.txt: Unknown Block: No Verified Bot
wpbot Neutral

wpbot

wpbot is an AI crawler from Unknown used for AI crawler discovery, assistant retrieval, AI search support, or model-related web access.

Family: wpbot Type: Ai Purpose: Ai Crawl
Not verified robots.txt: Unknown Block: Depends Observed
WPMU DEV Broken Link Checker Safe

WPMU DEV Broken Link Checker

WPMU DEV Broken Link Checker is a monitoring bot from WPMUDEV used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: WPMUDEV Type: Monitoring
Verified robots.txt: Unknown Block: No Verified Bot
WPMUDEV Safe

WPMUDEV Uptime Monitor 5.0

WPMUDEV Uptime Monitor 5.0 is a monitoring bot from WPMUDEV used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: WPMUDEV Type: Monitoring
Verified robots.txt: No Block: No Verified Bot
WPScan Caution

WPScan

WPScan is a security scanner used for security scanning, malware checks, vulnerability assessment, certificate review.

Operator and family: WPScan Type: Vulnerability Scanner Purpose: Vulnerability Scanning
Verified robots.txt: Unknown Block: Monitor Documented
WRTNBot Neutral

WRTNBot

WRTNBot is an AI search crawler from Wrtn used for AI search indexing, answer retrieval, citation discovery.

Operator and family: Wrtn Type: Ai Purpose: Ai Search
Not verified robots.txt: Unknown Block: Depends Observed
xAI-Grok Neutral

xAI-Grok

xAI-Grok is an AI crawler from xAI used for AI crawler discovery, assistant retrieval, AI search support, or model-related web access.

Operator: xAI Family: xAI Grok Type: Ai Purpose: AI Search Or Agentic Browsing
Not verified robots.txt: Unknown Block: Depends Observed
Xenu Link Sleuth Neutral

Xenu Link Sleuth

Xenu Link Sleuth is a validator bot from Xenu used for markup validation, accessibility testing, structured-data review.

Operator and family: Xenu Type: Validator Purpose: Validation And Testing
Verified robots.txt: Unknown Block: Allow Documented
XY-Archive-Compliance Neutral

XY Archive Compliance Bot

XY Archive Compliance Bot is a security scanner from XY Archive Compliance used for security scanning, malware checks, vulnerability assessment, certificate review.

Operator and family: XY Archive Compliance Type: Security
Verified robots.txt: No Block: Depends Verified Bot
yacybot Neutral

yacybot

yacybot is a search crawler from YaCy used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: YaCy Type: Search Purpose: Indexing
Verified robots.txt: Yes Block: Depends Official Documented
YaDirectFetcher Safe

YaDirectFetcher

YaDirectFetcher is a monitoring bot from Yandex used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: Yandex Type: Monitoring
Verified robots.txt: Yes Block: Depends Official Documented
Yahoo Ad monitoring Neutral

Yahoo Ad Monitoring

Yahoo Ad Monitoring is a monitoring bot from Yahoo used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: Yahoo Type: Monitoring
Verified robots.txt: No Block: Depends Verified Bot
Yahoo-Blogs Neutral

Yahoo Blogs

Yahoo Blogs is a feed fetcher from Yahoo used for RSS or Atom feed polling, syndication refresh, subscription delivery.

Operator and family: Yahoo Type: Feed Fetcher Purpose: Blog Feed Fetching
Not verified robots.txt: Unknown Block: Depends Documented
YahooFeedSeeker Neutral

Yahoo FeedSeeker

Yahoo FeedSeeker is a feed fetcher from Yahoo used for RSS or Atom feed polling, syndication refresh, subscription delivery.

Operator and family: Yahoo Type: Feed Fetcher Purpose: Feed Fetching
Not verified robots.txt: Unknown Block: Depends Documented