Bot Directory

Public records for crawlers, AI agents, scrapers, scanners, and user-agent patterns.

AI Crawlers Search Bots Scrapers Scanners
Need live control?

Classify traffic and enforce rules before content is served.

Try Edge Free Open Edge BotDB
998 bots
DotBot Neutral

Moz dotbot

Moz dotbot is a search crawler from Moz used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: Moz Type: Search Purpose: Indexing
Verified robots.txt: No Block: Depends Verified Bot
View Record Open in Edge BotDB
rogerbot-crawler Neutral

Moz rogerbot

Moz rogerbot is a search crawler from Moz used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: Moz Type: Search Purpose: Indexing
Verified robots.txt: Yes Block: Depends Verified Bot
View Record Open in Edge BotDB
MRGbot Safe

MRGbot

MRGbot is a web scraper from MRG Web Services srl used for public web data collection, page extraction, content monitoring.

Operator and family: MRG Web Services srl Type: Scraper Purpose: Scraping
Verified robots.txt: Yes Block: Depends Verified Bot
View Record Open in Edge BotDB
msnbot Safe

MSN

MSN is a search crawler from Microsoft used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: Microsoft Type: Search Purpose: Indexing
Verified robots.txt: Yes Block: Depends Verified Bot
View Record Open in Edge BotDB
muckrack.com Safe

Muck Rack Scraper

Muck Rack Scraper is a feed fetcher from Muck Rack used for RSS or Atom feed polling, syndication refresh, subscription delivery.

Operator and family: Muck Rack Type: Feed Purpose: Feed Fetch
Verified robots.txt: No Block: Depends Verified Bot
View Record Open in Edge BotDB
MuckRack Caution

MuckRack

MuckRack is a web scraper from Muck Rack used for public web data collection, page extraction, content monitoring.

Operator and family: Muck Rack Type: Content Crawler Purpose: Content Discovery
Verified robots.txt: Unknown Block: Monitor Documented
View Record Open in Edge BotDB
MxToolbox Neutral

MxToolbox

MxToolbox is a monitoring bot used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: MxToolbox Type: Monitoring Bot Purpose: Uptime And Performance Monitoring
Verified robots.txt: Unknown Block: Allow Documented
View Record Open in Edge BotDB
MyCentralAIScraperBot Neutral

MyCentralAIScraperBot

MyCentralAIScraperBot is an AI training crawler from MyCentralAI used for AI model training, dataset discovery.

Operator and family: MyCentralAI Type: Ai Purpose: Ai Training
Not verified robots.txt: Unknown Block: Depends Observed
View Record Open in Edge BotDB
NagetBot Neutral

NagetBot

NagetBot is an AI crawler from Unknown used for AI crawler discovery, assistant retrieval, AI search support, or model-related web access.

Family: NagetBot Type: Ai Purpose: Ai Crawl
Not verified robots.txt: Unknown Block: Depends Observed
View Record Open in Edge BotDB
Nava/ Neutral

Nava Labs ASP (Dev)

Nava Labs ASP (Dev) is an AI crawler from Nava Labs used for assistant-driven browsing, page retrieval.

Operator: Nava Labs Family: Nava Type: Ai Purpose: Ai Assistant
Verified robots.txt: Unknown Block: Depends Verified Bot
View Record Open in Edge BotDB
naver-blueno Safe

naver-blueno

naver-blueno is a search crawler from Naver used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: Naver Type: Search Purpose: Indexing
Verified robots.txt: Yes Block: Depends Verified Bot
View Record Open in Edge BotDB
naverbot Safe

naverbot

naverbot is a search crawler from Naver used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: Naver Type: Search Purpose: Indexing
Verified robots.txt: Yes Block: Depends Verified Bot
View Record Open in Edge BotDB
NavuBot Neutral

NavuBot

NavuBot is an AI training crawler from HivePoint, Inc. used for AI model training, dataset discovery.

Operator: HivePoint, Inc. Family: Navu Type: Ai Crawler Purpose: Ai Training Crawling
Not verified robots.txt: Unknown Block: Depends Documented
View Record Open in Edge BotDB
Neevabot Safe

Neevabot

Neevabot is a search crawler from Neeva used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: Neeva Type: Search Purpose: Indexing
Verified robots.txt: No Block: Depends Verified Bot
View Record Open in Edge BotDB
Nessus Caution

Nessus

Nessus is a security scanner from Tenable used for security scanning, malware checks, vulnerability assessment, certificate review.

Operator and family: Tenable Type: Vulnerability Scanner Purpose: Vulnerability Scanning
Verified robots.txt: Unknown Block: Monitor Documented
View Record Open in Edge BotDB
Netcraft Safe

Netcraft

Netcraft is a monitoring bot used for security scanning, malware checks, vulnerability assessment, certificate review.

Operator and family: Netcraft Type: Security
Verified robots.txt: Unknown Block: No Verified Bot
View Record Open in Edge BotDB
netEstate NE Crawler Neutral

netEstate Imprint Crawler

netEstate Imprint Crawler is a search crawler from netEstate used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: netEstate Type: Search Purpose: Indexing
Verified robots.txt: No Block: Depends Verified Bot
View Record Open in Edge BotDB
Neticlebot Neutral

Neticlebot

Neticlebot is a monitoring bot from Neticle used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: Neticle Type: Monitoring Purpose: Media Monitoring
Not verified robots.txt: Unknown Block: Depends Documented
View Record Open in Edge BotDB
Netlab360 Caution

Netlab360

Netlab360 is a security scanner from Qihoo 360 Netlab used for security scanning, malware checks, vulnerability assessment, certificate review.

Operator and family: Qihoo 360 Netlab Type: Security Scanner Purpose: Security Scanning
Verified robots.txt: Unknown Block: Monitor Documented
View Record Open in Edge BotDB
netumo Safe

Netumo

Netumo is a monitoring bot used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: Netumo Type: Monitoring
Verified robots.txt: Unknown Block: No Verified Bot
View Record Open in Edge BotDB
Netvibes Neutral

Netvibes

Netvibes is a feed fetcher used for RSS or Atom feed polling, syndication refresh, subscription delivery.

Operator and family: Netvibes Type: Feed Fetcher Purpose: Feed Fetching
Not verified robots.txt: Unknown Block: Depends Documented
View Record Open in Edge BotDB
NETVIGIE Safe

NETVIGIE

NETVIGIE is a monitoring bot used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: Netvigie Type: Monitoring
Verified robots.txt: Unknown Block: No Verified Bot
View Record Open in Edge BotDB
Netweak Neutral

Netweak

Netweak is a monitoring bot used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: Netweak Type: Monitoring Purpose: Uptime Performance Monitoring
Verified robots.txt: Unknown Block: Depends Documented
View Record Open in Edge BotDB
NewRelicbot Safe

New Relic

New Relic is a monitoring bot used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: New Relic Type: Monitoring
Verified robots.txt: No Block: No Verified Bot
View Record Open in Edge BotDB