Bot Directory

Public records for crawlers, AI agents, scrapers, scanners, and user-agent patterns.

AI Crawlers Search Bots Scrapers Scanners
Need live control?

Classify traffic and enforce rules before content is served.

Try Edge Free Open Edge BotDB
998 bots
meta-externalfetcher Neutral

Meta-ExternalFetcher

Meta-ExternalFetcher is a search crawler from Meta used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: Meta Type: Search Purpose: Indexing
Verified robots.txt: Yes Block: Depends Verified Bot
View Record Open in Edge BotDB
meta-webindexer/1.1 Neutral

Meta-WebIndexer

Meta-WebIndexer is a search crawler from Meta used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: Meta Type: Search Purpose: Indexing
Verified robots.txt: Yes Block: Depends Verified Bot
View Record Open in Edge BotDB
Metorik Safe

Metorik

Metorik is an ecommerce crawler used for product discovery, catalog checks, marketplace crawling, price monitoring.

Operator and family: Metorik Type: Ecommerce
Verified robots.txt: Unknown Block: No Verified Bot
View Record Open in Edge BotDB
MgidBot Neutral

MgidBot

MgidBot is a web scraper from MGID used for public web data collection, page extraction, content monitoring.

Operator and family: MGID Type: Scraper Purpose: Scraping
Verified robots.txt: Yes Block: Depends Verified Bot
View Record Open in Edge BotDB
MicrosoftPreview Neutral

Microsoft Preview

Microsoft Preview is a web scraper from Microsoft used for public web data collection, page extraction, content monitoring.

Operator and family: Microsoft Type: Scraper Purpose: Scraping
Verified robots.txt: Yes Block: Depends Verified Bot
View Record Open in Edge BotDB
MidasNetworkBot Neutral

MidasNetworkBot

MidasNetworkBot is a feed fetcher from Midas Network used for RSS or Atom feed polling, syndication refresh, subscription delivery.

Operator and family: Midas Network Type: Feed Fetcher Purpose: News Feed Fetching
Not verified robots.txt: Yes Block: Depends Documented
View Record Open in Edge BotDB
mirrorweb Neutral

MirrorWebCrawler

MirrorWebCrawler is a search crawler from MirrorWeb Ltd used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: MirrorWeb Ltd Type: Search Purpose: Indexing
Verified robots.txt: No Block: Depends Verified Bot
View Record Open in Edge BotDB
MissinglettrBot Neutral

Missinglettr Bot

Missinglettr Bot is a web scraper from Missinglettr used for public web data collection, page extraction, content monitoring.

Operator and family: Missinglettr Type: Scraper Purpose: Scraping
Verified robots.txt: No Block: Depends Verified Bot
View Record Open in Edge BotDB
MistralAI-Index Safe

MistralAI-Index

MistralAI-Index is a search crawler from Mistral AI used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: Mistral AI Type: Search Purpose: Indexing
Verified robots.txt: Yes Block: Depends Official Documented
View Record Open in Edge BotDB
MistralAI-User Neutral

MistralAI-User

MistralAI-User is an AI crawler from Mistral AI used for assistant-driven browsing, page retrieval.

Operator and family: Mistral AI Type: Ai Purpose: Ai Assistant
Verified robots.txt: Yes Block: Depends Official Documented
View Record Open in Edge BotDB
MistralAI-User/1.0 Neutral

MistralAI-User 1.0

MistralAI-User 1.0 is an AI agent from Mistral AI used for AI crawler discovery, assistant retrieval, AI search support, or model-related web access.

Operator and family: Mistral AI Type: Ai User Agent Purpose: User Initiated Ai Fetching
Verified robots.txt: Yes Block: Depends Documented
View Record Open in Edge BotDB
MJ12bot Neutral

MJ12Bot

MJ12Bot is a search crawler from Majestic used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: Majestic Type: Search Purpose: Indexing
Verified robots.txt: Yes Block: Depends Verified Bot
View Record Open in Edge BotDB
ModularConnector Safe

Modular DS

Modular DS is a monitoring bot from Uniqoders Technologies SL used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: Uniqoders Technologies SL Type: Monitoring
Verified robots.txt: Unknown Block: No Verified Bot
View Record Open in Edge BotDB
MojeekBot Safe

Mojeek

Mojeek is a search crawler used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: Mojeek Type: Search Purpose: Indexing
Verified robots.txt: No Block: Depends Verified Bot
View Record Open in Edge BotDB
Mollie HTTP client/ Safe

Mollie Bot

Mollie Bot is a webhook callback from Mollie B.V. used for webhook delivery, payment notifications, service callbacks.

Operator and family: Mollie B.V. Type: Webhook
Verified robots.txt: Unknown Block: No Verified Bot
View Record Open in Edge BotDB
momenticbot Safe

MomenticBot

MomenticBot is a monitoring bot from Momentic used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: Momentic Type: Monitoring
Verified robots.txt: Unknown Block: Depends Verified Bot
View Record Open in Edge BotDB
Monitage Safe

Monitage

Monitage is a monitoring bot used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: Monitage Type: Monitoring
Verified robots.txt: No Block: No Verified Bot
View Record Open in Edge BotDB
monitis Safe

Monitis

Monitis is a monitoring bot used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: Monitis Type: Monitoring
Verified robots.txt: No Block: No Verified Bot
View Record Open in Edge BotDB
MonitoRSS Safe

MonitoRSS

MonitoRSS is a monitoring bot used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: MonitoRSS Type: Monitoring
Verified robots.txt: Yes Block: Depends Verified Bot
View Record Open in Edge BotDB
Monsidobot Safe

Monsidobot

Monsidobot is a monitoring bot from Acquia used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: Acquia Type: Monitoring
Verified robots.txt: No Block: No Verified Bot
View Record Open in Edge BotDB
MonSpark Safe

MonSpark

MonSpark is a monitoring bot used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: MonSpark Type: Monitoring
Verified robots.txt: No Block: No Verified Bot
View Record Open in Edge BotDB
montastic-monitor Safe

MontasticMonitor

MontasticMonitor is a monitoring bot from Montastic used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: Montastic Type: Monitoring
Verified robots.txt: No Block: No Verified Bot
View Record Open in Edge BotDB
MoodleBot Neutral

MoodleBot

MoodleBot is a tool from Moodle used for MoodleBot is used for Automated website access, content retrieval, or integration activity associated with the listed operator.

Operator and family: Moodle Type: Tool Purpose: Education Platform Fetching
Not verified robots.txt: Unknown Block: Depends Documented
View Record Open in Edge BotDB
MotoMinerBot Neutral

MotoMinerBot

MotoMinerBot is a search crawler from Motominer used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: Motominer Type: Search Purpose: Indexing
Verified robots.txt: No Block: Depends Verified Bot
View Record Open in Edge BotDB