Bot Directory

Public records for crawlers, AI agents, scrapers, scanners, and user-agent patterns.

AI Crawlers Search Bots Scrapers Scanners
Need live control?

Classify traffic and enforce rules before content is served.

Manus Bot Neutral

Manus Bot

Manus Bot is an AI crawler from Manus used for assistant-driven browsing, page retrieval.

Operator and family: Manus Type: Ai Purpose: Ai Assistant
Verified robots.txt: Unknown Block: Depends Verified Bot
Manus-User Neutral

Manus User

Manus User is an AI crawler from Manus used for assistant-driven browsing, page retrieval.

Operator and family: Manus Type: Ai Purpose: Ai Assistant
Not verified robots.txt: Unknown Block: Depends Observed
marfeel-audits Safe

Marfeel Audits Crawler

Marfeel Audits Crawler is a monitoring bot from Marfeel used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: Marfeel Type: Monitoring
Verified robots.txt: Unknown Block: Depends Verified Bot
mrfCompass Neutral

Marfeel Bot

Marfeel Bot is a monitoring bot from Marfeel used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: Marfeel Type: Monitoring Purpose: Publisher Analytics Fetching
Verified robots.txt: Unknown Block: Depends Documented
marfeel-flowcards Safe

Marfeel Flowcards Crawler

Marfeel Flowcards Crawler is a search crawler from Marfeel used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: Marfeel Type: Search Purpose: Indexing
Verified robots.txt: Unknown Block: Depends Verified Bot
marfeel-preview Safe

Marfeel Preview Crawler

Marfeel Preview Crawler is a search crawler from Marfeel used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: Marfeel Type: Search Purpose: Indexing
Verified robots.txt: Unknown Block: Depends Verified Bot
marfeel-social Safe

Marfeel Social Crawler

Marfeel Social Crawler is a search crawler from Marfeel used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: Marfeel Type: Search Purpose: Indexing
Verified robots.txt: Unknown Block: Depends Verified Bot
search.marginalia.nu Safe

Marginalia Search

Marginalia Search is a search crawler from Marginalia used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: Marginalia Type: Search Purpose: Indexing
Verified robots.txt: Unknown Block: Depends Verified Bot
MarketGoo Neutral

marketgoo

marketgoo is a web scraper used for public web data collection, page extraction, content monitoring.

Operator and family: marketgoo Type: Scraper Purpose: Scraping
Verified robots.txt: Yes Block: Depends Verified Bot
MarkMonitor Caution

MarkMonitor

MarkMonitor is a brand Protection Crawler used for MarkMonitor is used for Automated website access, content retrieval, or integration activity associated with the listed operator.

Operator and family: MarkMonitor Type: Brand Protection Crawler Purpose: General Bot Traffic
Verified robots.txt: Unknown Block: Monitor Documented
(dbot) Safe

Mars Finder

Mars Finder is a search crawler from Mars Flag used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: Mars Flag Type: Search Purpose: Indexing
Verified robots.txt: Yes Block: Depends Official Documented
MauiBot Neutral

MauiBot

MauiBot is a web scraper from Unknown used for public web data collection, page extraction, content monitoring.

Family: MauiBot Type: Scraper Purpose: Web Scraping
Not verified robots.txt: Unknown Block: Depends Observed
Mavifinds | Bot Safe

Mavifinds Bot

Mavifinds Bot is a security scanner from Mavifinds used for security scanning, malware checks, vulnerability assessment, certificate review.

Operator and family: Mavifinds Type: Security
Verified robots.txt: Unknown Block: No Verified Bot
mediaboardbot Safe

mediaboardbot

mediaboardbot is a monitoring bot from Mediaboard used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: Mediaboard Type: Monitoring
Verified robots.txt: No Block: No Verified Bot
MedialogiaBot Safe

Medialogia Bot

Medialogia Bot is a monitoring bot from Medialogia used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: Medialogia Type: Monitoring
Verified robots.txt: No Block: No Verified Bot
MediaMonitoringBot Neutral

MediaMonitoringBot

MediaMonitoringBot is a monitoring bot used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: MediaMonitoringBot Type: Monitoring
Verified robots.txt: Yes Block: Depends Verified Bot
Mediatoolkitbot Neutral

Mediatoolkitbot

Mediatoolkitbot is a monitoring bot from Determ used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: Determ Type: Monitoring
Verified robots.txt: Yes Block: Depends Verified Bot
MediavineMetadataParser Neutral

Mediavine Medatada Parser

Mediavine Medatada Parser is a search crawler from Mediavine used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: Mediavine Type: Search Purpose: Indexing
Verified robots.txt: Unknown Block: Depends Verified Bot
MediavineMetadataParser/ Neutral

Mediavine Metadata Parser

Mediavine Metadata Parser is an ad Crawler from Mediavine used for Mediavine Metadata Parser is used for Mediavine Metadata Parser fetches page metadata for Grow, bookmarking, sharing.

Operator and family: Mediavine Type: Ad Crawler Purpose: Metadata Parsing
Verified robots.txt: Unknown Block: Depends Documented
MegaIndex Neutral

MegaIndex

MegaIndex is a search crawler used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: MegaIndex Type: Search Purpose: SEO Crawling
Not verified robots.txt: Unknown Block: Depends Observed
CentComBot Neutral

MelonMesa Bot

MelonMesa Bot is a security scanner from MelonMesa used for security scanning, malware checks, vulnerability assessment, certificate review.

Operator: MelonMesa Family: MelonMesa Bot Type: Security
Verified robots.txt: Unknown Block: Depends Verified Bot
Meltwater Neutral

Meltwater

Meltwater is a monitoring bot used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: Meltwater Type: Monitoring
Verified robots.txt: Yes Block: Depends Verified Bot
meta-externalads/1.1 Neutral

Meta-ExternalAds

Meta-ExternalAds is a search crawler from Meta used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: Meta Type: Search Purpose: Indexing
Verified robots.txt: Yes Block: Depends Verified Bot
meta-externalagent Neutral

Meta-ExternalAgent

Meta-ExternalAgent is an AI training crawler from Meta used for AI model training, dataset discovery.

Operator and family: Meta Type: Search Purpose: Ai Training
Verified robots.txt: Yes Block: Depends Verified Bot