Search the Botcrawl Bot Directory database to identify AI crawlers, search bots, scrapers, scanners, monitoring bots, preview bots, and user-agent patterns.
meta-externalads/1.1
Neutral
The Meta-ExternalAds crawler crawls the web for use cases such as improving advertising and other business-related products and services.
meta-externalagent
Neutral
Use cases such as training AI models or improving products by indexing content directly.
meta-externalfetcher
Neutral
Crawler receives individual links at the user's initiative to support certain product features.
meta-webindexer/1.1
Neutral
Meta-WebIndexer is a search crawler from Meta used for search indexing, page discovery, or content rendering.
Metorik
Safe
Metorik is a commerce-related fetcher from Metorik used for commerce, checkout, advertising, or payment-related checks.
MgidBot
Neutral
MgidBot is used for detecting context categories of the content for advertising recommendations.
MicrosoftPreview
Neutral
MicrosoftPreview generates page snapshots for Microsoft products.
mirrorweb
Neutral
MirrorWebCrawler is a search crawler from MirrorWeb Ltd used for search indexing, page discovery, or content rendering.
MissinglettrBot
Neutral
Missinglettr will crawl specific blog posts on customers' sites to help turn them into social media campaigns.
MistralAI-Index
Safe
MistralAI-Index is a search crawler from Mistral AI used for search indexing, page discovery, or content rendering.
MistralAI-User
Neutral
Bot for user actions in le Chat by Mistral AI, for instance when asked to open a web page.
MJ12bot
Neutral
MJ12bot is the web crawler for Majestic. MJ12Bot does not currently cache web content or personal data. Instead it maps the link relationships between websites to build a search engine.
ModularConnector
Safe
Modular DS is a monitoring bot from Uniqoders Technologies SL used to check website availability, performance, or page health.
MojeekBot
Safe
Details and information for webmasters regarding Mojeekbot, the web crawler for the Mojeek search engine.
Mollie HTTP client/
Safe
Mollie B.V. is a payment service provider. We use webhooks to notify our merchants about updates to their payments.
momenticbot
Safe
MomenticBot is a monitoring bot from Momentic used to check website availability, performance, or page health.
Monitage
Safe
Uptime monitoring bot, part ofclicky.com web analytics.
monitis
Safe
Monitis is a monitoring bot from Monitis used to check website availability, performance, or page health.
MonitoRSS
Safe
MonitoRSS is a monitoring bot from MonitoRSS used to check website availability, performance, or page health.
Monsidobot
Safe
Monsidobot is a monitoring bot from Acquia used to check website availability, performance, or page health.
MonSpark
Safe
MonSpark is a website monitoring service. Its monitoring bot checks website availability, network conditions and TLS certificate validity.
montastic-monitor
Safe
MontasticMonitor is a monitoring bot from Montastic used to check website availability, performance, or page health.
MotoMinerBot
Neutral
MotoMinerBot is MotoMiner's web crawling bot. All vehicle detail pages we index are searchable via MotoMiner's search engine.
DotBot
Neutral
Dotbot is Moz's web crawler, it gathers web data for the Moz Link Index. This data we collect through Dotbot is available in the Links section of your Moz Pro campaign, Link Explorer, and.