Bot Directory

Search the Botcrawl Bot Directory database to identify AI crawlers, search bots, scrapers, scanners, monitoring bots, preview bots, and user-agent patterns.

MagiBot Safe

MagiBot

MagiBot is owned by Peak Labs which focuses on the research and development of information extraction and retrieval technology to transform knowledge in natural language into immeasurable.

Operator and family: Peak Labs Type: Search Purpose: Indexing
Verified robots.txt: No Block: Depends Verified Bot
Magnet.me-web/ Neutral

MagnetmeBot

MagnetmeBot checks the websites of our paying customers and ensures the job openings are being kept in sync.

Operator and family: Magnet.me Type: Search Purpose: Indexing
Verified robots.txt: No Block: Depends Verified Bot
Mail.RU_Bot Neutral

MailRUBot

The mail.ru bot is a mail fetcher on behalf of the Mail.ru email service.

Operator and family: Mail Russia Type: Scraper Purpose: Scraping
Verified robots.txt: Yes Block: Depends Verified Bot
MainWP Safe

MainWP

Website Managed - MainWP Control Dashboard for accessing MainWP child sites.

Operator and family: Direct Support / Website Managed Type: Monitoring
Verified robots.txt: No Block: No Verified Bot
Make/production Safe

Make Platform

Make Platform is a webhook callback service from Make used for service notifications, callbacks, or integration events.

Operator and family: Make Type: Webhook
Verified robots.txt: No Block: No Verified Bot
ManageWP Safe

ManageWP

The ManageWP webhooks integration to manage multiple WordPress websites with a single dashboard.

Operator and family: ManageWP Type: Webhook
Verified robots.txt: No Block: No Verified Bot
Manus Bot Neutral

Manus Bot

Manus Bot is an AI fetcher from Manus used for AI-related crawling, retrieval, search, or assistant activity.

Operator and family: Manus Type: Ai Purpose: Ai Assistant
Verified robots.txt: No Block: Depends Verified Bot
marfeel-audits Safe

Marfeel Audits Crawler

Marfeel Audits Crawler is a monitoring bot from Marfeel used to check website availability, performance, or page health.

Operator and family: Marfeel Type: Monitoring
Verified robots.txt: Unknown Block: Depends Verified Bot
marfeel-flowcards Safe

Marfeel Flowcards Crawler

Marfeel Flowcards Crawler is a search crawler associated with Marfeel, used for search indexing or content discovery.

Operator and family: Marfeel Type: Search Purpose: Indexing
Verified robots.txt: Unknown Block: Depends Verified Bot
marfeel-preview Safe

Marfeel Preview Crawler

Marfeel Preview Crawler is a search crawler associated with Marfeel, used for search indexing or content discovery.

Operator and family: Marfeel Type: Search Purpose: Indexing
Verified robots.txt: Unknown Block: Depends Verified Bot
marfeel-social Safe

Marfeel Social Crawler

Marfeel Social Crawler is a search crawler from Marfeel used for search indexing, page discovery, or content rendering.

Operator and family: Marfeel Type: Search Purpose: Indexing
Verified robots.txt: Unknown Block: Depends Verified Bot
search.marginalia.nu Safe

Marginalia Search

Marginalia Search is a noncommercial niche search engine focusing on old websites, personal websites, and blogs that suffer crippling discoverability problems in today's fiercely.

Operator and family: Marginalia Type: Search Purpose: Indexing
Verified robots.txt: No Block: Depends Verified Bot
MarketGoo Neutral

marketgoo

marketgoo is a crawler used for scraping, SEO analysis, or data collection from marketgoo.

Operator and family: marketgoo Type: Scraper Purpose: Scraping
Verified robots.txt: Yes Block: Depends Verified Bot
(dbot) Safe

Mars Finder

Mars Finder is a search crawler from Mars Flag used for search indexing, page discovery, or content rendering.

Operator and family: Mars Flag Type: Search Purpose: Indexing
Verified robots.txt: Yes Block: Depends Official Documented
MauiBot Neutral

MauiBot

MauiBot web-scraper identifier observed in user-agent databases and server logs.

Family: MauiBot Type: Scraper Purpose: Web Scraping
Not verified robots.txt: Unknown Block: Depends Observed
Mavifinds | Bot Safe

Mavifinds Bot

The Mavifinds Bot is part of a security service that monitors websites. It can automatically activate security measures, such as an under attack mode, in response to threats.

Operator and family: Mavifinds Type: Security
Verified robots.txt: No Block: No Verified Bot
mediaboardbot Safe

mediaboardbot

Mediaboard's web crawler (media monitoring) detects client mentions in online news and public content.

Operator and family: Mediaboard Type: Monitoring
Verified robots.txt: No Block: No Verified Bot
MedialogiaBot Safe

Medialogia Bot

Medialogia Bot is the web crawler for Medialogia, a Russian media monitoring company. It collects data from online news and social media for analysis.

Operator and family: Medialogia Type: Monitoring
Verified robots.txt: No Block: No Verified Bot
MediaMonitoringBot Neutral

MediaMonitoringBot

MediaMonitoringBot crawls and indexes news and media publishers websites for a new materials and try to match it against keywords provided by our customers (subscribers) and send them.

Operator and family: MediaMonitoringBot Type: Monitoring
Verified robots.txt: Yes Block: Depends Verified Bot
Mediatoolkitbot Neutral

Mediatoolkitbot

The Mediatoolkitbot is a media monitoring tool that crawls the open internet looking for phrases Determ users search for, helping marketers find relevant opportunities for advertising.

Operator and family: Determ Type: Monitoring
Verified robots.txt: Yes Block: Depends Verified Bot
MediavineMetadataParser Neutral

Mediavine Medatada Parser

Mediavine Medatada Parser is a search crawler from Mediavine used for search indexing, page discovery, or content rendering.

Operator and family: Mediavine Type: Search Purpose: Indexing
Verified robots.txt: No Block: Depends Verified Bot
MegaIndex Neutral

MegaIndex

MegaIndex crawler used for backlink, SEO, and competitive intelligence services.

Operator and family: MegaIndex Type: Search Purpose: SEO Crawling
Not verified robots.txt: Unknown Block: Depends Observed
CentComBot Neutral

MelonMesa Bot

This bot is used to aggregate data about a popular online multiplayer game from consenting hosts who have opted-in to this collection. The data that is aggregated is reflected in a panel.

Operator: MelonMesa Family: MelonMesa Bot Type: Security
Verified robots.txt: No Block: Depends Verified Bot
Meltwater Neutral

Meltwater

Meltwater is a monitoring bot from Meltwater used to check website availability, performance, or page health.

Operator and family: Meltwater Type: Monitoring
Verified robots.txt: Yes Block: Depends Verified Bot