Bot Directory

Search the Botcrawl Bot Directory database to identify AI crawlers, search bots, scrapers, scanners, monitoring bots, preview bots, and user-agent patterns.

Authory Neutral

Authory

The Authory bot visits websites to back up articles on behalf of journalists and other writers who use the service.

Operator and family: Authory Type: Feed Purpose: Feed Fetch
Verified robots.txt: No Block: Depends Verified Bot
AutomatonHeadlessChrome Safe

Automaton

Automaton is a monitoring bot from Automaton used to check website availability, performance, or page health.

Operator and family: Automaton Type: Monitoring
Verified robots.txt: No Block: No Verified Bot
Awario Neutral

AwarioSmartBot

AwarioSmartBot is a web crawlers sent by Awario to discover and collect new and updated web data (that is further used by Internet marketers from all over the world).

Operator and family: Awario Type: Feed Purpose: Feed Fetch
Verified robots.txt: No Block: Depends Verified Bot
Baidu-ADS Safe

Baidu ADS Server Proxy

Baidu ADS Server Proxy is a search crawler from Baidu used for search indexing, page discovery, or content rendering.

Operator and family: Baidu Type: Search Purpose: Indexing
Not verified robots.txt: No Block: Depends Observed
Baiduspider Safe

Baiduspider

Baiduspider is the search engine crawler for the search engine Baidu.

Operator and family: Baidu Type: Search Purpose: Indexing
Verified robots.txt: No Block: Depends Verified Bot
Barkrowler Neutral

Barkrowler

SEO web crawler to identify web page popularity and thematic.

Operator and family: Babbar Type: Search Purpose: Indexing
Verified robots.txt: Yes Block: Depends Verified Bot
BazQux Safe

Bazqux

The Bazqux Fetcher is how BazQux Reader grabs RSS/Atom feeds and com­ments when users choose to sub­scribe to your blog in BazQux Reader. Fetcher col­lects and pe­ri­od­i­cally re­freshes.

Operator and family: Bazqux Type: Feed Purpose: Feed Fetch
Verified robots.txt: No Block: Depends Verified Bot
BestChange Neutral

BestChange Bot

The BestChange bot downloads exchange rate information from 600 websites every 5 seconds.

Operator and family: BestChange Type: Search Purpose: Indexing
Verified robots.txt: No Block: Depends Verified Bot
Better Uptime Bot Safe

Better Uptime

The bot is used for monitoring infrastructure platforms.

Operator and family: Better Stack Type: Monitoring
Verified robots.txt: No Block: No Verified Bot
bnf.fr_bot Neutral

Bibliotheque Nacional de France Crawler

Bibliothèque nationale de France's mission is to collect, catalog, preserve, enrich and communicate the national documentary heritage.

Operator and family: Bibliothèque nationale de France Type: Search Purpose: Indexing
Verified robots.txt: No Block: Depends Verified Bot
bigsur.ai Neutral

Big Sur AI

Big Sur AI Crawler, crawlers users websites to enable AI-infused experiences.

Operator and family: Big Sur AI Type: Search Purpose: Ai Training
Verified robots.txt: No Block: Depends Verified Bot
BigScootsMonitor Safe

BigScootsMonitor

BigScootsMonitor is a monitoring bot from BigScoots used to check website availability, performance, or page health.

Operator: BigScoots Family: BigScootsMonitor Type: Monitoring
Verified robots.txt: No Block: No Verified Bot
BigUpDataBot Neutral

BigUpData Bot

BigUpData Bot is a crawler used for scraping, SEO analysis, or data collection from BigUpData.

Operator and family: BigUpData Type: Scraper Purpose: Scraping
Verified robots.txt: No Block: Depends Verified Bot
http://www.binarycanary.com Safe

BinaryCanary

BinaryCanary monitors websites for availability and performance issues.

Operator and family: Binary Canary Type: Monitoring
Verified robots.txt: No Block: No Verified Bot
adidxbot Neutral

Bing Ads

AdIdxBot is the crawler used by Bing Ads. AdIdxBot crawls ads and follows the websites from those ads for quality control. Just like Bingbot, AdIdxBot has both “desktop” and “mobile”.

Operator and family: Microsoft Type: Search Purpose: Indexing
Verified robots.txt: No Block: Depends Verified Bot
BingPreview Neutral

Bing Preview

BingPreview generates page snapshots for Bing. Note that BingPreview has desktop and mobile variants.

Operator and family: Microsoft Type: Scraper Purpose: Scraping
Verified robots.txt: No Block: Depends Verified Bot
bingbot Safe

BingBot

Bingbot is Microsoft Bing's crawler and handles most of Bing's routine web crawling for search discovery and indexing.

Operator and family: Microsoft Type: Search Purpose: Indexing
Verified robots.txt: Yes Block: Depends Verified Bot
Bitbucket-Webhooks Safe

Bitbucket

Bitbucket is a webhook callback service from Atlassian used for service notifications, callbacks, or integration events.

Operator and family: Atlassian Type: Webhook
Verified robots.txt: No Block: No Verified Bot
FastDAST Safe

Black Duck Fast Dynamic

Fast Dynamic DAST security scanning by Black Duck Software.

Operator and family: Black Duck Type: Security
Verified robots.txt: No Block: No Verified Bot
BLEXBot Neutral

BLEXBot

SEO PowerSuite Link Explorer (webmeup.com) is the world's freshest backlink index, and the primary source of backlink-related data for the SEO PowerSuite tools. We're dedicated to providing.

Operator and family: WebMeUp Type: Search Purpose: Indexing
Verified robots.txt: No Block: Depends Verified Bot
BlingERP Neutral

Bling ERP

Bling ERP is a commerce-related fetcher from Bling used for commerce, checkout, advertising, or payment-related checks.

Operator and family: Bling Type: Ecommerce
Verified robots.txt: No Block: Depends Verified Bot
36b1546a5700e52eb2972b3f92b314fa Safe

Blockaid

Blockaid is a security scanner from Blockaid used for security checks, malware review, or abuse prevention.

Operator and family: Blockaid Type: Security
Verified robots.txt: No Block: No Verified Bot
Blogtrottr Safe

Blogtrottr

Blogtrottr delivers updates from all of your favourite news, feeds, and blogs directly to your email inbox, giving you the flexibility to stay updated whilst on the go.

Operator and family: Blogtrottr Type: Feed Purpose: Feed Fetch
Verified robots.txt: No Block: Depends Verified Bot
BlogVault Safe

BlogVault

WordPress services like backup, security, monitoring etc.

Operator and family: BlogVault Type: Security
Verified robots.txt: No Block: No Verified Bot