Bot Directory

Search the Botcrawl Bot Directory database to identify AI crawlers, search bots, scrapers, scanners, monitoring bots, preview bots, and user-agent patterns.

Adyen Safe

Adyen

The Adyen webhooks integration sends HTTP requests to inform web servers about payment-related events.

Adyen Adyen Webhook Webhook
Verified robots.txt: No Block: No Verified Bot
AfterShip Magento-2 Connector Safe

AfterShip Magento 2 Connector

AfterShip Magento 2 Connector is a commerce-related fetcher from AfterShip used for commerce, checkout, advertising, or payment-related checks.

AfterShip AfterShip Ecommerce Ecommerce
Verified robots.txt: No Block: No Verified Bot
aa/1.0 Neutral

Agency Analytics Crawler

A web crawler by Agency Analytics that allows their clients to check their own sites for SEO.

Agency Analytics Agency Analytics Search Indexing
Verified robots.txt: No Block: Depends Verified Bot
AgencyToolsPro Neutral

AgencyToolsPro Uptime Monitoring Bot

Website monitoring bot, let's you know that your website is down.

AgencyToolsPro AgencyToolsPro Monitoring Monitoring
Verified robots.txt: No Block: Depends Verified Bot
AhrefsSiteAudit Neutral

Ahrefs Site Audit

Ahrefs Site Audit is a search crawler from Ahrefs used for search indexing, page discovery, or content rendering.

Ahrefs Ahrefs Search Indexing
Verified robots.txt: No Block: Depends Verified Bot
AhrefsBot Neutral

AhrefsBot

AhrefsBot is a Web Crawler that powers the 12 trillion link database for Ahrefs online marketing toolset. It constantly crawls web to fill our database with new links and check the status.

Ahrefs Ahrefs Search Indexing
Verified robots.txt: No Block: Depends Verified Bot
Akismet Safe

Akismet

The integration API for the Akismet spam filtering service.

Akismet Akismet Security Security
Verified robots.txt: No Block: No Verified Bot
AlertSite Safe

Alertsite by Smartbear

Alertsite by Smartbear is the HTTP monitoring probe that monitors its customers websites for availability and performance anomalies.

Smartbear Smartbear Monitoring Monitoring
Verified robots.txt: No Block: No Verified Bot
Algolia Crawler Safe

Algolia

Algolia is a search crawler from Algolia used for search indexing, page discovery, or content rendering.

Algolia Algolia Search Indexing
Verified robots.txt: No Block: Depends Verified Bot
AllAfrica Safe

All Africa Crawler

All Africa Crawler is a crawler used for scraping, SEO analysis, or data collection from AllAfrica Global Media.

AllAfrica Global Media AllAfrica Global Media Scraper Scraping
Verified robots.txt: No Block: Depends Verified Bot
AmazonAdBot Neutral

Amazon AdBot

Amazon AdBot is a crawler used by Amazon's advertising services. It visits advertiser landing pages to ensure they are compliant with advertising policies.

Amazon Amazon Search Indexing
Verified robots.txt: No Block: Depends Verified Bot
AmazonBuyForMe Neutral

Amazon Buy For Me

Amazon Buy For Me is a commerce-related fetcher from Amazon used for commerce, checkout, advertising, or payment-related checks.

Amazon Amazon Ecommerce Ecommerce
Verified robots.txt: No Block: Depends Verified Bot