Bot Directory

Cookiebot Safe

Cookiebot

Cookiebot scans the website for cookies and trackers to gather and provide the information on a cookie banner.

Operator and family: Cybot A/S Type: Security

Verified robots.txt: No Block: No Verified Bot

CookieYesbot Safe

CookieYesBot

CookieYesbot scans and identify cookies and related information on websites that use CookieYes platform.

Operator and family: CookieYes Type: Security

Verified robots.txt: No Block: No Verified Bot

Cotoyogi Neutral

Cotoyogi

By providing a centralized management environment for content and models, we will build a framework that promotes both data utilization and research and development of AI technology, and.

Operator and family: Research Organization of Information and Systems Type: Search Purpose: Ai Training

Verified robots.txt: Yes Block: Depends Verified Bot

Coveobot Safe

Coveo Bot

Coveo provide services to website, customer service and commerce solutions so they can feature relevant experiences to their end users; said services are based on a unified index which.

Operator and family: Coveo Type: Ecommerce

Verified robots.txt: No Block: Depends Verified Bot

Crawlson Safe

Crawlson

Crawlson is a search engine crawler for the crawlson.com search engine.

Operator and family: Crawlson Type: Search Purpose: Indexing

Verified robots.txt: Yes Block: Depends Verified Bot

Crazy Egg Safe

Crazy Egg Bot

Crazy Egg bot that takes screenshots of pages, collects assets, tests script installation.

Operator and family: CrazyEgg Type: Preview

Verified robots.txt: No Block: No Verified Bot

CriteoBot Neutral

CriteoBot

Criteo Crawler is a software that visits web pages and analyzes its content to serve relevant ads on them.

Operator and family: Criteo Type: Search Purpose: Indexing

Verified robots.txt: No Block: Depends Verified Bot

Criticalcss.com/ Neutral

Critical CSS Bot

Critical CSS Bot is an automated bot from Critical CSS.

Operator and family: Critical CSS

Verified robots.txt: No Block: Depends Verified Bot

cron-job.org Safe

cron-job.org

cron-job.org is an automated bot from cron-job.org.

Operator and family: cron-job.org Purpose: Site Owner Fetch

Verified robots.txt: No Block: No Verified Bot

customerio-webhooks Safe

Customer.io Webhooks

Customer.io Webhooks is a webhook callback service from Customer.io used for service notifications, callbacks, or integration events.

Operator and family: Customer.io Type: Webhook

Verified robots.txt: Unknown Block: Depends Verified Bot

cXensebot Neutral

Cxense

The Cxensebot performs SEO monitoring and analysis of customer webpages.

Operator and family: Cxense Type: Monitoring

Verified robots.txt: No Block: Depends Verified Bot

cybaa-agent Safe

Cybaa Agent

Cybaa Agent is a security scanner from Cybaa used for security checks, malware review, or abuse prevention.

Operator and family: Cybaa Type: Security

Verified robots.txt: Unknown Block: Depends Verified Bot

CybaaBot Safe

Cybaa Bot

Performs broad security research by crawling the most popular domains obtained from Crawler.Ninja.

Operator: Cybaa Family: Cybaa Bot Type: Security

Verified robots.txt: No Block: No Verified Bot

dash0-synthetic Safe

Dash0 Synthetic Monitoring

Dash0 Synthetic Monitoring is a monitoring bot from Dash0 used to check website availability, performance, or page health.

Operator and family: Dash0 Type: Monitoring

Verified robots.txt: Unknown Block: Depends Verified Bot

RSiteAuditor Neutral

DataForSEO

DataForSEO is using RSiteAuditor to scan websites for critical on-site SEO errors and provides aggregated data in a structured form to its customer through a RESTful API.

Operator and family: DataForSEO Type: Scraper Purpose: Scraping

Verified robots.txt: No Block: Depends Verified Bot

DataForSeoBot Neutral

DataForSEO Bot

DataForSEO Bot is a driving force of our leading product - Backlinks API, which has been developed with a single purpose: providing website owners, webmasters, and SEO professionals with.

Operator and family: DataForSEO Type: Scraper Purpose: Scraping

Verified robots.txt: No Block: Depends Verified Bot

Dataprovider Safe

Dataprovider.com

Dataprovider.com indexes the web and structures the data.

Operator and family: Dataprovider.com Type: Search Purpose: Indexing

Verified robots.txt: No Block: Depends Verified Bot

Daum/4.1 Safe

Daum

Daum is a search crawler associated with Daum, used for search indexing and content discovery.

Operator and family: Daum Type: Search Purpose: Indexing

Verified robots.txt: No Block: Depends Verified Bot

deadlinkchecker Safe

deadlinkchecker

Dead Link Checker is a service which crawls a customer's website reporting on any broken links (404, 500 etc) it finds.

Operator and family: DLC Websites Type: Search Purpose: Indexing

Verified robots.txt: No Block: No Verified Bot

lumar Safe

Deepcrawl

The DeepCrawl bot crawls the websites of its customers to collect performance analytics and suggest SEO optimizations.

Operator and family: Lumar Type: Search Purpose: Indexing

Verified robots.txt: No Block: No Verified Bot

DeepSeekBot Neutral

DeepSeekBot

Claimed DeepSeek crawler identifier found in third-party bot directories; no official public DeepSeek user-agent documentation was confirmed.

Operator and family: DeepSeek Type: Ai Purpose: AI Model Data Collection

Not verified robots.txt: Unknown Block: Depends Observed

Detectify Safe

Detectify

Detectify is a security scanner from Detectify used for security checks, malware review, or abuse prevention.

Operator and family: Detectify Type: Security

Verified robots.txt: Yes Block: Depends Verified Bot

Devin Neutral

Devin

Devin is a collaborative AI teammate built to help ambitious engineering teams achieve more.

Operator and family: Devin AI Type: Ai Purpose: Ai Assistant

Verified robots.txt: No Block: Depends Verified Bot

Diffbot Neutral

Diffbot

Diffbot is a search crawler from Diffbot used for search indexing, page discovery, or content rendering.

Operator and family: Diffbot Type: Search Purpose: Indexing

Verified robots.txt: Yes Block: Depends Official Documented