Bot Directory

Public records for crawlers, AI agents, scrapers, scanners, and user-agent patterns.

AI Crawlers Search Bots Scrapers Scanners
Need live control?

Classify traffic and enforce rules before content is served.

Try Edge Free Open Edge BotDB
998 bots
kagi-fetcher Neutral

kagi-fetcher

kagi-fetcher is an AI search crawler from Kagi used for AI search indexing, answer retrieval, citation discovery.

Operator and family: Kagi Type: Ai Purpose: Ai Search
Not verified robots.txt: Unknown Block: Depends Observed
View Record Open in Edge BotDB
kakaotalk-scrap Neutral

KakaoTalk Scrap

KakaoTalk Scrap is a web scraper from Kakao used for public web data collection, page extraction, content monitoring.

Operator: Kakao Family: KakaoTalk Scrap Type: Scraper Purpose: Scraping
Verified robots.txt: No Block: Depends Verified Bot
View Record Open in Edge BotDB
Kangaroo Bot Neutral

Kangaroo Bot

Kangaroo Bot is an AI crawler from Kangaroo used for assistant-driven browsing, page retrieval.

Operator and family: Kangaroo Type: Ai Purpose: Ai Assistant
Not verified robots.txt: Unknown Block: Depends Observed
View Record Open in Edge BotDB
KargoBot-Artemis Safe

KargoBot-Artemis

KargoBot-Artemis is a security scanner from Kargo used for security scanning, malware checks, vulnerability assessment, certificate review.

Operator and family: Kargo Type: Security
Verified robots.txt: No Block: No Verified Bot
View Record Open in Edge BotDB
netarkivindsamling Neutral

kb.dk_bot

kb.dk_bot is a search crawler from Netarkivet used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: Netarkivet Type: Search Purpose: Indexing
Verified robots.txt: No Block: Depends Verified Bot
View Record Open in Edge BotDB
kernel Neutral

Kernel Browsers

Kernel Browsers is an AI crawler from Kernel used for assistant-driven browsing, page retrieval.

Operator and family: Kernel Type: Ai Purpose: Ai Assistant
Verified robots.txt: Unknown Block: Depends Verified Bot
View Record Open in Edge BotDB
KeyCDN Neutral

KeyCDN

KeyCDN is a monitoring bot used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: KeyCDN Type: Monitoring Bot Purpose: Uptime And Performance Monitoring
Verified robots.txt: Unknown Block: Allow Documented
View Record Open in Edge BotDB
Kimi-User Neutral

Kimi User

Kimi User is an AI crawler from Moonshot AI used for assistant-driven browsing, page retrieval.

Operator and family: Moonshot AI Type: Ai Purpose: Ai Assistant
Not verified robots.txt: Unknown Block: Depends Observed
View Record Open in Edge BotDB
kinsta-bot Safe

Kinsta

Kinsta is a monitoring bot used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: Kinsta Type: Monitoring
Verified robots.txt: Unknown Block: No Verified Bot
View Record Open in Edge BotDB
Klaviyo/1.0 Neutral

klaviyo

klaviyo is a web scraper used for public web data collection, page extraction, content monitoring.

Operator and family: Klaviyo Type: Scraper Purpose: Scraping
Verified robots.txt: Unknown Block: Depends Verified Bot
View Record Open in Edge BotDB
KlaviyoAIBot Neutral

KlaviyoAIBot

KlaviyoAIBot is a search crawler from Klaviyo used for search indexing, URL discovery, page rendering, result freshness.

Operator: Klaviyo Family: KlaviyoAIBot Type: Search Purpose: Indexing
Verified robots.txt: Unknown Block: Depends Verified Bot
View Record Open in Edge BotDB
Known Agent Neutral

Known Agent

Known Agent is an AI crawler from Known Agents used for assistant-driven browsing, page retrieval.

Operator and family: Known Agents Type: Ai Purpose: Ai Assistant
Verified robots.txt: No Block: Depends Verified Bot
View Record Open in Edge BotDB
KunatoCrawler Neutral

KunatoCrawler

KunatoCrawler is an AI crawler from Kunato used for AI crawler discovery, assistant retrieval, AI search support, or model-related web access.

Operator and family: Kunato Type: Ai Purpose: Ai Crawl
Not verified robots.txt: Unknown Block: Depends Observed
View Record Open in Edge BotDB
laion-huggingface-processor Neutral

laion-huggingface-processor

laion-huggingface-processor is an AI training crawler from LAION / Hugging Face used for AI model training, dataset discovery.

Operator and family: LAION / Hugging Face Type: Ai Purpose: Ai Training
Not verified robots.txt: Unknown Block: Depends Observed
View Record Open in Edge BotDB
LAIONDownloader Neutral

LAIONDownloader

LAIONDownloader is an AI training crawler from LAION used for AI model training, dataset discovery.

Operator and family: LAION Type: Ai Purpose: Ai Training
Not verified robots.txt: Unknown Block: Depends Observed
View Record Open in Edge BotDB
LCC Neutral

LCC

LCC is an AI crawler from Unknown used for AI crawler discovery, assistant retrieval, AI search support, or model-related web access.

Family: LCC Type: Ai Purpose: Ai Crawl
Not verified robots.txt: Unknown Block: Depends Observed
View Record Open in Edge BotDB
LegalMonster Safe

LegalMonster

LegalMonster is a security scanner from Legal Monster ApS used for security scanning, malware checks, vulnerability assessment, certificate review.

Operator and family: Legal Monster ApS Type: Security
Verified robots.txt: Unknown Block: No Verified Bot
View Record Open in Edge BotDB
Leikibot Neutral

LeikiBot

LeikiBot is a search crawler from DoubleVerify used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: DoubleVerify Type: Search Purpose: Indexing
Verified robots.txt: No Block: Depends Verified Bot
View Record Open in Edge BotDB
https://www.letsencrypt.org Safe

Let’s Encrypt

Let's Encrypt is a monitoring bot used for security scanning, malware checks, vulnerability assessment, certificate review.

Operator and family: Let's Encrypt Type: Security
Verified robots.txt: No Block: No Verified Bot
View Record Open in Edge BotDB
Level9SearchBot Safe

Level9SearchBot

Level9SearchBot is a search crawler from Level 9 used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: Level 9 Type: Search Purpose: Indexing
Verified robots.txt: Unknown Block: Depends Verified Bot
View Record Open in Edge BotDB
https://www.loc.gov/programs/web-archiving/for-site-owners/ Neutral

Library Of Congress Web Archiving

Library Of Congress Web Archiving is a feed fetcher from United States Library of Congress used for RSS or Atom feed polling, syndication refresh, subscription delivery.

Operator and family: United States Library of Congress Type: Feed Purpose: Feed Fetch
Verified robots.txt: No Block: Depends Verified Bot
View Record Open in Edge BotDB
LinerBot Neutral

LINER Bot

LINER Bot is an AI training crawler used for AI model training, dataset discovery.

Operator and family: Liner Bot Type: Ai Purpose: Ai Training
Verified robots.txt: No Block: Depends Verified Bot
View Record Open in Edge BotDB
Linespider Safe

Linespider

Linespider is a search crawler from LINE Corporation used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: LINE Corporation Type: Search Purpose: Indexing
Verified robots.txt: Unknown Block: Depends Verified Bot
View Record Open in Edge BotDB
Linguee Bot Neutral

Linguee Bot

Linguee Bot is an AI crawler from Linguee used for AI crawler discovery, assistant retrieval, AI search support, or model-related web access.

Operator and family: Linguee Type: Ai Purpose: Ai Crawl
Not verified robots.txt: Unknown Block: Depends Observed
View Record Open in Edge BotDB