Bluesky
Neutral
Bluesky Link Preview Service
Bluesky social pulls links in advance to render webpage previews.
Verified robots.txt: No Block: Depends Verified Bot
Search the Botcrawl Bot Directory database to identify AI crawlers, search bots, scrapers, scanners, monitoring bots, preview bots, and user-agent patterns.
Bluesky
Neutral
Bluesky social pulls links in advance to render webpage previews.
BoardGamePrices Bot
Neutral
Price comparison site for board games. Need to crawl store pages for participating stores. All stores give permission to be crawled.
botify
Neutral
SiteCrawler, part of the Botify Analytics suite, gives enterprise SEO teams the power to evaluate the structure and content of their websites just like a search engine.
magpie-crawler
Neutral
The Magpie Crawler indexes content for its soical media monitoring solution.
Bravebot
Neutral
Brave search has a crawler to discover new pages and index their content.
brightbot
Safe
Brightbot is a monitoring bot from Bright Data used to check website availability, performance, or page health.
BrightEdge Crawler
Neutral
Autopilot is an SEO marketing automation tool that includes features for internal linking and image optimization. We crawl customer sites so that we can determine the best links to use on.
browserbase
Neutral
Browserbase is an AI agent from Browserbase used for AI-related crawling, retrieval, search, or assistant activity.
BufferLinkPreviewBot
Neutral
When Buffer users share links in their social media posts, their scraper helps create engaging previews.
BugsNag
Safe
BugsNag integration service, now Insight Hub, is used for error and performance monitoring of web applications.
Buttondown
Safe
The bot fetches RSS feeds to import into Buttondown newsletters.
Caliperbot/1.0
Neutral
Caliperbot crawls Conductor clients' and prospects' websites for HTML feature extraction to power Content Analytics features within our Searchlight web application.
CapitalOneBot
Neutral
Capital One Bot crawls dealer websites for getting the usage information for Capital One lead navigator button.
Catchpoint
Safe
Our mission has remained the same from day one: to prioritize monitoring and observability from the end-user perspective.
CCBot
Neutral
CCBot is a security scanner from Common Crawl used for security checks, malware review, or abuse prevention.
CertChief
Safe
Cert Chief is a certificate monitoring tool that periodically crawl web properties to check their configuration and reports problems and changes when they are detected.
channable/importer
Neutral
Channable is a crawler used for scraping, SEO analysis, or data collection from Channable.
channel3bot
Neutral
Channel3Bot visits public product pages to index their content, with the aim of driving traffic back to those websites.
ChargeBee
Safe
Chargebee provides a webhooks integration to notify web servers of payment events.
chatgpt-operator
Safe
ChatGPT-Operator is an AI agent from OpenAI used for AI-related crawling, retrieval, search, or assistant activity.
ChatGPT-User
Neutral
ChatGPT-User represents user-initiated browsing or retrieval activity from ChatGPT.
Checkly
Safe
Checkly is a high-programmability active monitoring solution. We support users in monitoring their websites and APIs. Puppeteer and Playwright (both supported) are browser automation tools.
CheckView
Neutral
CheckView is a monitoring bot from CheckView.io used to check website availability, performance, or page health.
chrome-privacy-preserving-prefetch-proxy
Safe
Chrome Privacy Preserving Prefetch Proxy is a preview or prefetch bot from Google used to fetch pages for previews, unfurls, or user-triggered rendering.