Bot Directory

Search the Botcrawl Bot Directory database to identify AI crawlers, search bots, scrapers, scanners, monitoring bots, preview bots, and user-agent patterns.

660 bots
PRTGCloudBot Safe

PaesslerCloudBot

The PaesslerCloudBot is used by Paessler PRTG to monitor websites for availability and performance.

Operator and family: Paessler Type: Monitoring
Verified robots.txt: No Block: No Verified Bot
PagePeeker Safe

PagePeeker

PagePeeker screenshot and website-thumbnail robot.

Operator and family: PagePeeker Type: Preview Purpose: Website Thumbnailing
Verified robots.txt: Yes Block: No Official Documented
PanguBot Neutral

PanguBot

Claimed Huawei PanGu crawler identifier used in third-party AI-agent directories.

Operator: Huawei Family: PanGu Type: Ai Purpose: AI Model Data Collection
Not verified robots.txt: Unknown Block: Depends Observed
Panopta Safe

Panopta

Panopta is an uptime monitoring service acquired by Fortinet.

Operator and family: Panopta Type: Monitoring
Verified robots.txt: No Block: No Verified Bot
parsely.com Safe

Parse.ly

Parse.ly is a monitoring bot from Parse.ly used to check website availability, performance, or page health.

Operator and family: Parse.ly Type: Monitoring
Verified robots.txt: Yes Block: No Verified Bot
ParticleNewsBot Neutral

ParticleNewsBot

Particle is an AI powered aggregator that collects news from many sources.

Operator: Particle Family: ParticleNewsBot Type: Search Purpose: Indexing
Verified robots.txt: No Block: Depends Verified Bot
PayPal Safe

PayPal

The PayPal webhooks is part of PayPal's Instant Payment Notification message service, automatically notifying merchants of events related to PayPal transactions.

Operator and family: PayPal Type: Webhook
Verified robots.txt: No Block: No Verified Bot
AdpResearchBot Neutral

payroll-bot

Payroll-bot is an AI crawler operated by ADP, Inc. to collect publicly available legal and payroll documentation.

Operator: ADP Family: payroll-bot Type: Search Purpose: Ai Training
Verified robots.txt: No Block: Depends Verified Bot
Perplexity-User Neutral

Perplexity-User

Perplexity-User is an AI agent from Perplexity used for AI-related crawling, retrieval, search, or assistant activity.

Operator and family: Perplexity Type: Ai Purpose: Ai Assistant
Verified robots.txt: Unknown Block: Depends Verified Bot
PerplexityBot Safe

PerplexityBot

PerplexityBot is an AI crawler from Perplexity used for AI-related crawling, retrieval, search, or assistant activity.

Operator and family: Perplexity Type: Ai Purpose: Ai Search
Verified robots.txt: Unknown Block: Depends Verified Bot
PetalBot Neutral

PetalBot

PetalBot is to access both PC and mobile websites and establish an index database which enables users to search the content of your site in Petal search engine and present content.

Operator and family: Huawei Type: Search Purpose: Indexing
Verified robots.txt: Yes Block: Depends Verified Bot
Pingdom.com_bot_version_1.4 Safe

Pingdom

Pingdom is a monitoring bot from Pingdom used to check website availability, performance, or page health.

Operator and family: Pingdom Type: Monitoring
Verified robots.txt: Yes Block: Depends Verified Bot
pingping.io/ Safe

PingPing

PingPing is a website monitoring service whose bots check website uptime and TLS certificate validity.

Operator and family: PingPing Type: Monitoring
Verified robots.txt: No Block: No Verified Bot
Pinterestbot Safe

PinterestBot

PinterestBot is a search crawler from Pinterest used for search indexing, page discovery, or content rendering.

Operator and family: Pinterest Type: Search Purpose: Indexing
Verified robots.txt: Yes Block: Depends Official Documented
Pocket Casts Feed Parser Safe

Pocket Casts Feed Parser

Fetches podcast feeds, for playback in the Pocket Casts apps.

Operator and family: Pocket Casts Type: Feed Purpose: Feed Fetch
Verified robots.txt: No Block: Depends Verified Bot
polar.sh webhooks Safe

Polar

The Polar webhooks integration sends HTTP requests to inform web servers about billing events.

Operator and family: Polar Type: Webhook
Verified robots.txt: No Block: No Verified Bot
Potions Safe

Potions

The Potions bot fetches product feeds and crawls data from its customers' websites, used for e-commerce related services.

Operator and family: Potions Type: Ecommerce
Verified robots.txt: No Block: Depends Verified Bot
Prerender Neutral

prerender

prerender is a crawler used for scraping, SEO analysis, or data collection from Prerender, LLC.

Operator: Prerender, LLC Family: LLC. Type: Scraper Purpose: Scraping
Verified robots.txt: No Block: Depends Verified Bot
PressEngineBot Neutral

PressEngine Bot

The PressEngine Bot verifies coverage created by video games press as genuine and their own creation. When a member of the video games press is granted a review key for a video game they.

Operator and family: PressEngine Type: Security
Verified robots.txt: Yes Block: Depends Verified Bot
PrintFriendly.com Safe

PrintFriendly

Extract Content to Show Print Friendly version. Publishers typically embed our button - https://www.printfriendly.com/button - so that their visitors can view a Print Friendly Page and/or.

Operator and family: PrintFriendly.com Type: Preview
Verified robots.txt: No Block: No Verified Bot
Pro-Sitemaps Safe

Pro Sitemaps

Pro Sitemaps is a monitoring bot from PRO Sitemaps used to check website availability, performance, or page health.

Operator and family: PRO Sitemaps Type: Monitoring
Verified robots.txt: No Block: No Verified Bot
productsup.io/crawler Neutral

Productsup Crawler

Productsup crawls websites to import additional product data.

Operator: Productsup Family: Productsup Crawler Type: Search Purpose: Indexing
Verified robots.txt: No Block: Depends Verified Bot
Verispider Safe

Project Honeypot

Project Honeypot is a security bot associated with Unspam Technologies, Inc, used for security checks, verification, anti-abuse checks, or vulnerability scanning.

Operator: Unspam Technologies, Inc Family: Inc. Type: Security
Verified robots.txt: No Block: No Verified Bot
ProjectShield-UrlCheck Safe

ProjectShield Url Check

ProjectShield Url Check is a security scanner from Google used for security checks, malware review, or abuse prevention.

Operator and family: Google Type: Security
Verified robots.txt: No Block: No Verified Bot