Bot Directory: Search Bots, Crawlers and User Agents

Need live enforcement?

Classify traffic and apply rules before content is served.

Try Edge Free Open Edge BotDB

998 bots

View by

Playpatrol Neutral

Playpatrol

Playpatrol is a testing from CreateIT used for markup validation, accessibility testing, structured-data review.

Operator: CreateIT Family: Playpatrol Type: Testing Purpose: Automated Testing

Verified robots.txt: Unknown Block: Depends Documented

View Record Open in Edge BotDB

Pocket Casts Feed Parser Safe

Pocket Casts Feed Parser

Pocket Casts Feed Parser is a feed fetcher from Pocket Casts used for RSS or Atom feed polling, syndication refresh, subscription delivery.

Operator and family: Pocket Casts Type: Feed Purpose: Feed Fetch

Verified robots.txt: No Block: Depends Verified Bot

View Record Open in Edge BotDB

PocketParser Caution

PocketParser

PocketParser is a web scraper from Pocket used for public web data collection, page extraction, content monitoring.

Operator and family: Pocket Type: Content Crawler Purpose: Content Discovery

Verified robots.txt: Unknown Block: Monitor Documented

View Record Open in Edge BotDB

Poggio-Citations Neutral

Poggio Citations

Poggio Citations is an AI search crawler from Poggio used for AI search indexing, answer retrieval, citation discovery.

Operator and family: Poggio Type: Ai Purpose: Ai Search

Not verified robots.txt: Unknown Block: Depends Observed

View Record Open in Edge BotDB

polar.sh webhooks Safe

Polar

Polar is a webhook callback used for webhook delivery, payment notifications, service callbacks.

Operator and family: Polar Type: Webhook

Verified robots.txt: Unknown Block: No Verified Bot

View Record Open in Edge BotDB

Poseidon Research Crawler Neutral

Poseidon Research Crawler

Poseidon Research Crawler is an AI training crawler from Poseidon Research used for AI model training, dataset discovery.

Operator and family: Poseidon Research Type: Ai Purpose: Ai Training

Not verified robots.txt: Unknown Block: Depends Observed

View Record Open in Edge BotDB

PostmanRuntime Caution

PostmanRuntime

PostmanRuntime is an HTTP Client from Postman used for PostmanRuntime is used for Automated website access, content retrieval, or integration activity associated with the listed operator.

Operator and family: Postman Type: Http Client Purpose: Automated Http Requests

Verified robots.txt: Unknown Block: Monitor Documented

View Record Open in Edge BotDB

Potions Safe

Potions

Potions is an ecommerce crawler used for product discovery, catalog checks, marketplace crawling, price monitoring.

Operator and family: Potions Type: Ecommerce

Verified robots.txt: Unknown Block: Depends Verified Bot

View Record Open in Edge BotDB

Prerender Neutral

prerender

prerender is a web scraper from Prerender, LLC used for public web data collection, page extraction, content monitoring.

Operator: Prerender, LLC Family: LLC. Type: Scraper Purpose: Scraping

Verified robots.txt: Unknown Block: Depends Verified Bot

View Record Open in Edge BotDB

PressEngineBot Neutral

PressEngine Bot

PressEngine Bot is a security scanner from PressEngine used for security scanning, malware checks, vulnerability assessment, certificate review.

Operator and family: PressEngine Type: Security

Verified robots.txt: Yes Block: Depends Verified Bot

View Record Open in Edge BotDB

PrintFriendly.com Safe

PrintFriendly

PrintFriendly is a link preview bot from PrintFriendly.com used for link unfurling, rich-card generation, social previews, messaging previews.

Operator and family: PrintFriendly.com Type: Preview

Verified robots.txt: Unknown Block: No Verified Bot

View Record Open in Edge BotDB

Pro-Sitemaps Safe

Pro Sitemaps

Pro Sitemaps is a monitoring bot used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: PRO Sitemaps Type: Monitoring

Verified robots.txt: Unknown Block: No Verified Bot

View Record Open in Edge BotDB

ProbelySPDR Neutral

Probely

Probely is a security scanner used for security scanning, malware checks, vulnerability assessment, certificate review.

Operator and family: Probely Type: Security Purpose: Security Scanning

Not verified robots.txt: Unknown Block: Depends Documented

View Record Open in Edge BotDB

productsup.io/crawler Neutral

Productsup Crawler

Productsup Crawler is a search crawler from Productsup used for search indexing, URL discovery, page rendering, result freshness.

Operator: Productsup Family: Productsup Crawler Type: Search Purpose: Indexing

Verified robots.txt: Unknown Block: Depends Verified Bot

View Record Open in Edge BotDB

Verispider Safe

Project Honeypot

Project Honeypot is a security scanner from Unspam Technologies, Inc used for security scanning, malware checks, vulnerability assessment, certificate review.

Operator: Unspam Technologies, Inc Family: Inc. Type: Security

Verified robots.txt: No Block: No Verified Bot

View Record Open in Edge BotDB

ProjectShield-UrlCheck Safe

ProjectShield Url Check

ProjectShield Url Check is a monitoring bot from Google used for security scanning, malware checks, vulnerability assessment, certificate review.

Operator and family: Google Type: Security

Verified robots.txt: Yes Block: No Verified Bot

View Record Open in Edge BotDB

Protopage Safe

Protopage

Protopage is a feed fetcher from Protopage Ltd used for RSS or Atom feed polling, syndication refresh, subscription delivery.

Operator and family: Protopage Ltd Type: Feed Purpose: Feed Fetch

Verified robots.txt: No Block: Depends Verified Bot

View Record Open in Edge BotDB

compatible; proximic Neutral

Proximic

Proximic is a search crawler from ComScore used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: ComScore Type: Search Purpose: Indexing

Verified robots.txt: Yes Block: Depends Verified Bot

View Record Open in Edge BotDB

PRTG Network Monitor Neutral

PRTG Network Monitor

PRTG Network Monitor is a monitoring bot from Paessler used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: Paessler Type: Monitoring Bot Purpose: Uptime And Performance Monitoring

Verified robots.txt: Unknown Block: Allow Documented

View Record Open in Edge BotDB

pulsetic.com Safe

Pulsetic

Pulsetic is a monitoring bot used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: Pulsetic Type: Monitoring

Verified robots.txt: No Block: No Verified Bot

View Record Open in Edge BotDB

PWABuilderHttpAgent Neutral

PWABuilder

PWABuilder is an automated bot from Microsoft used for PWABuilder is used for Automated traffic classification requires log review, user-agent comparison.

Operator and family: Microsoft

Verified robots.txt: Yes Block: Depends Verified Bot

View Record Open in Edge BotDB

qatech Safe