Bot Directory

Protopage Safe

Protopage

Protopage.com indexes RSS news headlines mostly from news sites.

Operator and family: Protopage Ltd Type: Feed Purpose: Feed Fetch

Verified robots.txt: No Block: Depends Verified Bot

compatible; proximic Neutral

Proximic

The Proximic crawler visites websites serving ads on behalf of them or their partners to determine which ads best fit a given website's content.

Operator and family: ComScore Type: Search Purpose: Indexing

Verified robots.txt: Yes Block: Depends Verified Bot

pulsetic.com Safe

Pulsetic

Website monitoring service. Uptime monitoring and down alerts.

Operator and family: Pulsetic Type: Monitoring

Verified robots.txt: No Block: No Verified Bot

PWABuilderHttpAgent Neutral

PWABuilder

PWABuilder (pwabuilder.com) is a free, open source developer tool from Microsoft that helps developers build progressive web apps and publish them in app stores. PWABuilder tool analyzes.

Operator and family: Microsoft

Verified robots.txt: Yes Block: Depends Verified Bot

qatech Safe

QA.tech

QA.tech is a monitoring bot from QA.tech used to check website availability, performance, or page health.

Operator and family: QA.tech Type: Monitoring

Verified robots.txt: Unknown Block: Depends Verified Bot

qstash Safe

QStash

QStash is a platform for building event-driven applications.

Operator and family: Upstash Type: Webhook

Verified robots.txt: Unknown Block: Depends Verified Bot

QualifiedBot Neutral

QualifiedBot

Bot crawls customer websites to provide information to customer hosted chatbots.

Operator: Qualified.com, Inc. Family: Inc. Type: Search Purpose: Indexing

Verified robots.txt: No Block: Depends Verified Bot

cf-qualys-scanner Safe

Qualys

Qualys Web Application Scanner is a cloud-based service that provides automated crawling and testing of custom web applications to identify vulnerabilities including cross-site scripting.

Operator and family: Qualys Type: Security

Verified robots.txt: No Block: No Verified Bot

SSL Labs Safe

Qualys SSL Scanner

SSL Labs / Qualys is used to test monitor ssl rating against their site.

Operator and family: Qualys Type: Monitoring

Verified robots.txt: No Block: No Verified Bot

Quantcastbot Neutral

Quantcastbot

Quantcast Bot is the name of a web crawler used by Quantcast for advertisement quality assurance and to understand page content for Interest-Based Audiences.

Operator and family: Quantcast Type: Search Purpose: Indexing

Verified robots.txt: No Block: Depends Verified Bot

Quartr Crawler Neutral

Quartr Crawler

Quartr uses a crawler to obtain and deliver investor relations material.

Operator: Quartr Family: Quartr Crawler Type: Search Purpose: Indexing

Verified robots.txt: No Block: Depends Verified Bot

Qwantbot Safe

Qwantbot

Qwantbot is Qwant’s search crawler, used to support Qwant search indexing and related Qwant bot variants.

Operator and family: Qwant Type: Search Purpose: Search Indexing

Verified robots.txt: Yes Block: Depends Verified Bot

Qwantbot-news Safe

Qwantbot-news

Qwantbot-news is a search crawler from Qwant used for search indexing, page discovery, or content rendering.

Operator and family: Qwant Type: Search

Verified robots.txt: Yes Block: Depends Official Documented

Rackspace Monitoring Safe

Rackspace

The Rackspace HTTP Monitor monitors customer websites for uptime and other issues.

Operator and family: Rackspace Type: Monitoring

Verified robots.txt: No Block: No Verified Bot

rakutenusabot-image Neutral

Rakuten Image extraction bot

Rakuten uses this bot to crawl product images so that we can display cashback deals for our merchants.

Operator and family: Rakuten Type: Scraper Purpose: Scraping

Verified robots.txt: No Block: Depends Verified Bot

Razorpay Safe

Razorpay Webhook

Razorpay sends webhooks back to ecommerce sites.

Operator and family: Razorpay Type: Webhook

Verified robots.txt: No Block: No Verified Bot

Readable Safe

Readable

Readable is a collection of text analysis tools, primarily focused on clarity and plain language. We spider customers' websites, find the content of each page, analyse it, and present that.

Operator and family: Added Bytes Ltd Type: Monitoring

Verified robots.txt: No Block: No Verified Bot

Recurly Webhook Safe

Recurly Webhooks

Recurly Webhooks is a webhook callback service from Recurly used for service notifications, callbacks, or integration events.

Operator and family: Recurly Type: Webhook

Verified robots.txt: Yes Block: Depends Verified Bot

RedactusWebClassifier Neutral

Redactus Web Classifier

Web classifier: fetches a site’s homepage to assign a category and rechecks periodically.

Operator and family: Redactus Type: Security

Verified robots.txt: No Block: Depends Verified Bot

redirect-pizza Safe

Redirect Pizza Destination Monitor

Redirect Pizza Destination Monitor is a monitoring bot from redirect.pizza used to check website availability, performance, or page health.

Operator and family: redirect.pizza Type: Monitoring

Verified robots.txt: Unknown Block: Depends Verified Bot

Reelevant Neutral

Reelevant

Reelavant allows customers to dynamically update content inside their emails. Needs to fetch images of their product at runtime and send them back to the users.

Operator and family: Reelevant Type: Scraper Purpose: Scraping

Verified robots.txt: No Block: Depends Verified Bot

Retool Neutral

Retool

Retool is a commerce-related fetcher from Retool used for commerce, checkout, advertising, or payment-related checks.

Operator and family: Retool Type: Ecommerce

Verified robots.txt: No Block: Depends Verified Bot

RetroListeCOM Safe

RetroListeCOM

RetroListeCOM is a service that tracks user counts on gaming-related websites, and its bot visits those sites to collect this data.

Operator and family: Niclas Papst Type: Monitoring

Verified robots.txt: No Block: No Verified Bot

RevvimGort Neutral

Revvim

Our bot crawls our customers' websites to identify SEO opportunities.

Operator and family: Revvim Type: Search Purpose: Indexing

Verified robots.txt: No Block: Depends Verified Bot