Bot Directory

Search the Botcrawl Bot Directory database to identify AI crawlers, search bots, scrapers, scanners, monitoring bots, preview bots, and user-agent patterns.

660 bots
GoogleOther-Video Neutral

GoogleOther-Video

GoogleOther-Video is an AI crawler from Google used for AI-related crawling, retrieval, search, or assistant activity.

Operator and family: Google Type: Ai Purpose: Ai Training
Verified robots.txt: Yes Block: Depends Official Documented
GoPay Safe

GoPay

GoPay is a commerce-related fetcher from GoPay.cz used for commerce, checkout, advertising, or payment-related checks.

Operator and family: GoPay.cz Type: Ecommerce
Verified robots.txt: No Block: No Verified Bot
GPT-Actions Neutral

GPT-Actions

GPT-Actions is an AI agent from OpenAI used for AI-related crawling, retrieval, search, or assistant activity.

Operator and family: OpenAI Type: Ai Purpose: Ai Assistant
Verified robots.txt: Unknown Block: Depends Verified Bot
GPTBot Neutral

GPTBot

GPTBot is OpenAI’s web crawler used to discover publicly available content that may support generative AI systems.

Operator and family: OpenAI Type: Ai Purpose: Ai Training
Verified robots.txt: Yes Block: Depends Official Documented
grafana/synthetic-monitoring Safe

Grafana’s Synthetic Monitoring

Grafana's Synthetic Monitoring agent provides probe functionality and executes network checks for monitoring remote targets.

Operator and family: Grafana Labs Type: Monitoring
Verified robots.txt: No Block: No Verified Bot
GrapeshotCrawler Neutral

Grapeshot

The Grapeshot bot, part of Oracle Advertising, crawles web pages for content analysis and classification.

Operator and family: Grapeshot Type: Search Purpose: Indexing
Verified robots.txt: No Block: Depends Verified Bot
Grok-DeepSearch Neutral

Grok-DeepSearch

Claimed Grok DeepSearch user-agent token for xAI search/research workflows.

Operator: xAI Family: xAI Grok Type: Ai Purpose: AI Search Or Deep Research
Not verified robots.txt: Unknown Block: Depends Observed
GrokBot Neutral

GrokBot

Claimed xAI/Grok bot identifier used in third-party AI crawler lists; sightings may be inconsistent.

Operator: xAI Family: xAI Grok Type: Ai Purpose: AI Search Or Agentic Browsing
Not verified robots.txt: Unknown Block: Depends Observed
GroovinaAdsbot Neutral

Groovinads

The Groovinads bot gathers data from e-commerce websites to support its ad services.

Operator and family: Groovinads Type: Ecommerce
Verified robots.txt: No Block: Depends Verified Bot
GTmetrix Safe

GTmetrix

GTmetrix is a free tool that analyzes a page's speed performance. Using PageSpeed and YSlow, GTmetrix generates scores for pages and offers actionable recommendations on how to fix them.

Operator and family: GTmetrix Type: Monitoring
Verified robots.txt: No Block: No Verified Bot
GuestpostsBot Safe

GuestpostsBot

GuestpostsBot is a Web Crawler that has several functions to facilitate the website owner who has registered his site on the guestposts.com.br platform to monitor his site. The bot.

Operator and family: Guest Posts Type: Monitoring
Verified robots.txt: No Block: No Verified Bot
Haiku-SearchBot Neutral

Haiku-SearchBot

Haiku-SearchBot is for user actions in Haiku. When users ask Haiku a question, it may visit a web page to help answer.

Operator: CLERK SAS Family: CLERK Type: Search Purpose: Indexing
Verified robots.txt: No Block: Depends Verified Bot
Hatena Safe

Hatena

Hatena's service automatically visits web pages to get the information it needs to use in the service.

Operator and family: Hatena Type: Search Purpose: Indexing
Verified robots.txt: Yes Block: Depends Verified Bot
HelloworkJobPostingBot Neutral

HelloWork

HelloWork is a French job board, and its bot aggregates job listings for its platform. It crawls company career pages and other sources to collect this information.

Operator and family: HelloWork Group Type: Search Purpose: Indexing
Verified robots.txt: No Block: Depends Verified Bot
HetrixTools Safe

HetrixTools

HetrixTools offers an Uptime Monitoring service, where our monitoring locations (bots) will check whether our users' websites are online or not. Similar to more known services such as.

Operator and family: HetrixTools Inc Type: Monitoring
Verified robots.txt: No Block: No Verified Bot
hey.com/imageproxy Neutral

HEY Email Privacy Proxy

HEY email stops spy pixels and prevents user IP tracking by proxying all HTML email images, fonts, and external assets.

Operator: HEY Family: HEY Email Privacy Proxy Type: Scraper Purpose: Scraping
Verified robots.txt: No Block: Depends Verified Bot
HIFIBot Neutral

HIFIBot

HIFIBot is an automated bot from HIFI.

Operator and family: HIFI
Verified robots.txt: No Block: Depends Verified Bot
Honeybadger Uptime Check Safe

honeybadgerbot

HoneybadgerBot is the bot used by the Honeybadger error and uptime monitoring service.

Operator and family: Honeybadger Industries Type: Monitoring
Verified robots.txt: No Block: No Verified Bot
hookdeck Safe

Hookdeck

Hookdeck is a webhook callback service from Hookdeck used for service notifications, callbacks, or integration events.

Operator and family: Hookdeck Type: Webhook
Verified robots.txt: Unknown Block: Depends Verified Bot
HostTracker Safe

HostTracker

The HostTracker monitor tracks website availability and performance for their customers.

Operator and family: HostTracker Type: Monitoring
Verified robots.txt: No Block: No Verified Bot
Hotjar Safe

Hotjar

Hotjar provides user analytics and feedback for website owners.

Operator and family: Hotjar Type: Feed Purpose: Feed Fetch
Verified robots.txt: No Block: No Verified Bot
hstspreload-bot Neutral

HSTS preload bot

Checks that links can be preloaded for Chromium.

Operator and family: Chromium Type: Security
Verified robots.txt: No Block: Depends Verified Bot
HubSpot Crawler Neutral

HubSpot Crawler

HubSpot offers a full platform of marketing, sales, customer service, and CRM software — plus the methodology, resources, and support — to help businesses grow better. Get started with free.

Operator and family: Hubspot Type: Search Purpose: Indexing
Verified robots.txt: No Block: Depends Verified Bot
HubSpot-FeedFetcher Safe

HubSpot Feed Fetcher

When posting to LinkedIn from Hubspot, images need to be pulled through to LinkedIn when published. The crawler performs this function.

Operator and family: HubSpot Type: Feed Purpose: Feed Fetch
Verified robots.txt: No Block: Depends Verified Bot