Perplexity-User
User-triggered Perplexity fetcher for live answers.
Perplexity search crawler used to surface and link websites in Perplexity search results.
User-agent: PerplexityBot
Disallow: /
(http.user_agent contains "PerplexityBot")
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} PerplexityBot [NC]
RewriteRule ^ - [F,L]
if ($http_user_agent ~* "PerplexityBot") { return 403; }
PerplexityBot is a search crawler. Perplexity says it is not used to crawl content for AI foundation model training and recommends allowing it in robots.txt if you want your site to appear in Perplexity search results.
Perplexity publishes IP ranges for PerplexityBot.
https://docs.perplexity.ai/docs/resources/perplexity-crawlers
User-triggered Perplexity fetcher for live answers.
coccocbot-fast is the HTTP request user-agent Cloudflare Radar shows for the verified Cốc Cốc crawler.
The YaCy project documents yacybot as the user-agent used by YaCy web crawlers.
You.com Search Engine Crawler
Zoominfobot is an indexing robot for a web search engine, similar to Google. Created by Zoom Information Inc.(www.zoominfo.com), Zoominfobot’s patented techn...
