Yext Inc
The Yext Crawler provides Yext customers with a tool to retrieve data from their own websites.
YextBotUser-agent strings can be copied. Use this record for identification, logging, filtering, and review. For allow lists or low-friction rules, pair the identifier with operator documentation, reverse DNS, IP ranges, request signatures, or another verification method when available.
Identification
User-Agent: YextBot
HTTP Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/87.0.4280.88 YextBot/Java Safari/537.36
Robots Token: YextBot
Purpose: indexing
Identity: verified-bot
Handling
Common Use: Search indexing, content discovery, rendering, or search-result freshness checks.
Detection Notes: Match `YextBot` as a case-insensitive substring in HTTP user-agent logs. Use bot_http_agent for full user-agent examples when the client sends a longer browser-like string. Do not treat a user-agent match alone as proof of identity for allow-listing.
Risk: neutral
Should Block: depends
Respects Robots: no
Rules
Block Snippet
User-agent: YextBot Disallow: /