Yext Inc

Neutral Block: Depends Verified robots.txt: No

The Yext Crawler provides Yext customers with a tool to retrieve data from their own websites.

Key facts

Operator
Yext
Family
Yext
Purpose
General Crawl
User-Agent
YextBot
Should you block it?
Depends
Verified
Yes
Respects robots.txt
No
Identity type
Unknown
Confidence
Medium
Last verified
2026-04-01
Last checked
2026-04-01

Bot details

Identity

User-Agent
YextBot
robots.txt token
YextBot
HTTP user-agent
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/87.0.4280.88 YextBot/Java Safari/537.36

Ownership

Operator
Yext
Family
Yext
Type
Scraper
Purpose
General Crawl

Verification and trust

Source type
Unknown
Confidence
Medium
Last verified
2026-04-01
Last checked
2026-04-01
Verification
Validate the identifying user-agent or signature against the operator documentation before creating hard allow rules.
Operator URL
https://www.yext.com
Source URL
https://www.yext.com
Spoofing risk
User-agent strings can be spoofed. For allow-listing or low-friction rules, pair the published identifier with operator documentation or cryptographic verification when available.

Blocking and detection

Robots

User-agent: YextBot
Disallow: /

Cloudflare

(http.user_agent contains "YextBot")
Advanced server rules

Apache

RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} YextBot [NC]
RewriteRule ^ - [F,L]

Nginx

if ($http_user_agent ~* "YextBot") { return 403; }

Notes

Known user-agent patterns: YextBot Known user-agent strings: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/87.0.4280.88 YextBot/Java Safari/537.36 Robots.txt handling in the directory: no.

Operator documentation: https://www.yext.com

Evidence and source

  • Validate the identifying user-agent or signature against the operator documentation before creating hard allow rules.
  • The Yext Crawler provides Yext customers with a tool to retrieve data from their own websites.
  • User-agent strings can be spoofed. For allow-listing or low-friction rules, pair the published identifier with operator documentation or cryptographic verification when available.

https://www.yext.com

Related bots

Similar purpose: General Crawl

Confidence: Medium

FishBot

FishBot crawls webpages to deliver Open Source AI for All

FishBot FishBot Ai Crawler General Crawl FishBot
Verified robots.txt: No Neutral Block: Depends
Confidence: Medium

YGS Group Falconer Scraper

A content based scraper only for partners we collaborate with who have given permission to have their website scraped.

YGS Group YGS Group Falconer Scraper Ai Crawler General Crawl ygs-scraper-bot
Verified robots.txt: No Neutral Block: Depends
Official Confidence: High Official Documented

YandexMobileBot

Yandex crawler for mobile-layout checks.

Yandex Yandex Search Crawler General Crawl YandexMobileBot
Verified robots.txt: No Safe Block: Depends
Confidence: Medium

YahooMailProxy

Yahoo Mail Proxy is a content fetch proxy that retrieves the page content of URLs that are embedded within emails sent to Yahoo Mail users.

Yahoo Yahoo Scraper Preview General Crawl YahooMailProxy
Verified robots.txt: No Neutral Block: Depends