YandexMobileBot
Yandex crawler for mobile-layout checks.
Yandex crawler that downloads pages to check accessibility for users.
User-agent: YandexAccessibilityBot
Disallow: /
(http.user_agent contains "YandexAccessibilityBot")
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} YandexAccessibilityBot [NC]
RewriteRule ^ - [F,L]
if ($http_user_agent ~* "YandexAccessibilityBot") { return 403; }
YandexAccessibilityBot is the Yandex robot that downloads pages to check their accessibility for users. Yandex says it can ignore the general Yandex Webmaster setting and does not take general robots.txt rules into account.
https://yandex.com/support/webmaster/en/robot-workings/check-yandex-robots
Yandex crawler for mobile-layout checks.
Yandex crawler for image indexing.
Yandex crawler for video indexing.
Yandex Calendar fetcher for user-requested calendar files.
Yandex's primary search indexing robot.
Helps process robots.txt to prevent page content from appearing in Yandex AI responses; does not make indexing requests.
Indexes content for search in languages other than Russian.
Helps process robots.txt to prevent page content from appearing in Yandex AI responses; does not make indexing requests.
FishBot crawls webpages to deliver Open Source AI for All
The Yext Crawler provides Yext customers with a tool to retrieve data from their own websites.
A content based scraper only for partners we collaborate with who have given permission to have their website scraped.
Yahoo Mail Proxy is a content fetch proxy that retrieves the page content of URLs that are embedded within emails sent to Yahoo Mail users.
