YandexVideo
Yandex crawler for video indexing.
Yandex image crawler used for Yandex Images indexing.
User-agent: YandexImages
Disallow: /
(http.user_agent contains "YandexImages")
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} YandexImages [NC]
RewriteRule ^ - [F,L]
if ($http_user_agent ~* "YandexImages") { return 403; }
YandexImages is the Yandex crawler that indexes images for Yandex Images. Yandex documents its user agent and says it takes general robots.txt rules into account.
https://yandex.com/support/webmaster/en/robot-workings/check-yandex-robots
Yandex crawler for video indexing.
Yandex crawler for mobile-layout checks.
Yandex's primary search indexing robot.
Yandex crawler for accessibility checks.
Yandex Calendar fetcher for user-requested calendar files.
Helps process robots.txt to prevent page content from appearing in Yandex AI responses; does not make indexing requests.
Indexes content for search in languages other than Russian.
Helps process robots.txt to prevent page content from appearing in Yandex AI responses; does not make indexing requests.
ZumBot is a web crawler that indexes webpages for Zum Open Internet Search.
Zoominfobot is an indexing robot for a web search engine, similar to Google.
You.com Search Engine Crawler
Yeti is the web crawler for Naver, a South Korean search engine.
