YandexImages
Yandex crawler for image indexing.
Yandex's main indexing robot for search.
User-agent: YandexBot
Disallow: /
(http.user_agent contains "YandexBot")
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} YandexBot [NC]
RewriteRule ^ - [F,L]
if ($http_user_agent ~* "YandexBot") { return 403; }
YandexBot is Yandex’s main indexing robot. Yandex documents its user agent and says it takes general robots.txt rules into account. Yandex recommends reverse and forward DNS checks to verify authenticity.
https://yandex.com/support/webmaster/en/robot-workings/check-yandex-robots
Yandex crawler for image indexing.
Yandex crawler for video indexing.
Yandex crawler for mobile-layout checks.
Yandex Calendar fetcher for user-requested calendar files.
Yandex crawler for accessibility checks.
Helps process robots.txt to prevent page content from appearing in Yandex AI responses; does not make indexing requests.
Indexes content for search in languages other than Russian.
Helps process robots.txt to prevent page content from appearing in Yandex AI responses; does not make indexing requests.
Zoominfobot is an indexing robot for a web search engine, similar to Google.
ZumBot is a web crawler that indexes webpages for Zum Open Internet Search.
You.com Search Engine Crawler
Yeti is the web crawler for Naver, a South Korean search engine.
