Bot Directory

Search the Botcrawl Bot Directory database to identify AI crawlers, search bots, scrapers, scanners, monitoring bots, preview bots, and user-agent patterns.

660 bots
YandexFavicons Safe

YandexFavicons

YandexFavicons is a preview or prefetch bot from Yandex used to fetch pages for previews, unfurls, or user-triggered rendering.

Operator and family: Yandex Type: Preview
Verified robots.txt: No Block: Depends Official Documented
YandexImageResizer Safe

YandexImageResizer

YandexImageResizer is a search crawler from Yandex used for search indexing, page discovery, or content rendering.

Operator and family: Yandex Type: Search Purpose: Indexing
Verified robots.txt: Yes Block: Depends Official Documented
YandexImages Safe

YandexImages

YandexImages is a search crawler from Yandex used for search indexing, page discovery, or content rendering.

Operator and family: Yandex Type: Search Purpose: Indexing
Verified robots.txt: Yes Block: Depends Official Documented
YandexMarket Safe

YandexMarket

YandexMarket is a search crawler from Yandex used for search indexing, page discovery, or content rendering.

Operator and family: Yandex Type: Search Purpose: Indexing
Verified robots.txt: Yes Block: Depends Official Documented
YandexMedia Safe

YandexMedia

YandexMedia is a search crawler from Yandex used for search indexing, page discovery, or content rendering.

Operator and family: Yandex Type: Search Purpose: Indexing
Verified robots.txt: Yes Block: Depends Official Documented
YandexMetrika Safe

YandexMetrika

YandexMetrika is a monitoring bot from Yandex used to check website availability, performance, or page health.

Operator and family: Yandex Type: Monitoring
Verified robots.txt: No Block: Depends Official Documented
YandexMobileBot Safe

YandexMobileBot

YandexMobileBot is a search crawler from Yandex used for search indexing, page discovery, or content rendering.

Operator and family: Yandex Type: Search Purpose: Indexing
Verified robots.txt: No Block: Depends Official Documented
YandexMobileScreenShotBot Safe

YandexMobileScreenShotBot

YandexMobileScreenShotBot is a preview or prefetch bot from Yandex used to fetch pages for previews, unfurls, or user-triggered rendering.

Operator and family: Yandex Type: Preview
Verified robots.txt: No Block: Depends Official Documented
YandexOntoDB Safe

YandexOntoDB

YandexOntoDB is a search crawler from Yandex used for search indexing, page discovery, or content rendering.

Operator and family: Yandex Type: Search Purpose: Indexing
Verified robots.txt: Yes Block: Depends Official Documented
YandexOntoDBAPI Safe

YandexOntoDBAPI

YandexOntoDBAPI is a search crawler from Yandex used for search indexing, page discovery, or content rendering.

Operator and family: Yandex Type: Search Purpose: Indexing
Verified robots.txt: No Block: Depends Official Documented
YandexPagechecker Safe

YandexPagechecker

YandexPagechecker is a search crawler from Yandex used for search indexing, page discovery, or content rendering.

Operator and family: Yandex Type: Search Purpose: Indexing
Verified robots.txt: Yes Block: Depends Official Documented
YandexPartner Safe

YandexPartner

YandexPartner is a search crawler from Yandex used for search indexing, page discovery, or content rendering.

Operator and family: Yandex Type: Search Purpose: Indexing
Verified robots.txt: No Block: Depends Official Documented
YandexRCA Safe

YandexRCA

YandexRCA is a search crawler from Yandex used for search indexing, page discovery, or content rendering.

Operator and family: Yandex Type: Search Purpose: Indexing
Verified robots.txt: No Block: Depends Official Documented
YandexRenderResourcesBot Safe

YandexRenderResourcesBot

YandexRenderResourcesBot is a search crawler from Yandex used for search indexing, page discovery, or content rendering.

Operator and family: Yandex Type: Search Purpose: Indexing
Verified robots.txt: No Block: Depends Official Documented
YandexScreenshotBot Safe

YandexScreenshotBot

YandexScreenshotBot is a preview or prefetch bot from Yandex used to fetch pages for previews, unfurls, or user-triggered rendering.

Operator and family: Yandex Type: Preview
Verified robots.txt: No Block: Depends Official Documented
YandexSearchShop Safe

YandexSearchShop

YandexSearchShop is a search crawler from Yandex used for search indexing, page discovery, or content rendering.

Operator and family: Yandex Type: Search
Verified robots.txt: No Block: Depends Official Documented
YandexSitelinks Safe

YandexSitelinks

YandexSitelinks is a monitoring bot from Yandex used to check website availability, performance, or page health.

Operator and family: Yandex Type: Monitoring
Verified robots.txt: Yes Block: Depends Official Documented
YandexSpravBot Safe

YandexSpravBot

YandexSpravBot is a search crawler from Yandex used for search indexing, page discovery, or content rendering.

Operator and family: Yandex Type: Search Purpose: Indexing
Verified robots.txt: Yes Block: Depends Official Documented
YandexTracker Safe

YandexTracker

YandexTracker is a monitoring bot from Yandex used to check website availability, performance, or page health.

Operator and family: Yandex Type: Monitoring
Verified robots.txt: No Block: Depends Official Documented
YandexUserproxy Safe

YandexUserproxy

YandexUserproxy is a search crawler from Yandex used for search indexing, page discovery, or content rendering.

Operator and family: Yandex Type: Search
Verified robots.txt: No Block: Depends Official Documented
YandexVerticals Safe

YandexVerticals

YandexVerticals is a search crawler from Yandex used for search indexing, page discovery, or content rendering.

Operator and family: Yandex Type: Search Purpose: Indexing
Verified robots.txt: Yes Block: Depends Official Documented
YandexVertis Safe

YandexVertis

YandexVertis is a search crawler from Yandex used for search indexing, page discovery, or content rendering.

Operator and family: Yandex Type: Search Purpose: Indexing
Verified robots.txt: Yes Block: Depends Official Documented
YandexVideo Safe

YandexVideo

YandexVideo is a search crawler from Yandex used for search indexing, page discovery, or content rendering.

Operator and family: Yandex Type: Search Purpose: Indexing
Verified robots.txt: Yes Block: Depends Official Documented
YandexVideoParser Safe

YandexVideoParser

YandexVideoParser is a search crawler from Yandex used for search indexing, page discovery, or content rendering.

Operator and family: Yandex Type: Search Purpose: Indexing
Verified robots.txt: No Block: Depends Official Documented