Search

1,298 Bot records
Results 125
About

About the Search

Search crawlers, AI agents, monitors, scrapers, preview bots, and other automated web clients.

Contribute
Search

kagi-fetcher

Kagi

kagi-fetcher is retained as an observed Kagi retrieval identity, but the exact token lacks sufficient current operator documentation for Verified status.

Search

KagiBot

Kagi

KagiBot crawls public pages for Kagi search indexing and follows robots.txt controls.

Search

Level9SearchBot

Level 9

Level9SearchBot is a versioned search crawler identity associated with Level 9.

Search

Linespider

LINE services

Linespider crawls public pages for search features across LINE services and follows the Robots Exclusion Protocol.

Search

LinkupBot

Linkup

Linkup.so search and AI-grounding crawler.

Search

LohiSoftBot

LohiSoft s.r.o.

LohiSoftBot is a search-index crawler with published source IPs and forward-confirmed reverse-DNS verification guidance.

Search

LyonlBot

Lyonl

LyonlBot identifies web crawling attributed to the Lyonl search service.

Search

LyonlBot-Image

Lyonl

LyonlBot-Image identifies image-oriented crawling attributed to Lyonl.

Search

LyonlBot-News

Lyonl

LyonlBot-News identifies news-oriented crawling attributed to Lyonl.

Search

MagnetmeBot

Magnet.me

MagnetmeBot is a search crawler from Magnet.me used for search indexing, URL discovery, page rendering, result freshness.

Search

Mars Finder

Mars Flag

Mars Finder enterprise site-search crawler.

Search

MetaJobBot

METAJob

MetaJobBot searches public employer sites for job vacancies to include in METAJob results.

Search

MojeekBot

Mojeek

MojeekBot is Mojeek's documented search crawler and follows the Robots Exclusion Standard.

Search

MRGbot

MRG Web Services srl

MRGbot is MRG Web Services' web crawler for its search/data corpus.

Search

MSN

Microsoft

MSN/msnbot is a historical Microsoft search crawler and is now retained as Verified Legacy rather than an active crawler.

Search

naver-blueno

NAVER

naver-blueno is retained as an Unverified Legacy token; NAVER's current documented preview robot is the separate `Blueno` identity.

Search

naverbot

NAVER

naverbot is retained as an Unverified Legacy token; NAVER's current documented search crawler is `Yeti`.

Search

Neevabot

Neeva

Neevabot is Neeva's former search crawler, retained as Verified Legacy after the Neeva search service shut down.

Search

NewsBank

NewsBank

NewsBank is a search crawler used for search indexing, URL discovery, page rendering, result freshness.

Search

NewsNow

NewsNowUK

NewsNow is a search crawler from NewsNowUK used for search indexing, URL discovery, page rendering, result freshness.

Search

Noorobot

Noor Digital Agency AB

Noorobot is a search crawler from Noor Digital Agency AB used for search indexing, URL discovery, page rendering, result freshness.

Search

NostoCrawlerBot

Nosto

NostoCrawlerBot is a search crawler from Nosto used for search indexing, URL discovery, page rendering, result freshness.