Search
360Spider
360Spider is a Qihoo 360 search crawler used for discovering and indexing public web content for 360 Search.
Search
Accoona-AI-Agent
Accoona-AI-Agent is a verified historical crawler identity retained for legacy server-log matching.
Search
Amzn-SearchBot
Amazon search crawler for Amazon search experiences, not generative-AI training.
Search
Applebot
Apple web crawler for Apple search experiences; crawled data may also support Apple foundation-model training.
Search
askFinz-Crawler
askFinz-Crawler is a askFinz search crawler used for general public-web and partner-site reading for the askFinz search index.
Search
askFinz-Maps
askFinz-Maps is a askFinz geographic crawler used for reading place and geographic reference pages.
Search
askFinz-News
askFinz-News is a askFinz news crawler used for reading news and article pages for askFinz.
Search
AspiegelBot
Legacy Aspiegel/Huawei crawler identity associated with the PetalBot search-crawler family.
Search
Baiduspider
Baiduspider is Baidu's web-search crawler family used to discover and index public web content.
Search
BingBot
Bingbot is Microsoft's primary crawler for discovering, rendering, and indexing web content for Bing Search.
Search
Cliqzbot
Cliqzbot is a crawler associated with Cliqz and used for search indexing; candidate traffic can include `Cliqzbot`
Search
cognitiveSEO Crawler
cognitiveSEO Crawler is an automated agent from cognitive SEO Internet Marketing Tools. used for seo link content analysis.
Search
ConvermaxBot
ConvermaxBot is an automated agent from Convermax Corp. used for site search indexing.
Search
CopyleaksBot
CopyleaksBot is a Copyleaks search crawler used for discovering and crawling publicly available webpages for Copyleaks search services.
Search
Cotoyogi
Cotoyogi is an automated agent from Research Organization of Information and Systems used for ai training.
Search
CriteoBot
CriteoBot is an automated agent from Criteo used for contextual advertising analysis.