Google desktop web crawler that indexes desktop versions of web pages.
User-agent: Googlebot-Desktop
Disallow: /(http.user_agent contains "Googlebot-Desktop")RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} Googlebot\-Desktop [NC]
RewriteRule ^ - [F,L]if ($http_user_agent ~* "Googlebot-Desktop") { return 403; }Google desktop web crawler that indexes desktop versions of web pages.
Googlebot-Desktop is operated by Google. Botcrawl classifies it as a search bot with a primary purpose of indexing.
Key profile signals: risk level: neutral; robots.txt: yes; verified: yes.
Verify the exact user-agent together with the published IP ranges at https://developers.google.com/static/search/apis/ipranges/googlebot.json.
Published IP range data is available for this bot at https://developers.google.com/static/search/apis/ipranges/googlebot.json.
Source documentation: https://developers.google.com/search/docs/crawling-indexing/googlebot
https://developers.google.com/search/docs/crawling-indexing/googlebot
Project Shield, created by Google Cloud and Jigsaw and powered by Google Cloud Armor, provides free unlimited protection against DDoS attacks, a type of digital attack…
Google Messages link-preview fetcher.
Generic Google crawler for public-content fetches outside core Search.
GoogleOther-Image is GoogleOther optimized for fetching publicly accessible image URLs.
GoogleOther-Video is GoogleOther optimized for fetching publicly accessible video URLs.
Google Pinpoint fetcher for user-supplied sources.
Google's main crawler for Search and related surfaces.
Google crawler used on owner request for Vertex AI Agents.
Zoominfobot is an indexing robot for a web search engine, similar to Google.
ZumBot is a web crawler that indexes webpages for Zum Open Internet Search.
You.com Search Engine Crawler
Yandex crawler for video indexing.