ProjectShield Url Check
Project Shield, created by Google Cloud and Jigsaw and powered by Google Cloud Armor, provides free unlimited protection against DDoS attacks, a type of digital attackā¦
Google Smartphone crawler that indexes mobile web pages using a smartphone user agent.
User-agent: Googlebot-Smartphone
Disallow: /
(http.user_agent contains "Googlebot-Smartphone")
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} Googlebot\-Smartphone [NC]
RewriteRule ^ - [F,L]
if ($http_user_agent ~* "Googlebot-Smartphone") { return 403; }
Google Smartphone crawler that indexes mobile web pages using a smartphone user agent.
Googlebot-Smartphone is operated by Google. Botcrawl classifies it as a search bot with a primary purpose of user triggered.
Key profile signals: risk level: neutral; robots.txt: yes; verified: yes.
Verify the exact user-agent together with the published IP ranges at https://developers.google.com/static/search/apis/ipranges/googlebot.json.
Published IP range data is available for this bot at https://developers.google.com/static/search/apis/ipranges/googlebot.json.
Source documentation: https://developers.google.com/search/docs/crawling-indexing/googlebot
https://developers.google.com/search/docs/crawling-indexing/googlebot
Project Shield, created by Google Cloud and Jigsaw and powered by Google Cloud Armor, provides free unlimited protection against DDoS attacks, a type of digital attackā¦
GoogleOther-Image is GoogleOther optimized for fetching publicly accessible image URLs.
GoogleOther-Video is GoogleOther optimized for fetching publicly accessible video URLs.
Google Messages link-preview fetcher.
Generic Google crawler for public-content fetches outside core Search.
Google's main crawler for Search and related surfaces.
Google Pinpoint fetcher for user-supplied sources.
Google crawler used on owner request for Vertex AI Agents.
User-authorized website monitoring, uptime checks, diagnostics, and compliance scanning for managed sites.
Research web crawler that indexes publicly available documents. Respects robots.txt and rate limits.
Spyglasses Bot evaluates website content to identify issues that would affect visibility by AI assistants.
AI assistant for e-commerce stores. Only crawls sites upon request from the site owner.
