ProjectShield Url Check
Project Shield, created by Google Cloud and Jigsaw and powered by Google Cloud Armor, provides free unlimited protection against DDoS attacks, a type of digital attackā¦
Verifies that Business Profile links are reachable, relevant, and return valid responses.
User-agent: Google-BusinessLinkVerification
Disallow: /
(http.user_agent contains "Google-BusinessLinkVerification")
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} Google\-BusinessLinkVerification [NC]
RewriteRule ^ - [F,L]
if ($http_user_agent ~* "Google-BusinessLinkVerification") { return 403; }
Known user-agent patterns: Google-BusinessLinkVerification Known user-agent strings: Google-BusinessLinkVerification Robots.txt handling in the directory: no.
Operator documentation: https://support.google.com/business/answer/13769188?hl=en
Project Shield, created by Google Cloud and Jigsaw and powered by Google Cloud Armor, provides free unlimited protection against DDoS attacks, a type of digital attackā¦
Google Messages link-preview fetcher.
Generic Google crawler for public-content fetches outside core Search.
GoogleOther-Image is GoogleOther optimized for fetching publicly accessible image URLs.
GoogleOther-Video is GoogleOther optimized for fetching publicly accessible video URLs.
Google AdSense uses a crawler called Google-Display-Ads-Bot to verify your site when you add a site to AdSense.
Google-CWS is the Chrome Web Store user-triggered fetcher for URLs developers provide in extension and theme metadata.
Google-hosted browsing agent triggered by user requests.
User-authorized website monitoring, uptime checks, diagnostics, and compliance scanning for managed sites.
Research web crawler that indexes publicly available documents. Respects robots.txt and rate limits.
Spyglasses Bot evaluates website content to identify issues that would affect visibility by AI assistants.
AI assistant for e-commerce stores. Only crawls sites upon request from the site owner.
