Google Chrome Web Store

Safe Block: Depends Verified robots.txt: No

Chrome Web Store fetcher that requests URLs developers provide in extension and theme metadata.

Fetches URLs referenced in Chrome Web Store extension or theme metadata.

Key facts

Source type: Unknown
Confidence: High
Last checked: 2026-04-03
Verification: Verify with Google’s published fetcher documentation and Google verification guidance.
Operator URL: https://chromewebstore.google.com/
Source URL: https://developers.google.com/crawling/docs/crawlers-fetchers/google-user-triggered-fetchers
Spoofing risk: User-agent strings can be spoofed. Verify with the operator’s published guidance before allow-listing.

Officially documented in Google’s user-triggered fetchers list.

User-agent: Google-CWS
Disallow: /

(http.user_agent contains "Google-CWS")

Advanced server rules

RewriteCond %{HTTP_USER_AGENT} Google-CWS [NC]
RewriteRule .* - [F,L]

if ($http_user_agent ~* "Google-CWS") { return 403; }

Google-CWS is the Chrome Web Store fetcher. It requests URLs developers supplied in the metadata of Chrome extensions and themes.

Verify with Google’s published fetcher documentation and Google verification guidance.
Officially documented in Google’s user-triggered fetchers list.
Fetches URLs referenced in Chrome Web Store extension or theme metadata.
User-agent strings can be spoofed. Verify with the operator’s published guidance before allow-listing.