Bot intelligence record

Google-Extended

Usually allow

Use the Google-Extended identifier to separate Google AI crawler, assistant, or AI search traffic from normal visitor requests in server logs.

Ai Ai Assistant Official Documented Confidence: High Verified: Yes robots.txt: Yes
Operator
Google
Family
Google
Type
Ai
Source type
Official
Last checked
2026-05-20

User-Agent Pattern

Google
Google-Extended
Verification note

User-agent strings are identification signals, not proof of identity. Confirm important allow, block, or rate-limit decisions with logs, DNS or IP evidence, request behavior, or operator documentation when available.

Robots.txt Snippet

Click snippet to copy 
User-agent: Google-Extended Disallow: /

Click the snippet to copy it, or highlight the text manually.

Handling Guidance

Depends

This bot is usually safe to allow when the request source is verified and the traffic matches your site policy.

AI crawling, assistant retrieval, AI search, or model-supporting discovery depending on the user-agent.

Record Details

Structured data
Operator
Google
Family
Google
Type
Ai
Purpose
Ai Assistant
Identity type
Official Documented
Confidence
High
Last verified
2026-04-01
Last checked
2026-05-20
Source type
Official
Verification
Control token only; it relies on existing Google crawler traffic rather than a separate HTTP user-agent.
Spoofing risk
User-agent strings for Google-Extended can be spoofed. Treat user-agent detection as a classification signal, then verify with published IP ranges, reverse DNS, signatures, operator documentation, or published operator documentation, IP ranges, reverse DNS, signatures, or other verified identity signals before allow-listing.

Notes

Google-Extended is listed in the Botcrawl directory as an AI control token from Google. The primary identifier for log review is Google-Extended.

Identification

  • User-agent pattern: Google-Extended
  • Family: Google
  • Type: AI
  • Kind: Control Token

Common use

AI crawling, assistant retrieval, AI search, or model-supporting discovery depending on the user-agent.

Verification and handling

Control token only; it relies on existing Google crawler traffic rather than a separate HTTP user-agent.

Directory guidance marks the risk level as Safe and the blocking decision as Depends. Do not rely on the user-agent string alone because user-agent strings can be copied or spoofed.

Robots.txt handling: Yes.

Evidence and Source

  • Control token only; it relies on existing Google crawler traffic rather than a separate HTTP user-agent.
  • Match `Google-Extended` as a case-insensitive substring in HTTP user-agent logs. Review bot_aliases for alternate names or product labels. Do not treat a user-agent match alone as proof of identity for allow-listing.
  • AI crawling, assistant retrieval, AI search, or model-supporting discovery depending on the user-agent.
  • User-agent strings for Google-Extended can be spoofed. Treat user-agent detection as a classification signal, then verify with published IP ranges, reverse DNS, signatures, operator documentation, or published operator documentation, IP ranges, reverse DNS, signatures, or other verified identity signals before allow-listing.

https://developers.google.com/crawling/docs/crawlers-fetchers/google-common-crawlers

Same family: Google

ProjectShield-UrlCheck Safe

ProjectShield Url Check

ProjectShield Url Check is a security scanner from Google used for security checks, malware review, or abuse prevention.

Operator and family: Google Type: Security
Verified robots.txt: No Block: No Verified Bot
GoogleOther-Image Neutral

GoogleOther-Image

GoogleOther-Image is an AI crawler from Google used for AI-related crawling, retrieval, search, or assistant activity.

Operator and family: Google Type: Ai Purpose: Ai Training
Verified robots.txt: Yes Block: Depends Official Documented
GoogleOther-Video Neutral

GoogleOther-Video

GoogleOther-Video is an AI crawler from Google used for AI-related crawling, retrieval, search, or assistant activity.

Operator and family: Google Type: Ai Purpose: Ai Training
Verified robots.txt: Yes Block: Depends Official Documented
Googlebot Safe

Googlebot

Googlebot is Google Search’s web crawler for discovering, rendering, and indexing web pages.

Operator and family: Google Type: Search Purpose: Indexing
Verified robots.txt: Yes Block: Depends Official Documented

Same operator: Google

GoogleAgent-Mariner Neutral

GoogleAgent-Mariner

Claimed Google Project Mariner user-agent token for agentic browser activity.

Operator: Google Family: Google Mariner Type: Ai Purpose: Agentic Browsing
Not verified robots.txt: Unknown Block: Depends Observed
Google-NotebookLM Neutral

Google-NotebookLM

Google-NotebookLM is an AI fetcher from Google used for AI-related crawling, retrieval, search, or assistant activity.

Operator and family: Google Type: Ai Purpose: Ai Assistant
Verified robots.txt: No Block: Depends Official Documented
Google-Pinpoint Safe

Google-Pinpoint

Google-Pinpoint is a search crawler from Google used for search indexing, page discovery, or content rendering.

Operator and family: Google Type: Search
Verified robots.txt: No Block: Depends Official Documented
Googlebot-News Safe

Googlebot-News

Googlebot-News is a search crawler from Google used for search indexing, page discovery, or content rendering.

Operator and family: Google Type: Search Purpose: Indexing
Verified robots.txt: Yes Block: Depends Official Documented

Similar purpose: Ai Assistant

TwinAgent Neutral

TwinAgent

TwinAgent is an AI fetcher from Twin used for AI-related crawling, retrieval, search, or assistant activity.

Operator and family: Twin Type: Ai Purpose: Ai Assistant
Verified robots.txt: No Block: Depends Verified Bot
SpyglassesBot Neutral

Spyglasses Bot

Spyglasses Bot evaluates website content to identify issues that would affect visibility by AI assistants.

Operator and family: Spyglasses Type: Ai Purpose: Ai Assistant
Verified robots.txt: No Block: Depends Verified Bot
Perplexity-User Neutral

Perplexity-User

Perplexity-User is an AI agent from Perplexity used for AI-related crawling, retrieval, search, or assistant activity.

Operator and family: Perplexity Type: Ai Purpose: Ai Assistant
Verified robots.txt: Unknown Block: Depends Verified Bot
Nava/ Neutral

Nava Labs ASP (Dev)

Bot to assist social workers in navigating safety net benefit websites.

Operator: Nava Labs Family: Nava Type: Ai Purpose: Ai Assistant
Verified robots.txt: No Block: Depends Verified Bot

Monitor This Bot In Edge

Botcrawl Edge

Use Botcrawl Edge to see matching traffic, create allow or block rules, and control this bot across connected sites.

Get Edge Now Open BotDB