Bot intelligence record

Google-Extended

Usually allow

Use the Google-Extended identifier to separate Google AI crawler, assistant, or AI search traffic from normal visitor requests in server logs.

Ai Ai Assistant Official Documented Confidence: High Verified: Yes robots.txt: Yes

User-Agent Pattern

Google
Google-Extended

Robots.txt Snippet

Copy-ready 
User-agent: Google-Extended Disallow: /

Handling Guidance

Depends

This bot is usually safe to allow when the request source is verified and the traffic matches your site policy.

AI crawling, assistant retrieval, AI search, or model-supporting discovery depending on the user-agent.

Record Details

Structured data
Operator
Google
Family
Google
Type
Ai
Purpose
Ai Assistant
Identity type
Official Documented
Confidence
High
Last verified
2026-04-01
Last checked
2026-04-30
Source type
Official

Notes

Google-Extended is listed in the Botcrawl directory as an AI control token from Google. The primary identifier for log review is Google-Extended.

Identification

  • User-agent pattern: Google-Extended
  • Family: Google
  • Type: AI
  • Kind: Control Token

Common use

AI crawling, assistant retrieval, AI search, or model-supporting discovery depending on the user-agent.

Verification and handling

Control token only; it relies on existing Google crawler traffic rather than a separate HTTP user-agent.

Directory guidance marks the risk level as Safe and the blocking decision as Depends. Do not rely on the user-agent string alone because user-agent strings can be copied or spoofed.

Robots.txt handling: Yes.

Evidence and Source

  • Control token only; it relies on existing Google crawler traffic rather than a separate HTTP user-agent.
  • AI crawling, assistant retrieval, AI search, or model-supporting discovery depending on the user-agent.

https://developers.google.com/crawling/docs/crawlers-fetchers/google-common-crawlers

Same family: Google

ProjectShield-UrlCheck Safe

ProjectShield Url Check

ProjectShield Url Check is a security scanner from Google used for security checks, malware review, or abuse prevention.

Google Google Security Security
Verified robots.txt: No Block: No Verified Bot
GoogleOther Neutral

GoogleOther

Generic crawler that may be used by various product teams for fetching publicly accessible content from sites. For example, it may be used for one-off crawls for internal research and.

Google Google Search Indexing
Verified robots.txt: Yes Block: Depends Official Documented
GoogleOther-Image Neutral

GoogleOther-Image

GoogleOther-Image is an AI crawler from Google used for AI-related crawling, retrieval, search, or assistant activity.

Google Google Ai Ai Training
Verified robots.txt: Yes Block: Depends Official Documented
GoogleMessages Safe

GoogleMessages

GoogleMessages is a preview or prefetch bot from Google used to fetch pages for previews, unfurls, or user-triggered rendering.

Google Google Preview Preview
Verified robots.txt: No Block: Depends Official Documented

Same operator: Google

GoogleOther-Video Neutral

GoogleOther-Video

GoogleOther-Video is an AI crawler from Google used for AI-related crawling, retrieval, search, or assistant activity.

Google Google Ai Ai Training
Verified robots.txt: Yes Block: Depends Official Documented
Google-Pinpoint Safe

Google-Pinpoint

Google-Pinpoint is a search crawler from Google used for search indexing, page discovery, or content rendering.

Google Google Search Search
Verified robots.txt: No Block: Depends Official Documented
Google-Agent Neutral

Google-Agent

Google-Agent is an AI fetcher from Google used for AI-related crawling, retrieval, search, or assistant activity.

Google Google Ai Ai Assistant
Verified robots.txt: No Block: Depends Official Documented
Googlebot Safe

Googlebot

Googlebot is Google Search’s web crawler for discovering, rendering, and indexing web pages.

Google Google Search Indexing
Verified robots.txt: Yes Block: Depends Official Documented

Similar purpose: Ai Assistant

SpyglassesBot Neutral

Spyglasses Bot

Spyglasses Bot evaluates website content to identify issues that would affect visibility by AI assistants.

Spyglasses Spyglasses Ai Ai Assistant
Verified robots.txt: No Block: Depends Verified Bot
Nava/ Neutral

Nava Labs ASP (Dev)

Bot to assist social workers in navigating safety net benefit websites.

Nava Labs Nava Ai Ai Assistant
Verified robots.txt: No Block: Depends Verified Bot
TwinAgent Neutral

TwinAgent

TwinAgent is an AI fetcher from Twin used for AI-related crawling, retrieval, search, or assistant activity.

Twin Twin Ai Ai Assistant
Verified robots.txt: No Block: Depends Verified Bot
Perplexity-User Neutral

Perplexity-User

Perplexity-User is an AI agent from Perplexity used for AI-related crawling, retrieval, search, or assistant activity.

Perplexity Perplexity Ai Ai Assistant
Verified robots.txt: Unknown Block: Depends Verified Bot

WordPress Bot Protection

Bot Blocker for WordPress

Monitor bot traffic, review live activity, and control AI crawlers, scrapers, scanners, spam bots, and fake trusted bots from one clean WordPress dashboard.

Buy Now Start Free Trial