Bot intelligence record

GoogleAgent URLContext

Review first

GoogleAgent URLContext is an AI crawler from Google used for assistant-driven browsing, page retrieval; it appears in server logs as `GoogleAgent-URLContext`.

Ai Ai Assistant Observed Confidence: High Verified: Yes robots.txt: Yes
Operator
Google
Family
Google
Type
Ai
Source type
Official
Last checked
2026-06-23

User-Agent Pattern

Google
GoogleAgent-URLContext
Verification note

User-agent strings are identification signals, not proof of identity. Confirm important allow, block, or rate-limit decisions with logs, DNS or IP evidence, request behavior, or operator documentation when available.

Robots.txt Snippet

Click snippet to copy 
User-agent: GoogleAgent-URLContext
Disallow: /

Click the snippet to copy it, or highlight the text manually.

Handling Guidance

Depends

Use this record as bot intelligence, then verify the request source and behavior before allowing, blocking, or rate limiting.

GoogleAgent URLContext is used for assistant-driven browsing, page retrieval, and automated web access on behalf of an AI product.

Record Details

Structured data
Operator
Google
Family
Google
Type
Ai
Purpose
Ai Assistant
Identity type
Observed
Confidence
High
Last verified
2026-06-23
Last checked
2026-06-23
Source type
Official
Verification
Verify GoogleAgent URLContext by matching `GoogleAgent-URLContext` to Google evidence, then checking reverse DNS, source-network ownership, signed request data, or published crawler documentation when available.
Spoofing risk
GoogleAgent URLContext has medium spoofing risk because the user-agent can be copied, even when the bot has strong source or documentation support.

Notes

  • GoogleAgent URLContext is an AI crawler from Google used for assistant-driven browsing, page retrieval, and automated web access on behalf of an AI product.
  • Its primary user-agent pattern is GoogleAgent-URLContext.
  • GoogleAgent URLContext is verified with High confidence. The identity type is Observed, and the evidence basis is official operator documentation.
  • GoogleAgent URLContext is marked as respecting robots.txt directives for crawler access control.
  • GoogleAgent URLContext should be handled according to the site owner’s AI crawler policy, with allow, block, or rate-limit rules applied deliberately.

Evidence and Source

  • Verify GoogleAgent URLContext by matching `GoogleAgent-URLContext` to Google evidence, then checking reverse DNS, source-network ownership, signed request data, or published crawler documentation when available.
  • GoogleAgent URLContext traffic is primarily detected by the `GoogleAgent-URLContext` user-agent pattern. Compare source IPs, reverse DNS, request paths, and crawl cadence with Google infrastructure before trusting the traffic.
  • GoogleAgent URLContext is used for assistant-driven browsing, page retrieval, and automated web access on behalf of an AI product.
  • GoogleAgent URLContext has medium spoofing risk because the user-agent can be copied, even when the bot has strong source or documentation support.

https://developers.google.com/crawling/docs/crawlers-fetchers/google-common-crawlers

Same family: Google

Google-Structured-Data-Testing-Tool Neutral

Google Structured Data Testing Tool

Google Structured Data Testing Tool is a validator bot from Google used for markup validation, accessibility testing, structured-data review.

Operator and family: Google Type: Validator Purpose: Validation And Testing
Verified robots.txt: Yes Block: Allow Documented
View Record Open in Edge BotDB
Google-Transparency-Report Caution

Google Transparency Report

Google Transparency Report is a security scanner from Google used for security scanning, malware checks, vulnerability assessment, certificate review.

Operator and family: Google Type: Security Scanner Purpose: Security Scanning
Verified robots.txt: Yes Block: Monitor Documented
View Record Open in Edge BotDB
GoogleDocs Caution

GoogleDocs

GoogleDocs is a web scraper from Google used for public web data collection, page extraction, content monitoring.

Operator and family: Google Type: Content Crawler Purpose: Content Discovery
Verified robots.txt: Yes Block: Monitor Documented
View Record Open in Edge BotDB
Google AppsViewer Caution

Google AppsViewer

Google AppsViewer is a web scraper from Google used for public web data collection, page extraction, content monitoring.

Operator and family: Google Type: Content Crawler Purpose: Content Discovery
Verified robots.txt: Yes Block: Monitor Documented
View Record Open in Edge BotDB

Same operator: Google

Google favicon Neutral

Google Favicon

Google Favicon is a search crawler from Google used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: Google Type: Search Engine Crawler Purpose: Search Indexing
Verified robots.txt: Yes Block: Monitor Documented
View Record Open in Edge BotDB
Google Search Console Neutral

Google Search Console

Google Search Console is a validator bot from Google used for markup validation, accessibility testing, structured-data review.

Operator and family: Google Type: Validator Purpose: Validation And Testing
Verified robots.txt: Yes Block: Allow Documented
View Record Open in Edge BotDB
Google Page Speed Insights Neutral

Google PageSpeed Insights

Google PageSpeed Insights is a performance Tool from Google used for uptime monitoring, synthetic checks, performance measurement, accessibility review.

Operator and family: Google Type: Performance Tool Purpose: Pagespeed Testing
Verified robots.txt: Yes Block: Depends Documented
View Record Open in Edge BotDB
Googlebot-Discovery Neutral

Googlebot Discovery

Googlebot Discovery is a search crawler from Google used for search indexing, URL discovery, page rendering, result freshness.

Operator and family: Google Type: Search Engine Crawler Purpose: Discovery Crawling
Verified robots.txt: Yes Block: Depends Documented
View Record Open in Edge BotDB

Similar purpose: Ai Assistant

QuillBot Neutral

QuillBot

QuillBot is an AI crawler used for assistant-driven browsing, page retrieval.

Operator and family: QuillBot Type: Ai Purpose: Ai Assistant
Not verified robots.txt: Unknown Block: Depends Observed
View Record Open in Edge BotDB
Shap-User Neutral

Shap User

Shap User is an AI crawler from Shap used for assistant-driven browsing, page retrieval.

Operator and family: Shap Type: Ai Purpose: Ai Assistant
Not verified robots.txt: Unknown Block: Depends Observed
View Record Open in Edge BotDB
Thinkbot Neutral

Thinkbot

Thinkbot is an AI crawler used for assistant-driven browsing, page retrieval.

Operator and family: Thinkbot Type: Ai Purpose: Ai Assistant
Not verified robots.txt: Unknown Block: Depends Observed
View Record Open in Edge BotDB
Trae Neutral

Trae

Trae is an AI crawler used for assistant-driven browsing, page retrieval.

Operator and family: Trae Type: Ai Purpose: Ai Assistant
Not verified robots.txt: Unknown Block: Depends Observed
View Record Open in Edge BotDB

Monitor This Bot In Edge

Botcrawl Edge

Use Botcrawl Edge to see matching traffic, create allow or block rules, and control this bot across connected sites.

Get Edge Now Open in Edge BotDB