Overview
AgentTrust Collector is a specialized discovery crawler operated by agenttru.st that identifies publicly advertised AI agents.
It starts with hostnames found in public Certificate Transparency logs and requests robots.txt plus a fixed set of well-known agent and MCP metadata documents. It does not scan ports, guess paths, follow ordinary site links, execute JavaScript, submit forms, attempt authentication, or probe for vulnerabilities.
Its documented HTTP user agent is AgentTrust-Collector/1.0 (+https://agenttru.st/crawler; purpose=AI Agent Discovery; contact=admin@agenttru.st), and its robots.txt token is agenttrust-collector.
AgentTrust Collector is verified with High confidence. The identity type is Officially documented, based on first-party crawler documentation.
It honors rules for the agenttrust-collector token. Verify requests with forward-confirmed reverse DNS under agenttru.st rather than trusting the user-agent string alone.
Identity
- User-Agent Pattern
-
AgentTrust-Collector - Aliases
- agenttrust-collector
- HTTP Agent Examples
-
AgentTrust-Collector/1.0 (+https://agenttru.st/crawler; purpose=AI Agent Discovery; contact=admin@agenttru.st) - Robots Token
- agenttrust-collector
- Identity Type
- Officially documented
- Evidence Method
- Verify AgentTrust Collector with forward-confirmed reverse DNS: resolve the request IP to a hostname ending in `agenttru.st`, then resolve that hostname and confirm it returns the original IP.
Classification
- Type
- AI
- Kind
- Crawler
- Family
- AgentTrust
- Purpose
- AI agent discovery
Behavior and handling
- Common Use
- AgentTrust Collector discovers publicly advertised AI agents, agent cards, and MCP server metadata.
- Detection Notes
- Match the `AgentTrust-Collector` user-agent, then verify the source with forward-confirmed reverse DNS under `agenttru.st`. Traffic outside the documented discovery paths, at materially higher rates, or without forward-confirmed DNS should not be treated as this crawler.
- Respects robots.txt
- Yes
- Spoofing Risk
- AgentTrust Collector has medium spoofing risk because its user-agent can be copied; use forward-confirmed reverse DNS under agenttru.st before trusting the identity.
- Risk
- Neutral
- Recommended Handling
- Depends
Rules and controls
- Robots.txt Snippet
-
User-agent: agenttrust-collector Disallow: / - Apache Rule
-
RewriteEngine On RewriteCond %{HTTP_USER_AGENT} AgentTrust-Collector [NC] RewriteRule .* - [F,L] - Nginx Rule
-
if ($http_user_agent ~* "AgentTrust-Collector") { return 403; }
Relationships
- Operator
- agenttru.st Checked 2026-08-06
Relationships without an Evidence link are normalized from the canonical directory record. They should not be interpreted as independent proof of physical presence or request origin.