Overview
AIWebIndex-Agent is a fetcher associated with Lyrenth, used for fetching a specific URL on demand for the AI-readable index.
Its primary user-agent pattern is AIWebIndex-Agent; related patterns include AIWebIndex-Agent/2.0; a representative HTTP user-agent is AIWebIndex-Agent/2.0 (+https://lyrenth.com/bot; on-demand index fetch).
AIWebIndex-Agent is verified with High confidence. The identity type is Verified Bot.
AIWebIndex-Agent is documented or strongly corroborated as honoring robots.txt directives.
The operator documents this separately from scheduled AIWebIndex crawling while applying the same robots.txt token and verification mechanisms.
Treat the user-agent as an identity claim. Corroborate it with operator-controlled IP ranges, reverse DNS, signed-request evidence, or current documentation when available.
Identity
- User-Agent Pattern
-
AIWebIndex-Agent - Aliases
- AIWebIndex-Agent/2.0
- HTTP Agent Examples
-
AIWebIndex-Agent/2.0 (+https://lyrenth.com/bot; on-demand index fetch) - Robots Token
- AIWebIndex
- Identity Type
- Verified bot
- Evidence Method
- Match `AIWebIndex-Agent` and validate the source using Lyrenth's published IP ranges, forward-confirmed reverse DNS, Web Bot Auth, or ownership-verification documentation.
Classification
- Type
- AI
- Kind
- Fetcher
- Family
- AIWebIndex
- Purpose
- User-triggered index fetching
Behavior and handling
- Common Use
- fetching a specific URL on demand for the AI-readable index
- Detection Notes
- Detect primarily by `AIWebIndex-Agent`; related patterns include `AIWebIndex-Agent/2.0`; representative HTTP user-agent: `AIWebIndex-Agent/2.0 (+https://lyrenth.com/bot; on-demand index fetch)`. User-agent strings can be copied, so corroborate with stronger evidence before granting trust.
- Respects robots.txt
- Yes
- Spoofing Risk
- AIWebIndex-Agent has medium spoofing risk because its user-agent can be copied; use published operator-controlled verification data when available.
- Risk
- Neutral
- Recommended Handling
- Depends
Rules and controls
- Robots.txt Snippet
-
User-agent: AIWebIndex Disallow: / - Apache Rule
-
RewriteCond %{HTTP_USER_AGENT} "AIWebIndex\-Agent" [NC] RewriteRule ^ - [F,L] - Nginx Rule
-
if ($http_user_agent ~* "AIWebIndex\-Agent") { return 403; }
Relationships
- Operator
- Lyrenth Checked 2026-08-02
Relationships without an Evidence link are normalized from the canonical directory record. They should not be interpreted as independent proof of physical presence or request origin.