About this bot
Inline-AI-Crawler is a knowledge ingestion crawler associated with Inline Technologies and used for publisher content ingestion.
The primary identifying User-Agent token is Inline-AI-Crawler. Representative HTTP request identity: Inline-AI-Crawler/0.1 (+https://getinline.io; support@getinline.io).
Directory status: Active. Identity classification: Verified with High confidence. Identity type: http message signature.
robots.txt handling depends on the request mode or how the URL was supplied. Use deterministic server-side controls when needed.
Risk classification: Neutral. Evaluate the request context and site policy before allowing, rate-limiting, or blocking.
How to identify this bot
- User-agent and crawler identifiers
-
Inline-AI-CrawlerInline-AI-Crawler/0.1 (+https://getinline.io; support@getinline.io)
Details
- Kind
- Knowledge ingestion crawler
- Family
- Inline Technologies
- Purpose
- Publisher content ingestion
- Common Use
- ingesting publisher-owned content to ground on-site AI assistant answers
- Respects robots.txt
- Depends
Verification
- Identity Type
- HTTP message signature
- Evidence Method
- Identify candidates by `Inline-AI-Crawler`, then verify with Inline’s Web Bot Auth signature or current published crawler IPs. Publisher-supplied sources and automatically discovered URLs have different robots semantics.
- Detection Notes
- Publisher-supplied sitemap/RSS/URL sources are direct publisher instructions and can be fetched independently of Disallow; automatically discovered URLs honor robots.txt. Prefer Web Bot Auth, or corroborate against Inline’s published production/staging crawler IPs.
Website controls
- Robots.txt Snippet
# robots.txt handling depends on the request mode or source. Use server-side controls when deterministic restriction is required.