HenkBot
AI search

HenkBot

HenkBot is Valyu’s documented web crawler for indexing public web content used by Valyu search and retrieval APIs; it identifies as `HenkBot`.

Operator Valyu

About this bot

HenkBot is a crawler associated with Valyu and used for ai-search-indexing.

The primary identifying User-Agent token is HenkBot. Representative HTTP request identity: Mozilla/5.0 (compatible; HenkBot/1.0; +https://valyu.ai/crawler).

Directory status: Active. Identity classification: Verified with High confidence. Identity type: official documented.

This identity is recorded as honoring robots.txt under HenkBot. Site-specific access policy should still be evaluated before enforcement.

Risk classification: Neutral. Recommended handling is to evaluate the request context before allowing, rate-limiting, or blocking.

How to identify this bot

User-agent and crawler identifiers
HenkBot Mozilla/5.0 (compatible; HenkBot/1.0; +https://valyu.ai/crawler)

Details

Kind
Crawler
Family
HenkBot
Purpose
AI search indexing
Common Use
HenkBot crawls and indexes public web content to support Valyu search, retrieval, and AI-agent data access.
Respects robots.txt
Yes

Verification

Identity Type
Officially documented
Evidence Method
Match `HenkBot` as the claimed identity signal and corroborate it with independent network, DNS, cryptographic, or request-context signals when stronger authentication is required.
Detection Notes
Match the `HenkBot` User-Agent token. A User-Agent match identifies a claim, not authenticated origin.
Website controls
Robots.txt Snippet
User-agent: HenkBot Disallow: /

Similar Bots

AI search

xAI-SearchBot

Unconfirmed

xAI-SearchBot is an observed/tentative identity associated by name with xAI search retrieval; current authoritative identity and authentication evidence remains insufficient.

AI search

WRTNBot

Wrtn

WRTNBot is Wrtn’s AI/search crawler used to retrieve public web content for Wrtn search and AI-assisted retrieval services.

AI search

LINER Bot

LINER

LinerBot crawls and indexes public web content for LINER search/source discovery. Unsupported model-training language has been removed.

AI search

Kernel Search

Kernel Technologies, Inc.

KernelSearchBot is the dedicated Kernel search/indexing crawler and is distinct from the Kernel Browsers automation platform.

AI search

DuckAssistBot

DuckDuckGo

DuckAssistBot/1.2 performs real-time retrieval for DuckDuckGo's AI-assisted answers, follows robots.txt, and is explicitly not used for AI-model training.

AI search

Direqt Anomura

Direqt

Anomura is Direqt's AI-search crawler identity. V17 preserves the documented UA but removes the unsupported claim that it ignores robots.txt.