DesearchBot
AI

DesearchBot

DesearchBot is a crawler operated by Desearch, used for indexing public pages for search and retrieval by people and AI agents; it identifies as `DesearchBot`.

Operator Desearch

About this bot

DesearchBot is a crawler associated with Desearch and used for ai search indexing.

The primary identifying User-Agent token is DesearchBot. Representative HTTP request identity: Mozilla/5.0 (compatible; DesearchBot/1.0; +https://www.desearch.ai/crawler).

Directory status: Active. Identity classification: Verified with High confidence. Identity type: http message signature.

This identity is recorded as honoring robots.txt under DesearchBot.

Risk classification: Neutral. Evaluate request context and site policy before allowing, rate-limiting, or blocking.

How to identify this bot

User-agent and crawler identifiers
DesearchBot Mozilla/5.0 (compatible; DesearchBot/1.0; +https://www.desearch.ai/crawler)

Details

Kind
Crawler
Family
Desearch
Purpose
AI search indexing
Common Use
indexing public pages for search and retrieval by people and AI agents
Respects robots.txt
Yes

Verification

Identity Type
HTTP message signature
Evidence Method
Validate the signed request identity when available and use the User-Agent token for discovery. Do not treat the User-Agent string alone as authentication.
Detection Notes
Match the `DesearchBot` User-Agent token. A User-Agent match identifies a claim, not authenticated origin.
Website controls
Robots.txt Snippet
User-agent: DesearchBot Disallow: /

Similar Bots

AI

Keenable-User

Keenable.ai, Inc.

Keenable-User fetches specific web pages at the request of users or AI agents.

AI

KeenableBot

Keenable.ai, Inc.

KeenableBot indexes public web pages for search used by AI agents and applications.

AI

AgentReadyWebsiteBot

Omnirithm Inc.

AgentReadyWebsiteBot tests whether requested websites expose information needed by AI agents.

AI

AgenstryBot

Agenstry

AgenstryBot checks public AI agent cards, discovery documents, and MCP endpoints.

AI

A1-User

A1

A1-User retrieves public pages when an A1 assistant user requests a web page.

AI

Gist Crawler

ProRata / Gist

The Gist Crawler retrieves licensed and public web content for Gist.ai search and retrieval.