Botcrawl Intelligence API

API

Use Botcrawl's bot intelligence feed and REST API to identify AI crawlers, search bots, scrapers, scanners, monitoring bots, preview bots, and user-agent patterns.

Create API Key View API Docs

Try it live

Identify a User-Agent

Paste a crawler, bot, or user-agent token and Botcrawl will return a limited public match. Create an API key for protected endpoints, bulk lookup, usage tracking, and full records.

Botcrawl's API is the machine-readable version of the Bot Directory database. Use it to identify bots, look up user-agent strings, download cached intelligence feeds, and build integrations around crawlers, AI bots, scrapers, scanners, and automated traffic.

Lookup a User-Agent

Send a user-agent string and Botcrawl returns likely bot matches, operators, categories, confidence, risk, and recommended handling.

GET /wp-json/botcrawl/v1/public-lookup?ua=GPTBot

List Bot Records

Fetch paginated bot records with sort and filter parameters for name, date, type, operator, category, risk, and recommended action.

GET /wp-json/botcrawl/v1/bots?sort=name&order=asc

Bulk Lookup

Send up to 100 user-agent strings in one request for log enrichment, dashboards, security tools, and enterprise workflows.

POST /wp-json/botcrawl/v1/bulk-lookup

API Keys

Botcrawl data endpoints use API keys. Send your key with an Authorization bearer header or X-Botcrawl-API-Key header.

Authorization: Bearer bc_live_...

Developer Endpoints

  • /wp-json/botcrawl/v1/public-lookup?ua=GPTBotLimited public user-agent lookup for demos and low-volume checks.
  • /wp-json/botcrawl/v1/statusAPI status, access mode, cache policy, and feed version.
  • /wp-json/botcrawl/v1/docsMachine-readable API documentation.
  • /wp-json/botcrawl/v1/openapiOpenAPI-style endpoint description for developers.
  • /wp-json/botcrawl/v1/search?q=googleSearch bot records by name, operator, category, and user-agent pattern.
  • /wp-json/botcrawl/v1/bulk-lookupIdentify up to 100 user-agent strings in one authenticated request.
  • /wp-json/botcrawl/v1/changesRetrieve recently updated bot records for sync workflows.
  • /wp-json/botcrawl/v1/usageReview current key plan, monthly usage, and request limits.
  • /bot-intel/v1/manifest.jsonStatic feed manifest with file URLs, checksums, and version data.