About this bot
bl.uk_ldfc_bot is a legal-deposit web-archive crawler associated with British Library.
The identifying User-Agent token is bl.uk_ldfc_bot. A representative versioned request identity is bl.uk_ldfc_bot/3.4.0-20220727 (+https://www.bl.uk/legal-deposit/web-archiving).
Directory status: Active. Identity classification: Verified with Medium confidence. Identity type: documented service plus network-corroborated User-Agent.
The British Library continues to operate legal-deposit web crawling, while current network-attributed traffic under this exact token resolves to British Library crawler infrastructure. The Library’s current public page does not publish the exact token or make a sufficiently specific robots.txt-compliance claim for this identity, so robots behavior remains unknown.
Verification: Match the versioned bl.uk_ldfc_bot identity and corroborate source origin with forward-confirmed British Library *.bl.uk crawler hostnames before granting identity-based trust.
How to identify this bot
- User-agent and crawler identifiers
-
bl.uk_ldfc_botbl.uk_ldfc_bot/3.4.0-20220727 (+https://www.bl.uk/legal-deposit/web-archiving) - Also known as
- British Library legal deposit crawler; bl.uk_ldfc_bot/3.4.0 British Library legal deposit crawler bl.uk_ldfc_bot/3.4.0
Details
- Kind
- Legal deposit archiver
- Family
- British Library
- Purpose
- Uk legal deposit web archiving
- Common Use
- preserving UK websites under British Library legal-deposit and web-archiving programs
- Respects robots.txt
- Unknown
Verification
- Identity Type
- documented-service-plus-observed-user-agent
- Evidence Method
- Match the versioned `bl.uk_ldfc_bot` identity and corroborate source origin using forward-confirmed British Library `*.bl.uk` crawler hostnames. The current British Library legal-deposit service is active, but the exact token is not currently published on its public service page.
- Detection Notes
- The exact versioned token is currently observed from British Library crawler infrastructure, while the British Library confirms that UK legal-deposit web crawling remains active. robots.txt behavior is not asserted from the current operator page.
Website controls
- Robots.txt Snippet
# robots.txt behavior is unconfirmed. Use server-side controls if restriction is required.- Apache Rule
RewriteCond %{HTTP_USER_AGENT} "bl\.uk_ldfc_bot" [NC] RewriteRule ^ - [F,L]- Nginx Rule
if ($http_user_agent ~* "bl\.uk_ldfc_bot") { return 403; }