Overview
Bibliothèque nationale de France Crawler is an archiver associated with Bibliothèque nationale de France, used for capturing public websites for the French national web archive.
Its primary user-agent pattern is bnf.fr_bot; related patterns include BnF crawler; Bibliothèque nationale de France crawler; a representative HTTP user-agent is Mozilla/5.0 (compatible; bnf.fr_bot; +https://www.bnf.fr/fr/capture-et-archivage-du-web).
Bibliothèque nationale de France Crawler is Verified from current authoritative identity documentation reviewed on 2026-08-07. The identity type is Official Documented.
Robots.txt behavior for Bibliothèque nationale de France Crawler is not confirmed. Do not treat a user-agent-only rule as guaranteed enforcement.
The operator and crawler identity are documented by the Bibliothèque nationale de France; current robots.txt enforcement details are not sufficiently explicit for a guaranteed claim.
Treat the user-agent as an identity claim. Corroborate it with operator-controlled IP ranges, reverse DNS, signed-request evidence, or current documentation when available.
Identity
- User-Agent Pattern
-
bnf.fr_bot - Aliases
- BnF crawler; Bibliothèque nationale de France crawler
- HTTP Agent Examples
-
Mozilla/5.0 (compatible; bnf.fr_bot; +https://www.bnf.fr/fr/capture-et-archivage-du-web) - Robots Token
- bnf.fr_bot
- Identity Type
- Officially documented
- Evidence Method
- Match `bnf.fr_bot` to the current operator documentation and corroborate the request with operator-controlled verification signals where available; User-Agent strings alone can be spoofed.
Classification
- Type
- Archive
- Kind
- Archiver
- Family
- Bibliothèque nationale de France
- Purpose
- Web archiving
Behavior and handling
- Common Use
- capturing public websites for the French national web archive
- Detection Notes
- Detect primarily by `bnf.fr_bot`; related patterns include `BnF crawler; Bibliothèque nationale de France crawler`; representative HTTP user-agent: `Mozilla/5.0 (compatible; bnf.fr_bot; +https://www.bnf.fr/fr/capture-et-archivage-du-web)`. User-agent strings can be copied, so corroborate with stronger evidence before granting trust.
- Respects robots.txt
- Unknown
- Spoofing Risk
- Bibliothèque nationale de France Crawler has medium spoofing risk because its user-agent can be copied; use published operator-controlled verification data when available.
- Risk
- Safe
- Recommended Handling
- Do not block
Rules and controls
- Robots.txt Snippet
-
# Robots.txt behavior is unconfirmed. Monitor first and use stronger controls when enforcement is required. - Apache Rule
-
RewriteCond %{HTTP_USER_AGENT} "bnf\.fr_bot" [NC] RewriteRule ^ - [F,L] - Nginx Rule
-
if ($http_user_agent ~* "bnf\.fr_bot") { return 403; }
Relationships
- Operator
- Bibliothèque nationale de France Checked 2026-08-07
Relationships without an Evidence link are normalized from the canonical directory record. They should not be interpreted as independent proof of physical presence or request origin.