Overview
IA Archiver is an archive crawler from Internet Archive / Alexa legacy used for web archiving, preservation crawls, historical capture, and public record indexing.
Its primary user-agent pattern is ia_archiver; related patterns include ia-archiver; ia_archiver-web.archive.org; Alexa IA Archiver.
IA Archiver is not independently verified with Medium confidence. The identity type is Documented, and the evidence basis is observed traffic patterns and user-agent evidence.
IA Archiver is marked as respecting robots.txt directives for crawler access control.
IA Archiver should be reviewed against site policy, source evidence, crawl rate, and requested paths before a permanent allow or block rule is created.
IA Archiver belongs to the Internet Archive bot family.
Identity
- User-Agent
ia_archiver- Aliases
- ia-archiver; ia_archiver-web.archive.org; Alexa IA Archiver
- HTTP Agent Examples
ia_archiver- Robots.txt Token
ia_archiver- Identity Type
- Documented
- Evidence Method
- Verify IA Archiver by matching `ia_archiver` to Internet Archive / Alexa legacy evidence, then checking reverse DNS, source-network ownership, signed request data, or published crawler documentation when available.
Classification
- Type
- Archiver
- Kind
- Archiver
- Family
- Internet Archive
- Purpose
- web-archiving
Behavior and handling
- Common Use
- IA Archiver is used for web archiving, preservation crawls, historical capture, and public record indexing.
- Detection Notes
- IA Archiver traffic is primarily detected by the `ia_archiver` user-agent pattern; related patterns include `ia-archiver; ia_archiver-web.archive.org; Alexa IA Archiver`. Compare source IPs, reverse DNS, request paths, and crawl cadence with Internet Archive / Alexa legacy infrastructure before trusting the traffic.
- Respects robots.txt
- Yes
- Spoofing Risk
- IA Archiver has medium spoofing risk because user-agent strings can be copied; pair the match with DNS, IP, behavior, or operator evidence.
- Risk
- Neutral
- Recommended Handling
- Depends
Rules and controls
- Robots.txt Snippet
User-agent: ia_archiver Disallow: /