SmarshBot
Archive

SmarshBot

SmarshBot is associated with Smarsh web-content capture for compliance archiving.

Operator Smarsh

About this bot

SmarshBot is an automated identity associated with Smarsh and used for capturing designated web content for communications compliance and archiving.

The primary recorded identity signal is SmarshBot. Representative HTTP identity: SmarshBot/1.0.

Directory status: Active. Identity classification: Verified with Medium confidence. Identity type: observed-operator-attributed.

robots.txt behavior is not currently confirmed.

Risk classification: Safe. Recommended handling: depends.

How to identify this bot

User-agent and crawler identifiers
SmarshBot SmarshBot/1.0

Details

Kind
Web archiving crawler
Family
Smarsh
Purpose
Compliance web archiving
Common Use
regulated web-content capture, supervision, and compliance archiving
Respects robots.txt
Unknown

Verification

Identity Type
observed-operator-attributed
Evidence Method
Match the observed `SmarshBot/1.0` identity and corroborate with configured Smarsh web-archiving activity or other operator/network context.
Detection Notes
The current Smarsh service role is well established, but the request identity is not treated as cryptographically authenticated.
Website controls
Robots.txt Snippet
# robots.txt behavior is unconfirmed. Do not rely on a robots.txt rule without verification.
Apache Rule
SetEnvIfNoCase User-Agent "SmarshBot" bot_match
Nginx Rule
if ($http_user_agent ~* "SmarshBot") { set $bot_match 1; }

Similar Bots

Archive

bl.uk_ldfc_bot

British Library

bl.uk_ldfc_bot is a network-corroborated British Library legal-deposit web-archive crawler identity.

Archive

Library Of Congress Web Archiving

Library of Congress Web Archiving

The Library of Congress uses web crawlers, primarily Heritrix-based workflows, to preserve selected web content and generally bypasses robots.txt for archival capture. A…

Archive

kb.dk_bot

Netarkivet

Netarkivet performs Danish legal-deposit web archiving with Heritrix. The service intentionally does not use robots.txt as an access-control policy.

Archive

Internet Archive – Archive-It

Archive-It

Archive-It is a hosted web-archiving service. Its crawlers normally respect robots.txt by default, but collection owners can configure robots handling and different crawl…

Archive

Internet Archive

Internet Archive

Internet Archive uses this crawler identity for web preservation and Wayback Machine archiving; it is not a feed-only fetcher.

Archive

Europarchive

European Archive

Europarchive is used for historical web archiving; it can appear in server logs as `europarchive.org`.