TikTokSpider
Scraper

TikTokSpider

TikTokSpider is retained Unverified as an observed ByteDance/TikTok crawler token; current operator-controlled evidence does not justify the previous universal robots.txt claim.

Operator ByteDance

About this bot

TikTokSpider is an automated identity associated with ByteDance and used for TikTokSpider is used for public web data collection, page extraction, content monitoring, and third-party crawler activity.

The primary recorded identity signal is TikTokSpider. Representative HTTP identity: TikTokSpider.

Directory status: Active. Identity classification: Unverified with Medium confidence. Identity type: observed.

robots.txt behavior is not currently confirmed; use server-side controls when enforcement is required.

Risk classification: Safe. Recommended handling: depends.

How to identify this bot

User-agent and crawler identifiers
TikTokSpider
Also known as
TikTok spider

Details

Kind
Fetcher
Family
ByteDance
Purpose
Scraping
Common Use
TikTokSpider is used for public web data collection, page extraction, content monitoring, and third-party crawler activity.
Respects robots.txt
Unknown

Verification

Identity Type
Observed
Evidence Method
Treat `TikTokSpider` as an identification signal only. Corroborate the claimed identity independently before applying trust-based rules.
Detection Notes
Match the `TikTokSpider` User-Agent token and consider recorded aliases where applicable. A User-Agent match identifies a claim, not authenticated origin.
Website controls
Robots.txt Snippet
# robots.txt behavior is unconfirmed. Do not rely on a robots.txt rule without verification.
Apache Rule
SetEnvIfNoCase User-Agent "TikTokSpider" bot_match
Nginx Rule
if ($http_user_agent ~* "TikTokSpider") { set $bot_match 1; }

Similar Bots

Scraper

Tumblr

Automattic

Tumblr is retained Unverified as an observed Tumblr application/fetch identity; the recorded versioned User-Agent lacks current operator-controlled documentation sufficient for promotion.

Search engine crawler

Toutiao

ByteDance

Bytespider is ByteDance’s Toutiao Search crawler for discovering and indexing public webpages; genuine requests can be corroborated through ByteDance network/rDNS signals.

Preview

TikTok

ByteDance

TikTok is retained Unverified as a preview/fetch identity; current operator-controlled evidence does not justify the previous universal robots.txt claim.

Scraper

Rakuten Image extraction bot

Rakuten

Rakuten Image extraction bot is a web scraper from Rakuten used for public web data collection, page extraction, content monitoring.

Scraper

ProperPicsBot

ProperPics

ProperPicsBot is a crawler associated with ProperPics and used for business contact prospecting; it can be identified by `ProperPicsBot`.

Scraper

Ozon Web Grabber

Ozon

Ozon Web Grabber is a web scraper from Ozon used for public web data collection, page extraction, content monitoring.