
Someone Scraped 5.6 Billion TikTok Videos and Put the Data on Hugging Face for Free
Decrypt reports that a developer going by "hashfunction," operating under the datasocial account, published a 460GB Parquet dataset of metadata for ~5.6 billion public TikTok videos (captions, hashtags, sound IDs, engagement counts, AI-generated flags) on Hugging Face under CC BY-NC 4.0. The data was obtained by hitting TikTok's private mobile API using generated Android device identities, reverse-engineered request signatures, and a spoofed TLS handshake, bypassing the ToS clause that bars automated scraping; datasocial.ai sells the scraper's source code for $1,699 and offers commercial access, creator profiles, and daily updates. The piece situates this against TikTok's gated official Research Tools and the ongoing Reddit v. Perplexity/SerpApi scraping lawsuit.
What it is
DataSocial's scraper (sold for $1,699) that produced the 5.6B-video Hugging Face dataset. The page gives the specific evasion mechanism and scale figures the summary doesn't: generated Android device identities, reverse-engineered request signatures, and a spoofed TLS handshake against TikTok's private mobile API, with no login/account required.
How it works
The write-up claims the system collected 3.23 billion creator profiles, 5.94 billion videos, and 2.8 billion comments in three weeks — no authenticated session, just device-identity spoofing plus matching TikTok's internal request-signing scheme and TLS fingerprint to pass as a genuine Android client.
Preconditions / Affected
Applies to anyone pulling data via TikTok's undocumented mobile API rather than its web/public endpoints. TikTok's Section 3.4 ToS bars automated extraction without written approval; this bypasses that via the private API rather than the public site, which the official gated Research Tools (academic/nonprofit, approval-required, limited to US/EEA/UK/Canada/Switzerland) are meant to control.
Caveats
- Figures (3.23B profiles, 5.94B videos, 2.8B comments, three-week collection time) come from DataSocial's own write-up, unverified by Decrypt.
- The dataset itself has no creator-handle column in the free Hugging Face release — user ID is only available via DataSocial's paid/commercial tier.
- A separate, unrelated 4.5-billion-video TikTok dataset is also already circulating on Hugging Face, so this scraping technique/exposure isn't unique to one actor.
- TikTok is not a party to the Reddit v. Perplexity/SerpApi litigation cited for legal context, and that case concerns scraping via Google search results, not a private mobile API — the legal-precedent angle doesn't directly transfer.
Organisations named here
- CC BY-NC 4.0
- datasocial.ai
- DMCA
Technologies named here
- TikTok
- Hugging Face
- DataSocial scraper
- TikTok Research Tools
- SerpApi
- Android
- Perplexity
- Parquet
Techniques named here
- Automated web/API scraping
- Anti-bot circumvention
Threat actors named here
- hashfunction
Related in Offensive Security