Skip to content

Back to the catalog

Malicious Model Detection with MLTracer Syscalls
binarly.iowriteup

Malicious Model Detection with MLTracer Syscalls

Binarly presents MLTracer, a pipeline that loads PyTorch/pickle model files in a Docker sandbox with strace and FakeNet-NG, flags suspicious syscalls, and labels behavior (reverse shell, downloader, exfiltration) with a SetFit classifier. Run over ~2.12M Hugging Face files, it found 218 confirmed malicious models. Hugging Face-hosted scanners (Protect AI, JFrog, ClamAV, VirusTotal) and ModelScan/SaferPickle missed 9–84% of them. The post also tabulates 21 static-scanner evasion techniques, two of them novel (CodeType/FunctionType indirect variant, EXT2 + copyreg.add_extension), and shows detection of a CVE-2025-32434 exploit that the HF scanners missed.

TL;DR Binarly's MLTracer sandboxes model loading and traces syscalls to catch malicious Hugging Face pickle models that static scanners miss, with a taxonomy of 21 scanner evasion techniques.
Open original ↗

Organisations named here

  • Alternative Execution Primitives
  • Binarly
  • MalHug
  • MLTracer
  • PickleBall
  • PickleCloak

CVEs named here

  • CVE-2025-32434

Technologies named here

  • Pickle-based model file formats
  • MLTracer
  • PyTorch torch.load
  • PyTorch torch::jit::Unpickler::rebuildTensorFromTypeV2
  • FakeNet-NG
  • Docker with closed user-defined bridge network for isolation
  • strace
  • ModelScan 0.8.8
  • Fickling
  • SaferPickle commit 88564e07dbefff58d1bc7571cce01accd394b102
  • picklescan
  • ModelScan
  • Promptfoo ModelAudit
  • SaferPickle
  • PyTorch weights-only unpickler
  • Safetensors
  • Research cited
  • Hugging Face
  • PyTorch
  • Python
  • Docker
  • VirusTotal
  • JFrog

Techniques named here

  • Load-time code execution via pickle deserialization
  • Pre-filtering with PyTorch weights-only unpickler
  • Static scanner evasion
  • Sandbox/analysis-environment detection as an evasion risk

Related in AI Security