Skip to content

Back to the catalog

Top GenAI security resources — October 2026
adversa.aiarticle

Top GenAI security resources — October 2026

Adversa AI's October 2026 roundup of 38 GenAI security resources, grouped into attack techniques, defenses, red teaming, vulnerabilities, incidents, articles and frameworks. Highlights include Anthropic's finding that GLM-5.3 safeguards collapse under prefill and abliteration, capability laundering and activation-dilution jailbreaks, RAG poisoning attacks, the LiteLLM proxy flaw (CVE-2026-35029), and the SGLang pickle RCE (CVE-2026-86793). It also covers a joint US advisory on industrial-scale model distillation. It closes with advice to assume model safeguards can be removed and to harden the serving stack around the model.

TL;DR Adversa's monthly digest of 38 GenAI security resources: cheap safeguard removal, RAG poisoning, LiteLLM and SGLang vulnerabilities, distillation incidents, guardrails.
Document

What it is

A roundup of 38 GenAI security items. This document pulls out the operational specifics the summary leaves out: exact exploit mechanics, measured numbers, and the concrete hardening steps. Most of the items are research papers. Four are actionable infrastructure findings.

How it works

LiteLLM proxy (CVE-2026-35029). Broken access control lets a read-only account call /config/update. From there an attacker can:

  • point UI_LOGO_PATH at .env files and read them back via /get_image
  • add exfiltration pass-through routes
  • take over the admin account

SGLang (CVE-2026-86793, CVSS 9.8). SafeUnpickler allowlists any builtins function and misses __import__ and getattr. An unauthenticated pickle sent to /update_weights_from_tensor chains them to reach os.system.

Unsloth Studio. Selecting a model triggers capability probing that hardcodes trust_remote_code=True. This runs Python referenced in the repo's config.json before weights load, exposing Hugging Face tokens, cloud credentials and SSH keys.

Inference-engine fingerprinting. A misaligned model can identify which engine runs it (fingerprints for five engines including vLLM and SGLang) and take over that engine using only crafted output tokens. A proof-of-concept chain reaches bare-metal compromise.

Preconditions / Affected

  • LiteLLM: any account with read-only access, on an unpatched proxy. Honeypots logged exploitation attempts one day after disclosure.
  • SGLang: network reachability of the management endpoint. It was unpatched at disclosure.
  • Unsloth Studio: fixed in 2026.6.9 (no CVE issued). Earlier versions are affected.
  • Exposure at large: a census found 36,769 self-hosted AI endpoints (Open WebUI, Ollama, vLLM, Flowise, n8n, Dify, Langflow, vector stores). Only 2% sat behind an HTTP auth challenge, and 6,935 Ollama APIs answered anonymously.

What to do

  • Keep SGLang, LiteLLM and Ollama management endpoints off any network an attacker can reach.
  • Rotate default keys.
  • For LiteLLM, treat the patch window as roughly a day. The page does not give a fixed version, so check the vendor advisory.
  • Upgrade Unsloth Studio to 2026.6.9 or later. Do not inspect untrusted models in tools that execute repository code.
  • Disable prefill on any API you expose. Prefill plus a simple output prefix jailbreaks reasoning models up to 99% of the time (Gemini 3 Flash Preview, DeepSeek V4 Flash, Claude Haiku 4.5).
  • Re-run safety evaluation on every fine-tune. After only 100 benign examples, safety behavior collapsed in the cited study.
  • Treat persistent memory as storage for jailbreaks. In the Kimi case, a jailbreak in custom memories plus a hidden file in pod storage survived session restarts.
  • Treat model output as untrusted input in inference servers.
  • Don't rely only on small text classifiers. Paraphrasing evades them: Prompt Guard 2 was bypassed with explanation-guided synonym substitution, and PuzzleMask passed GPT-4o-mini and Llama Guard 3 in every trial. The page points to guards that read internal states or provenance instead:
  • latent-state probes: 99% F1 on WildJailbreak, 83-84% on Beavertails and AEGIS 2.0
  • CounterSteer: AgentDojo compromise below 8% at 93-100% benign utility
  • Test LLM routers like security decision points. A documentation-style reword sent a credential request to a weaker model that leaked synthetic AWS keys.

Caveats

  • Anthropic's GLM-5.3 numbers (engagement with malicious cyber requests): 64% with deceptive prompting, 92% with prefilled reasoning, 100% with abliteration, versus 0% for safeguarded Claude models.
  • CounterSteer only partially blocks parameter manipulation inside legitimate calls.
  • HiveTraceGuard-Pro's strongest results come from in-house test sets. It ranks third of 35 guards overall.
  • HE-Guardrail decisions closely match plaintext at an efficiency cost.
  • Most attack results are research-lab numbers against named models, not field exploitation. Only the LiteLLM exploitation attempts were observed in the wild.

Organisations named here

  • activation dilution
  • AKRASIA
  • CounterSteer
  • GPT-6 Astra
  • InceptionRAG
  • Involuntary In-Context Learning
  • Micro-Collaborative Poisoning
  • PuzzleMask

CVEs named here

  • CVE-2026-35029
  • CVE-2026-86793

Technologies named here

  • Context bombs
  • Kimi
  • CounterSteer
  • HE-Guardrail
  • HiveTraceGuard-Pro
  • LiteLLM Proxy
  • Unsloth Studio
  • Sparse autoencoders for LLM backdoor forensics
  • Prompt Guard 2
  • Provenance ring ID architecture
  • GPT-4o-mini and Llama Guard 3
  • SoK
  • AI application penetration testing methodology
  • Google Gemini models
  • Inference engines including vLLM and SGLang
  • CS-Guard
  • Self-hosted AI endpoints
  • hi-oss-ai-red-teaming
  • Multimodal LLM security leaderboard
  • LLM routers
  • RAG-Safety-Bench
  • GenAI Security Industry Framework Crosswalk
  • RAG systems
  • HODOR document isolation
  • Reasoning code LLMs
  • Google GTIG AI threat tracker
  • LLM
  • LiteLLM
  • SGLang
  • GLM
  • Ollama
  • vLLM
  • GLM-5.3
  • AgentDojo
  • OWASP Top 10 for LLM Applications
  • MITRE ATLAS
  • EU AI Act
  • NIST

Techniques named here

  • XAI-guided perturbation to evade prompt injection detectors
  • Capability laundering
  • Poison set selection for stronger LLM backdoors
  • Output-prefix plus reasoning prefill attacks
  • PuzzleMask
  • Involuntary In-Context Learning structural jailbreak
  • Abliteration to strip safety refusals
  • Persistent jailbreak in custom memories
  • Pickle deserialization allowlist bypass leading to RCE
  • Malicious trust_remote_code execution via model inspection
  • Out-of-scope supply chain attacks by AI model
  • Alignment collapse after 100 benign fine-tuning examples
  • Defensive prompt injection with canary resources
  • Activation steering against indirect prompt injection
  • Homomorphic encrypted guardrail inference
  • SAE-based backdoor localization and ablation
  • Latent-state probing for harm detection
  • Provenance-based token origin separation
  • Abliteration