Reva: Real-Time Vulnerability Detection through Contextual Probing of Large Language Models

What it claims

REVA screens a single C/C++ function for vulnerabilities in about one second. It runs one partial forward pass of an open-weight LLM over the function plus pre-built security context, then a 0.57M-parameter attention probe turns the hidden states into a risk score. On the PrimeVul test set (435 pairs), REVA matches or beats the agentic detectors VulTriage and VulAgent when all run on the same LLM. On every LLM tested it has a lower false positive rate than VulTriage. The authors, Lbath and Amini (Université Grenoble Alpes), label these results preliminary.

How it works

flowchart LR
  TM[Project threat model<br/>built once by agents] --> P[Prompt]
  SG[Semgrep findings] --> P
  CG[Callers, callee signatures] --> P
  FC[File includes, types, macros] --> P
  F[New function] --> P
  P --> L[LLM pass to deepest<br/>probed layer, no text]
  L --> S[Code-token states,<br/>4 layers x 1,024 dims]
  S --> A[8-head attention probe]
  A --> R[Risk score, median threshold]

1. An LLM agent workflow builds a threat model once per project. It covers attacker-controlled inputs, trust boundaries, and each module's role, sinks and guards.
2. For each function, tree-sitter pulls call sites, callee signatures, and file-level includes, types, macros and siblings. Semgrep findings are added. This repository context exists for 84% of training functions. The rest get only the function and its Semgrep findings. The median prompt is 3,840 tokens, 601 of them code.
3. The LLM runs only up to the deepest of four evenly spaced probed layers (38% to 85% of depth, chosen by cross-validation). Later layers are skipped and no tokens are generated.
4. Only the code tokens' hidden states are kept. Each is normalized, projected to 1,024 dimensions and concatenated across the four layers.
5. The probe is trained on PrimeVul training functions. Its loss combines per-function classification with a ranking term that pushes each vulnerable function above its patched twin. Pairs are used only in training. At inference each function is scored alone.

Evidence

Test accuracy (each function scored independently), same LLM for every method:

• GLM-5.3-Flash: REVA 60.0% (CI 57.8–62.2), VulTriage 59.2%, VulAgent 52.8%. The gap to VulTriage is not significant (CI −2.1 to +3.8).
• Qwen3.8-27B: REVA 59.5%, VulTriage 53.9%, VulAgent 54.5%.
• Muse-Glimmer-30B: REVA 58.9%, VulTriage 53.6%, VulAgent 52.2%.
• An ensemble of Muse-Glimmer-30B and GLM-5.3-Flash reaches 60.7%.

REVA's FPR is 40.0–41.1%. VulTriage's is 46.0–75.6%. REVA reverses labels on 1.8–3.0% of pairs, against 6.9–8.3% for VulTriage. Its accuracy varies by 1.1 points across LLMs, against 5.6 for VulTriage.

On the same prompt, asking the LLM for a Yes/No verdict scores 7.4 to 8.7 points below the probe on the test set. Semgrep, Cppcheck and Flawfinder alone score 50.8–51.3%, which is chance.

Ablations were run on the Muse-Glimmer-30B validation set. Mean pooling instead of attention costs 4.2 points. A single layer, no standardization, or no ranking loss each cost 0.4–1.5 points, none significant. Dropping repository context costs 1.0 point, and using the function alone costs 1.9 points. Neither is significant. Holding out whole projects lowers CV accuracy by 1.3–2.7 points.

How to use it

REVA needs hidden states, so it cannot run behind an API. It needs self-hosted open weights. The LLM and Semgrep rules load once at startup, and threat models are built ahead of time.

With Muse-Glimmer-30B on one H200, a function takes 0.93 s, of which 0.46 s is context extraction. At $3.50/hour for a rented H200, 1,000 functions cost $0.9–$1.3. The same work costs $9–$61 with VulTriage and $63–$307 with VulAgent at public API rates.

A probe must be trained on labeled functions. The page does not mention released code, weights or a licence.

Limits

• Test accuracy is about 6 points below cross-validation (60.0% vs 66.2% on GLM-5.3-Flash). The authors attribute this to PrimeVul's chronological split shifting toward newer code.
• Only about 22% of pairs have both versions labeled correctly, and roughly 40% of benign functions are still flagged.
• The evaluation covers only PrimeVul's C/C++ functions, which are short. Context showed no significant benefit there. The authors expect it to matter more for interprocedural bugs, but that is untested.
• Agentic times were measured under parallel load and overstate single-request latency. Functions with no verdict count as benign.
• On GLM-5.3-Flash the agentic baselines ran in FP8 while REVA ran in BF16 (CV 66.2% vs 65.9%).
• VulTriage's published 63.6% on GPT-4o could not be replicated.
• Comparisons on larger LLMs and calibrated escalation inside coding agents are listed as future work. The fetched text is cut off partway through that section.