Reva: Real-Time Vulnerability Detection through Contextual Probing of Large Language Models

What it is

REVA (REal-time Vulnerability Analysis) is a function-level vulnerability detector from Amine Lbath and Massih-Reza Amini (Université Grenoble Alpes, CNRS), posted to arXiv in October 2026. It runs one forward pass of an open-weight LLM over a prompt containing the function plus security context, generates no text, and feeds the hidden states of the code tokens to a small trained probe that outputs a risk score. It is a preliminary result on C/C++ code only.

How it works

The prompt has labeled sections: a per-project threat model (built once ahead of time by an LLM agent workflow), Semgrep findings, call sites and callee signatures, and the file's includes, types and macros (extracted with tree-sitter). The median prompt is 3,840 tokens, 601 of them code. Repository context exists for 84% of training functions; otherwise the prompt holds the function and Semgrep findings.

The pass stops after the deepest of four probed layers, evenly spaced from 38% to 85% of model depth and chosen by cross-validation. The code-token states are normalized, projected to 1,024 dimensions and concatenated across layers. An 8-head attention probe (0.57 M parameters) pools them. Training combines a per-function classification loss with a ranking loss that puts each vulnerable function above its benign twin. Pairs are used only in training; at test time each function is scored alone.

Preconditions / Affected

• Needs access to hidden states, so it cannot run behind a hosted API. The authors ran it on a rented H200 GPU.
• Needs a probe trained on labeled functions (PrimeVul's training set here) and a precomputed threat model per project.
• Timing was 0.93 s per function on Muse-Glimmer-30B, 0.46 s of which was context extraction.

Results

On PrimeVul's test set (435 pairs), accuracy is 58.9% (Muse-Glimmer-30B), 59.5% (Qwen3.8-27B) and 60.0% (GLM-5.3-Flash). Agentic baselines run on the same LLMs score as follows:

On GLM-5.3-Flash the 0.8-point lead over VulTriage has a 95% CI of [−2.1, +3.8], so it is a tie. REVA is significantly ahead in five of six same-LLM comparisons. It has a lower false positive rate than VulTriage on every LLM (40.0% to 41.1% versus 46.0% to 75.6%).

Cost per 1,000 functions is $0.9 to $1.3 for REVA, $9 to $61 for VulTriage and $63 to $307 for VulAgent at public API rates. Running the probe on two LLMs together reached 60.7% at 1.78 s.

Ablations on the Muse-Glimmer-30B validation set:

• Asking the same LLM for a Yes/No verdict on the same prompt falls to chance, 7.4 to 8.7 points below the probe on the test set.
• Semgrep, Cppcheck and Flawfinder alone score 50.8% to 51.3%.
• Mean pooling instead of attention pooling costs 4.2 points.
• Dropping repository context costs 1.0 point and keeping only the function costs 1.9; neither is significant.

Caveats

• Absolute accuracy is about 60%, with a roughly 40% false positive rate. It is a screening signal, not a verdict.
• Test accuracy is 6 points below cross-validation (60.0% versus 66.2% on GLM-5.3-Flash). The authors attribute this to PrimeVul's chronological split and a shift toward newer code.
• With no project shared between training and evaluation folds, cross-validation accuracy drops only 1.3 to 2.7 points.
• The repository context did not help measurably on PrimeVul's short functions. The authors expect it to matter for interprocedural bugs but did not show it.
• VulTriage's published GPT-4o figure of 63.6% could not be replicated by the authors, so the comparison uses open LLMs.
• Evaluation covers PrimeVul only and three open models; larger LLMs are listed as future work. The text provided cuts off partway through the future-plans section.