JevOut: Natural Context Can Flip Decision Models

What it claims

The paper introduces JevOut, a probability-guided optimizer that redirects "decision models" (systems mapping language to a distribution over finite typed options) from a correct answer to a pre-chosen wrong option, using only short, fluent context additions that leave source, question, choices, and gold answer intact. On Jev it flips 312 of 508 initially-correct decisions (TFR 61.4%) within 64 accepted target evaluations; 229 of those reach ≥0.7 probability on the fixed wrong option. Successful contexts add a median of 31 words. Three other systems — OpenSourceJev, Von, and a plain Qwen scorer — hit TFRs of 64.9%–73.2%.

How it works

For each initially-correct item, the target is fixed as the most probable wrong option (Eq. 1), then held constant. The optimizer maximizes a log-probability margin between the target and its strongest competitor, so it gets a graded signal before the selected option actually changes.

flowchart TD
  A[Initially correct item] --> B[Fix most-probable wrong option as target]
  B --> C[Proposer: add one fluent sentence + insertion point]
  C --> D{Constraint check:<br/>natural fit, answer preserved,<br/>no explicit selection cue}
  D -->|reject, no target call| C
  D -->|accept| E[Target model returns probabilities]
  E --> F[Score by target margin, archive]
  F -->|Gibbs-weighted resample<br/>+25% restarts from original| C
  F --> G{Target selected?}
  G -->|yes| H[Targeted flip]

Budget is 16 proposal slots × 4 iterations, capped at 64 accepted target evaluations. A separate coherence/answer-preservation check runs without seeing the target or its scores. Two training recipes (V1: next-addition; V2: whole successful context) fine-tune the proposer on development trajectories to generate flipping contexts directly.

Evidence

• Optimization vs. single-shot: one target-aware addition flips 16.9% of Jev decisions; a neutral addition flips 2.2% — the iterative optimizer is what uncovers the 61.4%.
• Probability vs. label feedback: on a matched 140-item subset, using probabilities to allocate proposals beats label-only by 7.1 points at the same 64-eval budget.
• Transfer: frozen contexts redirect other models without refinement; proposer training improves direct generation on held-out Jev items with no iterative feedback.
• Data: seven datasets — MMLU-Pro, SuperGPQA, MuSR, ToMBench, LAR-ECHR, SATA-Bench (multi-answer), BFCL V4 (tool routing) — each 50 dev + 100 held-out (350 dev / 700 eval total). Primary target is Jev 1.13.0; OpenSourceJev runs on Qwen3-1.7B Q8.

How to use it

Code at https://github.com/xzx34/JevOut, homepage https://xzx34.github.io/jevout/. If you consume a decision model's probability distribution to route requests, pick tools, or trigger actions, this is a direct attack surface: an upstream component that appends "background" or "retrieved evidence" can steer a high-confidence wrong choice without touching the query.

Limits

Single-author preprint (USC, arXiv 2026-09-24); the fetched text cuts off mid-Setup, so full hyperparameters, ablations, and per-dataset breakdowns aren't visible here. Attacks assume access to the model's option probabilities (a white-box-ish signal) and a proposer/base model to generate and vet additions. No licence is stated on the page. Note the model names (Jev, Von) appear to be paper-specific labels — verify what real systems they map to before citing.