LLM2Jev: LLMs Are Already Jev-Style Decision Models -- When and How to Fine-Tune Them

What it claims

The core claim is that a frozen causal LLM can already serve as a Jev-style decision model: it returns a categorical distribution over K predefined options without generating free text. The trick is to read the next-token probabilities over bracketed numeric IDs.

The paper frames this against community models on the public JevBench leaderboard, all built on Qwen3.5-4B:

The fine-tuned Hopper and reflex land near the frozen SemIf readout, which is the gap the paper investigates.

How it works

Prompt. The prompt uses the default chat template with thinking disabled:

State: {s}
Options:
[1] {o_1} . . . [K] {o_K}
Select one option. Answer only with its bracketed numeric identifier.

The assistant turn is prefilled with Best answer: [. Option k corresponds to the completion k].

No cap on option count. The closing bracket makes the suffixes prefix-free (1] is not a prefix of 12]). That removes the 16/26-option cap of letter readouts.

Trie depth depends on the tokenizer.

• Qwen splits digits, so the trie depth is at most ⌊log₁₀K⌋+2.
• Phi-4 groups digits, so any integer up to 999 is one token and the depth is 2.

Training-free scoring. Each candidate's score sₖ is the sum of log-probabilities over its suffix tokens. The scores are softmaxed over the K candidates. There is no temperature scaling and no permutation averaging.

Parallel suffix scoring. The prompt prefix is prefilled once and its KV cache reused. Candidate suffixes are teacher-forced in parallel, batched by length. A 151-way task needs one prefill plus at most 3 short forward passes on digit-level tokenizers, or 1 on multi-digit tokenizers.

Joint versus isolated scoring. All options sit in one prompt, so each is scored in context of the full slate. Per-option scorers lose that context: jev-local with frozen Qwen3.5-9B gets 74.9% on public JevBench, against 81.4% for listwise readouts on Qwen3.5-4B.

Fine-tuning (full or LoRA). The method uses one trained loss plus three anchors:

• L_tree: the negative log-likelihood of the gold candidate under probabilities renormalized over the legal tokens at each trie node. It is computed only along the gold path, so it stays cheap at K=151.
• L_mass (KL anchor): keeps the total legal-vs-illegal probability at trie nodes close to the frozen base model.
• L_out (KL anchor): keeps the distribution over illegal tokens at trie nodes close to the base model. Both L_mass and L_out are applied at gold-path nodes and 3 sampled off-path nodes.
• L_pos (KL anchor): keeps the full next-token distribution close to the base model at the reply start and after the answer.

The anchors exist because L_tree alone can suppress the absolute probability of valid output formats without being penalized.

During training, option order is randomly shuffled so the numeric IDs carry no fixed meaning. The paper reports this improves consistency under cyclic option rotations on MMBench.

Evidence

• Models and setup: Qwen3.5-4B and Qwen3-0.6B, six training data compositions, ablations over anchor strength and LoRA vs full fine-tuning.
• Evaluation: external benchmarks, public JevBench, general capability, multimodal image benchmarks, and conversational behavior.
• Calibration: reported as ECE and Brier score on raw distributions, with no post-hoc temperature.
• Truncation: the fetched text cuts off in the method section. LLM2Jev's own result tables are not included, beyond the 81.4% listwise figure above.

How to use it

No code, weights or licence are given in the fetched text. The recipe is fully specified by the prompt, the prefill and the scoring rule above, so it can be applied to any causal LLM without architecture or tokenizer changes. Per the paper:

• Fine-tune only when the training data covers a known weakness of the base model, such as a small backbone or many-option intent routing.
• For strong models, use LoRA with tight anchors.
• For small models, use milder regularization.

Limits

• Public items only. All JevBench numbers are diagnostic, from the 231 public items (48 easy, 72 standard, 111 hard). They are not official leaderboard standings, since the held-out and sealed sets are private.
• Position bias remains. Joint scoring is sensitive to option order. Inference-time permutation averaging is deliberately omitted for cost, so some position bias remains, especially training-free.
• Fine-tuning can hurt. Applied indiscriminately to strong models on broad benchmarks, fine-tuning gives diminishing returns and can cause negative transfer.
• Narrow testing. Only two Qwen backbones are evaluated.