Fine-tuning a 350M Model for Better Structured Outputs in 100 GRPO Steps

What it claims

A recipe, not a released model. GRPO with a LoRA adapter on LiquidAI/LFM2.5-350M, using about 500 augmented Nemotron prompts for 100 steps, raises the IFStruct pass rate from 22.6% to 29.7% (2,000 samples). The largest gain is on bare-list outputs, which nearly double. YAML barely moves. The training fits a free-tier 16 GB Colab or Kaggle GPU. The authors say this is not the pipeline behind the RL model in the IFStruct release blog, and the post does not try to reproduce that model's score.

How it works

Data augmentation. The augmentation is aimed at the gap between the Nemotron data and IFStruct:

• 40% of prompts get a "return the output inside a fenced code block" instruction.
• A disjoint 20% are rewritten as top-level-array tasks: the schema is wrapped in an array with a required item count.

LoRA. Settings are r=16 and alpha=32. The targets include the LFM-specific modules out_proj, in_proj, w1, w2 and w3 alongside q/k/v_proj. That trains about 6M parameters, or 1.66% of the model.

Rewards. Three rewards, each scored in [0, 1], are combined as a weighted sum with weights 1.0 / 0.5 / 2.0:

flowchart LR
  C[Completion] --> F["format: 1.0 requested form / 0.2 wrong-but-parseable / 0 unparseable"]
  C --> N["field count: 1.0 exact, linear decay"]
  C --> S["schema validation: violation count, gated on required-key coverage"]
  F -->|x1.0| R[Weighted reward]
  N -->|x0.5| R
  S -->|x2.0| R

GRPO config. lr 5e-5, 10 warmup steps, 8 generations per prompt, batch size 4, gradient accumulation 8, steps_per_generation=2, max_completion_length=1024, temperature 1.1 to keep groups varied, beta (KL) 0.01. The LoRA is merged with merge_and_unload() before GGUF conversion.

Evidence

Both runs use the same llama.cpp stack with BF16 GGUF on an M5 Max MacBook with 36 GB, and the same 2,000 IFStruct test samples.

• Baseline reproduction. The locally measured base score is 22.6%, against the 21.1% reported in the IFStruct blog.
• Reference model. Qwen3.5-2B scores 33.15%, as reported in that blog. It was not rerun here.
• Error profile. "Unclosed code block" (317) and "missing code block" (170) drop out of the top errors after training. "Required field missing" rises slightly (7,228 → 7,331), and so does "wrong item count" (738 → 890).
• Gains are uneven by entity. interview_transcript_segment falls from 26.2% to 16.2%, and camera_review goes from 7.2% to 6.0%. Recipe (10.0%) and GPU review (7.4%) stay very low.

How to use it

The full pipeline is in the accompanying notebook. Evaluation uses uv, llama.cpp (installed via brew install llama.cpp) and the Liquid4All/ifstruct repo.

Serve the base model:

llama-server \
  -hf LiquidAI/LFM2.5-350M-GGUF:BF16 \
  -c 32768 \
  -np 4 \
  -ngl 99 \
  --alias LiquidAI/LFM2.5-350M \
  --host 127.0.0.1 \
  --port 8080

Convert the merged checkpoint:

git clone --depth 1 https://github.com/ggml-org/llama.cpp
pip install ./llama.cpp/gguf-py
mkdir -p models
python llama.cpp/convert_hf_to_gguf.py \
  PATH_TO_YOUR_MERGED_MODEL \
  --outfile ./models/lfm25-350m-grpo-bf16.gguf \
  --outtype bf16

Run the evaluation:

uv run ifstruct-eval \
  --model lfm25-350m-grpo-structured-output \
  --base-url http://localhost:8081/v1 \
  --api-key dummy \
  --dataset data/test.jsonl \
  --results-file results/lfm25-350m-grpo.json \
  --n-threads 4 \
  --max-tokens 2048 \
  -v

Limits

• Single run. Results come from one run with no seeds or variance reported.
• JSON-only rewards. The rewards target JSON, so YAML gains are negligible.
• Missing fields remain the main failure. The dominant error, required fields missing at over 7k occurrences, is essentially unchanged by training.
• Regressions. Some entity types get worse after training.
• Still behind the 2B model. The tuned model scores 29.7% against Qwen3.5-2B's 33.15%.
• No weights. The post publishes no trained weights or licence information.