Skip to content

Back to the catalog

LineupRL: Verifiable Reinforcement Learning for Time Series Captioning via Caption-to-Series Identification
arxiv.orgpaper

LineupRL: Verifiable Reinforcement Learning for Time Series Captioning via Caption-to-Series Identification

LineupRL is an RLVR (reinforcement learning with verifiable rewards) pipeline for training time series captioning models, where the reward comes from a frozen LLM verifier that must match a generated caption back to its source time series among distractors, rather than judging caption quality directly. The authors show a 3B vision-language model trained with LineupRL outperforms SFT and RL baselines across two captioning benchmarks plus downstream forecasting/reconstruction tasks, and even beats the 72B VLM whose captions the SFT baseline distills from, using 1/24 the parameters, while resisting reward hacking.

TL;DR A caption-to-series identification reward turns RLVR into an effective, reward-hacking-resistant trainer for time series captioning VLMs, letting a 3B model beat a 72B teacher.
Open original ↗
Document

What it claims

LineupRL trains a time series captioner with a reward that needs no reference captions. A frozen LLM is shown the caption and K real candidate series as raw numbers, and must pick the series the caption describes. A 3B VLM trained this way beats SFT and RL baselines on every metric across BEDTime and CaTS-Bench, and on caption-only forecasting and reconstruction. It also beats the 72B teacher that the SFT baseline distils from. The verifier is a 14B off-the-shelf LLM, because matching is easier than writing questions or grading captions.

How it works

flowchart LR
  S[Series x] --> R[Render line chart]
  R --> P[VLM captioner]
  P --> C[G captions]
  S --> D[8 nearest real series<br/>by mean/std/min/max,<br/>overlap ≤ ηT]
  D --> N[Redraw K−1 distractors each step]
  C --> V[Frozen LLM verifier<br/>raw numbers, no chart]
  N --> V
  V -->|x at each of K positions| A[Accuracy × 2 = reward]
  A --> U[RLOO update]
  U --> P
  • Captioner input. The captioner reads a rendered chart (axis ticks, no per-point labels), so input size does not grow with series length.
  • Verifier input. The verifier never sees the chart. The authors say a VLM's weak chart reading would add noise to the reward.
  • Distractors decide the reward's quality.
  • The distractors are the K−1 real series of the same length that are nearest in (mean, std, min, max), excluding any that overlap x on more than a fraction η of steps.
  • In training, distractors are redrawn each step from the 8 nearest admissible neighbours, so a caption cannot overfit one fixed set.
  • The obvious shortcut fails. Building distractors from x itself (shuffle, reverse, cut-and-paste) lets the verifier find x 72.5% of the time without reading the caption.
  • Position bias. The verifier is queried K times, with x at each position. The reward is r = 2·accuracy.
  • Why it resists hacking. Text addressed to the verifier changes no answer, and neither does generic description. A fabricated value that matches a distractor actively costs reward.
  • Optimiser. RLOO (REINFORCE leave-one-out) over G captions per series.

Evidence

  • SFT failure. Fine-tuning the 3B VLM on 72B-teacher captions does not raise entailment of reference captions on BEDTime. On CaTS-Bench it drops series-to-caption identification below the untuned model. The authors call this "ungrounded captioning": fluent captions that copy reference style but miss the series' features.
  • Evaluation axes:
  • one-directional NLI entailment against reference captions
  • caption→series and series→caption identification against fixed nearest-neighbour candidates, with a frozen LLM that may differ from the training verifier
  • a decoder that forecasts or reconstructs values from the caption alone, using a chronological train/test split
  • Qualitative. In a case study, the trained captioner describes the trend and states values at key points.

The fetched text ends partway through the evaluation protocol. No result tables or per-metric numbers are available here.

How to use it

The code is at https://github.com/HaochenZhang717/LineupRL-official-repo. The page gives no install commands, released weights or licence. To reproduce the method you need:

  • a VLM captioner
  • an off-the-shelf LLM verifier (the paper uses 14B)
  • a pool of real series of matching lengths to draw distractors from

Limits

  • Scope. The method is defined only for univariate series.
  • Missing details. The fetched text does not name the verifier or captioner models, and does not give K, η or the compute cost.
  • Metric overlap with training. The caption→series metric is the same task as the training reward, with a different verifier and candidate set. That makes it partly self-aligned. Weight the entailment and forecasting/reconstruction results more heavily.
  • One-directional entailment. Captions can say more than the reference without penalty, including incorrect extra content.
  • Results not checked here. The headline claims could not be verified against tables, because the text was truncated before the results section.

Technologies named here

Techniques named here

  • LineupRL
  • Caption-to-series identification reward
  • Supervised fine-tuning for captioning

Related in training