CellTrack-Bench: Microscopic Cell Tracking Benchmark for Vision-Language Models

What it claims

CellTrack-Bench (Biohub, arXiv October 2026) recasts cell tracking as a division of labour. Algorithmic trackers handle the "mechanical" bulk of links, and a vision-language model handles the "cognitive" cases: divisions, cells near the field-of-view edge, and state changes. The benchmark tests that role with three tasks on the same local frame-to-frame transition. In cell linking, the model predicts correspondences directly. In track repair, it gets a tracker's edge set and returns links to add and remove. In track inspection, it flags tracker decisions for human review without editing them. Segmentation is given, so only association is scored.

How it works

The source data is 10 Cell Tracking Challenge (CTC) datasets plus 5 bacterial datasets (35 sequences). Sequences are split before any mining to avoid temporal leakage. For CTC data, sequence 01 is validation and 02 is test. Bacterial datasets are split into identifier-ordered halves. All design choices, including the harness, were made on validation only.

Hard cases come from errors shared by three trackers with different inductive biases. Each tracker is trained on two of three 5-dataset folds and evaluated on the held-out fold, using the provided masks:

• Hungarian matching on centroid distance: CTC LNK 0.888.
• Trackastra (transformer over segment tokens): LNK 0.963.
• HOCT (higher-order, cells and edges as tokens): LNK 0.946.

Links are ranked by how many trackers got them wrong (0 to 3). Ties are broken by hardness h(e), the mean error of the HOCT and Trackastra link scores. The Hungarian tracker counts toward the error tally but not toward hardness, because its cost is not a calibrated probability. The top K links per dataset are kept: K=64 for the hard subset and K=256 for the extended set. Each problem is a compact neighbourhood of 5 source cells containing the error, plus their candidate targets. The candidate graph includes artificial appear and disappear nodes.

Evidence

The paper tests whether tracker errors are correlated before relying on shared failures. All three trackers fail together on 0.28% of 1.5M candidate links (4,222 links). Independence would predict 0.0022%. Given one tracker's error, another tracker is 11.3 to 49.6 times more likely to make the same error. The Hungarian baseline's LNK of 0.888 shows that proximity alone solves most links. This motivates the benchmark's proximity-debiased metrics, which score only associations a nearest-neighbour rule would not already get right.

The primary metric is dataset-macro-averaged per-item F1. The protocol also reports division-specific measures, a CTC-comparable graph score, dataset-stratified results, and the output-token budget. Harness variables studied on validation are cell presentation, coordinate format, multi-turn interaction, and segmentation-outline hints.

How to use it

Code, stored model outputs, and a pipeline that regenerates the benchmark from the source datasets are at https://github.com/royerlab/celltrack-bench. Rebuilding the benchmark requires the CTC and bacterial source datasets. The stored outputs allow re-scoring without re-running the 22 models.

Limits

The fetched text ends partway through the metrics section. It contains no VLM results, no harness ablation outcomes, and no failure analysis, so it does not show how any of the 9 closed-source or 13 open-weight models scored. The introduction says 13 of the 15 datasets contribute problems, without saying in this excerpt which two are excluded. Problems are local, adjacent-frame transitions rather than full time-lapse sequences, so scores do not measure end-to-end tracking. Detection quality is out of scope because ground-truth segmentation is supplied. The authors describe the benchmark as a diagnostic, not a new tracker or training method.