M-PFT: Angular Margin for Calibrated Prompt-Tuning of Fine-Tuned Vision-Language Models

What it claims

M-PFT is a training-time regularizer for CLIP-style prompt tuning. It adds two angular losses to cross-entropy so that prompt-tuned models stop being underconfident on base classes and overconfident on novel classes. The authors attribute the two failures to two geometric distortions. Embedding collapse lowers θjk, the angle between tunable class embeddings, and drives base underconfidence. Semantic drift raises θj, the angle between a tunable embedding and its zero-shot counterpart, and drives novel overconfidence. The method enlarges the margin θjk − θj. The paper is dated October 2026, and the authors are from MBZUAI, the University of Moratuwa and Albukhary International University.

How it works

Prompt tuning follows CoOp. M learnable context tokens are prefixed to each class-name embedding, the text encoder produces φk, and only the tokens are trained on few-shot base-class data. The total loss is L_CE + L_min-max + L_anchor:

• L_min-max minimizes the maximum pairwise cosine similarity among tunable class embeddings, which raises θjk.
• L_anchor minimizes the angle between each φk and the frozen zero-shot embedding zk from the template "a photo of a [CLS]", which lowers θj.

The authors claim the two terms produce no opposing gradients, unlike the statistical auxiliary losses in TCPT and zero-shot normalization methods (Tab. 4, Fig. 4). The page also contrasts M-PFT with test-time calibration methods (C-TPT, A-TPT, AGI-TPT), which add inference-time cost. KgCoOp, cited in related work, already pulls learned prompts toward hand-crafted text embeddings. The fetched text does not say how L_anchor differs from it beyond being angular.

Evidence

The motivating analysis (Fig. 3) covers 10 datasets: Caltech101, DTD, EuroSAT, Aircraft, Food101, Flowers102, OxfordPets, StanfordCars, SUN397 and UCF101.

• Base classes: ECE rises with the mean cosine similarity between tunable class embeddings, with linear fit ECE = 5.41x − 1.42.
• Novel classes: ECE falls as tunable embeddings stay closer to zero-shot, with linear fit ECE = −54.94x + 46.13. This is the strong signal.

The authors concede that the cross-dataset base correlation is weak. They cite Tab. 4, where raising θjk brings base ECE to 2.49, but the fetched text does not give the comparison value.

Fig. 1 compares M-PFT with TCPT and DAC using CoOp on 10 fine-grained datasets. It scores each method as 1 − ECE/ECE_worst, so results are relative to the worst method rather than absolute ECE. The other baselines named are CAC, DOR and margin-based zero-shot normalization.

Limits

The fetched text stops at the start of Section 3.2. It omits the exact loss formulas, the loss weights, the backbones, the per-method and per-dataset results tables, the accuracy numbers, and the overhead measurements. The code link is mentioned but no URL appears. The nine prompt-tuning methods tested are not listed in the fetched text, though related work discusses CoOp, CoCoOp, KgCoOp, ProDA, ProGrad, MaPLe, PromptSRC, HiCroPL and MMRL. The claim of "comparable accuracy" cannot be checked from this excerpt. On base classes, the claimed result is "competitive" calibration, not a consistent improvement.