Skip to content

Back to the catalog

When Should LLMs Abstain? Chain-of-Self-Questioning for Selective Risk Control
arxiv.orgpaper

When Should LLMs Abstain? Chain-of-Self-Questioning for Selective Risk Control

This paper introduces Chain-of-Self-Questioning (CoSQ), a prompt-only framework that conditions an LLM's answer commitment on an explicit self-assessment of whether it has sufficient factual support to answer. Tested across eleven open-weight and hosted model families on the 817-item TruthfulQA multiple-choice set and a secondary Natural Questions Short-Answer evaluation, the best variant (Grounded-CoSQ at τ=0.90) cuts the mean wrong-commitment rate by 32.1% relative to chain-of-thought prompting while also raising answered accuracy, holding consistently across all models and thresholds tested.

TL;DR A prompt-only self-questioning technique lets LLMs decide when to abstain rather than confidently hallucinate, cutting wrong-commitment rates ~32% on TruthfulQA across eleven models.
Open original ↗
Document

What it claims

CoSQ is a prompt-only wrapper that makes an LLM commit to an answer only after an explicit self-assessment of whether it has the information needed. The headline configuration, Grounded-CoSQ at τ=0.90, cuts the mean unconditional wrong-commitment rate from 13.1% under chain-of-thought prompting to 8.9% (the 32.1% relative reduction the summary cites), while raising answered accuracy from 86.9% to 89.7% and still answering 87.6% of questions. Both gains hold across all eleven models and every threshold tested.

How it works

Three variants trade coverage against reliability, giving neighboring operating points on the same abstain curve:

  • Grounded-CoSQ (τ=0.90): 87.6% coverage, the reliability/accuracy sweet spot above.
  • Critical-CoSQ: 88.6% coverage — answers slightly more, still beats the baseline.
  • Adaptive-CoSQ: 86.5% coverage — most conservative of the three.

All three remain more reliable than the CoT baseline; the threshold τ is the knob you tune for how costly an unsupported commitment is versus a referral. This is the detail the summary flattens: it is not a single method but a tunable family, so an adopter picks the variant/threshold matching their abstain economics.

Evidence

  • Primary: 817-item TruthfulQA multiple-choice validation set, seventeen conditions evaluated, final balanced-option protocol (option balancing matters here — TruthfulQA MC is sensitive to position/option artifacts, so the "balanced" qualifier is load-bearing).
  • Numbers above are means across eleven open-weight and hosted model families.
  • Secondary: Natural Questions Short-Answer, described only as "convergent open-form evidence" — no numbers given for it on this page.

Limits

  • Abstract only on the fetched page: no per-model breakdown, no NQ figures, no example prompts, no code or data links surfaced, and the eleven model families are not named. You cannot verify the per-model consistency claim from what is shown.
  • Absolute reliability is modest: even the best variant still wrong-commits at 8.9%, so this reduces but does not eliminate confident errors.
  • The framing assumes an unsupported commitment is more costly than referral/review — the gains are irrelevant if you cannot act on abstentions.
  • TruthfulQA MC and NQ-Short only; no reasoning, coding, or long-form generation tested.

Organisations named here

  • Natural Questions
  • TruthfulQA

Techniques named here

  • Chain-of-Self-Questioning
  • Grounded-CoSQ
  • Critical-CoSQ
  • Adaptive-CoSQ

Related in research