Who Is Sycophantic to Whom? Confident LLMs, Human Deference, and the Collapse of Trust Calibration

What it claims

HARICHAN (Human–AI Reliance in CHAt iNteractions) finds that learners accept a confidently stated falsehood about 65% of the time, and that this rate stays the same at every falsification level. Users do not become more sceptical as the assistant gets worse. Their confidence also stops separating right answers from wrong ones. The authors (Deepthika S., Pawni Yadav and B. Sankar, IISc Bangalore, October 2026) call this human-side deference, the mirror image of model sycophancy. They argue that sycophancy evaluations should measure both directions, and that trustworthiness belongs to the human–AI pair rather than to the model alone. Their design proposals are to make reliability legible per claim, to route verification to sources other than the assistant, and to measure trust through behaviour rather than ratings. They explain the effect with a "calculator fallacy": users give a fluent, probabilistic system the trust that deterministic tools earn through verification.

flowchart TD
  A[Learner doubts an answer] --> B{Can output be checked?}
  B -->|Deterministic tool| C[Compare to expectation]
  C --> D[Trust calibrates]
  B -->|Conversational AI| E[Ask the same assistant]
  E --> F[Confident restatement]
  F --> A

How it works

Each participant chatted with five assistants, all built on GPT-5-mini, one per science topic. Each assistant's briefing had 0, 25, 50, 75 or 100% of its facts replaced with plausible falsehoods. Every assistant spoke with full confidence. After each topic came a surprise quiz. Each item offered four options (the correct fact, the planted falsehood and two distractors) and asked for a confidence rating.

The researchers coded every transcript for which facts the assistant actually stated. This separates exposure to a falsehood from acceptance of it. The Human Deference Index adapts baseline-corrected sycophancy metrics from model evaluation to the human side.

Evidence

The study had 32 participants, 104 quizzes and 1,040 answers.

• Scores and choices. Mean scores fell from 7.4 to 2.9 out of 10. Participants picked the planted option on 60% of falsified items and on 11% of correctly taught ones (odds ratio 12.5).
• Confidence. Absorbed falsehoods were held at a mean confidence of 8.4/10. The modelled confidence advantage of right over wrong answers disappeared. The AUROC of confidence for correctness fell from 0.70 to chance.
• Trust ratings. Post-task trust ratings barely moved across reliability levels.
• Checking behaviour. Participants returned to the chat more when the assistant was worse, but returning went with more absorption, not less. About 30% of all turns pasted quiz questions back to the assistant being checked.
• The model's side. The assistant was almost never sycophantic and never conceded under pushback.

Limits

The fetched text covers the abstract, introduction and related work only. The results tables, the Human Deference Index formula, the statistical models and the authors' own limitations section are not available here.

The design itself sets some bounds. The sample is 32 people. The tasks are science topics only. Only one model was used, GPT-5-mini, prompted to be fully confident. Each participant completed up to five topics, not all five, since 104 quizzes is fewer than 32 × 5. The study does not test whether hedging or uncertainty display would change the 65% adoption rate.