Detecting and Understanding Ambiguity in Visualizations with Role-Playing Vision Language Model Interpretations

What it claims

Chan, Kuo, Lee, Tang and Ma (University of California, Davis) present a persona-conditioned VLM pipeline for interpretive ambiguity. Interpretive ambiguity is the potential for one chart to support substantively different claims about its visible patterns in response to the same question. The paper appears in IEEE Transactions on Visualization and Computer Graphics (arXiv v1, October 2026). The authors frame the output as diagnostic. An ambiguity profile does not declare a chart ambiguous or not. It also does not estimate how often real audiences would read it a given way.

How it works

Comparison happens at level L3 of Lundgard and Satyanarayan's four-level model: chart-grounded claims about extrema, outliers, correlations and comparisons. An L4 takeaway is a persona's full contextual answer and may contain several supporting L3 statements. A statement pair is two L3 statements from different personas for the same chart and question.

The four stages:

1. Chart profiling. A VLM writes an L1 description (title, type, axes, variables, encodings), a one-phrase topic, and selected L3 tasks: distribution, clusters, correlation, trends or anomalies. It produces one open-ended question per task.
2. Persona selection. The pool is 194,728 filtered PersonaHub descriptions. Topic and persona embeddings are compared, and the relevance cutoff is set per chart by an LLM judge with binary search. Candidates are then clustered, and the personas closest to the centroids are kept for diversity.
3. Chart interpretation. Chart type routes each chart to the VLM that scored best on the authors' comprehension benchmark for that type. Each persona answers each question. Every L3 statement is tagged with Target, Pattern and Scope.
4. Statement-pair labeling. A "labeling council" of LLM judges labels each pair Compatible, Disjoint or Opposing. The per-question label distribution is the ambiguity profile.

flowchart LR
  A[Chart image] --> B[Chart profile: topic, questions]
  B --> C[Persona retrieval + diversity]
  B --> D{Route by chart type}
  C --> D
  D --> E[Per-type VLM role-plays persona]
  E --> F[L3 statements with TPS]
  F --> G[Labeling council]
  G --> H[Ambiguity profile]

The labels derive from NLI. Neutral was renamed Disjoint to stress shared grounding in one chart. Contradiction was broadened to Opposing to cover conflicting higher-level framing. Equivalent and Entailment were merged into Compatible after qualitative analysis, because the pairs have no premise-hypothesis direction. Labels describe semantic relationships, not factual correctness. Correct chart reading is therefore a prerequisite the pipeline does not itself check.

The workflow is described as model-agnostic. The page text does not name the VLMs selected.

Limits

The authors state that personas are diagnostic probes, not stand-ins for human participants or representative samples. They cite Hu and Collier, who found that persona prompting helps little when persona variables explain little human variation. Only Disjoint and Opposing pairs signal potential ambiguity, and those signals are meant for follow-up with real audiences.

The fetched text ends partway through Section 3.3.1. It contains no VLM benchmark scores, no agreement figures against the human annotations, no chart corpus size, and no case-study results. Persona filtering details are in the supplemental material, and no code or release location appears in the excerpt.