CircleClar

What Is a Clarification?

Estimating Model-Dependent Input Ambiguity

Jingqin Tang*1Tim Tomov*1,2,3Sebastian Schmidt1Dominik Fuchsgruber1,2Stephan Günnemann1,2,3

1School of Computation, Information & Technology, Technical University of Munich2Munich Data Science Institute3Munich Center for Machine Learning

*Equal contribution

LP4FM Workshop @ NeurIPS 2026

Paper Code BibTeX

Abstract

In shortWhether a question is ambiguous depends on the model answering it: a model can only face the interpretations it knows. We use clarifications to find out which interpretations a particular model knows: we define what makes a clarification valid, prove that valid clarifications correspond one-to-one to valid answers, and build CircleClar, a black-box method that keeps an answer only if the model returns to it through an answer → clarification → answer cycle.

Read the full abstractHide the full abstract

Ambiguous inputs can admit multiple interpretations, each potentially yielding a different answer. Existing work largely treats this ambiguity as a property of the input, yet for a particular language model, only some alternatives may be known and answerable. We therefore distinguish world ambiguity, capturing all alternatives admitted by an input, from model ambiguity, capturing those accessible to the responding model. To characterize these alternatives, we formalize natural-language clarifications through three desiderata: uniqueness, faithfulness, and answer equivalence. We show that these desiderata induce a one-to-one correspondence between valid answers and equivalence classes of clarifications, implying that any interpretation can be represented either by its answer or by the clarifications that uniquely express it. We leverage this correspondence to introduce CircleClar, a black-box method that estimates the set of answers available to a model by checking consistency along an answer–clarification–answer cycle. Across open-domain QA and executable Text-to-SQL, CircleClar achieves strong recovery of these answer sets. In QA, the recovered support yields the strongest overall performance in model-dependent ambiguity detection and enables effective selection among answering, clarifying, and abstaining. Together, these results demonstrate the practical value of a model-dependent view of ambiguity and its characterization through an answer-clarification correspondence.

Overview

Figure 1 from the paper

Ambiguity is usually treated as a property of the input. We study the ambiguity a model actually faces, given what it knows.

The world answer support is fixed by the input, while the model-realizable support depends on the model’s knowledge. Switch between models to see how the same question becomes more or less ambiguous.

x“Who voiced the snake in The Jungle Book?”

World answers W(x)

  • y₁Sterling Holloway1967 animated film
  • y₂Scarlett Johansson2016 live-action film
  • y₃Joseph J. Terry2010 animated series

World ambiguity
|W(x)| = 3
Model ambiguity
Appropriate response

Results

We evaluate four models (Qwen3-4B, Qwen3-30B-A3B, GPT-5.4-mini, GPT-5.5) on open-domain QA, using 1,000 questions from AmbigQA and AmbigQA-HQ, a manually curated set of 100 questions with more reliable annotations, and on executable Text-to-SQL with 96 requests from AMBROSIA. The reference for each model is its own realizable answer set M(x): the valid answers it reliably reproduces when given their ground-truth clarifications.

Recovering model-realizable answers

Set F1 ± std. / EM (%) · best · second-best

What we measure. How closely each method recovers the model’s answer set M(x). Per question, set F1 scores the overlap between the predicted and the reference set; exact match (EM) counts questions where both sets are identical. Baselines: ICE with a threshold oracle-tuned per model and dataset, Sampling-50 (50 samples), and 1-Enumeration (asking once for all plausible answers). CircleClar uses a fixed τ = 0.6.

Swipe to see all models, or tap the table to enlarge →

Table 1. Mean per-question set F1 with standard deviation and exact set match for ICE, Sampling-50, 1-Enumeration, and CircleClar on AmbigQA-HQ, AmbigQA, and AMBROSIA, across four models. Green and orange mark the best and second-best scores. View full size

CircleClar performs strongly across datasets, achieving particularly large gains over baselines on smaller models and on the curated AmbigQA-HQ benchmark. Most of the gain comes from precision: the consistency check removes candidates whose cycle does not close, like Neel Sethi in the method demo. As models grow, they propose fewer invalid candidates and the gap to simple enumeration narrows.

See the paper for experimental setup and consistency-filtering analysis.

Detecting model-dependent ambiguity

What we measure. Whether a question is ambiguous for this model, i.e. whether it realizes at least two answers, |M(x)| ≥ 2. CircleClar’s score is simply the size of its estimate |M̂(x)|; we compare against ICE, CLARA-D, INTENT-SIM and APA† using AUROC.

AUROC (%) · 50 is chance · bold: best per row

AUROC heatmap on AmbigQA-HQ and AmbigQA comparing CircleClar, ICE, CLARA-D, INTENT-SIM, and APA. CircleClar has the highest AUROC in all four HQ settings and two of four AmbigQA settings. View full size

CircleClar is strongest overall; its clearest gains occur on AmbigQA-HQ. Most baselines stay close to chance: they target world ambiguity |W(x)|, which is not what the model faces, as the model switcher illustrates.

Answer, clarify, or abstain?

What we measure. The estimated set directly suggests a response: abstain if it is empty, answer if it holds one answer, clarify if it holds several. We report balanced accuracy over the three actions, comparing CircleClar with direct generic and model-dependent decision prompts and BAG, which bases its decision on previously sampled responses.

Balanced accuracy (%) · error bars: 95% bootstrap CIs · dashed line: constant policy

Balanced accuracy of three-way response selection for four models on two QA datasets. CircleClar is compared with BAG and direct model-dependent and generic decision prompts, with 95% bootstrap confidence intervals and a constant-policy baseline. View full size

CircleClar leads in most settings when deciding whether to abstain, clarify, or answer: deriving the action from the recovered set is more reliable than predicting it in one shot. It is the same rule that Models A–D follow in the problem demo.

BibTeX

@misc{tang2026clarification,
  title  = {What Is a Clarification? Estimating Model-Dependent Input Ambiguity},
  author = {Tang, Jingqin and Tomov, Tim and Schmidt, Sebastian and
            Fuchsgruber, Dominik and G{\"u}nnemann, Stephan},
  year   = {2026}
}

Scroll to inspect the figure. Press Escape to close.

Open original PDF