RESEARCH / TECHNICAL REPORTS
← Technical reportsRESEARCH NOTE / 16

Data agents

Closing Information Gaps in Interactive Text-to-SQL

Strong models fail at interactive data tasks not because they cannot write SQL, but because they commit before acquiring the evidence. PASCAL makes acquisition a controlled, budgeted first step.

AI fundamentals

Our foundational AI research spans agents, language and data—general-purpose capabilities to be reused in space scenarios.

Foundation models · agents · data methods

THE CENTRAL IDEA

Do not make the model a better SQL writer; make it a disciplined evidence acquirer. Interpose a source-aware acquisition layer before SQL generation, schedule every query and question by cost-normalized priority, and let the model commit only when the acquired evidence is sufficient.

Closing Information Gaps in Interactive Text-to-SQL

The challenge: the bottleneck has shifted

In interactive text-to-SQL, the evidence needed to write a correct query is not given upfront. It is distributed across the database schema, external knowledge bases and the user, and must be acquired through environment interactions under a finite budget. The paper's central observation: strong LLMs often fail not from deficient SQL synthesis, but from premature commitment—they start writing SQL before acquiring task-critical evidence.

This reframes where capability work should go. As model synthesis ability keeps improving, the marginal failure moves upstream to acquisition: what to look up, who to ask, and when to stop gathering and commit. Teams that keep investing only in generation quality are optimizing the wrong term.

The Information Sufficiency Hypothesis

Through controlled trajectory analysis, the authors formalize the observation as the Information Sufficiency Hypothesis: given sufficient task-relevant evidence, current models are largely capable of producing correct SQL; failures concentrate where evidence is missing. They complement the hypothesis with a five-class gap taxonomy that structures the acquisition space—making "what is missing" an explicit, classifiable object rather than a vague intuition.

The taxonomy is the diagnostic deliverable: each failure can be attributed to a gap class, and each gap class implies a different acquisition action. Diagnosis before therapy—this is what turns an agent framework from a bag of prompts into an engineered system.

Three mechanisms of PASCAL

Guided by this diagnosis, the authors build PASCAL, a training-free acquisition-control data agent. First, a source-aware information-acquisition layer interposed before SQL generation: the agent explicitly distinguishes schema lookups, knowledge-base retrieval and user clarification as different evidence sources with different costs. Second, cost-normalized priority scheduling: candidate acquisition actions are ranked by expected evidence value per unit of interaction cost, so the finite budget is spent on the most decisive gaps first. Third, training-free composition: PASCAL wraps existing LLM backbones without fine-tuning, so the framework transfers across models as they improve.

The training-free property matters as much as the scheduling. An acquisition layer that requires fine-tuning would have to be retrained for every backbone upgrade; a control layer that lives outside the model inherits each new generation of models for free.

priority(a) = expected evidence gain(a) / interaction cost(a)

Results: a local 35B beats frontier models

On BIRD-Interact, PASCAL with a local 35B backbone achieves 51.0% Phase-1 and 37.0% two-phase success rate, outperforming the frontier-model leaderboard results by up to 14.7 percentage points. The comparison carries the paper's thesis in one line: a mid-size local model with disciplined acquisition beats frontier models that commit early.

For deployment, the 35B result is the headline: acquisition control is exactly the kind of capability that lets an on-premises or onboard model punch above its weight class, without sending data to a frontier API.

Where the gains concentrate

Gains concentrate on BI tasks requiring cross-source evidence acquisition—tasks where the answer depends on combining schema facts, external knowledge and user intent. Where evidence is fully present in the prompt, acquisition control adds little; where evidence must be gathered, it is decisive. The gains map is thus itself a guide: it tells adopters which workloads justify an acquisition layer and which do not.

A useful mental model: PASCAL does not raise the model's ceiling; it raises the floor of the evidence the model stands on. The two are easy to confuse in aggregate benchmarks and must be separated when planning adoption.

Applicability and boundaries

The framework presumes an interactive setting with a queryable environment: databases that can be probed, knowledge bases that can be retrieved, users who can clarify. Where the environment cannot answer—or answers unreliably—priority scheduling cannot manufacture evidence. The five-class taxonomy, likewise, is a structuring of observed gaps rather than a completeness proof; new deployment domains should expect to extend it.

The authors are careful not to claim a universal agent recipe: the evidence supports acquisition control for interactive data tasks on the evaluated benchmark and backbones, and the design principle—diagnose the gap, then schedule acquisition by cost-normalized priority—is the transferable part.

Engineering mapping and evaluation boundaries

Mapped to an operational data stack, the acquisition layer sits between the user's question and the generation model: it reads the gap taxonomy, prices candidate probes against the remaining interaction budget, and only releases the question to SQL generation when evidence is sufficient. For space-operations data agents—querying telemetry stores, mission databases and procedure manuals—the same discipline applies verbatim: acquire first, commit second, and never let a fluent answer outrun its evidence.

A concrete evaluation plan for adoption: instrument the target workload with the gap taxonomy and measure the current failure mix; deploy the acquisition layer in shadow mode, logging which probes it would have scheduled; then compare success rate and interaction cost per solved task against the incumbent pipeline, reporting per-gap-class outcomes rather than only pooled averages. Only after the acquisition policy survives workloads the taxonomy never saw should it take over the live path.

SOURCE PAPER & REFERENCE MATERIAL

When the Bottleneck Shifts: Diagnosing and Closing Information Gaps in Interactive Text-to-SQL

Mengyi Yan, Zhinian Ma, Yue Wang, Yaoshu Wang, Xiaoke Zhu, Zizhe Wang, Haoyi Zhou, Zhumin Chen

Proceedings of EMNLP 2026 (Main Conference) · ACL Anthology 2026.emnlp-main.465

Continue to the next report →