CS Paper Radar

Do the Numbers Match the Plot? Statistical-Visual Verification for Automated Exploratory Data Analysis

Zixun Fu, Jian Huang, Maojun Sun, Yancheng Yuan

cs.HC cs.CV ·submitted 2026-08-12 03:44 UTC · v1

Abstract

Automated exploratory data analysis (EDA) systems can execute statistics correctly while producing misleading interpretations. For example, correlations may be driven by influential observations, and mean differences may arise from skewness, overlap, or sparse subgroups rather than broad distributional shifts. We present EDAgent, a sequential framework that performs targeted visual verification before report synthesis. For each finding, the system records the statistical result, generates a visualization request identifying relevant distributional features, creates the corresponding chart, and uses the observed pattern to retain, qualify, or reconsider the interpretation. We call this process statistical-visual verification. EDAgent includes VisPlan, an 8B-parameter model that converts visualization rationales into detailed chart plans. We evaluate 87 end-to-end reports on EDA-Bench using a six-dimension rubric and GPT-5.1 multimodal evaluation. Maximum-score proportions reached 87.4% for factuality and visual consistency, 93.1% for statistical-visual cross-validation, and 96.6% for analytical narrative. On 166 open-ended VisEval tasks, VisPlan improved the mean overall score from 4.55 (Qwen3-8B) to 4.62, with the largest gain in aesthetics. These descriptive results indicate that visual verification can make automated EDA reports more transparent about how statistical findings relate to underlying data structure.

Closest papers

Nearest neighbours by meaning across the whole index.

ChartRevive: Reconstructing Data Visualizations from Chart Images Using MLLM

cs.HC ·2026-09-22 ·Yuki Ueno, Aditeya Pandey
benchmark similarity 0.73
Abstract · Accepted as a Poster Session at IEEE VIS 2026 VISxGenAI Workshop

Static chart images are widely used in scientific publications, business reports, and presentations, yet recovering both the underlying data and visual design from chart images remains a labor-intensive manual process, making them difficult to reuse. While prior work has primarily focused on data extraction, the extraction of visual design specifications, including colors, marker shapes, and axis configurations, remains underexplored. To identify a suitable model for chart reconstruction, we systematically benchmark five multimodal large language models (MLLMs) across five basic chart types on both data and design extraction tasks. Our evaluation shows that textual and categorical information can generally be extracted reliably, whereas numeric and spatial information remain challenging. Among the evaluated models, GPT-5.4 achieves the best overall performance and is adopted as the backbone of our system. Guided by these findings, we present ChartRevive, a mixed-initiative system that combines MLLM-based extraction with an interactive verification interface, supporting users to efficiently inspect, correct, and refine reconstructed charts through overlay-based verification and real-time rebuilding.

0.10

DEEPCHART: How Far are LLMs from Faithful Data-Science Chart Generation?

cs.AI ·2026-08-27 ·Jiahui tang, Kuicai Dong, Dexun Li, Hongchao Gu, Haocheng Yu +5 more
top venuebenchmarkcode/data similarity 0.71
Abstract

Faithful chart generation in real-world data-science workflows requires grounding visualizations in scattered evidence, computing chart-ready quantities, and rendering them accurately. Modern LLMs can produce visually plausible, instruction-compliant charts, yet data-level hallucinations remain difficult to detect in long, noisy, and multimodal contexts. To measure this gap, we introduce DEEPCHART, an expert-annotated benchmark of 1,482 task-conditioned chart-generation instances drawn from real-world scientific papers, financial filings, and ecosystem reports. DEEPCHART formulates chart generation as an Extract--Reason--Visualize pipeline and evaluates source-data extraction, derived-data reasoning, and chart rendering stage by stage. Experiments with state-of-the-art models show that visually plausible charts often conceal data-level hallucinations, with extraction and reasoning errors common in realistic long and multimodal settings. These findings suggest that larger context windows alone are insufficient; faithful chart generation also requires reliable evidence extraction and quantitative reasoning before rendering. Our benchmark and associated resources are available at https://github.com/tangdouer1005/DeepChart.

0.37

InsightChain: Optimized Chain-of-Insight Analytics for LLM-driven Data Visualization

cs.CL cs.HC ·2026-09-06 ·Hanya Sun, Chen Zhang, Sheng Liang, Yongyue Zhang, Yong Liu
top venue similarity 0.71
Abstract · EMNLP 2026 Findings

Large language models (LLMs) are increasingly used for automated data visualization, yet existing approaches often frame visualization generation as a single-step mapping from user query to figure or code, overlooking the iterative analytical reasoning process of expert analysts. We present InsightChain, a four-stage visualization prompting pipeline (Explore--Focus--Test--Present) that emulates expert analytical workflows, together with VG-COPRO, a vision-guided automatic prompt optimization (APO) method adapted to jointly optimize such multi-stage, executable pipelines. To address the evaluation gap for complex data visualization, we introduce the Insight Progression Metric (IPM), a rubric combining four text-based dimensions with a vision-based dimension. We assess IPM through a 100-chain human pilot and an expanded 300-chain agent-based evaluation spanning all ten domains. Experiments on public datasets show that InsightChain consistently outperforms competing prompting baselines. Existing APO methods fail to yield consistent gains on this multi-stage task, whereas VG-COPRO improves performance in both in-domain and cross-domain settings.

0.30

Beyond Static Charts: Can Language and Vision Language Models Generate Interactive Data Visualization Interfaces?

cs.CL ·2026-08-11 ·Mizanur Rahman, Aaryaman Kartha, Enamul Hoque Prince
benchmarkcode/data similarity 0.69
Abstract

Data visualization is central to analytical reasoning, but real-world analysis increasingly requires language-driven interactive interfaces rather than static charts. Although recent large language and vision language models (LLMs/VLMs) have shown promise in generating static charts from natural language, their ability to generate interactive data visualization interfaces remains largely unexplored due to the lack of benchmarks. We introduce VIS-GEN, a benchmark for evaluating how well LLMs/VLMs can generate interactive visualization interfaces from natural language queries. VIS-GEN comprises 3,042 samples covering diverse analytical intents, including data filtering, temporal analysis, and visualization editing, each paired with dataset metadata and natural language queries that are designed to reflect realistic, goal driven data exploration scenarios. We benchmark 14 state-of-the-art open-source and closed-source LLMs/VLMs, revealing large performance gaps and frequent failures on queries involving implicit intent, multiple interaction alternatives, and complex editing operations, highlighting interactive interface generation as a key open challenge beyond static chart synthesis. To address this, we propose a structured multi stage interface generation framework that decomposes the task into visualization design representation, generation of multiple interface candidates, constraint-aware critique, and self-refinement. This approach improves the best models pass rate by 15.9 percentage points, demonstrating a practical path toward more reliable language-driven interactive visualization systems. We release VIS-GEN at https://github.com/vis-nlp/VIS-GEN.

0.24

Faithful Chart Generation for Multimodal Deep Research: Frame-Evidence Co-Adaptation

cs.HC cs.AIcs.CLcs.GR ·2026-09-30 ·Yuxin Yue, Yingchen Zhang, Ruqing Zhang, Jiafeng Guo, Maarten de Rijke +1 more
similarity 0.69
Abstract

Analytical charts in multimodal deep research encode quantitative claims, requiring every visualized value to be faithfully grounded in supporting evidence. Unlike retrieved images that mainly provide contextual information, charts require numerical fidelity: visualized values should not only match retrieved evidence quantitatively but also preserve its original meaning and scope. However, achieving such fidelity remains challenging because current systems usually construct visualization plans before knowing what quantitative evidence can actually be retrieved from the web. As a result, predefined plans may require entities, temporal ranges, or comparison dimensions that the retrieved evidence only partially supports. Existing approaches mainly address this issue through post-hoc verification after chart plans are fixed, enabling unsupported values to be identified but leaving the underlying visual frames unchanged. To address this challenge, we propose Frame-Evidence Co-Adaptation (FECA), an evidence-adaptive visual planning framework for multimodal deep research. Inspired by the bidirectional sensemaking process in Data-Frame Theory, FECA models chart generation as an iterative interaction between visual frames and retrieved evidence. Each visual frame is adaptive: the frame guides evidence acquisition, while retrieved evidence determines whether the frame should be accepted, revised, or dropped before rendering. By coupling visualization planning with evidence availability, FECA shifts chart generation from fixed-plan verification to adaptive evidence-grounded visual reasoning. Experiments on 100 real-world research topics show that FECA substantially improves numerical fidelity while preserving report quality and chart utility.

0.19

VisAudit: Evaluating Multimodal Agents for Visual Diagnosis and Repair

cs.LG ·2026-10-01 ·Shicheng Liu, Adam Kahirov, Qi Zhang, Zhimin Hu, Song Wang +3 more
benchmark similarity 0.68
Abstract

Multimodal agents are increasingly used for data visualization tasks but remain limited in autonomous review. Unlike humans, they may fail to recognize when a visualization is incorrect, determine what to change, repair it without disrupting correct content, and verify whether the intervention succeeded. Existing benchmarks largely evaluate predefined individual capabilities such as chart generation, instruction-guided editing, or defect detection, and therefore do not capture this gap in autonomous review. We introduce VisAudit, a benchmark for evaluating visualization diagnosis, repair, and verification. Given a rendered chart and configurable auxiliary evidence, including its source data table, intended text summary, and visualization code, an agent iteratively diagnoses potential defects, modifies and executes visualization code, inspects execution and visual feedback, and determines when no further intervention is needed. VisAudit defines three tracks spanning diagnosed repair, autonomous repair, and open-world verification, and contains 1,900 flawed instances across 21 chart types and 10 flaw categories, together with 300 initially correct charts. We construct the benchmark through controlled perturbations of validated source visualizations, with systematic verification and human-aligned quality control to ensure that injected defects are well-defined and recoverable from the available evidence. Experiments with leading multimodal models reveal a substantial gap from reliable autonomous review: the strongest evaluated model fully recovers only $47.4\%$ of flawed charts in the autonomous-repair setting.

0.12

Do you "see" what I "see"? A Multi-panel Visual Question Answer Dataset for Large Language Model Chart Analysis

cs.DL cs.AIcs.IR ·2026-10-09 ·Daniel J. Evans, J. P. Naiman, JooYoung Seo, J. Stephen Downie
publishedcode/data similarity 0.66
Abstract · 16 pages, 4 figures, accepted to The 30th International Conference on Theory and Practice of Digital Libraries, 2026

Digital libraries increasingly host scientific publications whose figures encode primary research findings, yet making this visual content searchable, accessible, and reusable remains an open challenge. Large language and multimodal models (LLMs/LMMs) provide the possibility to automate many figure parsing tasks such as determining data trends or extracting data from plots. In this work, we consider if these models can reliably support task-oriented visual data analysis when presented with synthetic, but realistic, scientific figures that contain multiple charts. Our Visual Question Answering (VQA) process scaffolds our tasks along a four-level description model from basic conceptual answers to interpretive and analytical understanding. Additionally, we outline our process for synthetically generating our single and multi-panel (charts in which there are multiple panels present) plots and testing the data with various LMMs using quantitative analysis. In general, we find LMMs become less accurate with increasing figure complexity as the number of panels, number of plot elements, and difficulty of questions increase. Finally, because LMM accuracy is expected to change rapidly, our analysis also considers cost relative to performance as a metric to inform future user decisions. We provide the detailed question-answer pairs used in this analysis, along with the code used to generate random plots, question-answer pairs and detailed bounding-box metadata for chart elements. The code and data are available at https://github.com/ReadingTimeMachine/LLM_VQA_MultiPanel and https://huggingface.co/datasets/ReadingTimeMachine/visual\_qa\_multipanel

0.68

ExCritique: An Interactive Visual Interface to Evaluate AI-Generated Explanations at Scale

cs.HC ·2026-11-09 ·Meheraj Hossain, Avinash Reddy Mudireddy, Naimul Hoque
similarity 0.66
Abstract

Vision-language Models (VLMs) can generate natural language explanations for complex decision-making scenarios. However, these explanations can sometimes be misleading, sounding convincing while containing incorrect or unsupported information. Automated explanation-quality metrics help researchers and practitioners evaluate large collections of explanations, but their scores alone provide limited support for identifying failure cases and understanding the reliability of these metrics. This paper presents a case study on how interactive visualizations can help researchers and practitioners evaluate both AI-generated explanations and automated explanation-quality metrics across large datasets. Based on formative interviews with nine AI researchers and practitioners, we designed an interactive visualization tool named ExCritique, situated in the task of Visual Question Answering (VQA). ExCritique visualizes the distributions of multiple automated quality metrics, allows users to explore thousands of data items, filter items and inspect the evidence behind individual scores. We compared ExCritique with a baseline interface without interactive visualizations in a controlled study involving 16 AI researchers and practitioners. Our findings suggest that the interactive visualizations in ExCritique helped users identify explanation-quality issues more efficiently and better understand automated quality metrics, supporting an effective way to measure the reliability of AI-generated explanations.

0.21