CS Paper Radar

Where LLMs Fail with Visualization DSLs

Chang Han, Andrew McNutt, Katherine Isaacs

cs.HC cs.CL ·submitted 2026-10-01 15:30 UTC · v2

Abstract

As LLMs take up the role of authoring charts using visualization domain-specific languages (DSLs), the human constraints that shaped those languages may no longer apply, as what is easy for a person is not necessarily easy for a model. To understand how LLMs might work better with DSLs, we explore where and how they fail with current DSL designs. We evaluate 10 JSON-style visualization DSLs with 41 tasks across 3 LLMs, then assess the generated specifications with JSON and rendering checks, and qualitative coding of failed cases. Analyzing how this specification generation process fails, we identify four recurring failure patterns, link each to specific DSL features, and discuss design considerations for future DSL designs.

Comment
VIS 2026 VISxGenAI, 6 pages, 3 figures

Closest papers

Nearest neighbours by meaning across the whole index.

Vibe Analysis: Exploring LLM Adoption by Data Visualization Practitioners

cs.HC cs.AI ·2026-09-25 ·Shani C Spivak, Aditi Krishna, Mahsan Nourani, Melanie Tory
published similarity 0.68
Abstract

Large language models (LLMs) are enticing in their promise to support data visualization (Vis) through faster and simpler workflows for data prep, analysis, and visualization creation. Yet LLMs are notoriously error-prone and not built for data visualization tasks. Few studies have explored LLM adoption among Vis practitioners. To fill this gap, we conducted semi-structured interviews with members of the Data Visualization Society, a global community of data visualization designers. Our findings show that Vis designers actively use LLMs for both creative and technical aspects of the visualization process. A new visualization workflow is emerging, a process we call vibe analysis, analogous to vibe coding. Some key challenges raised by participants parallel those of vibe coding, while others are Vis-specific, like gaps in Vis knowledge and chart verification. This work opens up opportunities for research combining LLM-mediated work with Vis tools that incorporate data visualization guidance, constraints, and best practices.

0.14

Beyond Static Charts: Can Language and Vision Language Models Generate Interactive Data Visualization Interfaces?

cs.CL ·2026-08-11 ·Mizanur Rahman, Aaryaman Kartha, Enamul Hoque Prince
benchmarkcode/data similarity 0.65
Abstract

Data visualization is central to analytical reasoning, but real-world analysis increasingly requires language-driven interactive interfaces rather than static charts. Although recent large language and vision language models (LLMs/VLMs) have shown promise in generating static charts from natural language, their ability to generate interactive data visualization interfaces remains largely unexplored due to the lack of benchmarks. We introduce VIS-GEN, a benchmark for evaluating how well LLMs/VLMs can generate interactive visualization interfaces from natural language queries. VIS-GEN comprises 3,042 samples covering diverse analytical intents, including data filtering, temporal analysis, and visualization editing, each paired with dataset metadata and natural language queries that are designed to reflect realistic, goal driven data exploration scenarios. We benchmark 14 state-of-the-art open-source and closed-source LLMs/VLMs, revealing large performance gaps and frequent failures on queries involving implicit intent, multiple interaction alternatives, and complex editing operations, highlighting interactive interface generation as a key open challenge beyond static chart synthesis. To address this, we propose a structured multi stage interface generation framework that decomposes the task into visualization design representation, generation of multiple interface candidates, constraint-aware critique, and self-refinement. This approach improves the best models pass rate by 15.9 percentage points, demonstrating a practical path toward more reliable language-driven interactive visualization systems. We release VIS-GEN at https://github.com/vis-nlp/VIS-GEN.

0.24

Design Reflections on Transition to LLM-Aided Novel Visualizations

cs.HC ·2026-09-10 ·Richard Brath
similarity 0.64
Abstract · In Proceedings of The First Reflection in Creative Experience (RiCE) Workshop (RiCE W1) arXiv:2607.24558

This study examines data visualization design evolution over 12.5 years, reflecting on the impact of Large Language Models over the last 3.75 years. Using a longitudinal corpus of 55 visualizations from a single-subject design record, the study identifies how LLMs have aided design-space exploration: reducing coding effort, enabling new design opportunities, shock, excitement, accomplishments, and shifts to the design process.

0.21

ggaction: A Grammar of Graphical Actions

cs.HC cs.GR ·2026-09-13 ·Hyeon Jeon, Jinwook Seo
code/data similarity 0.63
Abstract · 21 pages

A chart may be declarative; authoring it is not. Visualization grammars often describe charts as finished specifications, whereas people construct them through a sequence of authoring actions. This mismatch can make visualization code difficult for humans to interpret and for machines to generate from human intent. ggaction addresses this gap by modeling the chart authoring process itself. In ggaction, individual authoring actions are abstracted as functions, and the authoring process is expressed as a chain of these functions. This representation more closely aligns chart designers' authoring intent with code specifications, making the code easily understandable to both humans and machines, including language models. Through a series of evaluations, we show that ggaction is sufficiently expressive to capture common chart authoring intents and outperforms widely used visualization grammars, including Vega-Lite and ggplot2, in both human and machine interpretability. ggaction is available at github.com/ggaction/ggaction.

0.24

Harnessing LLMs Without Surrendering Control: Delegation Boundaries in Visual Data Storytelling Authoring

cs.HC ·2026-09-22 ·Zhuojun Jiang, Yuki Ueno, Chris Bryan
publishedcode/data similarity 0.61
Abstract · Accepted as a Short Paper at IEEE VIS 2026

Despite the emergence of large language models (LLMs) for visual data storytelling workflows, there are open questions about how authors decide what activities or tasks to entrust to them and what should be "protected" or maintained under human control. To investigate this, we interviewed a cohort of 12 expert visual data storytellers. Our analysis shows that participants rarely treated LLMs as autonomous storytellers. Instead, they tend to selectively delegate execution-oriented tasks to LLMs while retaining control over activities that shape narrative intent and story meaning. Our findings show that LLM assistance is most productive after human seeding and constraint-setting, and that it shifts labor from production to verification. We discuss design implications for boundary-aware authoring tools, data-grounded generation, low-fidelity ideation, and reporting practices for LLM-based visualization research. Supplemental materials for this paper are available at https://osf.io/hcnp6.

0.42

Semantic Validity vs. Graphical Arrangement in Visualization Grammars

cs.HC ·2026-11-16 ·Cynthia A. Huang, Matthew Kay
similarity 0.59
Abstract · 5 pages (4 + references). Accepted at the Grand Unified Grammar of Graphics (GUGOG) workshop at IEEE VIS 2026

Systems for modular, declarative and compositional specification for data visualizations, otherwise known as visualization grammars, often claim Wilkinson's Grammar of Graphics as motivation, inspiration or foundation. However, despite the shared intellectual lineage, visualization grammars can vary greatly in design scope and focus. In particular, systems that strongly adhere to domain-specific notions of correctness must trade-off expressivity for validity. This trade-off between semantics and graphical flexibility is currently under-articulated, leading to difficulty integrating various GoG-inspired visualization contributions, especially between InfoVis and Statistical Graphics. We propose three new types of visualization contributions to support such integration: Semantic Validity Theories, Semantic Visualization Systems and Graphical Arrangement Systems. The first two prioritize the adherence of data visualizations to correctness conditions, such as statistical or mathematical transformations and data semantics, whilst the third largely leave validity concerns to the user. We illustrate these new contribution types using examples from two recent grammar of graphics inspired papers, and show how they supported more nuanced notions of expressivity and usability in evaluation and design of visualization systems.

0.19

RaivenTracks: Branching Provenance for Conversational Visualization Workflows

cs.HC ·2026-08-14 ·Ella Hugie, Alexandra Irger, Grace Guo, Kenneth Moreland, David Pugmire +2 more
similarity 0.59
Abstract · *Ella Hugie and Alexandra Irger are co-first authors

As AI agents increasingly participate in scientific workflows, scientists are shifting from direct authorship toward oversight, inspection, and steering. LLM-driven visualization systems are a promising interface for this hand-off, yet they remain largely stateless, forcing users to reconstruct context across refinements and offering little support for revisiting prior decisions or exploring alternatives. We present RaivenTracks, a workflow-aware extension of the Raiven DSL-mediated visualization pipeline that treats validated visualization specifications as persistent, branchable checkpoints. Because each checkpoint is a verifiable RaivenDSL specification rather than a dialogue transcript, restoring a node recompiles a known artifact rather than re-interpreting prior context. RaivenTracks contributes a two-level state management architecture that pairs a persistent, branchable version tree with a fine-grained undo/redo stack over runtime visualization settings, across both InfoVis and SciVis backends. A formative pilot study with three visualization researchers shows early promise, with all participants adopting the version tree for branching and recovery, and surfaces design directions for tree navigation, node labeling, and scalability that inform a planned controlled comparison against Raiven without version history. We frame branchable conversational visualization history as a step toward provenance support for future scientist-in-the-loop oversight of AI-driven scientific workflows.

0.04

The CAPED Framework: Characterizing User Skills in Chart Authoring through the Lens of Tasks and Tools

cs.HC ·2026-10-26 ·Bongshin Lee, Zhicheng Liu, John Thompson, Chenglong Wang
published similarity 0.59
Abstract · Accepted at IEEE VIS 2026, to appear in IEEE Transactions on Visualization and Computer Graphics (TVCG)

In this paper, we introduce a new framework for characterizing the user skills required for chart authoring. Despite growing attention to authoring systems, we lack a conceptual framework that characterizes the diverse and multifaceted skills involved and helps identify opportunities for system design and development. Our initial investigation revealed that skill is a complex and elusive concept, and that existing theories on skills from other disciplines are too high-level to capture the importance of tasks and tools in data visualizations. To address this gap, we formulate the CAPED framework that characterizes skill as the ability to create a chart involving five dimensions pertinent to tasks and tools: input data, output chart, paradigm, abstraction, and expression. We demonstrate the framework's descriptive power by highlighting different skills involved in similar authoring situations and identifying the types of knowledge associated with chart authoring. We further present examples illustrating how the framework can be used to identify new research opportunities in supporting users with varying skills.

0.37