CS Paper Radar

Model Literacy: An Extra Summative Evaluation Factor for Visual Analytics

Lei Xia, Siyu Wu, Haodian Li, Ye Sun, Liang Zhou, Lei Shi

cs.HC ·submitted 2026-08-19 09:22 UTC · v1

Abstract

Understanding and enhancing visual analytics (VA) performance is important for maximizing their impact. Existing studies have successfully applied well-established summative evaluation methods from information visualization to the VA context, yet the recent emphasis on an extra data analysis/modeling stage in the VA pipeline poses an additional challenge. Inspired by the modern concept of visualization literacy, this paper examines model literacy, namely users' knowledge of the analysis model used in a VA technique, as an additional factor for VA performance. Results from a controlled study on the visual analysis of multidimensional data with two dimensionality-reduction models indicate a positive correlation between model-task accuracy and VA-task accuracy. The study involves two common dimensionality-reduction models, PCA and t-SNE. The correlation is stronger for PCA than for t-SNE in the current task design, a pattern consistent with the possibility that VA effectiveness is more closely associated with model literacy when model outputs are less directly readable from the visualization. Completion-time evidence does not show a stable efficiency gain, suggesting that differences in model intuitiveness may help explain when model knowledge shortens task completion and when it involves additional interpretive effort. The findings of this study suggest ways to further enrich VA evaluation methods and provide directions for developing more rigorous model-literacy assessment instruments.

Closest papers

Nearest neighbours by meaning across the whole index.

VLMProbe: An Open-source Library for VLM Probing and Interpretation

cs.CV cs.AI ·2026-10-21 ·Gexin Huang, Haisheng Fu, Yanting Yang, Chen Zhao, Ruinan Jin +2 more
code/data similarity 0.62
Abstract · 43

Understanding how vision-language models use visual information requires connecting internal representations to model behavior. We present vlmprobe, an open-source library that integrates lens readouts, attention attribution, causal interventions, and sparse autoencoder (SAE) analysis within a common model interface. Shared adapters, calibration and checkpoint utilities, evaluation pipelines, and visualizations support systematic analysis across architectures. Using this framework, we compare methods across vision-language, language, and vision models through matched case studies and controlled evaluations of pretrained sparse dictionaries. The comparisons show that representation readouts and spatial explanations depend on the analysis objective and observation site, and that local attribution estimates can disagree with measured intervention effects. SAE evaluations further expose distinct trade-offs in reconstruction fidelity, concept readout, and spatial localization. Together, these results support combining multiple forms of evidence to evaluate an interpretation. vlmprobe provides a reusable basis for selecting methods, comparing their conclusions, and testing hypotheses about model behavior. The code is available on github.com/ubc-tea/vlm-probe

0.27

More Accurate, Less Human: Gestalt Grouping in Vision Models

cs.CV cs.LG ·2026-08-10 ·Sudhanva Manjunath Athreya, Sai Phani Kumar Malladi
benchmark similarity 0.61
Abstract · Accepted to VISxVision 2026, a workshop at IEEE VIS 2026. Includes appendix with per-task stimuli, metric derivations, and full per-model re…

Human vision organizes what it sees into wholes: same-colored points group into series, similar marks cohere into categories, and shapes complete into recognizable objects. These are the Gestalt operations that visualization design builds on. Whether vision models organize visual content this way has not been systematically tested. We introduce a behavioral battery that scores models against human data from prior perception studies on four grouping tasks: mark-color odd-one-out, color-series counting, silhouette recognition, and object odd-one-out. We apply it to 45 models across five training families: supervised, self-supervised, and contrastive vision-language encoders, open-weight VLMs, and closed foundation models. The battery reveals that agreement with human responses captures aspects of perceptual organization that conventional performance metrics fail to distinguish, with several closed models exhibiting substantially lower alignment than their benchmark accuracy would suggest. Scoring against published perception data therefore gives visualization research a reusable yardstick, requiring no new user study, for auditing whether the models now entering visualization pipelines organize what they see the way their human audience does.

0.13

Divided Attention Amplifies the Importance of Expectation-Aligned Visualization Design

cs.HC ·2026-08-10 ·Jiho Kim, Anna L. Chinni, Karen B. Schloss, Michael Gleicher
published similarity 0.59
Abstract · 33 pages, 22 figures (11 pages, 5 figures for the main text; 22 pages, 17 figures for the supplementary material), to be published in IEEE T…

Studies have shown that visualization design affects interpretability when visualization interpretation is the user's sole task. However, in real-world settings, users often engage with visualizations while performing concurrent tasks, such as when users simultaneously monitor alerts or respond to messages. Such divided attention may alter how users interpret visualizations, potentially increasing the importance of designs that align with viewer expectations. We investigated this possibility through two experiments comparing visualization interpretation under single-task and dual-task conditions. Specifically, we examined how well-established inferred mappings between color, spatial position, and semantic concepts affect interpretation when users perform a concurrent task, both with unlimited viewing time (Exp. 1) and under limited viewing time (Exp. 2). Our results show that divided attention amplifies the performance gap between expectation-aligned and expectation-violating designs, affecting response time, interpretation accuracy, and the ability to produce a judgment under time constraints. To explain these results, we model the user's decision-making process using a Linear Ballistic Accumulator (LBA) framework. Our findings highlight the increased importance of aligning visualization designs with viewer expectations under divided attention and introduce a process-oriented modeling approach to understanding how expectation and multitasking shape visualization interpretation.

0.22

InsightChain: Optimized Chain-of-Insight Analytics for LLM-driven Data Visualization

cs.CL cs.HC ·2026-09-06 ·Hanya Sun, Chen Zhang, Sheng Liang, Yongyue Zhang, Yong Liu
top venue similarity 0.58
Abstract · EMNLP 2026 Findings

Large language models (LLMs) are increasingly used for automated data visualization, yet existing approaches often frame visualization generation as a single-step mapping from user query to figure or code, overlooking the iterative analytical reasoning process of expert analysts. We present InsightChain, a four-stage visualization prompting pipeline (Explore--Focus--Test--Present) that emulates expert analytical workflows, together with VG-COPRO, a vision-guided automatic prompt optimization (APO) method adapted to jointly optimize such multi-stage, executable pipelines. To address the evaluation gap for complex data visualization, we introduce the Insight Progression Metric (IPM), a rubric combining four text-based dimensions with a vision-based dimension. We assess IPM through a 100-chain human pilot and an expanded 300-chain agent-based evaluation spanning all ten domains. Experiments on public datasets show that InsightChain consistently outperforms competing prompting baselines. Existing APO methods fail to yield consistent gains on this multi-stage task, whereas VG-COPRO improves performance in both in-domain and cross-domain settings.

0.30

Exploring Avatar-Based Representations of Desktop Analytical Workflows for Asymmetric Collaborative Visual Analytics

cs.HC ·2026-09-14 ·Tiansu Chen, Yalong Yang, Wai Tong
similarity 0.57
Abstract

Collaborative visual analytics increasingly occurs across asymmetric desktop-VR settings, with desktop analysis offering precision and efficiency and VR providing spatial and embodied affordances. However, maintaining workspace awareness remains challenging because desktop collaborators are often represented in VR only through indirect cues such as perspective sharing, shared visualization state, or lightweight cursor traces, which do not convey their ongoing analytical activity in a VR-native manner. To address this gap, we present Desk2Avatar, which explores avatar-based representations of desktop analytical workflows in VR. We introduce two representation strategies, Distanced and Embodied, inspired by remote pointing and direct manipulation. We conducted a within-subject study with 18 participants comparing these strategies against a depth-adaptive cursor baseline. Our findings show that avatar-based representations improved collaborator awareness and attentional guidance over cursor cues, while excessive embodiment introduced occlusion, distraction, and additional workload. Finally, we discuss design implications for future desktop-to-VR representations, focusing on balancing collaborator presence, attentional guidance, and workspace readability.

0.24

What Visual Representations Do World Action Models Need?

cs.CV cs.RO ·2026-10-09 ·Kai Liu, Jiangran Lyu, Tingyu Deng, Yixin Zheng, Jingtian Wan +7 more
code/data similarity 0.57
Abstract · 24 pages. Project page: https://kailiu18.github.io/R-2WAM.github.io/

World Action Models (WAMs) inherit powerful priors from pretrained video generation models, but their variational autoencoder (VAE) representations are optimized for visual reconstruction rather than manipulation. External visual representations can improve manipulation, but existing methods couple representation choices with different integration mechanisms, obscuring their individual effects. We systematically compare visual representations, five integration mechanisms, and feature combinations under controlled settings. Our study yields three main findings. First, strong general-purpose visual representations provide broader robustness gains and can match or outperform representations pretrained for specific visual capabilities. Second, channel-wise joint prediction achieves the highest average success rate; further analyses suggest that future-feature prediction and the action expert's effective use of added visual information contribute to its gains. Third, additional sources should be selected for the task-relevant information they contribute beyond existing representations, as the same source can help one combination but harm another. However, even the best evaluated composition provides a smaller additional gain than the initial improvement from adding a strong general-purpose representation to the VAE-only baseline. We instantiate the resulting Representation Recipe as R$^2$-WAM. It achieves state-of-the-art performance on RoboCasa GR1 Tabletop with an 84.7% success rate and reaches 92.8% on RoboTwin 2.0, improving over the VAE-only baselines by 5.9 and 2.2 percentage points, respectively. Taken together, these findings suggest that future work should prioritize stronger general-purpose visual representations and more effective mechanisms for integrating them into WAMs, with feature composition serving as a supplementary route to improvement.

0.52

From Perception to Integration: Revisiting the Internal Dynamics of Reasoning in Vision-Language Models

cs.CV cs.LG ·2026-09-28 ·Rong Yu Xu, Prayag Tiwari, Shaolei Zhang
code/data similarity 0.57
Abstract · 15 pages, 4 figures. Code: https://github.com/allenxu09/from-perception-to-integration

Vision-language models (VLMs) can answer simple visual questions, but often struggle when one question requires several visual judgments. We study this gap with controlled tasks for feature binding, numerosity, spatial relations, and amodal completion, together with a Composite task that combines them. Matched counterfactual image pairs isolate changes in the visual evidence needed to answer. Across four models, direct answers, hidden-state readouts, and state interventions show that the individual judgments can be made without explicit reasoning and that intervening on the corresponding states can affect the answer. During reasoning, the Composite answer becomes decodable from hidden states and usable from shortened traces, often before the model stops on its own. We train a small detector to predict this readiness and stop reasoning at that point. On MMStar and RealWorldQA, this reduces mean reasoning tokens by 79.1% and 74.5%, while average accuracy rises by 3.13 and 3.30 percentage points, respectively. These findings connect the internal development of answer readiness to a practical rule for allocating reasoning computation.

0.33

Evaluating Human-LLM Alignment on Visualization Preference

cs.HC ·2026-11-04 ·Tanmay Bakshi, Chenglong Wang, Simra Shahid
similarity 0.57
Abstract · IEEE VIS 2026 (VISxGenAI Workshop)

There are often multiple valid visualizations, spanning different chart types, visual encodings, and aesthetic choices, that can all support a user's goal of analyzing a dataset. Determining which of these candidate visualizations is most appropriate, however, remains challenging. Prior work has leveraged expert-curated taxonomies to guide visualization design and subsequent quality evaluation, a practical evaluation model must also account for human-centric factors. Latest visualization tools integrate Large Language Models (LLMs) to generate and refine visualizations, using LLM-as-a-judge to provide design feedback. Without a deeper understanding of whether, and in what ways, human and LLM perceptions of visualizations align, there is a real risk of deploying low-quality agents that do not reflect human judgment. We study this question through a pilot experiment - we formulate the problem as a preference task over curated chart pairs and compare the judgments of state-of-the-art vision models with those of a panel of human annotators. We establish a cautionary tale: such judges risk filtering out informative, compelling displays. We find that frontier models collapse human subjectivity, a rich signal that reveals contrasting design choices, into near-deterministic verdicts.

0.19