CS Paper Radar

Notrix: Understanding Machine Learning Solutions Across Computational Notebooks at Scale

Xiaotian Su, Hongxin Fu, Xiaoyu Zhang, April Yi Wang

cs.HC ·submitted 2026-09-18 13:49 UTC · v1

Abstract

Computational notebooks make problem-solving visible, but typically only one notebook at a time. Meanwhile, in data science platforms like Kaggle, one competition can accumulate hundreds of notebooks. Effective collection-level analysis requires characterizing recurring solution patterns across all notebooks, as well as isolating specific notebooks for closer examination and learning. However, standard notebooks provide no common basis for this. Their workflows are nonlinear, cells declare no intent, and identical code can serve different ends, leaving hundreds of notebooks as separate documents. In this paper, we present Notrix, an interactive visual analytics tool for profiling hundreds of notebooks as one collection. Inspired by a formative study (N = 11), Notrix classifies every cell into one of thirteen machine learning (ML) stages, turning each notebook into a stage sequence, and clusters those sequences by structure rather than by code. To keep the representation constant as the scope narrows from the whole collection to a single cell, Notrix features three coordinated views---Workflow, Structural Matrix, and Detail---that appear at all four levels of granularity. In a within-subject study (N = 17) using two Kaggle collections of over 400 notebooks each, we observed participants answered questions about all notebooks more accurately with Notrix (median 88% vs. 50%) while opening 80% fewer notebooks per minute. Notably, four of the fourteen answered it without opening a single notebook (interaction logs, N = 14). Participants also reported significantly lower mental demand, temporal demand, and stress with Notrix (Holm-Bonferroni adjusted).

Closest papers

Nearest neighbours by meaning across the whole index.

Walk Before You Run: The Importance of Data Exploration for Data Analysis Agents

cs.DB cs.AI ·2026-08-17 ·Yike Yuan, Virum Ranka, Tina Lasisi, Lin Ma
publishedbenchmark similarity 0.64
Abstract · 9 pages, 6 figures. Accepted to VLDB 2026 Workshop: DASHSys: Systems for Data-centric Agents with Human-in-the-loop

LLM-based data-analysis tools are increasingly used to help users analyze messy spreadsheets and workbooks, from answering questions over uploaded files to generating code, summaries, and visualizations. These systems are often evaluated by the correctness of their final downstream answers. However, reliable data analysis also depends on an earlier step: understanding what the dataset contains before solving the requested task. For complex workbooks, this Data Exploration step includes identifying the logical tables behind physical sheets, interpreting column semantics, recovering keys and relationships, and detecting quality issues. In current tools and benchmarks, this step is usually left implicit, creating a gap between downstream task performance and the dataset understanding needed for reliable, human-checkable analysis. Our key contribution is to identify this overlooked gap, make Data Exploration a first-class evaluation target, and show through downstream experiments that stronger Data Exploration support improves task performance. To evaluate dataset understanding directly, we introduce two benchmark settings: a real multi-sheet workbook benchmark based on a Vitamin D study dataset, and an extension of DSBench with schema-fixed Data Exploration artifacts. In both settings, systems are evaluated by the quality of a structured artifact capturing tables, columns, semantic roles, relationships, and profiling signals. Our results show that strong LLMs and data-analysis agents still miss important logical structure even when they read spreadsheet content. Furthermore, explicit Data Exploration support often improves downstream correctness, suggesting it should be treated as a first-class, inspectable stage in LLM data-analysis workflows and a natural human-in-the-loop checkpoint where domain experts can review and correct the artifact before downstream analysis proceeds.

0.41

InsightChain: Optimized Chain-of-Insight Analytics for LLM-driven Data Visualization

cs.CL cs.HC ·2026-09-06 ·Hanya Sun, Chen Zhang, Sheng Liang, Yongyue Zhang, Yong Liu
top venue similarity 0.59
Abstract · EMNLP 2026 Findings

Large language models (LLMs) are increasingly used for automated data visualization, yet existing approaches often frame visualization generation as a single-step mapping from user query to figure or code, overlooking the iterative analytical reasoning process of expert analysts. We present InsightChain, a four-stage visualization prompting pipeline (Explore--Focus--Test--Present) that emulates expert analytical workflows, together with VG-COPRO, a vision-guided automatic prompt optimization (APO) method adapted to jointly optimize such multi-stage, executable pipelines. To address the evaluation gap for complex data visualization, we introduce the Insight Progression Metric (IPM), a rubric combining four text-based dimensions with a vision-based dimension. We assess IPM through a 100-chain human pilot and an expanded 300-chain agent-based evaluation spanning all ten domains. Experiments on public datasets show that InsightChain consistently outperforms competing prompting baselines. Existing APO methods fail to yield consistent gains on this multi-stage task, whereas VG-COPRO improves performance in both in-domain and cross-domain settings.

0.30

Designing Accessible and Rigorous ML Experiments: No-Code Visual Workflows and Structured Guidance for Beginners

cs.HC ·2026-11-10 ·Henrik H. J. Detjen, David Pfrang, Max Westphal, Melissa Finster, Dörte Corr +1 more
similarity 0.59
Abstract

Designing machine learning (ML) experiments requires beginners to plan a methodologically sound workflow and implement it in a concrete environment. We evaluated two forms of support in two within-participant studies with 18 predominantly technically trained participants: mlguide, which provides structured, evidence- backed guidance for workflow design, and AI Sandbox, a visual no-code environment for workflow implementation. After using mlguide, participants produced better high-level workflow designs, but their scores on a quiz about specific methodological choices did not improve significantly. Compared with Google Colab, AI Sandbox received higher hedonic user experience ratings, while perceived workload and acceptance did not differ significantly. These findings suggest that different forms of interactive support help with differ- ent aspects of beginner ML experimentation. The paper contributes a reusable evaluation task with instructions, a modified dataset, and a quiz, as well as empirical evidence on structured guidance for workflow design and on the experience of implementing workflows in visual and notebook-based environments.

0.36

polyview: A Python package for multi-view machine learning

cs.AI ·2026-10-05 ·Gwendal Debaussart-Joniec, Argyris Kalogeratos
similarity 0.58
Abstract

Multi-view learning jointly exploits multiple complementary representations of the same data and has become increasingly important in machine learning. However, the Python ecosystem lacks actively maintained, unified tooling for end-to-end multi-view workflows. In this paper, we present polyview, a Python package that provides tools for multi-view embedding, clustering, fusion, and view augmentation, as well as for handling incomplete views, all compatible with scikit-learn. The library offers a unified interface for composing heterogeneous multi-view workflows, including seamless transitions between multi-view and single-view stages. It is built around a core set of classes and utilities that enable composition of different methods and straightforward implementation of new ones. We illustrate the package on five real multi-view datasets and compare its components based on canonical correlation analysis with those of two established libraries. polyview aims to be both a practical toolkit for benchmarking and prototyping multi-view methods and a foundation for future research and development in this area.

0.18

Cellery: Gradual, Composable Structure for Spreadsheets

cs.HC ·2026-09-02 ·Jacob Yim, Sathwika Peechara, Brian Hempel, Michael Coblenz
top venue similarity 0.57
Abstract · To appear at the 39th Annual ACM Symposium on User Interface Software and Technology (UIST '26)

Structure in spreadsheets is typically restricted to ranges of cells and flat tables. However, replacing the open-ended two-dimensional grid with a more structured alternative risks limiting users' freedom and impeding exploratory workflows. How can we enable expressive structure in spreadsheets while preserving their flexibility? We propose a model that allows users to gradually add structure on top of the traditional grid. To identify common structural patterns, we conducted a content analysis of 30 real-world spreadsheets. Based on our findings, we developed Cellery, a novel spreadsheet system that makes these structures explicit. In Cellery, users gradually define the structure of their spreadsheets using named groupings of cells, composing together four structural primitives: Atoms, Lists, Tables, and Blocks. Once defined, users can directly manipulate structures, dynamically reference them in formulas, and incorporate them in reusable abstractions. We evaluate Cellery with an expressiveness study reproducing spreadsheets from our sample, demonstrating that it is capable of representing types of structure that traditional spreadsheets cannot. In a preliminary user study of five participants, we find that they successfully use Cellery to design spreadsheets with explicit structure, and we uncover opportunities for further work on the formula language.

0.28

Point, Revise, Review: Grounded Agentic Analysis in Reactive Notebooks with marimo-lens

cs.HC ·2026-09-17 ·Péter Ferenc Gyarmati, Trevor Manz, Dominik Moritz, Mennatallah El-Assady
code/data similarity 0.57
Abstract · 5 pages, 4 figures. Accepted for presentation at the VISxGenAI Workshop, IEEE VIS 2026. Source code: https://github.com/marimo-team/marimo-l…

In a computational notebook, a human can point to a rendered result and ask about "this," while an agent acts through cells, dependencies, and runtime state. When what the human sees and what the agent operates on are disconnected, the human must describe what they mean and trace what the agent did in prose that strips away situated visual and computational context. We present marimo-lens, an extension to the reactive Python notebook marimo for grounded human-agent analysis. Lens connects a human's marked output and note to its producing cell and relevant contributing computation, giving the agent computational context for the request. It surfaces agent activity, returns selected results to the notebook, and preserves the initiating selection for human review and reopening. We illustrate the lifecycle through an exploratory human-agent analysis of a real-world open dataset, showing how visually situated questions lead to computational inspection, notebook action, and returned evidence for human review.

0.39

PandasCorpus: A Resource of Real-World Pandas Workflows and Usage Patterns

cs.SE cs.LG ·2026-08-13 ·Syrym Abdikhan, Mazhar Hameed
datasetcode/data similarity 0.57
Abstract

Pandas has emerged as the de facto library for data processing and machine learning, widely used for tasks, such as data loading, transformation, and analysis. Despite its ubiquity, there has been limited systematic investigation into how Pandas is used in real-world projects and how typical workflows are composed in practice. To address this gap, we introduce PandasCorpus, a dataset curated from GitHub repositories that captures real-world Pandas workflows at scale. In this work, a workflow refers to Pandas-based code contained in Jupyter notebooks, a prevalent medium for writing, executing, and sharing data analysis code. The dataset comprises 139k notebooks from approximately 100k repositories and captures more than 4M Pandas API calls spanning 136 distinct operations. Beyond dataset construction, we characterize workflows using structural and Pandas-specific features and analyze notebook evolution between 2015 and 2025. Our study examines code executability, notebook size, and recurring sequences of Pandas operations, providing empirical insights into how Pandas is used in practice. The resulting corpus offers a reusable resource for studying data analysis workflows, Pandas usage patterns, and library-aware code composition. Both the dataset and the extraction pipeline are publicly available via GitHub and Zenodo.

0.20

Token Signatures of Code: Comparing Coding Behaviors Across Large Language Models

cs.CL cs.LGcs.SE ·2026-08-12 ·Junpeng Wang, Yuzhong Chen, Menghai Pan, Uday Singh Saini, Yiwei Cai
similarity 0.56
Abstract · 11 pages, 9 figures

The evaluation of large language models (LLMs) on coding tasks has primarily focused on performance metrics such as pass@k. As LLMs continue to advance, many models now meet baseline performance requirements, reducing the discriminative power of performance-based evaluation alone. Yet a key question remains largely unexplored: how do LLMs differ in their coding behavior? We propose CLIC (Code Learning for Identification and Comparison), a visual analytics approach that characterizes LLM coding behavior through token-frequency analysis. CLIC represents each code sample as a feature vector of token frequencies and trains an interpretable decision tree to separate two LLMs' code sets. Beyond classification accuracy, we define two new metrics: robustness, which measures whether the two LLMs remain distinguishable as their most-discriminative tokens are progressively removed, and concentration, which measures whether the difference is driven by a few dominant tokens or spread across many. Interpreting numerous pairwise comparisons (across LLM pairs, tasks, and tokenization levels) and tracing the full analytical chain form an inherently multi-scale, hypothesis-driven exploration task. We therefore develop an interactive visual analytics system to navigate the comparison landscape, identify pairs of interest, and drill down into discriminative tokens and their code contexts. Case studies comparing 10 LLMs across 22 Kaggle ML tasks reveal actionable insights for LLM selection and prompt engineering.

0.10