CS Paper Radar

Designing Accessible and Rigorous ML Experiments: No-Code Visual Workflows and Structured Guidance for Beginners

Henrik H. J. Detjen, David Pfrang, Max Westphal, Melissa Finster, Dörte Corr, Markus Wenzel

cs.HC ·submitted 2026-11-10 00:02 UTC · v1

Abstract

Designing machine learning (ML) experiments requires beginners to plan a methodologically sound workflow and implement it in a concrete environment. We evaluated two forms of support in two within-participant studies with 18 predominantly technically trained participants: mlguide, which provides structured, evidence- backed guidance for workflow design, and AI Sandbox, a visual no-code environment for workflow implementation. After using mlguide, participants produced better high-level workflow designs, but their scores on a quiz about specific methodological choices did not improve significantly. Compared with Google Colab, AI Sandbox received higher hedonic user experience ratings, while perceived workload and acceptance did not differ significantly. These findings suggest that different forms of interactive support help with differ- ent aspects of beginner ML experimentation. The paper contributes a reusable evaluation task with instructions, a modified dataset, and a quiz, as well as empirical evidence on structured guidance for workflow design and on the experience of implementing workflows in visual and notebook-based environments.

Closest papers

Nearest neighbours by meaning across the whole index.

Feeling Easier, Working Harder: Cognitive Load in AI-Assisted Code Comprehension

cs.SE cs.AI ·2026-11-09 ·Maryangela White, Maximilian Rose, Matthew Buckman, Amy McDonnell, Francisco Ortega +1 more
top venue similarity 0.64
Abstract · Accepted to ICSE 2027

Software developers spend significant time reading and understanding code, increasingly supplemented by generative AI. While these tools aim to reduce the difficulty of reading code, interacting with them introduces a dual-task environment that may impose extraneous cognitive load that is challenging to capture. This study investigates how access to generative AI affects developers' real-time cognitive workload, code comprehension performance, and self-perceived code comprehension strategies. We conducted a 3x2 within-subjects study with 20 novice CS students reviewing Java code snippets of varying lengths with or without AI assistance, followed by an AI-unassisted comprehension quiz. Cognitive workload was continuously captured using a Detection-Response Task (tDRT) during both reading and quiz phases, and compared to post-quiz NASA-TLX and SEQ scores. AI assistance significantly increased objective cognitive workload, raising tDRT reaction times by 11.2%, yet participants' perceived workload remained unchanged. Code comprehension scores were high under both conditions, with no evidence that AI assistance improved comprehension at the difficulty levels evaluated; a ceiling effect limited power to detect benefits on harder tasks. Qualitatively, participants reported trusting the AI conditionally -verifying its output against their own reading for more complex snippets, but directly trusting it for simpler code- and, while many felt their reading process was unchanged, a notable subset shifted toward skimming or reading the AI's explanations in place of the code itself. These findings reveal that developers do not accurately perceive the cognitive resources they expend when managing AI tools. This study provides evidence of added cognitive effort associated with using generative AI, pointing towards a need to account for this dual-task penalty in software engineering tools and education.

0.61

Exploring multimodal AI--XR interactions for software development: From expert perspectives to semantic code outlines

cs.HC cs.SE ·2026-11-18 ·Ilya Zakharov, Alexey Markin, Konstantin Ostroukhov, Alexey Perminov, Roham Koohestani +2 more
survey similarity 0.63
Abstract · Submitted to HIMAS 2027: Human-Centered Interactive Multimodal AI Systems, co-located with ACM IUI 2027

AI tools can generate code quickly, but developers still need to understand and check the results. We explore how the enriched multimodal context derived from the extended reality (XR) workspace might help developers manage this information. We first interviewed 13 experts in AI, XR, and related fields to identify opportunities and design challenges (Study 1). These interviews informed two prototypes explored with 12 professional developers (Study 2): four tried a version where the code faded to become invisible after it was looked at, and eight tried a version that replaced viewed code sections with short descriptions of their purpose. In the second prototype, head direction helped determine when to offer a change, while manual input let developers apply it and reopen the original code. A separate open-ended demonstration with seven software-engineering researchers provided complementary feedback. Participants' accounts suggested that descriptions could provide an overview of unfamiliar code and serve as reminders when revisiting it. However, difficulty comparing descriptions with source code, headset limitations, and controls that deviates from regular IDE routines could introduce additional effort. These exploratory findings identify requirements for attention support: make summaries easy to recognize and access, preserve straightforward inspection of the original code, and retain familiar navigation and editing practices. The study contributes to human-centered multimodal systems by showing how the coordination of input channels and AI-generated representations must account for users' attention, verification needs, and established ways of working.

0.13

"ChatGPT, what am I missing?": Designing AI Workflows around Professional Task Structure to Shape Analytic AI Use

cs.HC ·2026-09-15 ·Zilin Ma, Suzi Jazmati, Marco Chimenton, Yiyang Mei, Jacqueline Lane +2 more
similarity 0.61
Abstract

General-purpose AI lets users choose what support to request, but leaves them to structure the support a professional task requires. We examine how interactive workflows can embed professional task structure without prescribing how users engage with AI. We designed two scaffolded interfaces around the same negotiation scaffold: one presented a completed AI analysis, while the other supported user-directed, incremental development. A four-condition randomized experiment with 800 participants compared these interfaces with no-AI and an AI chat interface. AI-supported conditions improved preparation coverage over unaided work; the scaffolded workflows further improved coverage over chat. Although the scaffolded workflows produced similar coverage, the user-directed workflow elicited a broader repertoire of analytic requests and lower subjective effort. Professional scaffolding therefore depends not only on displayed structure but on how workflows organize users' engagement with it. Effective professional AI must structure how users and AI build analysis together.

0.19

Cognitive Inclusivity in AI Coding Assistants: Evidence from GitHub Copilot

cs.SE ·2026-10-30 ·Grace Kerr, Aarushi Parasrampuria, Faith Culas, Kelly Blincoe
similarity 0.61
Abstract

AI coding assistants are commonly used for daily software development tasks. While prior research has examined their impact on productivity and code quality, less is understood about how these tools support diverse ways of thinking. Evidence from non-AI software engineering tools suggests that mismatches between tool design and users' cognitive styles can introduce inclusivity barriers, but it remains unclear whether similar issues arise in AI coding assistants. We investigate how developers with different cognitive styles experience and interact with these tools and whether they exhibit cognitive inclusivity challenges. We conducted a controlled laboratory study with 20 student participants using GitHub Copilot to complete code generation, debugging, and refactoring tasks. Using think-aloud protocols, semi-structured interviews, and inductive thematic analysis, we identify patterns in user interaction. We then apply the GenderMag framework to examine how these patterns differ across cognitive styles. We identify five inclusivity issues concerning how Copilot explains suggestions, supports validation, facilitates learning, exposes its features, and influences the problem-solving process. Our findings suggest that common interaction paradigms in AI coding assistants can privilege certain cognitive styles while introducing additional cognitive effort for others. We derive design implications for more inclusive AI coding assistants.

0.52

AI literacy over tool design: a mixed-methods study of scaffolded versus unrestricted generative AI in programming education

cs.CY ·2026-09-15 ·Sepinoud Azimi
similarity 0.61
Abstract

Generative AI has become a routine resource in programming education, and most institutional responses to it are attempts at control, either by restricting access or by offering students a controlled version of the technology. This paper reports a seven-week mixed-methods pilot study in a master's-level data analytics course, in which 33 students were randomly assigned either to a scaffolded AI Study Coach embedded in the notebook-based laboratory sessions or to unrestricted use of AI tools of their own choosing. The Coach offered stepwise hints, did not generate code, limited the number of hints per session, and required a short reflection at the end of each session. The design assumed, in line with scaffolding theory and recent experimental evidence, that guided and limited support would build confidence and reduce over-reliance, and that the scaffolded group would learn more. Assignment performance did not differ between the conditions. Students in the Coach condition reported higher confidence but managed the hint budget poorly, while students in the unrestricted condition were satisfied with their tools and uneasy about how much they depended on them. In interviews, students in both conditions identified awareness of their own reliance on AI as the most valuable outcome of the course. Students who had formulated their own rules for when to use AI performed better in both conditions, and those with the best understanding of how the models work, in every case self-taught, used the tools most deliberately and achieved the highest scores. The design of the tool mattered less than the students' capacity to govern their own use of it, a capacity that is at present acquired by chance. The paper argues that the appropriate response is structural: assessment that grades the reasoning behind AI-assisted work, and AI literacy taught explicitly as a core skill.

0.27

Work2Skill: Capturing Human GUI Experience for Reusable Agent Skill Induction

cs.AI cs.HC ·2026-10-09 ·Yuqi Huang, Hui Li, Yifei Cao, Zikun Cai, Tianci Xie +7 more
code/data similarity 0.60
Abstract

Computer-use agents can perform increasingly complex GUI tasks, yet unfamiliar software and workflows requiring human domain knowledge remain challenging. Human demonstrations convey procedural knowledge through the same GUI interaction space, but reuse requires interpreting actions in context and separating workflow rules from task-specific details. We thus introduce Work2Skill, a framework that captures human GUI experience for reusable agent skill induction. WorkRecorder provides the framework's infrastructure for collecting this experience, aligning screen recordings, mouse and keyboard events, and available accessibility-tree information. Using WorkRecorder, we construct WorkDemo-350, the first synchronized multimodal dataset of human desktop GUI demonstrations for reusable agent skill induction and downstream evaluation. Its 350 demonstrations are screened for completeness and task success. We then propose an agentic skill-induction method that inspects evidence on demand to produce reusable skills capturing workflow rules, step dependencies, and completion checks. When the same four execution models are equipped with skills induced by each method under shared execution settings, Work2Skill achieves the highest scores and success rates. Mean score improvements over the corresponding No Skill baseline are 4.67--10.54 points on MyPCBench Bounded Action and 3.11--7.50 points on a 286-task subset of OSWorld-Verified. Our code and data are available at https://github.com/china-qijizhifeng/Work2Skill.

0.30

Look Before You Prompt, and After: Scaffolding Human-AI Collaboration in Software Tutorial Creation

cs.SE cs.HC ·2026-09-03 ·Avinash Bhat, Vy Bui, Jin L. C. Guo
similarity 0.60
Abstract · 27 pages, 8 figures, 5 tables. Under Review at TOSEM Special Issue on Human AI Collaboration in Software Engineering

With LLMs, creating software tutorials now involves steering the model's output and shaping it into a coherent, accurate learning resource, yet existing LLM tools offer writers little support for this work. By analyzing interviews with technical writers ($N=17$), we identify three requirements for how they assemble and structure multiple LLM responses, curate the context the model uses, and verify the generated content. We designed a tool called dBlocks with the following features: blocks to scope content, a context manager to edit context, and inline execution to verify code. Following a human-centered design method, we iteratively refined the design through a user study ($N=5$). In a within-subjects lab study ($N=16$) comparing dBlocks with participants' preferred workflows for LLM-assisted authoring, participants reported significantly higher confidence in the tutorials they produced with dBlocks. In addition, the tool reduced friction in verification, with writers verifying code as they drafted rather than deferring or skipping it, and helped them avoid searching long chat histories by scoping their work into blocks that kept each tutorial section and its LLM conversation together. More broadly, our work offers implications for tools that scaffold human-AI collaboration in SE workflows and shows how human-centered design can guide the development of LLM-integrated tools.

0.37

WIP: Exploring Student-Created AI Agents and Instructional Videos to Foster Engagement in Computing Education

cs.CY ·2026-11-17 ·Lucas G. Zangari, Olufisayo Omojokun, Pedro Guillermo Feijóo-García
published similarity 0.60
Abstract · This work was presented at 2025 ASEE/IEEE Frontiers in Education (FIE)

This work-in-progress (WIP) research-to-practice study examines the use of student-created artifacts to promote active learning in a large undergraduate Software Design and Engineering (SWE) course. Active learning strategies have been shown to significantly enhance student engagement, comprehension, and motivation, particularly in computing education. In this work, we explore the use of student-created artifacts to foster student engagement in higher-level Computer Science (CS) courses. Specifically, we examine the experiences of undergraduate and graduate CS students who participated in the creation of two types of artifacts: instructional video tutorials and conversational AI agents. The goal being to understand how designing these artifacts could support students in applying and learning course content. Students raised valuable insights regarding both experiences. Concerning the creation of instructional videos, they appreciated the interactivity brought through multimedia to their learning experiences, although some found the strategy to be time-consuming. Similarly, when designing conversational AI agents, students enjoyed the novelty and practical application of the task, while some encountered challenges related to the complexity of tools such as \textit{Google Dialogflow ES}. We report on the potential of student-created artifacts to foster deeper engagement with course material, encouraging both creativity and critical thinking in computing education.

0.17