CS Paper Radar

Vibe-GUIDE: A Graph-based User Interface in IDEs for Oversight in Vibe Coding

Chifang Chou, Sam Yu-Te Lee, Rudrajit Choudhuri, Kwan-Liu Ma

cs.HC ·submitted 2026-09-20 20:23 UTC · v1

Abstract

In agentic coding, developers shift from implementing changes themselves to specifying intent, evaluating the agent's work, and making approval decisions. However, delegating implementation can introduce cognitive debt that erodes project comprehension over time, constraining developers' ability to provide oversight. In this work, we investigate the role of persistent shared representations in supporting project comprehension and oversight of coding agents. We present Vibe-GUIDE, an agentic coding interface built around a structural, live, manipulable, and adaptive graph representation organized by functional modules. We evaluated our interface in a randomized between-subjects study comparing how 16 developers completed three cumulative coding tasks using our interface or a Chat-only baseline. We found that Vibe-GUIDE can support project comprehension and sustained task performance while keeping developers cognitively involved in oversight. These findings show how persistent shared representations can complement natural-language interaction and help developers maintain the understanding needed to oversee agent-generated changes as projects evolve.

Comment
14 pages, 3 figures, 4 tables

Closest papers

Nearest neighbours by meaning across the whole index.

From Watching to Steering: Enabling Human-Coding Agent Collaboration through Execution Traces

cs.HC cs.AI ·2026-11-03 ·Xiaoke Yan, Jiaying Lei, Danqing Shi, Zichen Wang, Nan Cao
similarity 0.73
Abstract · 16 pages,10 figures

Recent advances in large language models (LLMs) have enabled coding agents to autonomously perform complex software development tasks, yet their long-running execution remains difficult for users to inspect, understand, and steer. We present Vibetrace, an interactive trace visualization and intervention system for agentic programming built upon OpenCode. Vibetrace introduces a human-agent collaboration paradigm of Observe, Understand, Intervene, and Evolve. It (1) visualizes agent workflows as interactive execution traces for inspecting actions, intermediate states, and failures; (2) enables users to fork execution from prior steps and provide targeted corrections without restarting validated progress; and (3) distills traces and human feedback into reusable skills for continual agent evolution. On SWE-bench Verified, distilled skills reduce input tokens by 23.0% and cost by 15.5%, while improving pass rate from 65.0% to 68.0%. User studies further demonstrate improved accuracy and efficiency over conventional log-based interaction.

0.08

The Work Behind Delegation: A Framework for Supervising AI Coding Agents

cs.HC ·2026-09-21 ·Yeon Su Park, Nadia Arvi, Hae Ri Lee, Sehoon Lim, Qianou Ma +1 more
highly cited similarity 0.73
Abstract · Under Review

As AI coding agents carry out development tasks with greater autonomy, developers are shifting from direct implementation toward supervising delegated work. Yet existing research offers limited understanding of how developers organize supervisory activities into connected workflows. Drawing on observations and workflow diagrams from 19 experienced developers, we reconfigure Sheridan's framework of human supervisory control into seven stages and the connecting loops for supervising AI coding agents. We applied the framework to public developer discussions on Reddit and found that supervisory demands extend across stages and that developers manage them by concentrating effort in planning, delegating supervisory work to other agents, and turning recurring guidance into reusable assets. Our framework provides a useful analytical lens for understanding how developers supervise AI coding agents by capturing how supervision is structured in agentic software development.

0.42

Developers as Managers of AI Agents: A Workplace Shadowing Study

cs.SE ·2026-10-24 ·Yi-Hung Chou, June Hyung Kim, Vasisht Kartik, Victoria Jackson, Tobias Schimmer +2 more
similarity 0.71
Abstract

As AI coding agents become more autonomous, developers increasingly direct, coordinate, and check their work. Yet little is known about these practices in everyday professional development. We shadowed 11 professional developers across approximately 15 hours of work, coded their activities and task transitions, analyzed 255 submitted prompts, and conducted 4.6 hours of follow-up interviews. Developers spent an average of 33.4% of observed time formulating prompts or checking AI responses, with task-switching rates ranging from 0 to 101.6 switches per hour. Beyond producing artifacts, prompts supported understanding, requirements clarification, agent configuration, and coordination. We identified three recurring patterns: \textit{Supervisor}, involving direct artifact inspection; \textit{Delegator}, involving dialogue-based coordination and delegated implementation and checking; and \textit{Learner}, involving agents primarily to build understanding. Developers sought technical information from agents and organizational context, authority, and alignment from colleagues. Our findings suggest future studies of how developers manage attention across concurrent agent tasks and tools that support verification, coordination, and learning.

0.18

Beyond the Prompt: Linking What Developers Ask, Do, and Understand with Coding Agents

cs.SE ·2026-09-27 ·Yunhan Qiao, Summit Haque, Christopher Hundhausen
similarity 0.70
Abstract

Coding agents can now change code for developers, who describe goals, supply context, and respond to the agent's work. Yet prompts, screen activity, and task success each tell only part of this story. We present Say, Do, Understand, an end-to-end workflow for analyzing what developers write to an agent, what they do while it works, and what they can explain afterwards. The workflow has five stages (Capture, Prepare, Analyze, Integrate, and Interpret) and three instruments: a prompt codebook, a scheme for coding screen-recorded activities and events, and separate rubrics for explaining the process and the solution. We applied it in an observational study of ten experienced developers who used GitHub Copilot on an unfamiliar codebase. Crossing task performance with understanding produced four personas. The two measures agreed for eight developers but split for two: one passed most tests but could not explain the solution, and another passed few tests but explained it well. In this sample, the personas that most often asked the agent to check its work spent the least time testing on their own. These patterns are descriptive and do not generalize beyond the sample. We recommend the workflow to computing educators, industry practitioners, and researchers to adapt and evaluate human--AI communication in software engineering.

0.19

Spec-Driven Development as Prescribed and as Practiced

cs.SE cs.AI ·2026-08-25 ·Shyam Agarwal, Anmol Singhal, Andrew Macvean, Bogdan Vasilescu, Travis Breaux
highly cited similarity 0.70
Abstract

AI coding agents implement software from naturallanguage instructions, but the intent behind that work is lost when stated only in chat sessions. Spec-driven development (SDD) addresses this by having developers and agents write specifications that guide implementation, yet evidence on its use comes largely from tool vendors. We study SDD in practice through a thematic analysis of Reddit discussions, interviews with 21 practitioners, including eight tool makers, and 266 active and popular repositories in SpecMine, a corpus of 470,795 spec files from 73,030 public GitHub repositories that we release. Practitioners agreed that specs should capture intent, but many also included implementation details, used SDD selectively and mainly for new features, and reported more reliable agent behavior alongside greater planning and review costs. Our findings show that implementation detail may help agents while making specs prone to decay, that tool-specific record structures shape what people and agents see, and that participants described spec-code checks mainly when changes are completed rather than over time. We also find that individual benefits may come from clarifying intent before delegation, while team benefits may depend on SDD becoming a shared process. We derive implications for tool builders, practitioners, and researchers.

0.36

Rasoi: Visual Diffs for Evaluating Structural Changes Made by Coding Agents

cs.SE cs.HC ·2026-11-12 ·Riya Sahni, Aryan Kaul, Anushka Bhatnagar, Xuanming Zhang, Vivian Liu +2 more
similarity 0.70
Abstract

Coding agents let developers modify software rapidly through natural language. However, as agents take on larger changes, developers can lose track of how the codebase is evolving. These edits may look correct on the surface while restructuring the system in unexpected ways: duplicating logic, changing modules outside the requested scope, removing functionality developers meant to keep, or introducing architectural drift. We investigate how visual diffs can help developers understand and evaluate the structural changes that coding agents make. We present Rasoi, a design probe that maps how a codebase's architecture evolves after each agent-made edit, letting developers inspect which architectural elements and relationships were added, modified, or removed across multiple levels of abstraction. We deployed Rasoi in a week-long study with 12 developers working on their own codebases across varied technical domains. We found that visual diffs helped developers see whether changes stayed within scope, notice unexpected restructuring, and understand how modified parts related to the system as a whole. Our findings suggest that the value of oversight tools for coding agents lies less in making developers trust the agent and more in helping developers feel confident in their own evaluation of agent-made changes. We discuss implications for visual tools that support code comprehension and human oversight in AI-assisted software development.

0.23

Tracybot: Preserving Developer Intent by Tracing Agent-Generated Code to Prompts

cs.SE cs.HC ·2026-11-06 ·Răzvan Albu, Aliaksei Khval, Maxine Orlova, Danis Music, Filipe Araújo Veras Rosa +2 more
surveycode/data similarity 0.68
Abstract

Large language models (LLMs) are now widely used in software development, and LLM-based coding agents are given increasing autonomy. With coding agents, many design decisions are made in the conversation between the developer and the agent. Typically, the developer starts a conversation with an agent and states their intent, the agent proposes options if needed, and the developer plans the change accordingly. Then, the resulting code is committed. However, in such scenarios, the decisions and the intent behind the changes remain in the conversation, with no link to the code they produced. In this paper, we present Tracybot, an open-source tool that integrates with VS Code, Git, and coding agents to keep track of relevant conversations in an LLM-assisted development process. Tracybot captures each change as a Tasklet which consists of optional planning prompts used to plan a change, together with a build prompt that produces it. Tracybot has two main features: (i) it maps line-level code changes made by coding agents to the Tasklets that produced them, and (ii) it allows the developer to export a Tasklet as a decision record when needed, for example when the decision concerns the architecture of the project. We developed the tool over two iterations of a design science study, while focusing the evaluation on the usability and user-friendliness of the tool. In a focus group (7 participants) and a survey (24 participants), we found that Tracybot was perceived as useful for activities like code review, and most participants found our Tasklet design and presentation to be a clear way to represent the developer's intent. Tool: https://github.com/TracyTeam/tracybot. Video: https://youtu.be/Vg_a1F5eO04.

0.21

Harvesting SWE Benchmarks from Wild Vibe Coding Sessions

cs.SE cs.AI ·2026-11-02 ·Ao Lu, Yuanhao Zeng, Shaocheng Lan, Sijie Xu, Jiacheng Bao +3 more
benchmarkcode/data similarity 0.68
Abstract · Code: https://github.com/bytedance/SessionFarmer ; Benchmark: https://huggingface.co/datasets/ByteDance/SessionFarmer-CropChat

Development teams need reproducible tasks from their own repositories and workflows to evaluate coding agents. Conversational developer-agent collaboration ("vibe coding") produces abundant coding sessions and repository checkpoints, but requirements evolve across turns, unrelated conversations co-occur with commits, and engineering efforts can span multiple sessions. We present SessionFarmer, an automated framework that turns these records into self-contained, executable software engineering tasks. It analyzes dialogue and commit history to filter unrelated sessions and group multi-turn, potentially multi-session interactions into work episodes centered on one engineering goal. For each episode, it reconstructs a hermetic execution container, synthesizes a task statement from the full context, and authors repository-grounded engineering-quality rubrics and unit tests. Closed-loop test refinement uses requirement-breaking mutants and a blind solver to reduce false rejections of valid alternatives and false acceptances of incomplete solutions. We construct CropChat, an open benchmark of 242 tasks across 61 public repositories and 14 programming languages, and CropEnterprise from proprietary enterprise logs. In a CropEnterprise audit, the original developers endorse the vast majority of reconstructed episode boundaries, task statements, and quality rubrics. Full-episode context covers substantially more user requirements than single-turn or single-session inputs. Test refinement cuts false-rejection rates and raises balanced accuracy against reference patch labels by 7-9 percentage points across solvers. Across 12 model-harness configurations on CropChat, the strongest setup achieves 39.3% Pass@1, while harness choice shifts Pass@1 by up to 8.4 percentage points for the same model.

0.28