CS Paper Radar

The Work Behind Delegation: A Framework for Supervising AI Coding Agents

Yeon Su Park, Nadia Arvi, Hae Ri Lee, Sehoon Lim, Qianou Ma, Juho Kim

cs.HC ·submitted 2026-09-21 07:57 UTC · v1
highly cited

Abstract

As AI coding agents carry out development tasks with greater autonomy, developers are shifting from direct implementation toward supervising delegated work. Yet existing research offers limited understanding of how developers organize supervisory activities into connected workflows. Drawing on observations and workflow diagrams from 19 experienced developers, we reconfigure Sheridan's framework of human supervisory control into seven stages and the connecting loops for supervising AI coding agents. We applied the framework to public developer discussions on Reddit and found that supervisory demands extend across stages and that developers manage them by concentrating effort in planning, delegating supervisory work to other agents, and turning recurring guidance into reusable assets. Our framework provides a useful analytical lens for understanding how developers supervise AI coding agents by capturing how supervision is structured in agentic software development.

Comment
Under Review

Closest papers

Nearest neighbours by meaning across the whole index.

From Delegation to Action Approval: An Exploratory Study of Developers' Practices in Collaborating with AI Coding Agents

cs.SE ·2026-11-06 ·Yinan Wu, Dominik Wermke, Ze Shi Li, Kathryn Thomasset Stolee, Bowen Xu
similarity 0.75
Abstract

Developers now collaborate with AI coding agents that write code and take actions on their behalf. Beyond reviewing this code, developers decide what to delegate and which agent actions to approve. Some of these actions, such as running shell commands or reading files, can have effects that never appear in the resulting code. How developers make these decisions and how much effort this requires are not well understood. To better understand this collaboration, we surveyed 203 developers who use AI coding agents. We found that careful code review does not necessarily extend to action approval: 61% of developers who carefully reviewed most or all generated code still often approved at least one type of agent action without close review. Developers aware of security or privacy incidents or concerning practices did so more often than those who were not. We also found that autonomy varied more between developers than between development activities. Although developers reported a lighter workload, supervising agents remained mentally demanding. When asked what would make them grant more autonomy, developers wanted more than reliability: more than half of their responses called for agents that are bounded, observable, or reversible. Our findings suggest that as agents take on more of the coding, the challenge shifts toward supporting the developers who supervise them.

0.10

Developers as Managers of AI Agents: A Workplace Shadowing Study

cs.SE ·2026-10-24 ·Yi-Hung Chou, June Hyung Kim, Vasisht Kartik, Victoria Jackson, Tobias Schimmer +2 more
similarity 0.75
Abstract

As AI coding agents become more autonomous, developers increasingly direct, coordinate, and check their work. Yet little is known about these practices in everyday professional development. We shadowed 11 professional developers across approximately 15 hours of work, coded their activities and task transitions, analyzed 255 submitted prompts, and conducted 4.6 hours of follow-up interviews. Developers spent an average of 33.4% of observed time formulating prompts or checking AI responses, with task-switching rates ranging from 0 to 101.6 switches per hour. Beyond producing artifacts, prompts supported understanding, requirements clarification, agent configuration, and coordination. We identified three recurring patterns: \textit{Supervisor}, involving direct artifact inspection; \textit{Delegator}, involving dialogue-based coordination and delegated implementation and checking; and \textit{Learner}, involving agents primarily to build understanding. Developers sought technical information from agents and organizational context, authority, and alignment from colleagues. Our findings suggest future studies of how developers manage attention across concurrent agent tasks and tools that support verification, coordination, and learning.

0.18

Vibe-GUIDE: A Graph-based User Interface in IDEs for Oversight in Vibe Coding

cs.HC ·2026-09-20 ·Chifang Chou, Sam Yu-Te Lee, Rudrajit Choudhuri, Kwan-Liu Ma
similarity 0.73
Abstract · 14 pages, 3 figures, 4 tables

In agentic coding, developers shift from implementing changes themselves to specifying intent, evaluating the agent's work, and making approval decisions. However, delegating implementation can introduce cognitive debt that erodes project comprehension over time, constraining developers' ability to provide oversight. In this work, we investigate the role of persistent shared representations in supporting project comprehension and oversight of coding agents. We present Vibe-GUIDE, an agentic coding interface built around a structural, live, manipulable, and adaptive graph representation organized by functional modules. We evaluated our interface in a randomized between-subjects study comparing how 16 developers completed three cumulative coding tasks using our interface or a Chat-only baseline. We found that Vibe-GUIDE can support project comprehension and sustained task performance while keeping developers cognitively involved in oversight. These findings show how persistent shared representations can complement natural-language interaction and help developers maintain the understanding needed to oversee agent-generated changes as projects evolve.

0.43

When Review Alone No Longer Scales: Layered Supervision in AI-Assisted Software Engineering

cs.SE ·2026-08-26 ·Markus Stolze, Mirco Strässle
top venue similarity 0.69
Abstract · Accepted for publication at the 20th International Symposium on Empirical Software Engineering and Measurement (ESEM 2026), Software Enginee…

AI-assisted development tools enable software engineers to generate implementations at substantially higher speed and volume than in traditional workflows. Software teams have long relied on guardrails -- standing control mechanisms such as code review, linting, testing, and CI/CD pipelines -- to maintain quality and coordination. High-throughput AI-assisted generation increases pressure on these guardrails -- straining their capacity to keep pace with the volume and rate of generated changes -- and reshapes how organizations supervise development workflows, yet relatively little is known about how existing guardrails evolve in response. We conducted a qualitative interview study with five software engineering practitioners, situated within a broader practitioner survey. Our findings indicate that organizations distribute the work of supervision across multiple guardrail layers: preventive guardrails (produced by externalizing architectural intent and conventions into machine-interpretable form), executable guardrails (linting, testing, and CI/CD repurposed as scalable supervision infrastructure), and human oversight (shifting from line-by-line inspection toward supervisory interpretation focused on architectural reasoning, explainability, and long-term maintainability). We characterize this as a transition from review- centric guardrails toward layered supervision, in which no single guardrail carries the supervision load alone.

0.35

A Phased Workflow for Operating LLM-Based Coding Agents

cs.SE ·2026-08-31 ·Ante Kapetanovic, Tomislav Duricic, Andro Mercep, Emanuel Lacic
top venue similarity 0.68
Abstract · 2 pages, industry paper, to appear in proceedings of the 35th ACM International Conference on Information and Knowledge Management (CIKM '26…

LLM-based coding agents combine a foundation model with a harness that shapes agent behavior. For non-trivial tasks, how practitioners structure their work with the coding agents determines whether reliable results follow. We report on a phased workflow for operating coding agents developed by the AI research team at Infobip. The workflow structures agent-assisted development into four phases where human effort is front-loaded and delegation increases as artifacts mature. Context management is the central concern, addressed through four strategies applied at each phase to counter known failure modes. From practitioner experience, we observe that upstream errors in research and planning can compound across later phases, while correcting generated code can introduce bloat and fragility. This motivates front-loading human review. We identify two open problems: the absence of metrics for workflow effectiveness and the gap between formalized context management components and the workflow-level patterns that practitioners need.

0.37

Deliberate Pacing for Human Oversight in Agentic Software Development

cs.SE ·2026-11-06 ·Tobias Reichel, Michèle Stassen, Anamaria Mojica-Hanke, Maria Teresa Baldassarre, Sebastian Baltes +2 more
position similarity 0.67
Abstract

Agentic AI increasingly enables software development activities to proceed without continuous human involvement. While this autonomy may accelerate development, it also raises questions about when developers can meaningfully oversee software development done by agents. We propose deliberate pacing: intentionally adjusting when and how autonomous development proceeds to preserve opportunities for meaningful oversight. Drawing on an exploratory multivocal literature review of 20 sources, we identify three factors motivating this vision: changing relative costs of human involvement, the timing of oversight opportunities, and interdependencies between pacing decisions across the software development life cycle. We argue that neither uninterrupted autonomy nor frequent human intervention is universal. Instead, pacing should balance opportunities to understand and influence consequential decisions against costs in human attention, development time, and rework. We outline a research agenda to investigate when deliberate pacing is beneficial, how intervention timing affects oversight, and how pacing decisions interact across development activities.

0.11

From Watching to Steering: Enabling Human-Coding Agent Collaboration through Execution Traces

cs.HC cs.AI ·2026-11-03 ·Xiaoke Yan, Jiaying Lei, Danqing Shi, Zichen Wang, Nan Cao
similarity 0.67
Abstract · 16 pages,10 figures

Recent advances in large language models (LLMs) have enabled coding agents to autonomously perform complex software development tasks, yet their long-running execution remains difficult for users to inspect, understand, and steer. We present Vibetrace, an interactive trace visualization and intervention system for agentic programming built upon OpenCode. Vibetrace introduces a human-agent collaboration paradigm of Observe, Understand, Intervene, and Evolve. It (1) visualizes agent workflows as interactive execution traces for inspecting actions, intermediate states, and failures; (2) enables users to fork execution from prior steps and provide targeted corrections without restarting validated progress; and (3) distills traces and human feedback into reusable skills for continual agent evolution. On SWE-bench Verified, distilled skills reduce input tokens by 23.0% and cost by 15.5%, while improving pass rate from 65.0% to 68.0%. User studies further demonstrate improved accuracy and efficiency over conventional log-based interaction.

0.08

Specifying and Maintaining Agentic Workflows: An Empirical Study of GitHub Agentic Workflows

cs.SE ·2026-09-23 ·Jasem Khelifi, Issam Oukhay, Ali Ouni, Mohammed Sayagh, Mohamed Aymen Saied
similarity 0.67
Abstract

Agentic workflows shift software development from prompting AI agents for individual tasks to defining recurring work that agents execute automatically. GitHub Agentic Workflows (gh-aw) enables this approach through Markdown files that combine natural-language instructions with configuration and compile into executable GitHub Actions workflows. Unlike conventional workflows that primarily prescribe scripted operations, these files delegate tasks requiring interpretation to AI agents. They also couple agent instructions with execution triggers, making those instructions operational specifications for repeated repository activities. However, how developers structure and maintain these specifications, and which execution requirements and safeguards they express, remains insufficiently understood. In this paper, we examine the structure, evolution, and instruction content of gh-aw Markdown files to inform how practitioners define and maintain agent-run work. We analyze 1,248 files from 276 repositories, 20,841 commit-file events, and 288 resolved instruction-label sets from a sample of 294 files. Our results show that workflow instructions extend beyond short prompts, with a median of 556.5 words and code blocks in 62.1% of files. Among files with at least 120 days of observed activity, 78.2% still receive updates in month 4, while size-normalized churn decreases after the first month. Tasks, outputs, constraints, and process instructions each appear in over 93% of labeled workflows, yet only 9.4% explicitly address prompt-injection defense. LLM classification achieves an F1 score of 0.818 and Cohen's Kappa of 0.715 against the resolved human labels. These findings suggest that developers should account for the evolution of workflows copied or referenced across repositories and consider adding prompt-injection defenses, resource budgets, and evidencecredibility checks where applicable.

0.33