CS Paper Radar

Is This Code Mine? Student Ownership When AI Agents Write the Spec and the Code

Gustavo Pinto, Enne Rebeca Silva de Freitas, Danilo Monteiro Ribeiro

cs.SE ·submitted 2026-11-16 18:02 UTC · v1
survey

Abstract

Software engineering courses are starting to ask students to build entire systems with AI coding agents, but little is known about what students actually do with them and whether they still own what they deliver. If the agent writes both the code and the decisions behind it, the artifacts we grade no longer show what students learned. We studied an assignment in which 18 pairs of students built applications with agents under Spec-Driven Development, combining the mined history of their repositories, the prompts they recorded, and a survey of 33 students. The agents wrote 69\% of the specifications meant to keep students in control, sometimes from a prompt of a few words. Students locate their ownership in the ideas, crediting themselves with half of them but a fifth of the code, yet most could not explain that code to a colleague or modify it without the agent, and what they report having written is unrelated to what their own commits declare. The three teams that met the 100,000-line target did so with generated code the application never uses, in one case proposed by the agent itself, and one team rewrote its history to backdate its specifications. Teaching with agents should therefore assess what students specify, review, and can explain, rather than what they ship, and enforce the engineering process with tools rather than trusting it to students or agents.

Comment
Submitted to ICSE SEET; 12 pages

Closest papers

Nearest neighbours by meaning across the whole index.

Practical Implementation Report on Introducing Spec-Driven Development Using AI Agents in Software Development PBL

cs.SE ·2026-08-31 ·Hidetake Tanaka, Hiroshi Igaki, Kazumasa Shimari, Kiyoshi Honda, Naoki Fukuyasu
published similarity 0.66
Abstract · Accepted for publication in the 38th International Conference on Software Engineering Education and Training (CSEE&T 2026) : 18 pages, 1 tab…

In recent years, autonomous AI agents such as GitHub Copilot and Claude Code have been rapidly gaining popularity. This study reports on the practical implementation of Spec-Driven Development, a software development methodology premised on AI agents, within a Software Development Project-Based Learning (SDPBL) course for third-year undergraduate students. We defined a workflow consisting of four phases, namely investigation, planning, implementation, and review. We also established an environment tailored for the SDPBL course where AI agents generate documentation and code during each phase. We analyzed the results from three perspectives, namely students' subjective AI usage, implementation throughput, and code comprehension. The analysis reveals that AI usage patterns varied across development phases and teams. Moreover, while AI agent utilization increased implementation throughput, it also tended to encourage students to proceed with development without fully understanding the code. This study demonstrates that regular verification of code comprehension by instructors and appropriate feedback are essential for maintaining educational effectiveness when introducing SDD into SDPBL.

0.59

Hidden in Plain Sight: an Empirical Study about The Instruction Stacks of Coding Agents

cs.SE ·2026-10-27 ·Felipe de Sant'Anna Paixão, Dhyego Tavares, Eduardo Almeida, Iftekhar Ahmed
code/data similarity 0.63
Abstract

Coding agents send their models more than the user's prompt: they add instructions, tool definitions, and environmental context that form an instruction stack. This stack guides execution and changes across releases, yet remains largely unexposed to users and insufficiently characterized in research. We present a longitudinal, cross-agent study of the instruction stacks captured in the first model requests of 1,886 releases of Claude Code, Codex, Gemini CLI, Qwen Code, and OpenCode. Under controlled conditions, we analyze their composition and evolution and trace changes in the four open-source agents to documented motivations. Our findings show why examining the system prompt or reporting the model alone provides an incomplete account of agent inputs. Tool definitions dominate three agents' latest requests, while occasional concentrated revisions alter instructions governing tool use, attribution, and coding style. Such revisions can complicate how researchers identify agent-authored contributions and interpret generated code. We also found that only 48% of examined change units directly explain the motivation for stack revisions. Across agents, shared capabilities emerge despite increasingly distinct wording. These findings expose an evolving, incompletely documented layer of agent design. The findings motivate documenting captured instruction stacks alongside agent releases, models, and user prompts, and preserving the rationale behind their revisions.

0.31

Who Is Doing the Thinking? Characterising Human Agency in First-Year Students' AI Interactions

cs.SE cs.HC ·2026-11-03 ·Ali Shakiba, Thomas Chaffey
similarity 0.63
Abstract · 11 pages, 2 figures, 3 tables

When students use generative AI to produce their work, the learning process becomes invisible. Established frameworks such as Bloom's taxonomy then classify the cognitive level of what the AI produced, not what the student contributed. We propose the Human Agency Depth Taxonomy (HADT), a preliminary six-tier taxonomy that classifies the engineering contribution a student makes in a prompt, independent of the AI's output. HADT combines three dimensions: contextual bandwidth, engineering abstraction and student agency. We applied HADT to 962 prompts from 105 students' structured AI Journals in an AI-permissive, first-year software engineering course of 145 students, and examined how agency changed over ten journalled weeks. Most prompts (84.2%) carried some student contribution beyond delegation. Agency tracked the development lifecycle. Architecture-discovery prompts dominated the requirements phase (48% of turns) and fell to 5-12% from week 9, while prompts analysing existing systems rose during build and verification (odds ratio 2.88, with turns clustered by student). Two-thirds of the variance in tier lay within students, so agency depended more on the phase of work than on the student. A turn-level rise in delegation in deadline weeks did not hold once turns were clustered by student. Explicitly specifying constraints on the AI's output was rare throughout, which we interpret as a knowledge bottleneck for novices rather than a gap in instruction. We discuss implications for course and assessment design, and for using HADT as a formative lens on how students engage with AI.

0.21

Code, Commits, and Complexity in the GenAI Era: A Longitudinal Study of Team-Based Capstone Projects

cs.CY cs.SE ·2026-11-05 ·Asma Shakil, Claus Brabrand, Paul Denny
published similarity 0.63
Abstract · Accepted to SIGCSE 2027

Generative AI tools are now widely available to students, but little is known about how team-based project codebases have changed since they appeared. This matters when project courses use repository activity as evidence of progress and learning. Existing studies have largely examined self-reports, controlled tasks, or short individual assignments, leaving open questions around how student codebases evolve in authentic project settings. We present a ten-semester analysis of N=260 capstone team repositories from 2021 to 2025, spanning pre- and post-AI cohorts. We examine three dimensions: code size (lines of code), code characteristics (comment density, error-handling density, and cyclomatic complexity), and development behaviour (commit volume, edit size, and commit frequency). We find that end-of-semester codebases roughly triple, from about 3.4 KLOC in 2021--2023 to about 10 KLOC in 2024--2025. Code characteristics show a clear decoupling: cyclomatic complexity and error-handling density increase, while comment density does not. Later cohorts commit far more often and in greater volume, but individual commit sizes are largely unchanged, suggesting a shift in development rhythm rather than edit granularity. Most changes appear from 2024 onwards and are not uniformly monotonic. We did not directly observe AI use, so we report these as changes that coincide with increased GenAI availability rather than as effects of AI use. Our findings highlight the need to reconsider how student-produced artefacts are used and assessed in project courses.

0.40

What Open-Source Projects Tell Their Coding Agents: A Census of AGENTS.md and CLAUDE.md

cs.SE cs.AI ·2026-10-22 ·Pawel Zarzycki
similarity 0.62
Abstract · Corrected abstract metadata formatting and restored missing numerical values; paper unchanged

Coding agents can be guided with instruction files in a repository. We examine this emerging practice through a census of 22,854 popular, active GitHub repositories in thirteen languages, treating each repository's root 'AGENTS$.$md' and 'CLAUDE$.$md' as one instruction set. Agent instructions appear in 26.3% of repositories and 59.6% of the most-starred; where both files exist, 'CLAUDE$.$md' usually points to 'AGENTS$.$md'. We analyse 341,852 statements using model-assisted annotation and human review. The annotations reveal a practical mix of project knowledge and rules about code, building and workflow. Collaboration rules name concrete requirements, from test plans to commit formats, yet 74.0% of directives give no reason. Projects disagree on AI attribution: commit-message rules more often prohibit it than require it, while pull-request rules more often require disclosure. A third of instruction sets have a generator-associated signature, a tool-inserted block, shared text or an AI-marked creation commit; four in five are edited after creation. Prespecified tests associate repository popularity with rules on pull requests and AI contributions. Exploratory analyses link activity to process-oriented content and show that LLM and agent projects are over-represented among adopters. The models agree much more closely with each other than with the human reviewer's topic assignments, a caution for automated annotation. We release the annotations, taxonomies and pointers to the analysed file versions.

0.42

When Instruction Files Go Stale: Documentation Debt and Hidden Cost in Coding Agents

cs.SE cs.AI ·2026-11-06 ·Tural Mehtiyev, Biruk Tadesse, Lawrence Arkoh, Wesley Assunção
code/data similarity 0.62
Abstract · 11 pages, 5 figures, 3 tables. Replication package: https://doi.org/10.5281/zenodo.23992004

LLM-based coding agents such as Claude Code are guided by repository instruction files (also called context files or agent configuration files) like AGENTS.md and CLAUDE.md, which are read in every agent session. As any documentation artifact, these files can fall behind the code and become stale (e.g., a test folder moves, but the file still points to the old one). In technical debt terms, such a stale instruction is documentation debt, and its interest is the cost it adds to every later agent session. Evaluations that check only task success can miss this cost, and the stale file can persist. In this study, we ask what leads agents to act on a stale instruction, how it affects their behavior and at what cost, and how they recover. We study three real-world stale instructions from active open-source repositories in depth, each a different kind of guidance: a test command, a code location, and a procedure for adding a component. Agents fix 22 maintenance tasks under three versions of each instruction (left stale, corrected to the current path, and removed), with the code and the task held fixed, and we analyze 732 trajectories of two models at the level of individual tool calls. We find that agents follow stale guidance when it is relevant to their tasks, sometimes even when correct information is available elsewhere. On the stale test command, Claude Sonnet 5.5 needed about two extra tool calls per run, a third more than with the corrected command; in the other cases, we could not detect a difference. Although agents generally recovered from the immediate failures, none repaired the underlying instruction file. Correcting the instructions eliminated operations on obsolete paths, while removing them prevented most. Our findings show that task success alone can hide the costs of documentation debt, highlighting the need to evaluate agent behavior beyond final outcomes.

0.25

From Delegation to Action Approval: An Exploratory Study of Developers' Practices in Collaborating with AI Coding Agents

cs.SE ·2026-11-06 ·Yinan Wu, Dominik Wermke, Ze Shi Li, Kathryn Thomasset Stolee, Bowen Xu
similarity 0.61
Abstract

Developers now collaborate with AI coding agents that write code and take actions on their behalf. Beyond reviewing this code, developers decide what to delegate and which agent actions to approve. Some of these actions, such as running shell commands or reading files, can have effects that never appear in the resulting code. How developers make these decisions and how much effort this requires are not well understood. To better understand this collaboration, we surveyed 203 developers who use AI coding agents. We found that careful code review does not necessarily extend to action approval: 61% of developers who carefully reviewed most or all generated code still often approved at least one type of agent action without close review. Developers aware of security or privacy incidents or concerning practices did so more often than those who were not. We also found that autonomy varied more between developers than between development activities. Although developers reported a lighter workload, supervising agents remained mentally demanding. When asked what would make them grant more autonomy, developers wanted more than reliability: more than half of their responses called for agents that are bounded, observable, or reversible. Our findings suggest that as agents take on more of the coding, the challenge shifts toward supporting the developers who supervise them.

0.10

A Gated Spec-Driven Workflow: An Audit-Trail Study of Coding-Agent Work

cs.SE ·2026-11-03 ·Rodrigo Figueiredo Abdo, Pedro Carvalho Brom
similarity 0.61
Abstract · 15 pages, 2 figures, 6 tables

Spec-driven development (SDD) asks a coding agent to work from specifications, plans, and task lists rather than from the chat transcript. We study one such workflow for Claude Code, in which almost every control is an instruction carried out by the same model that it constrains. The user must approve each stage before the next begins. A reviewer subagent checks each artefact against a fixed, stage-specific defect list, and an executor subagent may only touch the files listed in its task. After each block of tasks, the orchestrator runs a file-inventory check to detect undeclared changes. We analyse the logs and transcripts of nine features in seven projects, all run by the authors, ranging from a Fortran solver and Python programs to books and this paper. Of these, five finished with every planned task done, including the Fortran solver feature, and 288 of the 581 planned tasks were completed block by block. The executor kept to its declared files: it left only one undeclared change in the deliverable folder, and the check caught it, although its reports record other departures from the protocol. The weak point was the orchestrator, which runs and judges its own checks. It recorded undeclared changes in 11 of 247 block checks, mostly from the authors' in-round requests and the environment, and stopped work in only four.

0.23