CS Paper Radar

Who Is Doing the Thinking? Characterising Human Agency in First-Year Students' AI Interactions

Ali Shakiba, Thomas Chaffey

cs.SE cs.HC ·submitted 2026-11-03 12:22 UTC · v1

Abstract

When students use generative AI to produce their work, the learning process becomes invisible. Established frameworks such as Bloom's taxonomy then classify the cognitive level of what the AI produced, not what the student contributed. We propose the Human Agency Depth Taxonomy (HADT), a preliminary six-tier taxonomy that classifies the engineering contribution a student makes in a prompt, independent of the AI's output. HADT combines three dimensions: contextual bandwidth, engineering abstraction and student agency. We applied HADT to 962 prompts from 105 students' structured AI Journals in an AI-permissive, first-year software engineering course of 145 students, and examined how agency changed over ten journalled weeks. Most prompts (84.2%) carried some student contribution beyond delegation. Agency tracked the development lifecycle. Architecture-discovery prompts dominated the requirements phase (48% of turns) and fell to 5-12% from week 9, while prompts analysing existing systems rose during build and verification (odds ratio 2.88, with turns clustered by student). Two-thirds of the variance in tier lay within students, so agency depended more on the phase of work than on the student. A turn-level rise in delegation in deadline weeks did not hold once turns were clustered by student. Explicitly specifying constraints on the AI's output was rare throughout, which we interpret as a knowledge bottleneck for novices rather than a gap in instruction. We discuss implications for course and assessment design, and for using HADT as a formative lens on how students engage with AI.

Comment
11 pages, 2 figures, 3 tables

Closest papers

Nearest neighbours by meaning across the whole index.

Judgment-Centred Software Engineering Education: A Post-Hype Review and Framework for AI-Augmented Learning

cs.SE cs.CY ·2026-09-24 ·Qusay H. Mahmoud
similarity 0.66
Abstract · 22 pages, 1 figure, 3 tables

Generative artificial intelligence has moved from a disruptive novelty to a recurring part of software-development and computing-education workflows, while software agents are beginning to act across repositories, command lines, browsers, tests, and other tools. The educational problem is no longer whether students should be allowed to generate code, but whether software-engineering (SE) programs can preserve and assess human understanding while preparing students to work responsibly with increasingly capable AI systems. This paper presents a structured integrative review of research and practice from 2023 through 23 September 2026, supplemented by established work on AI literacy, technical debt, and human-AI collaboration. The evidence supports a conditional conclusion: GenAI can improve access to explanations, feedback, practice, and short-term task completion, but learning outcomes depend on prior knowledge, scaffolding, verification, task design, and assessment. We examine student help-seeking and authorship, faculty assessment and policy demands, and the wider SE risks created when generated artifacts persist across repositories, teams, architectures, and deployed systems. We extend the concept of comprehension debt: the deferred learning and maintenance cost that arises when AI-assisted production outpaces a learner's or team's ability to explain, test, modify, and justify the resulting software. We then refine the AI-Augmented Software Engineering Education (AASEE) framework into five non-linear integration levels and four cross-cutting evidence obligations: explain, verify, modify, and account. The framework links delegation to evidence, governance, and recovery mechanisms appropriate to its consequences.

0.33

Surrendering and Reclaiming Judgment: How Software Engineering Students Experience AI Reliance

cs.SE ·2026-11-09 ·Nathan Cassee, Umit Akirmak, Luiz Guerra, Margaret-Anne Storey, Neil Ernst
similarity 0.65
Abstract · Submitted to TOSEM

Software architecture is one of the skills novice developers must master to succeed in industry, and as AI disrupts software development, it is also transforming how students learn architecture. Existing work on how students and novices work with AI has mostly focused on small programming exercises or on students' perceptions of AI, so little is known about how students experience AI as they learn software architecture in larger team projects. We report on an exploratory case study of 43 senior software engineering students who used AI throughout a semester-long, team-based software architecture project. We combined surveys with focus groups and analyzed the focus group transcripts and the open survey responses through Braun and Clarke's reflexive thematic analysis. We constructed two themes. The first, Surrendering Judgment to AI, traces how participants' reliance on AI deepened as they used it to architect and implement their systems, until they surrendered their judgment and became stuck in a codebase they had created but could not understand, which made them rely on AI even more. Participants described feeling ashamed, frustrated, and emotionally disconnected from their work, a toll that, in our reading, left them with even less reason to break their reliance. The second, Applying Human Judgment, centers on participants who kept or reclaimed their judgment: the knowledge and experience it rested on, and the strategies they used to apply it. Participants' accounts suggest that relaxed deadlines and a lecture and lab on working with AI gave them room to acquire knowledge and reclaim judgment. Given the expectations placed on junior developers and the widespread adoption of AI, we believe these first-hand accounts can help software engineering education and industry adopt AI in ways that preserve novice developers' judgment, ownership, and motivation.

0.28

Making the Invisible Visible: A Framework for Reflective AI Use in Software Engineering Education

cs.SE cs.CYcs.HC ·2026-09-28 ·Ali Shakiba, Thomas Chaffey
publishedhighly cited similarity 0.65
Abstract · 9 pages, 2 figures, Accepted for publication in the Proceedings of the 37th Annual Conference of the Australasian Association for Engineerin…

Generative AI (GenAI) is increasingly embedded in software engineering education, supporting activities such as requirements development, design exploration, documentation, and prototyping. However, educators often have visibility only into final artefacts, with limited insight into how students evaluate, verify, and refine AI-generated outputs during the learning process. This creates challenges for assessing evaluative judgement and responsible AI-assisted practice. This paper introduces the AI Journal, a structured reflection framework designed to make student-GenAI interaction visible in first-year software engineering education. The framework combines execution tracking, which records prompts, outputs, intent, and interaction context, with cognitive auditing, which captures verification strategies, intervention decisions, confidence judgements, critical learning moments, and reflections on AI-supported work. Deployed in a first-semester software engineering course, the AI Journal enabled visibility into aspects of student learning not observable through artefact-based assessments alone. Preliminary observations suggested variation in verification practices, intervention strategies, and perceptions of AI-supported work. Critical learning moments frequently occurred when students evaluated contextual suitability, feasibility, and requirements alignment rather than identifying obvious errors. The AI Journal demonstrates a practical, lightweight, and model-agnostic approach for making AI-assisted learning processes visible. By foregrounding verification, intervention, and reflection, it shifts attention from product-focused assessment toward evaluative judgement and responsible AI-assisted practice.

0.51

When AI Can Build It, What Should We Assess? Rethinking Evidence of Engineering Competence

cs.SE cs.HC ·2026-10-26 ·Ali Shakiba
similarity 0.63
Abstract · 11 pages, 1 figure, 4 tables

Generative AI can produce code, tests, documentation, and technical explanations that resemble artefacts traditionally used as evidence of student competence, weakening the inference from artefact quality to the engineering capability behind it. We report the redesign and first deployment of a 13-week, AI-permissive, software-intensive engineering course in which students may delegate substantial work to AI while retaining responsibility for planning, verification, review, integration, and explanation. The redesign couples an engineering accountability workflow, which structures human-AI work and preserves human decision authority, with an assessment evidence architecture comprising six complementary channels: product, process/origin, verification, explanation, reflection, and individual assurance. A collect--cross-check--escalate (CCE) reasoning logic connects these channels rather than treating any single artefact or trace as sufficient evidence of competence. By the end of Week 11, the deployment involved 87 students organised into 17 teams, with 957 individual weekly Git repositories and 34 group-project Git repositories supporting repeated assessment and engineering activity. The first deployment yielded four practical lessons: AI assistance changes where rubric criteria carry evidential weight; multiple evidence channels require deliberate cross-checking and meaningful complementarity; the evidential strength realised in practice may differ from that intended in the assessment design; and automation scales evidence collection more readily than interpretation. The contribution is a conceptually grounded, deployed design for making engineering responsibility and competence more visible when AI can perform substantial artefact production.

0.20

Code, Commits, and Complexity in the GenAI Era: A Longitudinal Study of Team-Based Capstone Projects

cs.CY cs.SE ·2026-11-05 ·Asma Shakil, Claus Brabrand, Paul Denny
published similarity 0.63
Abstract · Accepted to SIGCSE 2027

Generative AI tools are now widely available to students, but little is known about how team-based project codebases have changed since they appeared. This matters when project courses use repository activity as evidence of progress and learning. Existing studies have largely examined self-reports, controlled tasks, or short individual assignments, leaving open questions around how student codebases evolve in authentic project settings. We present a ten-semester analysis of N=260 capstone team repositories from 2021 to 2025, spanning pre- and post-AI cohorts. We examine three dimensions: code size (lines of code), code characteristics (comment density, error-handling density, and cyclomatic complexity), and development behaviour (commit volume, edit size, and commit frequency). We find that end-of-semester codebases roughly triple, from about 3.4 KLOC in 2021--2023 to about 10 KLOC in 2024--2025. Code characteristics show a clear decoupling: cyclomatic complexity and error-handling density increase, while comment density does not. Later cohorts commit far more often and in greater volume, but individual commit sizes are largely unchanged, suggesting a shift in development rhythm rather than edit granularity. Most changes appear from 2024 onwards and are not uniformly monotonic. We did not directly observe AI use, so we report these as changes that coincide with increased GenAI availability rather than as effects of AI use. Our findings highlight the need to reconsider how student-produced artefacts are used and assessed in project courses.

0.40

Is This Code Mine? Student Ownership When AI Agents Write the Spec and the Code

cs.SE ·2026-11-16 ·Gustavo Pinto, Enne Rebeca Silva de Freitas, Danilo Monteiro Ribeiro
survey similarity 0.63
Abstract · Submitted to ICSE SEET; 12 pages

Software engineering courses are starting to ask students to build entire systems with AI coding agents, but little is known about what students actually do with them and whether they still own what they deliver. If the agent writes both the code and the decisions behind it, the artifacts we grade no longer show what students learned. We studied an assignment in which 18 pairs of students built applications with agents under Spec-Driven Development, combining the mined history of their repositories, the prompts they recorded, and a survey of 33 students. The agents wrote 69\% of the specifications meant to keep students in control, sometimes from a prompt of a few words. Students locate their ownership in the ideas, crediting themselves with half of them but a fifth of the code, yet most could not explain that code to a colleague or modify it without the agent, and what they report having written is unrelated to what their own commits declare. The three teams that met the 100,000-line target did so with generated code the application never uses, in one case proposed by the agent itself, and one team rewrote its history to backdate its specifications. Teaching with agents should therefore assess what students specify, review, and can explain, rather than what they ship, and enforce the engineering process with tools rather than trusting it to students or agents.

0.31

Practical Implementation Report on Introducing Spec-Driven Development Using AI Agents in Software Development PBL

cs.SE ·2026-08-31 ·Hidetake Tanaka, Hiroshi Igaki, Kazumasa Shimari, Kiyoshi Honda, Naoki Fukuyasu
published similarity 0.63
Abstract · Accepted for publication in the 38th International Conference on Software Engineering Education and Training (CSEE&T 2026) : 18 pages, 1 tab…

In recent years, autonomous AI agents such as GitHub Copilot and Claude Code have been rapidly gaining popularity. This study reports on the practical implementation of Spec-Driven Development, a software development methodology premised on AI agents, within a Software Development Project-Based Learning (SDPBL) course for third-year undergraduate students. We defined a workflow consisting of four phases, namely investigation, planning, implementation, and review. We also established an environment tailored for the SDPBL course where AI agents generate documentation and code during each phase. We analyzed the results from three perspectives, namely students' subjective AI usage, implementation throughput, and code comprehension. The analysis reveals that AI usage patterns varied across development phases and teams. Moreover, while AI agent utilization increased implementation throughput, it also tended to encourage students to proceed with development without fully understanding the code. This study demonstrates that regular verification of code comprehension by instructors and appropriate feedback are essential for maintaining educational effectiveness when introducing SDD into SDPBL.

0.59

Reclaiming Epistemic Agency: A Critical Framework for Human-Generative AI Co-Agency in Education

cs.CY ·2026-08-27 ·Biranchi Poudyal
similarity 0.62
Abstract

Generative artificial intelligence (GenAI) has been primarily framed as an impartial educational tool. However, this framing overlooks an even larger shift: the reassignment of epistemological authority from teachers to students to machines. This paper presents a conceptual evaluation of the extent to which GenAI redistributes students' and teachers' ability to act in classrooms to produce knowledge, validate each other's claims, and create evidence of student learning while collaborating with and competing against humans. This evaluation draws on various theoretical paradigms, including Distributed Agency, Self-Determination Theory, Society 5.0, and Technology Integration Paradigms, including TPACK and SAMR. While all of the theoretical paradigms evaluated are relevant to the role of agency within education mediated by AI, none of them address the ongoing disparity regarding equitable distribution of power, ownership of the data used to mediate interaction, and accountability in relation to human-mediated interactions. As such, this paper introduces the Ecological Co-Agency Framework, which defines agency in terms of relational, regulatory, and pedagogical processes, conditioned by a defined commitment to human accountability for epistemological claims.

0.17