CS Paper Radar

What Will Remain Human in Software Architecture? A Focus Group Report

Uwe van Heesch, Olaf Zimmermann, Christian Kohls

cs.SE cs.AI ·submitted 2026-09-24 06:36 UTC · v1

Abstract

AI development agents are increasingly used to support and partially automate software architecture tasks. To explore how practitioners perceive this shift, specifically what changes, what remains, and what new responsibilities emerge, we conducted a focus group at the 31st European Conference on Pattern Languages of Programs, People, and Practices (EuroPLoP 2026). Twenty-two participants from industry and academia discussed current practices, trust and validation strategies, the boundaries of AI autonomy, governance challenges, and implications for education. Among others, we found broad consensus that architectural decision-making, accountability, and the authoring of architectural guardrails remain fundamentally human tasks. A central emergent concept was harness engineering: the discipline of building the system that governs AI-assisted system creation, comprising validation mechanisms, knowledge lay- ers, and company-specific standards. The participants agreed that criticality, understood as the combination of uncertainty and cost of change, serves as the universal criterion for calibrating human oversight. A further concern was cognitive debt: the progressive erosion of human understanding of the system when AI-assisted decisions are accepted without full intellectual engagement. In this report, we present the findings of the focus group and describe directions for future work.

Comment
21 pages, EuroPLoP 2026, no figures

Closest papers

Nearest neighbours by meaning across the whole index.

Judgment-Centred Software Engineering Education: A Post-Hype Review and Framework for AI-Augmented Learning

cs.SE cs.CY ·2026-09-24 ·Qusay H. Mahmoud
similarity 0.70
Abstract · 22 pages, 1 figure, 3 tables

Generative artificial intelligence has moved from a disruptive novelty to a recurring part of software-development and computing-education workflows, while software agents are beginning to act across repositories, command lines, browsers, tests, and other tools. The educational problem is no longer whether students should be allowed to generate code, but whether software-engineering (SE) programs can preserve and assess human understanding while preparing students to work responsibly with increasingly capable AI systems. This paper presents a structured integrative review of research and practice from 2023 through 23 September 2026, supplemented by established work on AI literacy, technical debt, and human-AI collaboration. The evidence supports a conditional conclusion: GenAI can improve access to explanations, feedback, practice, and short-term task completion, but learning outcomes depend on prior knowledge, scaffolding, verification, task design, and assessment. We examine student help-seeking and authorship, faculty assessment and policy demands, and the wider SE risks created when generated artifacts persist across repositories, teams, architectures, and deployed systems. We extend the concept of comprehension debt: the deferred learning and maintenance cost that arises when AI-assisted production outpaces a learner's or team's ability to explain, test, modify, and justify the resulting software. We then refine the AI-Augmented Software Engineering Education (AASEE) framework into five non-linear integration levels and four cross-cutting evidence obligations: explain, verify, modify, and account. The framework links delegation to evidence, governance, and recovery mechanisms appropriate to its consequences.

0.33

Understanding-First AI Engineering: Purposes, Reference Architecture, and Architectural Patterns for Human Understanding

cs.AI cs.SE ·2026-11-09 ·Liming Zhu, Qinghua Lu, Ming Ding
hot direction similarity 0.67
Abstract

AI systems can produce reliable results without developing human understanding. However, human understanding remains valuable for shaping subsequent AI-supported work, forming preferences, governing collective choices and satisfying curiosity, even when human participation is unnecessary for correcting errors. We propose understanding-first AI engineering, which treats purpose-specific human understanding as a first-class design principle guiding requirements, design and operation. The purposes of understanding are organised into four complementary rationales: reflective agency, effective action, collective governance and epistemic value. A reference architecture connects production, verification, use, and learning and exploration through shared records, supporting understanding through explanation and participation. Five candidate architectural patterns guide choices about parallel learning, explanatory models, goal exploration, audience-specific views and distributed expertise. Qualitative scenario analysis of two hypothetical use cases demonstrates how the purpose classification, reference architecture and selected patterns guide investigation, participation and decision processes while retaining operational controls. The cases illustrate design consequences and limitations rather than establish empirical effectiveness. The approach makes human understanding an explicit engineering commitment whose justification need not depend on improving the original AI task.

0.25

AI Agents Push Humans Out of the Loop

cs.AI cs.HC ·2026-08-24 ·Margaret Mitchell, Avijit Ghosh, Samir Passi
positionhighly cited similarity 0.66
Abstract

AI agents pose significant risks as they are granted increasing autonomy. A commonly proposed solution is human oversight and keeping a ''human in the loop'', but this is not a simple solution: Not only do current approaches to AI agent design impede effective human oversight, but the cognitive capacities required for it are also themselves degraded by extended use of AI systems. This position paper argues that current approaches to the development and deployment of AI agent systems do not support effective human oversight -- they contribute to its degradation. To address this, a top priority in the advancement of AI agents should be supporting the situated goals and cognitive requirements of effective human oversight, treating the human needs of overseers at the same level of importance as AI agent capability. To put this idea into practice, we connect work on automation and human-computer interaction to AI agent processes, outlining design-level affordances and organizational protocols that (1) support overseers in exercising critical judgement and (2) counteract the skill atrophy that arises from extended use of automation. We urge developers and deployers to adopt these or similar approaches. Without explicit support for the cognitive demands of effective human-agent interaction, AI agent systems will continue to passively incentivize the degradation of the very human skills they rely on.

0.41

Technical Debt Risks in AI-Assisted Software Development: Insights from Practitioners

cs.SE ·2026-11-11 ·Nazia Jahan, Nafisa Alam, Sourov Debnath, Arafat Rahman, S. M. Waesh Ahmed +2 more
similarity 0.66
Abstract

AI coding assistants increasingly support software development, raising concerns about technical debt associated with AI-generated code and development practices. We investigate these concerns through a sequential mixed-methods study involving 137 survey respondents and 20 follow-up interviews with software professionals in Bangladesh. Drawing on ten established technical debt dimensions, we examine perceived risks, contributing circumstances, maintenance consequences, validation practices, and developer understanding. Respondents expressed the strongest concerns about People Debt (73.0\%), Management Debt (70.1\%), and Architecture Debt (68.6\%), suggesting that risks extend beyond code quality. Participants associated ambiguous requirements, insufficient project context, and unrequested AI-generated changes with architectural inconsistencies, rework, and maintenance difficulties. Their experiences revealed contrasting outcomes: AI assistance supported refactoring, maintainability improvements, and testing, but also introduced rework and context misalignment. Although 89.8\% reported regularly verifying AI-generated code, participants expressed concerns about AI dependence, reduced code understanding, and review effort. They also emphasized the need for stronger architectural awareness, requirements clarification, and validation support in coding assistants. Our findings suggest that managing technical debt risks requires attention to project context, implementation decisions, and developer understanding alongside generated code quality.

0.11

Cognitive Debt in the GenAI Era - Listening to Practitioners' Voices on the Loss of Understanding

cs.SE ·2026-10-27 ·Theocharis Tavantzis, Bianca Trinkenreich, Michaela Greiler, Júlia Condé Araújo, Marina Condé Araújo +4 more
similarity 0.65
Abstract · 12 pages, 3 figures. Manuscript under review at ICSE-SEIP

Fast-paced AI-assisted software development risks reducing developers' understanding of the systems they build. This growing gap between what the system does and what developers understand about it, described as ``cognitive debt'', is not new, but AI has exacerbated it. This has led to a recent surge in blog posts and online discussions, in which practitioners recognize and write about cognitive debt, and share their experiences well before academic studies can capture them. To understand their concerns and inform future research and practice, we conduct a grey literature review of 76 sources, focusing on how cognitive debt is conceptualized and manifests, its contributing factors, its consequences, and the practices to prevent, detect, and mitigate it. We find that cognitive debt can accumulate unnoticed behind working code and passing tests, particularly when developers accept AI-generated solutions without building understanding. Practitioners report consequences such as harder maintenance and debugging, fragmented knowledge in the team, and reduced productivity. They propose practices such as documenting design rationale, review-sized changes, comprehension gates, and knowledge-sharing rituals, along with signals like reliability metrics to detect it. Our findings give software teams a structured view of emerging practical approaches in AI-assisted development and stress the need to preserve human understanding alongside faster code production.

0.55

An Autonomy Aware Metamodel for Human AI Collaboration in Software Engineering

cs.SE ·2026-09-06 ·Md Mahade Hasan, Md Toufique Hasan, Zheying Zhang, Kalle Kulonen, Konsta Kalliokoski +3 more
similarity 0.65
Abstract

Artificial Intelligence (AI) is shifting software engineering from tool-supported processes towards AI-first collaboration, where authority is dynamically distributed across human and artificial actors. However, existing method engineering approaches assume static, human-centric control and provide limited support explicitly capturing evolving autonomy. This paper presents a vision for autonomy-aware method engineering by proposing a metamodel that treats autonomy not as a fixed property of an actor, but as a derived, situation-dependent authority assignment determined by task, context, and collaboration pattern. The metamodel formalizes autonomy through four authority dimensions: task execution, task decomposition, task initiation, and collaboration reconfiguration. Through an analytical instantiation with a multi-agent requirements analysis tool, we illustrate how the metamodel supports dynamic authority assignment. This work provides a conceptual foundation for governance-aware, adaptable, and AI-first software engineering methods.

0.28

Key Institutional Considerations for Designing Effective Independent Verification Organizations

cs.AI ·2026-11-06 ·Uma Kalkar, Rumman Chowdhury, Reva Schwartz, Gabriella Waters
similarity 0.65
Abstract · 37 pages, 3 tables

Independent verification organizations (IVOs) have emerged as a popular, bipartisan solution for facilitating high-quality independent evaluations of AI technologies. But momentum is outpacing design: there is emerging consensus on the necessary operating conditions for robust IVOs, but little discourse on institutional and policy specifications to ensure those conditions are met. IVOs are currently envisioned primarily as a mechanism operating at the AI frontier and model level to evaluate catastrophic risks. Little work has yet been done on how to establish or enforce an IVO ecosystem that is equipped to handle myriad risks (i.e., beyond catastrophic ones) and operate at the system and application level across the deployment and use stages of the AI lifecycle. This paper argues that IVO design should ensure the capacity for evaluating AI models, systems, and applications in context for both frontier companies and enterprise companies & AI-adopting organizations. Additionally, this paper provides early considerations to advance IVO discourse from normative to operative, and from existential to everyday risks. Drawing from existing institutional constructs in financial services, education, public health, and science, we outline key considerations on whom IVOs serve, what access they require, how they interact with external governing structures, how they are funded, and how they fit into the wider assurance ecosystem. Much of the structure and knowledge necessary to holistically design IVOs and their ecosystem does not yet exist, even as companies rush to ink agreements with their chosen evaluators.

0.27

Governing Bring Your Own AI: A Parameterized Maturity Model

cs.CR ·2026-09-04 ·Dare Bello, John Hastings
survey similarity 0.64
Abstract · 8 pages, 3 figures, 5 tables

Employees are increasingly using personally owned generative AI tools such as ChatGPT, Gemini, and Claude for their daily work. This practice is known as Bring Your Own AI (BYOAI), which is a distinct form of Shadow AI in which employee-authenticated personal accounts are used outside of enterprise identity and security controls. Existing frameworks were designed for AI tools managed by organizations, and their coverage does not extend to unmanaged AI tools used with a personal account. In addressing these issues, we developed a governance model through a systematic review of the literature that produces a risk taxonomy and a framework-engagement profile. We also developed a parameterized governance model that measures how much a level of governance maturity reduces residual risk. A five-level maturity ladder is coupled to a technical control architecture through a chain in which the coverage of the control layer influences the security outcomes. Our study of a curated corpus of 30 records (24 research studies and 6 framework documents) indicated that the most prominent categories identified were data exposure and compliance, and framework engagement was inconsistent. Three mutually supporting pillars (technical, governance, and human) were established to support safeguards. Additionally, the results of the model demonstrated that prohibition-based solutions will result in residual risk levels close to those achieved through baseline solutions. Under the specified parameterization, layered control-based solutions substantially reduce modeled exfiltration risk and increase enforceable coverage.

0.36