CS Paper Radar

Model-Based Agentic Software Engineering

James C. Davis, Kelechi Kalu, Huiyun Peng, Parth V. Patil

cs.SE ·submitted 2026-08-25 21:43 UTC · v1
highly cited

Abstract

Coding agents increase implementation capacity without automatically making project intent, system structure, or acceptance evidence explicit. As implementation becomes abundant relative to engineering judgment, the scarce work shifts toward choosing useful abstractions, producing evidence, and determining which obligations govern acceptance. Existing workflows address parts of this gap through larger prompts, repository retrieval, or perchange review, but still require agents and engineers to reconstruct consequential properties. As an alternative, we present Model-Based Agentic Software Engineering (MAGE). MAGE is a framework and a theory for building trustworthy autonomy from commodity intelligence. MAGE addresses a representation problem and an authority problem: it externalizes the smallest purposeful representation needed to answer an engineering question, then gives settled obligations proportionate authority through constraints, sensors, validators, and gates. It keeps uncertain intent open and turns recurring reconstruction and judgment into durable engineering structure that later work can inherit. We developed MAGE from a longitudinal case and refined it through six independently reported industrial accounts. Across these sources, MAGE explains how externalized knowledge, bounded action, independent evaluation, and retained human authority can compose into a governed engineering environment, and proposes tests of when that environment turns commodity intelligence into durable engineering progress.

Closest papers

Nearest neighbours by meaning across the whole index.

Reality Is the Final Verifier: On Two Key Gaps in Agentic Software Engineering

cs.SE cs.AI ·2026-09-10 ·Alexander Krentsel, Shubham Agarwal, Mert Cemri, Shu Liu, Sidharth Sankhe +3 more
similarity 0.69
Abstract

Software development follows an implementation-verification loop in which developers or agents iteratively revise an implementation until an evaluator, such as a test suite, accepts it. The evaluator checks the implementation against a set of requirements under a model of the deployment environment. Yet even a formal proof that the implementation satisfies the requirements under the model cannot guarantee acceptable behavior after deployment. Requirements only approximate stakeholder intent, and the model only approximates the real deployment environment. We call these together - requirement gap and model gap - the two-gap framework, which unifies the main failure modes of agentic software engineer-ing: reward hacking exploits omissions in the requirements or model, while hallucination widens the gaps by fabricating requirements or environment assumptions. Because neither gap can generally be certified closed in an open, changing world, the goal shifts from closing them to continuously narrowing them. We therefore propose an assurance-revision loop that uses deployment evidence to revise the requirements, model, or evaluator when stakeholders reject the resulting behavior. We then cast assured agentic development as a resource-allocation problem over human judgment, agent capability, and compute. The two principal bottlenecks mirror the two gaps: human judgment for the requirement gap and faithful, costly evaluation for the model gap. Reality remains the final verifier: acceptable behavior under actual deployment conditions is the ultimate test, while predeployment evaluations remain proxies for it.

0.39

Let the Agent Do It? How Software Practitioners Understand and Make Permission Decisions in Agentic AI Assistants

cs.SE ·2026-10-05 ·Larissa Salerno, Haoyu Gao, Gregory Gay, Alexander Serebrenik, Philipp Leitner
similarity 0.68
Abstract · 21 pages, 6 figures

Agentic AI assistants increasingly act on developers' behalf by modifying files, executing commands, and accessing external resources. These actions often require permission, yet little is known about how practitioners make permission decisions while still benefiting from agent autonomy. To address this gap, we conducted a sequential mixed methods study, interviewing 18 practitioners who use AI agents and then surveying 115 practitioners based on the interview findings. We find that practitioners often understand agent behaviour through what they can directly observe and review, while decisions, data use, and other activity behind the scenes remain less clear. This uncertainty also shapes permission decisions, which depend on the scope and risk of an action, whether it fits the task, familiarity with the agent, and the environment in which it operates. Practitioners respond by adjusting how closely they oversee agents, from setting limits in advance to monitoring execution and reviewing work afterwards. How much scrutiny they apply depends on factors such as trust, task importance, time pressure, and the consequences of an action. Our findings suggest that permission systems should make consequential actions easier to review, distinguish what an agent is allowed to do from what the user intended, make reversibility clearer, avoid treating repeated approvals as stable preferences, and distinguish rejecting a single action from rejecting an entire approach.

0.15

A-SAW: The Agentic Software Architecture Workflow

cs.SE ·2026-10-09 ·Finn Klatt, Luca Di Grazia, Johannes Schneider
similarity 0.68
Abstract

AI assistance in software engineering focuses on implementation and verification, leaving the design phase, and in particular the maintenance of architecture documentation, comparatively underserved. Architecture as Code makes architectural artifacts versionable, diffable, and machine-readable, but it does not reduce the effort of keeping them updated, so design documentation still ends up as a fragmented, author-dependent, and only partially trusted record of why a system looks the way it does. In this paper we introduce the Agentic Software Architecture Workflow (A-SAW), a process-level artifact that extends Architecture as Code with a persistent agent layer: an IDE-integrated agent drafts architecture documents from a version-controlled C4 model, a typed element catalog, and instrumented templates. Every draft passes a mandatory human review gate before it is accepted. We evaluate it in a large industrial organization along two tracks: a rubric assessment of 45 architecture documents including 13 matched source-regeneration pairs, and 13 task-anchored practitioner interviews. Our results show that the workflow collapses the between-author variance that made quality unpredictable. Intrinsic quality also rises, by a median of +19 points across the matched pair. The cohort has a mean of 58% and 89% bundle templates, catalog, agent, and a richer regeneration context. So they bound the agent's contribution from above rather than isolating it. In the agent-generated documents, completeness (100%), consistency (93%), and machine-readability (99%) reach or approach the rubric maximum, while traceability reaches 79%. Eleven of the 13 participants rejected agent output that no human had reviewed, and one warned that polished output may lead reviewers to skip the review. Our work supports keeping human review mandatory and concentrating it where the agent must infer beyond the repository.

0.20

Agent Trajectories as Evidence for Software Quality Assurance

cs.SE cs.AI ·2026-11-17 ·Jonathan Ullrich, Julien Siebert, Michael Kläs, Adam Trendowicz, Andreas Jedlitschka
similarity 0.68
Abstract · 3 pages, submitted to ICSE2027 AMPLIFY Workshop https://conf.researchr.org/home/icse-2027/amplify-2027#co-create-software-engineering-resear…

Autonomous software agents increasingly plan, implement, and test software with limited human oversight. Their decisions may introduce implicit assumptions that final code and passing tests do not adequately expose. We propose analyzing observable agent trajectories alongside software and agent specifications and AI generated artifacts as evidence for targeted quality assurance. The aim is to prioritize consequential decisions for scrutiny, investigate their possible contribution to software problems, and inform improvements to specifications and development practices. Industrial experience motivates the idea; available execution records and preliminary analysis methods make it plausible. However, trajectories alone do not establish that decisions are justified, outcomes acceptable, or corrective changes effective. We ask how this evidence can support review, diagnosis, and continuous quality improvement, and when simpler QA approaches suffice. Perspectives from requirements engineering, verification and validation, agent systems, causal inference, empirical software engineering, and human factors could help develop and evaluate the idea.

0.16

Mind the Intent Gap: A Methodology for Collaborative Agent-Driven Software Engineering

cs.SE ·2026-10-27 ·Nicolas Tacheny, Tom Mens, Saïd Mahmoudi
position similarity 0.67
Abstract

AI agents have entered software teams, first as tools and now as team members. They produce in minutes what used to take days, yet the expected gain in delivery often fails to appear. We argue that they are entering processes designed for people, which quietly relied on people to cover three of their weaknesses. The team's intent was never fully written down, because colleagues filled the gaps. An AI agent fills them too, though by guessing, and drifts from what the team had in mind. The reasoning behind each deliverable was already lost from one step to the next, and AI agents now inflate and deflate the documents passed along. The reasoning people did alone was partly private and now moves into AI tools that keep it private by nature. We present CAIA, for Collaborative Agentic Intent Alignment, a methodology for software teams that brings agents into the way they work. A knowledge base describes the product that a project maintains. For each Change, a micro-team aligns on its intent before agents evolve the product and propose updates to the Knowledge Base. The proposed CAIA methodology emerged from an industrial need and is already in use at Ni2, with a platform built to support it. We present the methodology and outline our plans to evaluate it further.

0.26

Responsibility Boundaries for Agentic Systems: A Comparative Evaluation of Probabilistic Agentic Programming

cs.SE cs.AIcs.PL ·2026-10-23 ·Khadim Lo, Soukaina Alaoui Ismaili, Jean-Sébastien Dessureault
similarity 0.67
Abstract · 43 pages, 8 figures, 9 tables

Language-model agents combine reasoning, tools, memory, delegation, and external effects. Authority is often distributed across prompts, host code, configuration, and logs. Probabilistic Agentic Programming (PAP) makes responsibility boundaries explicit programming abstractions connecting mandate, authority, effects, and evidence. We compare PAP with programming paradigms, agent-oriented and BDI systems, workflow and policy languages, and LLM tool-use frameworks along seven dimensions. A mediation kernel formalizes checks before tool execution, operation-bound approval, resource limits, and evidence. Inductive proofs establish conditional safety for arbitrary model responses, assuming correct guards and complete mediation. A backward simulation matches graph executions to source executions in a defined sequential Touba subset. Its human checkpoints release program continuations; operation-bound approval remains a separate kernel contract. Finite-state exploration checks the model. Scripted enforcement tests exercise selected refusal paths and permitted controls. An exploratory study combines seven business cases, including two three-agent workflows, seven models from three providers, five methods, and five repetitions per case and method. Shared English instructions and exact checks of stored outcomes yield 1,225 assessed observations. Native Touba completes all 35 observations on six models and 30 on the remaining model; structured Touba ranges from 21 to 35. Native matches the highest observed count for each model, including ties with the frameworks. All paired case intervals include zero. These observed counts describe the evaluated tasks; the runtime authority boundary remains fixed. The results connect responsibility contracts, execution rules, and evidence.

0.15

A Case Study in Assuring AI-Written Software

cs.SE cs.AIcs.MA ·2026-10-06 ·Lindsey Ferris, Sierra Bonilla
published similarity 0.67
Abstract · Accepted to the NeurIPS 2026 Meta-Agents Workshop

Software-engineering agents can enable people without formal software training to build systems they could not otherwise implement and simultaneously can produce more code than even experts can meaningfully inspect. In both cases, exhaustive code review is not reliable as the sole basis for human control. We report a case study of a production healthcare platform built through coding agents and governed by an operator without formal software-engineering training. Over time, its workflow grew into a human-led meta-agent system where one agent wrote code, other agents supervised and reviewed it, and project rules carried lessons forward. The operator found that tests, monitors and reviewing agents used to supervise the system were fallible. Some monitors measured proxies rather than outcomes, some audits failed silently, missing checks disappeared from reported results and one automated repair caused operational disruption. In this case, human control depended on keeping the intended outcome, the evidence used to judge it, the agents' permissions and the final human decision were all tied to the same underlying objective.

0.33

SCARVA: Interpretable Agentic Reasoning in Software Supply-Chain Analysis

cs.CR ·2026-11-19 ·Tim Barsch, Bruno Nogueira, Tobias Wienand, Jan Fopma, Flavio Toffalini
similarity 0.67
Abstract

Complex applications may inherit security vulnerabilities from their dependencies, though a vulnerable dependency does not necessarily endanger the application. Existing tools flag dependencies associated with public vulnerabilities, and more advanced ones analyze the call graph to assess whether the vulnerable code is reachable. However, these approaches are limited to specific programming languages and ecosystems. LLM agents are promising candidates for this task, yet they hallucinate, take shortcuts, and reward hack. As these behaviors are opaque, the agents' conclusions may not be trusted. We present SCARVA a framework designed to produce interpretable reports. SCARVA combines advanced agent reasoning with an evidence-collection methodology that assesses the agent's analysis and improves its trustworthiness. Specifically, the ANALYST performs a software security assessment while SCARVA records every tool call, file access, and web request. Then, one or more independent JUDGE agents inspect the ANALYST's result and produce an evidence-backed report. We evaluate SCARVA on 54 manually curated cases, balanced across exploitable, non-exploitable, and vulnerability-free projects, using both open-weight and frontier models. The best ANALYST model, Qwen 3.8, reaches 82% balanced accuracy and surpasses the evaluated proprietary models. Human annotation validates the JUDGE findings: Claude Opus 5.5 performs best, agreeing with annotators in 94% of claim checks. Finally, we discuss the most common failure modes of agents on supply-chain vulnerability assessment.

0.11