CS Paper Radar

EA-Ops: Git-Native Architecture as Code for Continuous Enterprise Architecture Governance

Vahid Tavakkoli, Kabeh Mohsenzadegan, Kyandoghere Kyamakya

cs.SE ·submitted 2026-09-24 22:15 UTC · v1
benchmarkcode/datahot direction

Abstract

Enterprise architecture (EA) repositories frequently separate architecture models from the engineering workflow used to change software and infrastructure. This article presents EA-Ops, an open-source Git-native Enterprise Architecture-as-Code framework that represents architecture facts as YAML, validates typed relationships against an ArchiMate 3.2 profile, enforces organization-specific governance rules, performs graph-based change-impact analysis, and publishes human-facing reports and a static interactive portal from the same reviewed source. We evaluate EA-Ops with a reproducible GitHub Actions harness. Eight independently injected structural, semantic, and governance fault classes were executed across 30 trials each; all 240 trials matched ground truth exactly, with precision, recall, and $F_1$ of 1.000. Scalability experiments with 30 measured repetitions reached 50,000 objects and 100,000 relationships: median validation time was 52.582~s, median impact traversal was 627.000~ms, and peak resident-set size was 919.2~MB. A ten-scenario Metroville digital-permit reference architecture produced exact validation outcomes and exact impact-set agreement with an independent breadth-first-search oracle in every scenario. The configured 100,000-object end-to-end benchmark generated its model successfully but exceeded the 180-minute CI budget during the performance stage; no timing result is extrapolated. At 50,000 objects, Markdown report generation rather than semantic validation is the dominant scaling bottleneck. The results support Git-native continuous governance as a practical EA operating model at tens-of-thousands-of-object scale while defining clear limits and optimization targets for larger repositories.

Closest papers

Nearest neighbours by meaning across the whole index.

SWE-Prometheus: Measuring Engineering Governance Improvements in Real-World Repositories

cs.AI cs.SE ·2026-09-24 ·Jiajun Wu, Leixin Sun, Zihan Tan, Yitao Liu, Zhenjun Zhao +10 more
benchmark similarity 0.66
Abstract

Large language model based coding agents have made substantial progress on repository-level software engineering tasks. Existing repository benchmarks, however, usually start from a human-identified issue and evaluate whether a patch satisfies a functional signal. We present SWE-Prometheus, a benchmark for the broader task of improving repository engineering governance. Each task provides a fixed snapshot and an open-ended objective, requiring the agent to identify risks, prioritize interventions, and verify the resulting changes. SWE-Prometheus evaluates six governance dimensions through paired evidence, clean-environment probes, behavior gates, and two independent teacher ratings of the same evidence. The benchmark contains 60 repositories; ten models are evaluated on a shared 22-repository public subset, where mean Normalized Governance Improvement ranges from 0.0568 to 0.5760 and observed behavior-breakage rates range from 0% to 23%. On a frozen ten-repository batch, a repository-blind template obtains mean NGI 0.272, but its gains concentrate in Tests & CI, Quality Gates, and Documentation; it improves Reproducible Environment and Dependency & Security on none of the repositories. This baseline makes the distinction between adding governance artifacts and producing execution-backed improvements measurable. The no-op condition has median NGI zero and standard deviation 0.073; two teachers agree exactly on 57 of 60 dimension scores for the same no-op evidence. For the two highest conditional-mean systems, common-valid NGI is similar, while full-pool comparisons that include behavior failures favor Kimi-K3. These results show why repository-governance evaluation should report improvement, behavior preservation, evidence quality, and coverage together.

0.35

ArchGuard-APR: Architecture-Constraint-Aware Multi-Agent Program Repair Using Repository Graphs and Execution Feedback

cs.SE ·2026-10-24 ·Majid Feiz Deilami
code/data similarity 0.62
Abstract · 29 pages, 10 tables, 7 figures. Exploratory feasibility evaluation. Software, experimental artifacts, and reproducibility materials are avai…

Repository-level automated program repair must reason about more than a localized failing statement. A candidate change can satisfy a narrow test while violating dependency boundaries, modifying an unsafe path, or relying on context that omits relevant repository structure. ArchGuard-APR is an architecture-aware repair prototype that integrates deterministic repository analysis, immutable repository graphs, explicit architecture profiles, graph-guided context retrieval, role-based repair orchestration, patch-policy enforcement, isolated execution feedback, provider cost controls, and canonical source-free evidence. We evaluate the implemented pipeline through an exploratory paired feasibility study, not a confirmatory effectiveness experiment. Six executions were planned; four completed, and two black-4 conditions ended in pre-provider infrastructure attrition. Across two valid paired tasks, baseline and ArchGuard conditions each produced 0/2 successful repairs, and all four completed outputs were classified as invalid patches. The completed executions consumed 35,431 input tokens and 2,737 output tokens, for a total of 38,168 tokens, at a total cost of USD 0.1296325. No effectiveness advantage was observed, and the sample provides no useful inferential basis. The study instead establishes the engineering feasibility of composing architecture constraints with a reproducible repair pipeline and identifies patch validity and infrastructure robustness as the principal targets for future work.

0.42

Correct Code, Broken Contributions? SWE-CC: Benchmarking Repository Policy Compliance for Coding Agents

cs.SE cs.AI ·2026-10-05 ·Hai Dang Truong, Rayner Goh, Thanh Le-Cong, Yintong Huo
benchmarkcode/data similarity 0.61
Abstract · 39 pages, 6 figures. Benchmark and source code: https://github.com/dangtruong01/swe-cc-arxiv

Autonomous coding agents now resolve a substantial share of real-world GitHub issues. However, passing functional tests differs fundamentally from producing a high-quality contribution acceptable for merging. Mature open-source projects publish repository-specific contribution policies, spanning style, git, testing workflows, to ensure code quality and long-term maintainability. Because existing benchmarks evaluate patches solely on unit tests, agent compliance with repository governance remains unknown. In this paper, we introduce SWE-CC, a benchmark evaluating code and process compliance in autonomous software engineering. We develop a semi-automated pipeline that converts developer documentation across 12 open-source repositories into 823 machine-checkable atomic policies. SWE-CC introduces two features: 1) lightweight, deterministic checker functions that represent each policy, 2) a comprehensive auditing mechanism that inspects both agent runtime behaviors and final deliverables. We evaluate the compliance of agent workflows in 500 end-to-end software contribution tasks extended from SWE-bench Verified. Our evaluation of four LLMs under two agent scaffolds shows that modern agents suffer from coding compliance issues: although agents produce functionally correct patches, they still violate 43.1 percent of applicable project policies, with nearly half of all violations occurring during intermediate execution steps. These results show that functional correctness does not guarantee real-world readiness, highlighting that future software engineering agents must reliably conform to repository governance to enable safe and trustworthy deployment.

0.56

Specification-delta-driven data governance: an empirical study of the «spec-delta» as the unit of change in lakehouse data platforms

cs.AI cs.PL ·2026-08-20 ·Pablo Ramirez Amador
similarity 0.60
Abstract

Spec Driven Development SDD has consolidated the idea that the specification rather than the code should be the primary artefact governing AI assisted work. Tools such as GitHub Spec Kit, and proposals such as Constitutional SDD, have formalised this principle in the software domain, while the executable data-contracts literature has extended it to schema and quality enforcement at run time. Nevertheless, the treatment of the specification delta OpenSpec's core idea that every change should produce a reviewable increment of requirements as the unit of change in data platforms remains empirically unexplored, even though many data-platform changes are contractual (new datasets, service-level agreements, metric semantics, access policies) rather than purely code changes. This work formalises the spec-delta concept, proposes a taxonomy of data platform changes according to their suitability for incremental specification, and defines a controlled experiment comparing a spec-delta-driven workflow against a conventional code pull-request workflow without a delta. The response variables are discovery to deployment time, the density of defects reaching the Silver and Gold lakehouse layers, cross-tool metric divergence, and reviewer cognitive load measured with NASA TLX. The paper explicitly reserves a demonstration-and-laboratory section for instantiation on a real lakehouse environment. The contribution is not a tool but reproducible evidence and an applicability guide that helps to avoid the up front over specification antipattern.

0.23

From Architecture to Binary: Ensuring Cross-Domain Consistency in Model-Based Airborne Software Development

cs.SE ·2026-08-28 ·Nils Schlautmann, Viktor Sinitsyn, Benjamin Engelhard, Florian Holzapfel
similarity 0.60
Abstract · 10 Pages, Preprint for 45th AIAA DATC/IEEE Digital Avionics Systems Conference (DASC). Accepted for Publication

This paper presents an airborne software development approach for manned and unmanned aerial vehicles aimed at reducing inconsistencies across system, model-based functional, and embedded software domains. In environments influenced by standards such as ARP-4754B and DO-178C, these inconsistencies typically stem from insufficient enforcement across domain boundaries rather than missing process definitions. Building on a previously proposed toolchain centered on a relational interface database, we identify recurring failure modes and propose a repository-centered implementation to address them, tailored to small, resource-constrained teams operating without heavyweight process overhead. Each domain is assigned a primary repository with cross-repository references and dedicated CI pipelines that generate, update, and validate the exchanged artifacts. Automated interface updates, differential change notifications, and consistency checks propagate changes with minimal manual effort and surface inconsistencies before the time-consuming code-generation and compilation steps. An initial implementation in an ongoing experimental project is described, with qualitative feedback from its early use.

0.31

A-SAW: The Agentic Software Architecture Workflow

cs.SE ·2026-10-09 ·Finn Klatt, Luca Di Grazia, Johannes Schneider
similarity 0.60
Abstract

AI assistance in software engineering focuses on implementation and verification, leaving the design phase, and in particular the maintenance of architecture documentation, comparatively underserved. Architecture as Code makes architectural artifacts versionable, diffable, and machine-readable, but it does not reduce the effort of keeping them updated, so design documentation still ends up as a fragmented, author-dependent, and only partially trusted record of why a system looks the way it does. In this paper we introduce the Agentic Software Architecture Workflow (A-SAW), a process-level artifact that extends Architecture as Code with a persistent agent layer: an IDE-integrated agent drafts architecture documents from a version-controlled C4 model, a typed element catalog, and instrumented templates. Every draft passes a mandatory human review gate before it is accepted. We evaluate it in a large industrial organization along two tracks: a rubric assessment of 45 architecture documents including 13 matched source-regeneration pairs, and 13 task-anchored practitioner interviews. Our results show that the workflow collapses the between-author variance that made quality unpredictable. Intrinsic quality also rises, by a median of +19 points across the matched pair. The cohort has a mean of 58% and 89% bundle templates, catalog, agent, and a richer regeneration context. So they bound the agent's contribution from above rather than isolating it. In the agent-generated documents, completeness (100%), consistency (93%), and machine-readability (99%) reach or approach the rubric maximum, while traceability reaches 79%. Eleven of the 13 participants rejected agent output that no human had reviewed, and one warned that polished output may lead reviewers to skip the review. Our work supports keeping human review mandatory and concentrating it where the agent must infer beyond the repository.

0.20

Rasoi: Visual Diffs for Evaluating Structural Changes Made by Coding Agents

cs.SE cs.HC ·2026-11-12 ·Riya Sahni, Aryan Kaul, Anushka Bhatnagar, Xuanming Zhang, Vivian Liu +2 more
similarity 0.59
Abstract

Coding agents let developers modify software rapidly through natural language. However, as agents take on larger changes, developers can lose track of how the codebase is evolving. These edits may look correct on the surface while restructuring the system in unexpected ways: duplicating logic, changing modules outside the requested scope, removing functionality developers meant to keep, or introducing architectural drift. We investigate how visual diffs can help developers understand and evaluate the structural changes that coding agents make. We present Rasoi, a design probe that maps how a codebase's architecture evolves after each agent-made edit, letting developers inspect which architectural elements and relationships were added, modified, or removed across multiple levels of abstraction. We deployed Rasoi in a week-long study with 12 developers working on their own codebases across varied technical domains. We found that visual diffs helped developers see whether changes stayed within scope, notice unexpected restructuring, and understand how modified parts related to the system as a whole. Our findings suggest that the value of oversight tools for coding agents lies less in making developers trust the agent and more in helping developers feel confident in their own evaluation of agent-made changes. We discuss implications for visual tools that support code comprehension and human oversight in AI-assisted software development.

0.23

AgentTorrent: A Dataset of Context and Harness Engineering Artifacts on GitHub

cs.SE ·2026-11-11 ·Sebastian Baltes, Seyedmoein Mohsenimofidi, Levi Böhme, Vahram Nersesyan, Muhammad Auwal Abubakar +3 more
code/data similarity 0.59
Abstract · 5 pages, 2 figures, 2 tables, under review

Over the past months, developers have moved from formulating detailed prompts to engineering the context that agentic coding tools read and to configuring the underlying harnesses. For example, developers commit context files, skills, hooks, and MCP configurations to their repositories, and these files control which instructions a harness reads and which actions it can take. Developers may also commit the specs, plans, and task files that agentic coding tools and spec-driven development frameworks write for agents to implement. The AgentTorrent dataset records these files for six proprietary and open-source agentic coding tools in 10,944 engineered software projects on GitHub. It contains 62,374 configuration artifacts, the files that these artifacts import, 27,593 agentic planning documents, and more than 1.2 million commits that credit an AI tool. All heuristics used for data collection are grounded in tool documentation or derived from GitHub repositories, and we evaluate their detection accuracy with each collection. We use open-weight models to classify whether GitHub repositories host engineered software projects. Unlike most datasets, AgentTorrent links configuration artifacts, imported files, agentic planning documents, and commits that credit an AI tool within the same repositories. Configuration artifacts, planning documents, and credited commits occur together in 388 repositories. For example, our dataset allows researchers to study whether a tool is credited in commits that add instructions for that same tool and how instruction files import and contradict each other. Our data collection pipeline supports regular updates and automatically checks vendor documentation and reports unknown artifacts and agentic coding tools. The dataset and the pipeline are open-source and are archived on Zenodo.

0.36