CS Paper Radar

Recovering Software Architecture Intent from Historical Work Items using Generative AI: A Mixed-Methods Industry Case Study

Dominik Storck, Tobias Eisenreich, Stefan Wagner

cs.SE ·submitted 2026-08-28 14:56 UTC · v1

Abstract

Software architecture is often only partially captured in code, while much of the design intent lives in evolving project artifacts. In agile projects, work items, user stories, and related tracking documents preserve valuable traces of that intent, but they rarely support direct architectural analysis. This work investigates the recovery of C4 architecture diagrams from historical agile work items using an LLM-based pipeline. The semi-automatic five-step workflow employs a prompt chain, bidirectional traceability, and Chain-of-Thought reasoning to transform unstructured Azure DevOps work items into visual artifacts. Evaluated on two industry projects, we use a mixed-methods design combining qualitative expert interviews with a quantitative stability analysis. Practitioners perceive the generated architectural baselines as accurate and highly useful for system comprehension. Strictly bound by their input data, the artifacts mirror the documented intent, thereby surfacing discrepancies and architectural drift when compared to the implemented reality. Quantitatively, the workflow exhibits high stability for architectural entities but lower stability for their relationships, with relative variance compounding across generation steps. The proposed workflow demonstrates the practical viability of LLM-assisted architectural recovery based on development process artifacts.

DOI
10.1145/3843775.3844549

Closest papers

Nearest neighbours by meaning across the whole index.

A-SAW: The Agentic Software Architecture Workflow

cs.SE ·2026-10-09 ·Finn Klatt, Luca Di Grazia, Johannes Schneider
similarity 0.68
Abstract

AI assistance in software engineering focuses on implementation and verification, leaving the design phase, and in particular the maintenance of architecture documentation, comparatively underserved. Architecture as Code makes architectural artifacts versionable, diffable, and machine-readable, but it does not reduce the effort of keeping them updated, so design documentation still ends up as a fragmented, author-dependent, and only partially trusted record of why a system looks the way it does. In this paper we introduce the Agentic Software Architecture Workflow (A-SAW), a process-level artifact that extends Architecture as Code with a persistent agent layer: an IDE-integrated agent drafts architecture documents from a version-controlled C4 model, a typed element catalog, and instrumented templates. Every draft passes a mandatory human review gate before it is accepted. We evaluate it in a large industrial organization along two tracks: a rubric assessment of 45 architecture documents including 13 matched source-regeneration pairs, and 13 task-anchored practitioner interviews. Our results show that the workflow collapses the between-author variance that made quality unpredictable. Intrinsic quality also rises, by a median of +19 points across the matched pair. The cohort has a mean of 58% and 89% bundle templates, catalog, agent, and a richer regeneration context. So they bound the agent's contribution from above rather than isolating it. In the agent-generated documents, completeness (100%), consistency (93%), and machine-readability (99%) reach or approach the rubric maximum, while traceability reaches 79%. Eleven of the 13 participants rejected agent output that no human had reviewed, and one warned that polished output may lead reviewers to skip the review. Our work supports keeping human review mandatory and concentrating it where the agent must infer beyond the repository.

0.20

When Relevant Code Is Wrong: Runtime-Aware Context for AI Coding Agents

cs.SE ·2026-10-28 ·Wesley K. G. Assunção, Ramesh Periyathambi
similarity 0.65
Abstract

AI coding agents increasingly rely on retrieved code and development history to guide software changes. However, highly relevant artifacts may describe implementations that are inactive or incompatible with the target deployment. Neither semantic similarity nor recency establishes whether retrieved information remains applicable under feature flags, architectural migrations, and deployment-specific configurations. We challenge relevance-oriented retrieval by introducing operational context validity, a task-, deployment-, and time-dependent criterion for determining whether software evidence can legitimately guide implementation decisions. We instantiate this new retrieval principle in CASE, an agent-agnostic framework that connects repository artifacts with runtime configurations and architectural evolution, classifies evidence as valid, invalid, or unknown, and generates context packages distinguishing actionable guidance from historical rationale and unresolved constraints. We investigate its feasibility through a reproducible preliminary evaluation involving six development tasks, 15 artifacts, and three deployments. Conventional retrieval exposes 16 incompatible artifact occurrences as actionable, whereas CASE excludes them while retaining all applicable artifacts under the prototype's reference rules. Recency filtering reduces valid recall to 0.43, while removing feature-flag evidence from CASE reduces it to 0.13. Although these results demonstrate feasibility rather than improvements in coding-agent correctness, they reveal limitations of relevance-based retrieval and motivate a new research direction: enabling coding agents to reason not only about which information is relevant, but also whether it remains operationally valid.

0.09

The Code of Theseus: Can Agents Rebuild Software from Recovered Design Documents?

cs.SE ·2026-10-14 ·Cezar Sas, Sandra Mitrović, Fabio Barth, Georg Rehm, Matteo Salani
similarity 0.65
Abstract · 12 pages, 3 figures, 7 tables

LLM agents increasingly design a system before implementing it, yet the data to train and evaluate them centres on implementation and testing: requirements and design artefacts are often missing, outdated or private. Source code, in contrast, is abundant. Prior work recovers architecture documentation by feeding LLMs the source code or letting agents explore it, which saturates the context, does not scale to many repositories, and yields a single artefact rather than aligned ones. We present a framework that recovers aligned stakeholder intent, requirements and architecture from a compact repository representation, a static dependency graph and key contextual files, and release the artefacts of 10,764 Java, Python, C/C++ and C# repositories as a dataset. An LLM judge rates the artefacts of three generators as clear and internally consistent, but less reliably correct and at the right level of abstraction. To test whether the artefacts carry the design, we pose a software Ship of Theseus: a coding agent rebuilds 29 Commit0 libraries from the intent alone or with the recovered requirements and architecture, given only an interface contract and part of the tests. The artefacts do not significantly change the held-out pass rate (33.7% vs. 35.8%), but bring the rebuilt module structure significantly closer to the original (module F1 from 0.54 to 0.63), with either generator and even with concise artefacts. Recovered artefacts thus convey design that tests do not capture, offering a scalable source of design-level data for agents that design before they implement, and the round trip measures this without human references.

0.22

RaivenTracks: Branching Provenance for Conversational Visualization Workflows

cs.HC ·2026-08-14 ·Ella Hugie, Alexandra Irger, Grace Guo, Kenneth Moreland, David Pugmire +2 more
similarity 0.63
Abstract · *Ella Hugie and Alexandra Irger are co-first authors

As AI agents increasingly participate in scientific workflows, scientists are shifting from direct authorship toward oversight, inspection, and steering. LLM-driven visualization systems are a promising interface for this hand-off, yet they remain largely stateless, forcing users to reconstruct context across refinements and offering little support for revisiting prior decisions or exploring alternatives. We present RaivenTracks, a workflow-aware extension of the Raiven DSL-mediated visualization pipeline that treats validated visualization specifications as persistent, branchable checkpoints. Because each checkpoint is a verifiable RaivenDSL specification rather than a dialogue transcript, restoring a node recompiles a known artifact rather than re-interpreting prior context. RaivenTracks contributes a two-level state management architecture that pairs a persistent, branchable version tree with a fine-grained undo/redo stack over runtime visualization settings, across both InfoVis and SciVis backends. A formative pilot study with three visualization researchers shows early promise, with all participants adopting the version tree for branching and recovery, and surfaces design directions for tree navigation, node labeling, and scalability that inform a planned controlled comparison against Raiven without version history. We frame branchable conversational visualization history as a step toward provenance support for future scientist-in-the-loop oversight of AI-driven scientific workflows.

0.04

GADR: Gathering Architecture Decision Records from Meeting Transcriptions

cs.SE cs.AI ·2026-08-18 ·Lucas Daniel Costa da Silva, Kiev Gama
similarity 0.62
Abstract

Existing LLM-based approaches to Architecture Decision Record (ADR) generation share a critical and largely unexamined assumption: that input is already reasonably structured. In practice, architectural decisions emerge from informal, noisy meetings where choices are implicit, fragmented, and entangled with off-topic dialogue, precisely the conditions under which single-pass prompting degrades. This paper presents GADR, a multi-agent, self-correcting workflow that extracts architectural decisions from raw meeting transcriptions and generates Nygard-formatted ADR drafts. A feasibility study comprising five real project meeting transcripts, expert review by four senior architects, and evaluation by fifteen students provides initial evidence that the agentic workflow captures most expert-identified decisions and produces drafts participants found clear and useful, outperforming zero-shot and few-shot baselines in stability and structural adherence. The study also addresses the underexplored trade-off of RAG-based enrichment improving ADR depth while simultaneously risking transcript-unfaithful content, raising open questions about traceability in automated architectural documentation that we believe is worth the community's attention.

0.28

From Component Snapshots to Lifecycle Traces: Agent-Based Software Composition Analysis

cs.SE ·2026-09-16 ·Chaofan Li, Zhengduo Xue, Chengxiang Li, Yutao Hu, Yueming Wu +1 more
similarity 0.62
Abstract

Software supply-chain security requires accurate identification of third-party components and an understanding of how they evolve from development to execution. Existing software composition analysis (SCA) approaches examine manifests, build environments, release artifacts, containers, or runtime states, but typically produce only stage-specific views of software composition. As dependencies are resolved, removed, repackaged, and transformed across lifecycle stages, a single snapshot cannot capture both where a component originates and where it ultimately ends up. Combining snapshots from multiple stages still leaves their cross-stage relationships unresolved. We present SCA-Agent, an agent-based approach to lifecycle-aware SCA that reconstructs evidence-backed component lifecycle traces across Code, Build, Release, Deploy, and Runtime. SCA-Agent adaptively explores project-specific analysis paths, gathers stage-specific evidence, and correlates observations across stages to recover component identities, versions, introduction paths, propagation relationships, and final lifecycle states. We evaluate SCA-Agent on 105 real-world projects from the Java, JavaScript, and Python ecosystems. SCA-Agent achieves the highest component detection F1 across all lifecycle stages and ecosystems. For vulnerability exposure assessment, it reaches an F1 score of 96.69%, exceeding the best traditional SCA tool by 18.76 percentage points. These results show that lifecycle-aware SCA supports traceable component provenance and more accurate software supply-chain risk assessment.

0.10

Large Language Models for Requirements Engineering: A Cross-Task Empirical Evaluation

cs.SE ·2026-08-21 ·Jacek Dąbrowski, Manjeshwar Aniruddh Mallya, Alessio Ferrari, Mohammad Amin Zadenoori, Yijun Yu
code/data similarity 0.61
Abstract

Requirements-related information is scattered across heterogeneous artefacts such as user feedback, developer discussions, and software repositories, making the extraction of actionable requirements knowledge labour-intensive and hard to scale. Large Language Models (LLMs) can support many Requirements Engineering (RE) activities, from classification and traceability identification to specification and explanation generation, but existing evidence is fragmented across tasks, artefact types, and evaluation settings, and studies rarely offer cross-task evaluations or replication packages. We present two complementary empirical studies evaluating LLMs across five RE-related activities. The first is a controlled experiment on five lightweight open-source LLMs for feedback-driven requirements classification and specification generation. The second is an exploratory industrial case study on two frontier LLMs for traceability link identification and traceability explanation generation using real project artefacts. Classification and traceability identification were assessed with quantitative metrics, and generation tasks through human evaluation. LLM performance is strongly task-dependent, ranging from moderate to high, and no single model consistently outperformed the others, indicating that effective adoption depends on selecting models and prompting strategies per task. Our contributions are: (i) the first cross-task empirical evaluation of LLMs spanning five RE-related activities, (ii) replication materials supporting reproducibility, and (iii) a broader understanding of the capabilities, limitations, and practical readiness of current LLMs for RE.

0.48

Architectural Degradation: How to Measure and to Remediate

cs.SE ·2026-10-01 ·Noman Ahmad, Ruoyu Su, Matteo Esposito, Andrea Janes, Valentina Lenarduzzi +1 more
similarity 0.61
Abstract

Context. Architectural degradation undermines software maintainability, evolvability, and quality. However, existing research remains fragmented across measurement approaches, metrics, tools, and remediation strategies, limiting our understanding of how these elements relate across the degradation lifecycle. Aim. We consolidate the state of the art on architectural degradation by examining how researchers measure it, which metrics and tools support its assessment, and how existing approaches address remediation. Method. We conducted a Multivocal Literature Review of 284 peer-reviewed and grey-literature studies. We supported screening, data extraction, and classification with a locally executed LLM-assisted pipeline combining Retrieval-Augmented Generation, multi-model validation, and human adjudication. We then analyzed the resulting taxonomies and their cross-dimensional relationships. Results and Conclusions. We identified 277 measurement approaches, 357 metrics, 238 tools, and 395 remediation approaches. Research strongly concentrates on static and structural analysis, structural metrics, and detection-oriented tools. In contrast, remediation spans heterogeneous code-level, architectural, and organizational interventions and shows substantially less consolidation. Overall, the field has developed a mature diagnostic apparatus but has made less progress in connecting degradation detection with effective remediation. Our results provide a structured view of the available techniques and identify the diagnosis-remediation gap as a key direction for future research.

0.16