CS Paper Radar

A Text Mining and Classification Approach for Analyzing Architecture Decision Records

Nicolás Miccio Palermo, Antonela Tommasel, J. Andrés Diaz-Pace

cs.SE ·submitted 2026-09-07 11:52 UTC · v1
publishedcode/data

Abstract

Architectural decision records (ADRs) have become a popular lightweight mechanism for documenting architectural knowledge in software projects. However, there is limited empirical evidence on the kinds of architectural concerns captured in ADRs and how well their contents align with established architectural knowledge concepts and documentation practices. In this paper, we propose an automated text-mining and classification approach for analyzing ADRs at scale. We apply this approach to a dataset of ADRs extracted from ~550 open-source repositories, combining topic modeling, LLM-based classification, and template compliance checks. Our analysis examines decision taxonomies and quality attributes, and the degree to which ADRs adhere to the MADR template. Our findings show that ADRs frequently capture existence, technology, and process-related decisions, while alternatives, decisions drivers, and some quality concerns remain under-documented. We also observe recurring mismatches between ADR contents and template sections. These insights into current documentation practices provide architects with valuable information to reflect on how ADRs are and should be used to effectively deal with architectural knowledge. Furthermore, our automated approach is adaptable to other architectural tasks.

Closest papers

Nearest neighbours by meaning across the whole index.

GADR: Gathering Architecture Decision Records from Meeting Transcriptions

cs.SE cs.AI ·2026-08-18 ·Lucas Daniel Costa da Silva, Kiev Gama
similarity 0.65
Abstract

Existing LLM-based approaches to Architecture Decision Record (ADR) generation share a critical and largely unexamined assumption: that input is already reasonably structured. In practice, architectural decisions emerge from informal, noisy meetings where choices are implicit, fragmented, and entangled with off-topic dialogue, precisely the conditions under which single-pass prompting degrades. This paper presents GADR, a multi-agent, self-correcting workflow that extracts architectural decisions from raw meeting transcriptions and generates Nygard-formatted ADR drafts. A feasibility study comprising five real project meeting transcripts, expert review by four senior architects, and evaluation by fifteen students provides initial evidence that the agentic workflow captures most expert-identified decisions and produces drafts participants found clear and useful, outperforming zero-shot and few-shot baselines in stability and structural adherence. The study also addresses the underexplored trade-off of RAG-based enrichment improving ADR depth while simultaneously risking transcript-unfaithful content, raising open questions about traceability in automated architectural documentation that we believe is worth the community's attention.

0.28

Discovering the Relation Between Architectural Models and Reference Architectures

cs.SE ·2026-11-10 ·Thi Dinh Tran, Maria Teresa Rossi, Ludovico Iovino, Leonardo Mariani
top venuecode/datahot direction similarity 0.64
Abstract · Accepted at the 49th IEEE/ACM International Conference on Software Engineering (ICSE 2027). 13 pages, 5 figures, 4 tables. \c{opyright} 2027…

Assessing whether concrete software architectures are consistent with their reference architecture (RA) is essential to ensure that architectural decisions scale coherently across projects, preserving the quality attributes the RA was designed to guarantee. In the absence of dedicated techniques, mapping the components of an architectural model onto a predefined RA can be addressed only with general-purpose model-comparison algorithms. Our experiments show that these algorithms are inaccurate when applied to RAs. To address this gap, we propose Reference Architecture Mapping Discovery (RAMD), an LLM-based workflow that combines model analysis with structured prompting, and refines results through an iterative RA compliance-checking mechanism. We evaluate RAMD on 64 open-source architectural models spanning three RAs from the IoT, self-driving, and smart parking domains. With domain knowledge and GPT-4o, RAMD reaches 85.8% mapping accuracy and outperforms the strongest baseline by 44.6 percentage points, while ensuring structural consistency with the RA.

0.89

Architectural Degradation: How to Measure and to Remediate

cs.SE ·2026-10-01 ·Noman Ahmad, Ruoyu Su, Matteo Esposito, Andrea Janes, Valentina Lenarduzzi +1 more
similarity 0.63
Abstract

Context. Architectural degradation undermines software maintainability, evolvability, and quality. However, existing research remains fragmented across measurement approaches, metrics, tools, and remediation strategies, limiting our understanding of how these elements relate across the degradation lifecycle. Aim. We consolidate the state of the art on architectural degradation by examining how researchers measure it, which metrics and tools support its assessment, and how existing approaches address remediation. Method. We conducted a Multivocal Literature Review of 284 peer-reviewed and grey-literature studies. We supported screening, data extraction, and classification with a locally executed LLM-assisted pipeline combining Retrieval-Augmented Generation, multi-model validation, and human adjudication. We then analyzed the resulting taxonomies and their cross-dimensional relationships. Results and Conclusions. We identified 277 measurement approaches, 357 metrics, 238 tools, and 395 remediation approaches. Research strongly concentrates on static and structural analysis, structural metrics, and detection-oriented tools. In contrast, remediation spans heterogeneous code-level, architectural, and organizational interventions and shows substantially less consolidation. Overall, the field has developed a mature diagnostic apparatus but has made less progress in connecting degradation detection with effective remediation. Our results provide a structured view of the available techniques and identify the diagnosis-remediation gap as a key direction for future research.

0.16

Spotlights: Discovering Improvement Opportunities in Software Repositories

cs.SE ·2026-09-17 ·Udi Barzelay, Ophir Azulai, Idan Friedman, Inbar Shapira, Foad Abo Dahood +7 more
similarity 0.62
Abstract

Coding agents and evolutionary code-search systems can improve implementations once a target and evaluation criterion have been specified. Applying these methods to an existing software repository raises an earlier question: which implementation choices are worth investigating for a high-level engineering objective? We introduce \emph{optimization-opportunity discovery}, the repository-level task of identifying candidate source regions, explaining how they relate to the objective, and proposing possible changes. The task takes as input a repository, an engineering objective, and optional runtime evidence such as offline telemetry observations or profiles. It does not require the user to specify a defect, bottleneck, or code location. We present \emph{Spotlights}, a system that performs this task through logical repository mapping, successive agent reviews, and optional research linking candidates to relevant techniques. We evaluate Spotlights across model serving, document retrieval, blockchain ordering, and document processing. Across three cases, it recovers seven of nine expert-selected targets. In the reliability study, 70\% of the top ten candidates meet the stated correctness and severity thresholds. Across five repeated retrieval runs, 73.6\% of candidate occurrences have a matching source region in all five runs. Spotlights also rediscovers the target of a withheld retrieval optimization and connects it to a relevant tiling technique. In an implementation study, a discovered change reduces end-to-end page-processing runtime by 10.6\% while preserving measured output quality. These results establish optimization-opportunity discovery as a distinct and empirically evaluable step between a broad engineering objective and subsequent implementation and validation.

0.08

Can LLMs Extract Architectural Design Decisions from Source Code Commits? - A Preliminary Exploratory Study

cs.SE cs.AI ·2026-09-03 ·Amey Karan, Rudra Dhar, Mohamed Soliman, Karthik Vaidhyanathan
publishedcode/data similarity 0.62
Abstract · Accepted at IdeaArch Workshop at ECSA 2026

Context: Architectural Design Decisions (ADDs) capture the rationale behind the structure and evolution of software systems but are rarely documented explicitly, and are often hidden inside source code commits. Recovering them is important for Architectural Knowledge Management (AKM). Problem: Extracting ADDs from commits is challenging due to their implicit and unstructured nature. Large Language Models (LLMs) have shown strong capabilities in understanding code and text, yet their effectiveness for this task remains underexplored. Study: We present a preliminary study using four LLMs (Gemini 3 Pro, DeepSeek R1, Kimi K2, Qwen3) with zeroshot and fewshot prompting on 30 developer-written ADDs from open-source projects. We score outputs with ROUGE-L, BLEU, METEOR, and BERTScore, and one author manually reviews the Gemini outputs. Results: All models reach a BERT-F1 above 0.81, and fewshot prompting improves alignment (Gemini BERT-F1: 0.828 to 0.847). However, the generated ADDs are often too long, implementation-focused, and miss the rationale behind the decision. This highlights opportunities for architecture-aware LLM systems and automated AKM.

0.71

A-SAW: The Agentic Software Architecture Workflow

cs.SE ·2026-10-09 ·Finn Klatt, Luca Di Grazia, Johannes Schneider
similarity 0.62
Abstract

AI assistance in software engineering focuses on implementation and verification, leaving the design phase, and in particular the maintenance of architecture documentation, comparatively underserved. Architecture as Code makes architectural artifacts versionable, diffable, and machine-readable, but it does not reduce the effort of keeping them updated, so design documentation still ends up as a fragmented, author-dependent, and only partially trusted record of why a system looks the way it does. In this paper we introduce the Agentic Software Architecture Workflow (A-SAW), a process-level artifact that extends Architecture as Code with a persistent agent layer: an IDE-integrated agent drafts architecture documents from a version-controlled C4 model, a typed element catalog, and instrumented templates. Every draft passes a mandatory human review gate before it is accepted. We evaluate it in a large industrial organization along two tracks: a rubric assessment of 45 architecture documents including 13 matched source-regeneration pairs, and 13 task-anchored practitioner interviews. Our results show that the workflow collapses the between-author variance that made quality unpredictable. Intrinsic quality also rises, by a median of +19 points across the matched pair. The cohort has a mean of 58% and 89% bundle templates, catalog, agent, and a richer regeneration context. So they bound the agent's contribution from above rather than isolating it. In the agent-generated documents, completeness (100%), consistency (93%), and machine-readability (99%) reach or approach the rubric maximum, while traceability reaches 79%. Eleven of the 13 participants rejected agent output that no human had reviewed, and one warned that polished output may lead reviewers to skip the review. Our work supports keeping human review mandatory and concentrating it where the agent must infer beyond the repository.

0.20

When Relevant Code Is Wrong: Runtime-Aware Context for AI Coding Agents

cs.SE ·2026-10-28 ·Wesley K. G. Assunção, Ramesh Periyathambi
similarity 0.61
Abstract

AI coding agents increasingly rely on retrieved code and development history to guide software changes. However, highly relevant artifacts may describe implementations that are inactive or incompatible with the target deployment. Neither semantic similarity nor recency establishes whether retrieved information remains applicable under feature flags, architectural migrations, and deployment-specific configurations. We challenge relevance-oriented retrieval by introducing operational context validity, a task-, deployment-, and time-dependent criterion for determining whether software evidence can legitimately guide implementation decisions. We instantiate this new retrieval principle in CASE, an agent-agnostic framework that connects repository artifacts with runtime configurations and architectural evolution, classifies evidence as valid, invalid, or unknown, and generates context packages distinguishing actionable guidance from historical rationale and unresolved constraints. We investigate its feasibility through a reproducible preliminary evaluation involving six development tasks, 15 artifacts, and three deployments. Conventional retrieval exposes 16 incompatible artifact occurrences as actionable, whereas CASE excludes them while retaining all applicable artifacts under the prototype's reference rules. Recency filtering reduces valid recall to 0.43, while removing feature-flag evidence from CASE reduces it to 0.13. Although these results demonstrate feasibility rather than improvements in coding-agent correctness, they reveal limitations of relevance-based retrieval and motivate a new research direction: enabling coding agents to reason not only about which information is relevant, but also whether it remains operationally valid.

0.09

ASNER: Software Architecture Recovery with Adaptive Semantic Neighborhoods and Evidence-Grounded Refinement

cs.SE ·2026-10-27 ·Wenhao Wang, Zhi Jin, Jia Li, Dongming Jin, Heng Tian +3 more
benchmarkcode/data similarity 0.60
Abstract

Automated software architecture recovery provides architectural context for software engineering and repository-level LLM agents. Existing methods combine structural and semantic signals, but functional relatedness does not necessarily imply shared architectural responsibility. Fixed thresholds or Top-K neighborhoods also ignore repository scale and local similarity distributions, introducing errors that propagate through clustering. Unconstrained LLM refinement may further alter correct assignments without sufficient evidence. We propose ASNER (Adaptive Semantic Neighborhoods and Evidence-Grounded Refinement for Software Architecture Recovery), based on responsibility-aware, project-adaptive relation modeling and evidence-grounded boundary verification. ASNER derives structured file responsibilities with an LLM, constructs semantic neighborhoods that adapt to repository scale and local similarity distributions, and augments a static dependency graph with selected semantic relations and weak directory cues. Community detection recovers the component backbone. A Responsibility-Ownership-Aware (ROA) Agent then examines uncertain boundary files using project-level responsibilities and traceable code evidence, changing assignments only when supported. Experiments on 12 open-source projects spanning C, C++, Java, and Python show that ASNER achieves the best macro-average on all five metrics on the common benchmark, improving ARI and c2c@66 over GAER-GAT by 19.5% and 32.9%, respectively. A downstream evaluation shows that ASNER-guided LocAgent improves file-, module-, and function-level code localization, demonstrating the value of its architectural context for repository-level software engineering tasks.

0.23