CS Paper Radar

ORCA: Observability-Grounded Program Repair for Microservice Incidents

Yuanchen Gao, Yifang Tian, Yiran Li, Charles Zhang, Hans-Arno Jacobsen

cs.SE ·submitted 2026-08-17 18:11 UTC · v1
benchmark

Abstract

Microservice failures are often diagnosed from operational telemetry. However, automated program repair systems usually start from issue reports, localized code context, or failing tests. This mismatch leaves a gap between telemetry-based diagnosis and patch generation. We present ORCA, an observability-grounded APR pipeline for microservice incidents. ORCA first distills the differences in paired failure and reference telemetry into a fault signature, then uses the signature to identify candidate code and deployment-configuration locations. Repair graph agents and an Exploration agent generate unified-diff patch candidates from these locations. ORCA evaluates generated patches with a Telemetry-Grounded Patch Verifier that separates patch validity, syntactic and semantic correctness, test-oracle integrity, and telemetry replay. On a 575-case benchmark, ORCA outperforms all evaluated baselines in terms of cost-effectiveness. Results show that operational telemetry can be transformed from diagnostic evidence into actionable repair context: paired telemetry supports repair-oriented localization, while repair graph agents convert localized code and configuration evidence into constrained patch-generation context for the LLM. Telemetry-grounded verification then exposes repair outcomes that issue- or test-only evaluation would miss.

Closest papers

Nearest neighbours by meaning across the whole index.

From Finding to Fixing: How Useful is Fault Localization for Agentic Program Repair?

cs.SE ·2026-10-19 ·Xinyu Shi, Zhou Yang, An Ran Chen
similarity 0.69
Abstract

Fault localization (FL) aims to identify code locations responsible for software failures and is commonly evaluated by how accurately it ranks known fault locations. Large language model (LLM) agents are changing program repair and are increasingly used to search and reason over repositories. Although recent LLM-based FL methods have improved localization accuracy, their impact on agentic repair remains unclear. To address this gap, we conduct a systematic empirical study of FL in agentic repair. We evaluate six recent FL methods on SWE-bench Verified and Defects4J with three LLMs under a common repair framework and analyze 16,248 repair trajectories. We also examine gold locations and bug-cause descriptions as forms of FL guidance across different repair contexts. The studied FL methods do not consistently improve repair success, and higher localization accuracy does not consistently lead to larger repair gains. Yet FL guides agents' early search and helps them reach relevant code sooner. Accurate locations can reduce repair token use by up to 39.2\%, but current FL methods often consume more tokens than they save during repair. Experiments with gold guidance suggest that FL has greater potential when bug inputs contain fewer location clues. Providing bug causes can further improve repair success beyond locations alone. We also find that ground truth from a single reference patch does not fully capture the locations used by successful repairs. These findings help clarify the value of FL in agentic repair and inform future work on its design and evaluation.

0.16

Patch correctness in automated program repair: A five-dimensional evidence framework and cross-benchmark audit from classical repair to coding agents

cs.SE ·2026-11-07 ·Shuai Wu, Yanna Feng, Yufang Li, Xue Li, Zhijun Wang +2 more
benchmarkcode/data similarity 0.67
Abstract · Technical report. 24 pages, 2 figures, 6 tables. Dataset and evidence matrix available at https://doi.org/10.5281/zenodo.23639376

Tests guide automated program repair and determine which candidate patches are reported as successful. Repository-level coding agents extend this feedback loop by generating tests and revising code after observing execution results. This technical report introduces a five-dimensional evidence profile and reporting protocol to audit how test observations support patch-correctness claims across classical repair systems and coding agents. We assemble an audit dataset of 39 benchmark and evaluation artifacts: 32 primary empirical studies, one institutional technical audit, and six prior taxonomic baselines. We code the 33 primary contributions along five dimensions: test provenance and exposure, behavioral scope, oracle basis and dependence, execution integrity, and label adjudication. The central distinction separates checks visible during repair search from evidence reserved for final assessment. Four worked comparative audits show how changed candidate populations, oracle rules, or execution budgets alter reported detection rates. Additional tests can expose missed faults while rejecting valid alternatives or reducing correct-repair yield under a fixed budget. Repository-level audits separate issue reproduction from candidate-patch discrimination and distinguish automated discrepancy detection from requirements-based adjudication. An evidence matrix links each coded decision to supporting passages and records shared benchmark ancestry. We specify a reporting protocol for repair benchmarks and maintenance pipelines that retains uncertain judgments and records validation cost alongside fault detection and valid-patch retention.

0.24

Experimental Settings in LLM-Based Program Repair: A Study of Inputs, Tool Access, Feedback, and Validation

cs.SE ·2026-09-16 ·Xushu Dai, Yicheng Cai, Nanqing Luo, Pei-Yu Tseng
benchmark similarity 0.65
Abstract · 11 pages, 1 figure, 3 tables

Evaluations of automated program repair (APR) systems commonly report the benchmark, the number of repaired defects, and the tests used for final patch validation, but these items no longer fully specify the repair task presented to a system. Recent LLM-based systems differ in the information supplied before repair, the repository and testing operations permitted during repair, and the feedback returned after unsuccessful attempts, allowing the same benchmark to instantiate substantially different repair tasks ranging from localized patch generation to repository-level diagnosis and iterative repair. We present a framework for explicitly specifying the experimental settings associated with reported APR results. We analyze reported experimental settings from systems evaluated on Defects4J and SWE-bench and characterize each result by its task unit, fault-localization assumptions, initial input, tool access, repair-time feedback, final validation, and resource budget. Our analysis shows that benchmark identity alone is insufficient to reconstruct the evaluated task or determine the appropriate scope of comparison across reported repair rates. We therefore introduce a machine-readable schema for specifying each experimental setting to improve reproducibility and make the scope of cross-system comparisons explicit.

0.19

Externalizing Requirement-to-Repair Artifacts as Observable Traces for LLM-Based Program Repair

cs.AI cs.SE ·2026-09-14 ·Zewen Tao, Shin-nosuke Ishikawa
code/data similarity 0.64
Abstract · Accepted at REALM Workshop, EMNLP 2026. Code: https://github.com/ForeverMJ/THEMIS

Repository-level repair requires not only correct patches but also inspectable records that explain how issue requirements are translated into code changes and post-edit evidence. We contribute THEMIS, a stage-aware repair workflow that externalizes this requirement-to-repair process through semantic interpretation, a runtime requirement-code graph, graph-derived Developer guidance, retained repair rationale and patches, and post-edit audit records. A retrospective audit of 300 SWE-bench Lite cases demonstrates that these artifacts provide broad support for cross-stage inspection: a complete Developer rationale is available for 288 cases, and 214 cases (71.3%) retain a complete audited field set connecting the selected stages. The retained records further enable systematic measurement of cross-stage correspondence: target symbols recur in 62.6% of Developer rationales and in 62.8% of patches, rising to 75.8% when related symbols are included. In a paired 100-case comparison, the relational workflow resolves 19 cases versus 9 for the direct same-input condition; because the two conditions also differ in Analyzer output, graph-derived distillation, and Judge records, we report this as preliminary, workflow-level evidence rather than a causal effect of the graph component. Together, these results show that THEMIS makes otherwise implicit requirement-to-repair transitions inspectable, enabling systematic examination of how repair decisions persist, align, and evolve across stages.

0.36

ArchGuard-APR: Architecture-Constraint-Aware Multi-Agent Program Repair Using Repository Graphs and Execution Feedback

cs.SE ·2026-10-24 ·Majid Feiz Deilami
code/data similarity 0.64
Abstract · 29 pages, 10 tables, 7 figures. Exploratory feasibility evaluation. Software, experimental artifacts, and reproducibility materials are avai…

Repository-level automated program repair must reason about more than a localized failing statement. A candidate change can satisfy a narrow test while violating dependency boundaries, modifying an unsafe path, or relying on context that omits relevant repository structure. ArchGuard-APR is an architecture-aware repair prototype that integrates deterministic repository analysis, immutable repository graphs, explicit architecture profiles, graph-guided context retrieval, role-based repair orchestration, patch-policy enforcement, isolated execution feedback, provider cost controls, and canonical source-free evidence. We evaluate the implemented pipeline through an exploratory paired feasibility study, not a confirmatory effectiveness experiment. Six executions were planned; four completed, and two black-4 conditions ended in pre-provider infrastructure attrition. Across two valid paired tasks, baseline and ArchGuard conditions each produced 0/2 successful repairs, and all four completed outputs were classified as invalid patches. The completed executions consumed 35,431 input tokens and 2,737 output tokens, for a total of 38,168 tokens, at a total cost of USD 0.1296325. No effectiveness advantage was observed, and the sample provides no useful inferential basis. The study instead establishes the engineering feasibility of composing architecture constraints with a reproducible repair pipeline and identifies patch validity and infrastructure robustness as the principal targets for future work.

0.42

AegisFlow: A Multi-Agent Agentic AI Framework for Autonomous Remediation and Self-Healing in Fragile Data Ecosystems

cs.AI cs.CL ·2026-10-03 ·Muhammad Bilal Awan, Zubair Hussain, Abdul Shahid
similarity 0.64
Abstract · 43 pages

Traditional data pipelines are notoriously brittle, often failing due to upstream schema drift, API contract changes, or website DOM modifications. Present observability tools only raise alerts but for human engineers, resulting in a high Mean Time to Repair (MTTR) and operational fatigue. In this paper we propose AegisFlow (Agentic Engine for Intelligent Self-healing and Graph-driven Operations for Workload remediation), a novel agentic framework that closes the loop between detection and resolution. AegisFlow uses a Watchdog agent to collect runtime telemetry and has a Repair agent to automatically create, test and deploy code patches based on Large Language Models (LLMs). The framework presents the non-intrusive execution model called Parallel Shadow Patching, a non-intrusive execution model based on the Monitor, Analyze, Plan, Execute, Knowledge (MAPE-K) loop to generate and verify patches in digital twin environments. Through experimental testing, we have evaluated AegisFlow across five common failure scenarios, and see 98.1 percent improvement in MTTR (from an average of 170 minutes per patch to 3.2 minutes) and a patch success rate of 92 percent . In particular, the system is successful in dealing with changes in the JSON schema (96 percent ) and punctuation drift (98 percent ), and is least successful in Shadow DOM cases (85 percent ). AegisFlow frees up about 98 percent of data engineering on-call time from firefighting and reallocates it towards innovation. The framework is deployment agnostic consisting of a system that can be deployed in a plugin fashion into an existing pipeline orchestration system with minimal uplift to the existing system.

0.19

When Should Dependency Updates Invoke Repair Agents? A Lightweight Routing Study

cs.SE ·2026-09-22 ·Liheng Fan, Jialun Yin, Yuzhi Chen
similarity 0.64
Abstract

Dependency-update pull requests are frequent and mostly routine, but a small subset requires non-trivial compatibility repair. Recent repository-level coding agents make such repair increasingly plausible, yet invoking them on every dependency update wastes model calls, CI time, repository context, and review attention. We frame this as a pre-agent routing problem: deciding which dependency-update pull requests should be escalated before downstream diagnosis or repair attempts. We introduce DepFixRouter, a lightweight router that ranks dependency updates by historical compatibility-repair likelihood using creation-time textual and metadata signals. On 497 labeled GitHub dependency-update candidates, only 72 require substantive repair. A creation-time-safe LinearSVC using only PR titles and bot/dependency flags reaches 0.488 repair F1 and captures 51.4% of repairs within the top 20% routed pull requests, improving calls per captured repair from 6.90 under route-all or random policies to 2.68. Retrospective full-history signals improve top-20% recall to 65.3%, revealing substantial hindsight leakage in pull-request histories rather than deployment-time routing utility. In a 60-case diagnosis-agent pilot, router-gated diagnosis reduces actual LLM calls by 66.7% and tokens by 66.1%, suggesting budgetaware escalation while measuring diagnosis rather than patch generation. DepFixRouter can serve as a lightweight escalation layer between routine dependency-update automation and expensive repository-level agents, enabling budget-aware maintenance without relying on retrospective repair evidence for deployment-time routing.

0.51

RegoMender: A Multi-agent Framework for Automated Rego Policy Repair

cs.SE cs.AIcs.PL ·2026-10-21 ·Ruijie Zhang, Huaxiao Yin, Li Hu, Fanqi Kong, Haozhe Liang +3 more
top venuebenchmark similarity 0.64
Abstract · 13 pages, 5 figures, accepted to ASE 2026, research Track

Policy-as-Code (PaC) has emerged as a foundational paradigm in cloud-native systems. However, repairing Rego policies remains challenging due to their strict syntax, declarative semantics, and specific verification requirements. Existing automated program repair (APR) methods are mainly designed for mainstream languages and struggle to accommodate the unique policy logic and repair workflows of Rego. Thus, Rego policy repair remains a manual process driven by trial and error. To address this, we introduce RegoBench, the first automated Rego policy benchmark comprising 1,159 real-world defects, and propose RegoMender, a feedback-guided multi-agent framework. Collaborating via a shared blackboard and an atomic toolkit, RegoMender dynamically executes patching, rewriting, or rolling back, to resolve complex repair scenarios. Evaluated on RegoBench, RegoMender achieves syntactic and test-based semantic success rates of 82.9\% and 77.9\%, respectively, outperforming the strongest baseline by 13.4\%. Unlike other approaches whose performance drops in complex scenarios, RegoMender maintains effectiveness across single-step and chained repair tasks. Furthermore, it generalizes strongly across HCL and SQL and achieves an 86.4\% repair rate on 132 real-world CVE vulnerabilities, proving its robustness and practical utility for security.

0.47