CS Paper Radar

Diagnosis as Percolation on an Evidence Graph: Root-Cause Recoverability in Network Fault Diagnosis

Jerome Henry, Swadhin Pradhan, Amine Choukir

cs.NI ·submitted 2026-11-14 19:28 UTC · v1

Abstract

Diagnosing failures from incomplete observations presents three challenges: determining whether sufficient evidence connects a possible cause to a symptom, distinguishing the true cause from plausible alternatives, and assessing when a diagnosis can be trusted. We introduce an evidence-graph framework that separates these challenges and uses percolation theory to analyze how missing information affects cause-symptom connectivity. In controlled simulations with an average degree of four, a giant connected component emerges when approximately 25% of evidence is retained, but a specific cause-alarm pair reaches 50% connectivity only after slightly more than half is retained. Even connectivity does not guarantee correct cause identification. We apply this framework to multi-LLM diagnosis of IEEE 802.11 packet captures. A final reconciliation pass improves evidence-grounding weighted F1 from 0.912 to 0.964, primarily by removing isolated citations rather than connecting previously separate evidence chains, consistently across three graph constructions. However, connectivity is not a reliable confidence measure: its association with diagnostic quality becomes statistically insignificant after controlling for capture length, citation count, within-capture dependence, and multiple comparisons. Further experiments show that directed and weighted edges help selectively, while simpler overlap measures or domain-specific methods sometimes perform better. These findings establish evidence graphs as a framework for separating evidence availability, causal distinguishability, and confidence reliability, while demonstrating that connected evidence alone cannot certify diagnostic correctness.

Comment
28 pages, 8 figures, 20 tables

Closest papers

Nearest neighbours by meaning across the whole index.

FaulT-Bench: Towards Benchmarking Network Troubleshooting LLM Agents under Unreliable User Tickets

cs.NI cs.AI ·2026-08-27 ·Kuan-Hao Tseng, Niruth Bogahawatta, Yasod Ginige, Kunjan Patel, Kosta Dakic +1 more
benchmark similarity 0.63
Abstract

LLM-based agents are increasingly proposed for network fault diagnosis, but existing benchmarks evaluate them only on accurate tickets and always assume a fault is present, conditions rarely met in practice. We present FaulT-Bench, a benchmark of 200 troubleshooting scenarios across eight network topologies, five reimplemented from public practitioner labs, spanning genuine faults, false fault reports, incorrect device attribution, and incorrect root-cause claims. To isolate how ticket wording affects diagnosis, we further rewrite 72 false-premise tickets into five reporter personas that vary reporter confidence and verifiable detail one factor at a time, holding the network state fixed. Our automated harness deploys each scenario in Kathará, lets agents interact through the NIKA tool interface, and scores free-text diagnoses with an LLM judge across outcome, fix, and reasoning quality. Evaluating SADE, ReAct, and Claude Code, we find all three are near-saturated on accurate tickets and robust to misdirection, yet degrade sharply when the network is healthy and the ticket is wrong, probing until a benign condition can be promoted to a root cause rather than concluding nothing is wrong. Persona rewrites show that how a ticket is written matters more than what it claims: a confidently wrong report is handled about as well as an accurate one, while a vague, underspecified report degrades performance sharply. The three agents also fail differently, from constant over-diagnosis to unanswered runs, at very different cost. These results position FaulT-Bench as a benchmark for developing agentic systems that can reason reliably over the noisy, unreliable tickets of real-world network troubleshooting.

0.40

Know the Shape, Find the Fault: Topology-Conditioned Diagnosis of Multi-Agent LLM Failures

cs.MA cs.AI ·2026-10-07 ·Xinwen Liu, Zhuocheng Pan, Isabella Zhu, Jawei Zhang, Xudong Liu +1 more
similarity 0.62
Abstract

Multi-agent LLM systems coordinate task execution through exchanges of information among agents. When coordination breaks down, similar symptoms in execution traces can reflect different problems in how information is passed, used, or verified. Communication topology captures how agents exchange information and provides structural cues for distinguishing coordination failure modes. Using these cues for diagnosis requires establishing how topology relates to failure patterns and recovering the relevant structure from execution traces that lack explicit topology labels. We analyze the relationship between communication topology and failure patterns and introduce MAScope, a two-stage framework for topology-conditioned diagnosis. Its Trace Structural Extractor TSE recovers communication topology from heterogeneous execution traces by grounding an interaction graph in message evidence. The Topology-Conditioned Judge TC-Judge then classifies failures using the trace, predicted topology, an empirical failure prior estimated from separate labeled traces, and a short description of topology-specific failure patterns. Under a fixed orchestration structure, the recovered topology can be reused across executions. Experimental results show a statistically significant association between communication topology and failure type, with $χ^2 = 409.9$ and $p = 1.2 \times 10^{-70}$. On the \num{851} MAST-clean traces, ground-truth topology context raises gpt-mini's Macro-F1 from $0.173$ to $0.350$. With predicted topology, the pipeline achieves $0.346$, approaching the trace-only gpt-5.4 baseline of $0.372$. For \num{1000} traces under a fixed orchestration structure, the projected pipeline cost, including one topology extraction, is approximately $6\%$ of repeated gpt-5.4 diagnosis cost. These results show that topology-conditioned context improves failure diagnosis and supports lower-cost deployment.

0.29

Context Graph: Explainable Multi-Agent Decision Chains in AI-RAN

cs.NI cs.AIcs.IT ·2026-11-12 ·Xijun Wang, Zhaoyang Liu, Xiang Chen, Howard H. Yang, Chenyuan Feng
similarity 0.61
Abstract · This work has been submitted to the IEEE for possible publication

AI-RAN distributes network control across multiple autonomous agents, namely xApps and rApps operating over shared radio resources. When an alert or degradation appears, operators currently lack a structured way to trace which decision caused it, what context triggered that decision, and how other agents interacted. Existing explainability tools focus on single-model feature attribution or linear event logging, neither of which captures cross-agent decision chains. We propose the Context Graph, a heterogeneous directed graph that organizes policies, measurements, decisions, and alerts into a queryable explanation layer. Its foundational primitive is the Decision-Event node acting as a causal junction, where incoming edges capture triggering context and outgoing edges track downstream consequences that are progressively upgraded through counterfactual verification. Instantiated over existing E2, A1, and O1 data flows, our proof-of-concept demonstrates that compliance and diagnosis queries are resolved at sub-millisecond latency over graphs of tens of thousands of nodes, while reliably distinguishing true causal impacts from spurious correlations.

0.17

Where Root Cause Analysis Fails: A Retrieval-Reranking Decomposition

cs.LG ·2026-09-29 ·Hada Melino Muhammad, Luan Pham, Laure Barrière, Sachin Shetty, Leonardo Pulga +1 more
top venuebenchmarkcode/data similarity 0.60
Abstract · Accepted at NeurIPS 2026 (Evaluations & Datasets Track)

Identifying the root cause of an anomaly among hundreds of sensors is critical for preventing safety incidents and costly downtime in complex monitored systems. Existing studies evaluate root cause analysis (RCA) methods using top@k accuracy. We show that this metric has a fundamental blind spot: it conflates two failure modes, retrieval failure, where the true cause is never considered, and reranking failure, where it is considered but ranked too low. In this work, we introduce a retrieval-reranking decomposition and audit four well-known benchmarks to expose this blind spot. Our experiments show that, on benchmarks with complex faults, statistical baselines mis-rank the true cause 79-100% of the time, and graph-based methods never clearly beat the best statistical baseline, whether their causal graphs are learned on short fault windows, on retrieved candidate pools guaranteed to contain the cause, or on multi-day normal-operation data. Meanwhile, on simple benchmarks where faults manifest significantly at their origin, retrieval is nearly solved (98-100%). Guided by the decomposition, we build a two-stage pipeline combining a multi-signal retriever with an LLM reranker that, as one fixed configuration, matches or exceeds the best baseline's top@1 accuracy on all six benchmark suites (by up to +12 points), with no causal graph or labeled data required. When all methods rank the same retrieved candidates with the true cause guaranteed present, adding a short system-description document lets the reranker lead the best baseline by +7 to +18 points on every benchmark. Code is available at https://github.com/cruiseresearchgroup/DecompRCA.

0.70

Large Language Models (LLMs) for Telecom Root Cause Analysis (RCA): A Structured Reasoning Framework for Evidence-Grounded Diagnosis

cs.AI ·2026-09-02 ·Hao Zhou, Mandar Kulkarni, Hao Chen, Yan Xin, Charlie +1 more
publishedsurvey similarity 0.58
Abstract

Root cause analysis (RCA) is a critical task in telecom network operations, but diagnosing performance degradations in modern 5G and emerging 6G networks remains challenging due to complex cross-layer dependencies. While large language models (LLMs) offer promising capabilities for reasoning and knowledge integration, directly applying vanilla LLMs to telecom RCA often leads to hallucination, unstable reasoning, and poor alignment with structured network evidence. This work first reviews the evolution of telecom RCA from rule-based and machine learning (ML) approaches to emerging LLM-enabled techniques, and provides an overview of recent paradigms, including structured reasoning, retrieval-augmented knowledge grounding, agentic orchestration, and verifiable reasoning. Building upon these insights, we propose a structured reasoning framework for LLM-enabled telecom RCA that aligns diagnostic reasoning with telecom-specific evidence and domain knowledge. The proposed approach first organizes heterogeneous network telemetry into canonical contexts, and then enforces decision-path reasoning during diagnosis, and finally generates evidence-grounded explanations for reliable fault identification. Experimental results on two 5G RCA datasets, TeleLogs and TelecomTS, demonstrate that the proposed framework consistently improves diagnostic accuracy and decision consistency compared with baseline techniques. These cross-dataset results highlight the importance of structured reasoning design for practical LLM-based RCA systems in next-generation telecom networks.

0.21

Every Alarm Has a Cause: An Open Generator of Telecom Alarm Storms with Exact Root Causes, Bound to NORIA-O, RFC 8632 and TMF642

cs.NI cs.AI ·2026-10-21 ·Madhulatha Mandarapu, Sandeep Kunkunuru
code/data similarity 0.58
Abstract · 22 pages, 5 figures, 7 tables. Generator: https://github.com/samyama-ai/telecom-synthetic-data-generator (Apache-2.0). The dataset deposit i…

Alarm root-cause analysis is studied on data that cannot be shared. Operator logs are confidential, so what the field publishes is a causal graph between alarm types rather than per-incident causes, a fixed scenario set, gated, or never released. Two recent benchmarks carry per-incident root causes at scale; neither states a download location or a licence. We give a generator rather than a dataset. It runs on real backbone topologies from the Internet Topology Zoo and records the exact root cause and the propagation path of every alarm -- a by-product of generation, not a labelling pass. Every emitted field binds to a term in NORIA-O, RFC 8632, RFC 8345 or TMF642, and an unknown term fails the run. We ship fixed configurations, splits, a scorer and nine baselines. A one-line rule -- rank the device that raised the earliest critical alarm -- takes top-1 at 0.650 against 0.575 for our own temporal method, and wins 13 of 15 runs over three backbones and five seeds. An LLM given the same view reaches 0.600, the same model allowed to query the graph 0.625, and the rule wins on real operator alarms too, at 0.773: a property of alarm streams, not of our generator. One fault class separates the table: when a site loses power its PoP reports nothing, so every method that ranks alarmed devices scores 0.000 or near it, while the two that walk the topology to a silent candidate take 1.000 and 0.833.

0.61

GALA: Graph-Augmented LLM Agents for Root Cause Analysis and Incident Response in Microservices

cs.SE cs.AI ·2026-08-10 ·Yifang Tian, Yaming Liu, Zichun Chong, Zihang Huang, Yiran Li +1 more
top venue similarity 0.57
Abstract · Proceedings of the 41st IEEE/ACM International Conference on Automated Software Engineering (ASE '26), October 12--16, 2026, Munich, Germany

Microservice root cause analysis (RCA) requires correlating failures across heterogeneous telemetry within complex service dependency graphs. Existing methods often rely on a single telemetry modality; recent LLM-based approaches can suffer from unconstrained exploration and hallucination; and most systems stop at fault ranking without producing actionable incident response. We present GALA+, a graph-augmented LLM agentic framework centered on graph-guided investigation, which uses service dependencies to bound exploration and refine diagnosis through localized multi-modal evidence. For initial hypothesis generation, GALA+ combines complementary telemetry signals with STRIX, a novel trace- and graph-structure-aware scoring module. GALA+ then produces ranked diagnoses, incident summaries, and stratified action recommendations. We further introduce SURE-Score, a human-guided evaluation framework co-developed with industry SRE experts for assessing RCA-specific output quality beyond conventional text similarity metrics. On two microservice benchmarks, GALA+ consistently achieves the strongest overall results, surpassing the best LLM-based baseline by more than 25 percentage points in AC@1, while also receiving the highest ratings from both SURE-Score and independent human SRE evaluation.

0.71

Beyond Fault Localization: A Trajectory-Level Study of LLM Agents for Microservice Root Cause Analysis

cs.SE ·2026-08-21 ·Qisheng Lu, Aoyang Fang, Junjielong Xu, Jin'ao Shang, Songhan Zhang +3 more
similarity 0.57
Abstract · 23 pages, 7 figures, 4 tables

Automated microservice root cause analysis (RCA) is usually evaluated by whether the final answer localizes the fault source. This is insufficient for on-call site reliability engineers (SREs), who need the evidence and reasoning behind the fault. Evaluating the process is hard because agent trajectories interleave free-form reasoning with heterogeneous tool calls and telemetry results, making it difficult to assess agents' behavior. To assess how well RCA agents ground diagnoses in evidence, we propose a trajectory-analysis methodology that characterizes evidence navigation and utilization, SRE-oriented diagnostic intent, and recurring failure modes. Our key insight is that fault-propagation paths give a causal reference for where agents investigate and how they use evidence. With it, we analyze 5,000 trajectories from ten framework-backbone configurations (nine agent frameworks, two backbone models) on RCABench. Correct localization can hide deficiencies in causal reconstruction. Successful investigations use retrieved evidence to guide later queries, consider broader diagnostic intents, and investigate more deeply. Failed investigations drift toward unrelated services or repeat shallow queries; recurring failures arise from omitted evidence, misinterpreted evidence, and unsupported reasoning. To test whether these patterns can guide improvement, we build a lightweight defense prototype targeting the three failure families. Across two held-out benchmarks, two backbone models, and six diagnostic frameworks, the failure patterns recur; the prototype reduces their prevalence and raises Acc@1 by 3.5-35.1 percentage points in every configuration, suggesting that the mined failures generalize and are actionable. Our findings motivate a process-level view of automated RCA, in which trajectories reveal whether a diagnosis is correct, well-grounded, and open to improvement.

0.28