CS Paper Radar

SmartANN: Object Causal Modeling Boosts Approximate Nearest Neighbor Diagnosis and Auto-Design

Yutong Zhou, Guoxin Kang, Lei Wang, Xueya Zhang, Qinwei Yang, Jianfeng Zhan

cs.DB ·submitted 2026-09-08 04:35 UTC · v1
code/data

Abstract

Approximate Nearest Neighbor (ANN) algorithms achieve high efficiency through interdependent phases across index construction and query execution. This coupling allows upstream performance loss to propagate downstream, affecting execution behavior and measurable outputs. Existing component-level analyses mainly compare isolated design choices, while end-to-end benchmarks report aggregate metrics; neither traces loss propagation across dependent phases, hindering root-cause attribution and automated redesign. We present SmartANN, a framework based on the object causal model (OCM) for ANN bottleneck attribution and automated redesign. SmartANN represents an ANN workflow as eight ordered, replaceable objects and diagnoses them with a sequential diagnose-and-replace loop. At each iteration, it identifies the first object deviating from expected behavior or output as a bottleneck. Because an upstream bottleneck can obscure downstream ones, SmartANN replaces it with a test oracle when available, or with an implementation producing a better outcome, then continues downstream diagnosis. From the diagnosed bottlenecks and failure causes, SmartANN composes compatible actions from a pluggable action library to generate an optimized end-to-end ANN design. We instantiate SmartANN for IVF-PQ and HNSW, covering partition-and-quantization and graph-based ANN families. Experiments on eight real-world datasets show that SmartANN improves Recall by 0.24--74.20%, and increases QPS by 28.8--256.5% at comparable Recall, with low diagnosis and auto-design overhead. The code is available at https://github.com/zhouyutong20/SmartANN.

Closest papers

Nearest neighbours by meaning across the whole index.

Incremental Consistency Execution for Autonomous Intelligent Systems

cs.AI ·2026-09-21 ·Cheng Li, Jiexiong Liu, Yixuan Chen, Ziheng Huang
hot direction similarity 0.60
Abstract

Long-horizon autonomous intelligent systems rely on heterogeneous components such as large language models, databases, external APIs, and rule engines, while their external states continuously change during execution. Re-executing the entire workflow after every change introduces substantial redundant computation. This paper proposes an incremental consistency execution method based on task fact contracts, field-level dependency masks, and state perturbation result invariant domains. After an initial verified execution, the system constructs conservative invariant domains for critical inputs and uses them to determine whether downstream results can be safely renewed without re-invoking expensive components. When re-execution is required, only the smallest affected output fields are recomputed, and an equivalence barrier prevents unnecessary downstream propagation. A submission-time version consistency gate further ensures the safety of side-effecting actions. Experiments on industrial fault diagnosis, enterprise analytics, and LLM-based multi-tool assistants show that the proposed method significantly reduces expensive component calls and end-to-end latency while maintaining high consistency and low incorrect-reuse rates.

0.34

HyperDBDiag: Multi-Root-Cause Identification and Operator-Level Localization for Database Performance Anomalies

cs.DB ·2026-11-14 ·Hu Wang, Zhengjin Yan, Hong Zhu, Qiong Zuo
code/data similarity 0.59
Abstract · 14 pages, 7 figures, 5 tables. Code and data: https://github.com/Qiuzhidewang/HyperDBDiag

Database performance anomalies in modern production environments are not always triggered by a single cause. Instead, they often arise from the concurrent interaction of multiple causes within the same time window, posing substantial challenges to accurate operator-level root-cause analysis. When multiple root causes interact through shared resources, metric anomalies may be amplified, attenuated, or masked. Consequently, salient local anomalies may no longer correspond directly to the actual root causes, making conventional single-path diagnosis and shallow correlation-based methods prone to misdiagnosis. To address this challenge, we propose HyperDBDiag, a method for multi-root-cause identification and operator-level localization of database performance anomalies. HyperDBDiag connects root causes, performance metrics, SQL templates, and execution operators in a knowledge graph, captures joint anomaly patterns across metrics with a metric hypergraph, and uses a constrained LLM to determine the final root-cause set and provide remediation suggestions. When finer-grained localization is possible, HyperDBDiag also identifies the associated anomalous SQL statements and execution operators. We evaluate HyperDBDiag on a database performance anomaly dataset constructed using DB-MAGS. Experimental results show that HyperDBDiag effectively reduces graph complexity while improving root-cause localization accuracy.

0.31

Where Root Cause Analysis Fails: A Retrieval-Reranking Decomposition

cs.LG ·2026-09-29 ·Hada Melino Muhammad, Luan Pham, Laure Barrière, Sachin Shetty, Leonardo Pulga +1 more
top venuebenchmarkcode/data similarity 0.59
Abstract · Accepted at NeurIPS 2026 (Evaluations & Datasets Track)

Identifying the root cause of an anomaly among hundreds of sensors is critical for preventing safety incidents and costly downtime in complex monitored systems. Existing studies evaluate root cause analysis (RCA) methods using top@k accuracy. We show that this metric has a fundamental blind spot: it conflates two failure modes, retrieval failure, where the true cause is never considered, and reranking failure, where it is considered but ranked too low. In this work, we introduce a retrieval-reranking decomposition and audit four well-known benchmarks to expose this blind spot. Our experiments show that, on benchmarks with complex faults, statistical baselines mis-rank the true cause 79-100% of the time, and graph-based methods never clearly beat the best statistical baseline, whether their causal graphs are learned on short fault windows, on retrieved candidate pools guaranteed to contain the cause, or on multi-day normal-operation data. Meanwhile, on simple benchmarks where faults manifest significantly at their origin, retrieval is nearly solved (98-100%). Guided by the decomposition, we build a two-stage pipeline combining a multi-signal retriever with an LLM reranker that, as one fixed configuration, matches or exceeds the best baseline's top@1 accuracy on all six benchmark suites (by up to +12 points), with no causal graph or labeled data required. When all methods rank the same retrieved candidates with the true cause guaranteed present, adding a short system-description document lets the reranker lead the best baseline by +7 to +18 points on every benchmark. Code is available at https://github.com/cruiseresearchgroup/DecompRCA.

0.70

Root-Cause Attribution Is a Search Problem: Continual Search for Long-Horizon Agent Failures

cs.AI cs.HCcs.LGcs.SE ·2026-09-11 ·Harsh Raj, David Lee, Anas Mahmoud, Renxiong Wang, Razvan-Gabriel Dumitru +5 more
benchmark similarity 0.57
Abstract

The increasing deployment of AI agents in long-horizon tasks yields massive execution logs. Diagnosing failures within these records is crucial for reliability, as it transforms outcome-level signals into actionable interventions. The sheer scale of the data renders human review impractical, driving the need for automated root-cause attribution (RCA). However, automated RCA methods using LLMs suffer from low diagnostic accuracy, especially as execution traces grow larger. They struggle because relevant information is often sparse, distributed across distant actions, and disconnected from the visible failure, reducing root-cause attribution to a massive search problem. Existing RCA methods typically rely on one-shot LLM judgments to diagnose failures from execution traces. While effective for shorter trajectories, these judges tend to settle on a plausible diagnosis early, leaving critical evidence in longer traces unexamined. We introduce Continual Search, an iterative framework that nudges the judge, over successive turns, to keep searching for unresolved diagnostic evidence. We evaluate Continual Search across four existing RCA benchmarks. Recognizing the lack of massive execution traces in current benchmarks, we introduce MegaRCA-Mix to evaluate RCA at scale. MegaRCA-Mix provides a challenging testbed of 50 human-annotated failure trials spanning long-horizon, execution-heavy tasks. Across multiple benchmark suites and model families, Continual Search consistently improves attribution performance. On MegaRCA-Mix, for example, it improves Opus-4.8's F1 score by 29\%, from $0.471$ to $0.608$. More interestingly, within the same model family, lower-tier models can even surpass their higher-tier counterparts, demonstrating that effective search supersedes raw model scale.

0.15

WISE: An Agentic Framework for Whole-page Multimodal Search Experience Attribution

cs.IR ·2026-11-04 ·Wenyi Xu, Renzhe Zhou, Songyang Li, Feiran Zhu, Ruoqi Zhao +4 more
benchmark similarity 0.57
Abstract

Algorithm-driven services, including search, recommendation, and advertising, rely on multi-stage model pipelines. While research prioritizes model optimization, post-deployment monitoring focuses on operational reliability. Yet operationally healthy systems may still produce poor-quality results, exposing a gap in existing evaluation and monitoring practices. However, a systematic problem formulation and benchmark for this task remain lacking. To bridge this gap, we formulate System Quality Diagnosis (SQD) for algorithm-driven services, a new task that extends quality evaluation beyond identifying poor outcomes to attributing their causes across system components. Focusing on industrial search, we propose WISE, an agentic framework for end-to-end quality diagnosis that discovers user-facing issues and traces their origins to underlying algorithmic components. To improve diagnostic reliability, we introduce ReTrace for constructing evidence-consistent trajectory supervision and Adaptive Harness-Policy Optimization (AHPO) for failure-guided optimization of the agent harness and model policy. To enable systematic evaluation, we construct WISE-BENCH, a real-world multimodal search benchmark that combines user-facing evidence, internal pipeline traces, and expert diagnosis annotations. Extensive experiments show consistent gains across model scales, with WISE achieving 75.20% end-to-end accuracy on Qwen3.5-35B-A3B. In production, WISE supports quality auditing, online quality assurance, and model optimization, reducing diagnosis costs and improving downstream relevance models.

0.10

DataTrace: Benchmarking Data Agents for Evidence-Grounded Root-Cause Localization

cs.DB ·2026-11-13 ·Tung Sum Thomas Kwok, Xiaotian Lin, Yuyu Luo
top venuebenchmarkcode/data similarity 0.57
Abstract · Accepted in KDD 2027 Datasets and Benchmarks Track

Database agents diagnose pipeline failures, but a correct root cause is trustworthy only when it is supported by evidence the agent inspected. Existing benchmarks usually grade the final query or answer but fail to distinguish an evidence-based diagnosis from an unsupported correct answer. We introduce DataTrace, a benchmark for evidence-grounded root-cause localization over typed graphs of database artifacts. DataTrace uses controlled fault injection to derive root causes and evidence paths within 83 runnable environments without human annotations, and scores root-cause correctness and coverage of the supporting path separately. We also introduce CoSE, a generator-solver protocol in which an external verifier admits reproducible faults and checks both the submitted answer and the inspected evidence. Our evaluation shows that seven proprietary models achieve average root-cause accuracy 48%, with a mean per-model conditional unsupported rate of 18%. Fine-grained analysis shows that this gap varies sharply with fault mechanism, dependency distance, and evidence modality. After in-loop training on a matched held-out batch, the share of grounded correct diagnoses rises from 83.7% to 94.6%, while root-cause accuracy rises from 29.2% to 36.7%. These results show that while final-answer accuracy can hide unsupported diagnoses, evidence coverage provides an actionable signal to improve database agents. Code and data are available at https://hkustdial.github.io/datatrace/.

0.41

Second-Order Problem Solving for Recursive Self-Improvement in Formal Verification

cs.AI ·2026-10-05 ·Yuxuan Jiang, Aditya Vempaty, Ashish Jagmohan
similarity 0.56
Abstract

Recursive self-improvement (RSI) enables agents to iteratively optimize their workflows via execution feedback. However, standard RSI typically operates as a first-order optimizer: it repeatedly patches surface-level parameters in response to immediate failure symptoms, often leading to trial-and-error thrashing without resolving underlying mechanisms. To address this limitation, we introduce SO-RSI, a framework that elevates workflow optimization to a second-order diagnostic inquiry, investigating why failures occur before committing to structural interventions. SO-RSI passively monitors execution traces for three structural anomalies (recurrence, opposing edits, and expectation mismatch) to trigger targeted mechanism investigations. By executing lightweight diagnostic probes and maintaining persistent inquiry memory across RSI rounds, SO-RSI accumulates causal evidence to guide systematic workflow edits rather than parameter patches. Across Lean 4 proof generation and Verus-based verifiable code generation, SO-RSI improves final held-out pass rates over Naive RSI by 21.8 and 25.8 percentage points under matched 24-hour search budgets. Behavioral analyses further confirm that SO-RSI substantially suppresses failure recurrence and eliminates unproductive zero-progress optimization loops.

0.20

RankEvolve: A Reliable Multi-Agent Auto-Research Harness for Evolving Ranking Models

cs.AI ·2026-09-30 ·Zheng Chen, Linfeng Liu, Hong Li, Hong Yan
benchmarkcode/data similarity 0.56
Abstract · 29 pages, 5 figures, 13 tables, 1 algorithm; includes appendices

Auto-research agents, LLM systems that propose, implement, train, and evaluate model changes across iterations, promise to automate applied ML's experimental loop. Over long horizons, execution accuracy is a binding constraint: a change can silently leak held-out data, omit normalization, disconnect a gradient, or leave a train/eval flag unwired, invalidating expensive runs and compounding error across iterations. We present RankEvolve, an auto-research framework for evolving generative ranking models. An Executable Operating Protocol (EOP) declares phases, gates, branches, and loops, and the runtime enforces the compiled state machine. A meta-meta-harness composes complete black-box coding-agent products, including Claude Code and Codex, as execution-graph nodes that review and repair one another's work. In a budget-matched evaluation, heterogeneous composition raises all-oracle execution accuracy from the best single-product baseline of 45.8 percent to 62.5 percent (paired +16.7 points, 95 percent CI [6.6, 26.7]) while achieving a 10.4 percent silent critical-defect rate. An implemented knowledge layer carries findings, including negative results, across iterations. In a twelve-iteration deployment on the open-source HSTU recommender, RankEvolve reported NDCG@10 of 0.2192 on MovieLens-20M LARGE (+4.48 percent over the published anchor) and 0.1948 on BASE (+2.80 percent). ExecML-HSTU, seeded by incidents from that deployment, provides the oracle benchmark for the execution-accuracy evaluation. A pre-specified LitGPT transfer split replicates the heterogeneous-composition effect beyond recommendation (+12.5 points, 95 percent CI [3.0, 22.0]), and a paired ablation isolates per-step from full-protocol instruction injection. These results characterize when runtime-controlled composition of coding-agent products improves execution accuracy.

0.35