CS Paper Radar

Distribution-Aware Distributed Database Testing (Extended Version)

Zhou Zhou, Si Liu, Hengfeng Wei, Min Zhang

cs.DB ·submitted 2026-09-16 11:36 UTC · v1
published

Abstract

Distributed database management systems (DDBMSs) introduce new challenges for assessing their reliability due to distribution-specific characteristics that affect query execution and optimization. Existing testing approaches, largely designed for centralized DBMSs, often fail to explore diverse distributed execution behaviors and suffer from low executability of generated test queries, thereby limiting their effectiveness in bug detection. We propose DAT (Distribution-Aware Testing), a novel automated approach for detecting query-processing bugs related to distribution strategies and distributed optimizations in DDBMSs, by systematically leveraging distribution-aware information throughout the testing pipeline. DAT builds on a set of techniques that capture diverse combinations of logical schemas and data distribution strategies, and performs guided query mutation to trigger a wide range of distributed query execution behaviors and optimizations, while improving query executability via historical feedback. We implement our approach in a tool, DistRanger, and evaluate it on four widely used production DDBMSs. It uncovers 31 previously unknown bugs, including 28 related to distributed query processing and optimization, and outperforms state-of-the-art testers.

Comment
18 pages, technical report for Distribution-Aware Distributed Database Testing (VLDB27)

Closest papers

Nearest neighbours by meaning across the whole index.

DBcover: A White-box SQL Test Generation Framework for Coverage Improvement

cs.DB cs.SE ·2026-08-26 ·Yankai Rong, Shuang Liu, Jinhao Dong, Qiang Yin, Wei Lu +2 more
top venue similarity 0.72
Abstract · Accepted to the ICSE 2026 Industry Challenge Track. 12 pages, 4 figures, 2 tables

Relational Database Management Systems (RDBMSs) are the backbone of modern data-intensive applications, making reliability and robustness critical. However, achieving high coverage in RDBMS testing remains challenging because of large codebases and complex execution logic. Traditional fuzzing relies on random SQL generation and cannot capture the correspondence between SQL inputs and internal execution paths, while symbolic execution suffers from prohibitive cost and scalability limitations. We propose DBcover, an LLM-driven white-box SQL test generation framework based on contextual reasoning. DBcover uses lightweight dynamic analysis to extract SQL-to-path correspondence and call graphs as global context, and collects source-level information around target functions as local context. These contexts are organized in a unified knowledge graph for efficient retrieval and reuse. DBcover then performs two-phase test generation: it first selects a semantically relevant seed whose execution path is close to the uncovered target, and then guides the LLM with global and local context to generate SQL test cases that trigger previously uncovered code regions. Experiments show that DBcover achieves 80.1% and 82.3% coverage on PostgreSQL and MySQL, and is also effective on the enterprise RDBMS KingbaseES, demonstrating its practical applicability to closed-source systems.

0.51

DBRepro: Automated Database Synthesis via a Hybrid Constraint-Solving Approach for Reproducing Slow Queries

cs.DB cs.SE ·2026-08-28 ·Zhaoyang Zhang, Shuang Liu, Dengfeng Xu, Wei Lu, Jianquan Leng +2 more
top venue similarity 0.71
Abstract · 13 pages, 10 figures, and 3 tables. Accepted for publication in the Industry Showcase Track of the 41st IEEE/ACM International Conference on…

Slow queries frequently cause severe performance bottlenecks in database management systems. Diagnosing their root causes online risks exacerbating resource contention, while data privacy regulations often prohibit copying production data to test environments. Synthesizing a proxy database from non-intrusive metadata that induces the query optimizer to generate the same physical execution plans is therefore critical for offline diagnosis. High-fidelity reproduction requires preserving global statistical distributions while enforcing exact local cardinalities. Existing data-driven and workload-aware approaches cannot satisfy both requirements simultaneously. We present DBRepro, an automated end-to-end framework that formulates database generation as a constrained distribution synthesis problem. DBRepro initializes a global distribution from lightweight column statistics, extracts execution constraints from target queries, and progressively adjusts the distribution to satisfy these constraints while preserving the global distribution. Experiments on TPC-H and SSB show that DBRepro reduces cardinality error by up to 20.3% over a data-driven baseline while maintaining identical plan consistency. Compared with a workload-aware baseline, it reproduces 15% more consistent execution plans and reduces latency proportion error by 21.5%. We further validate DBRepro on a nearly 1 TB real-world dataset managed by KingbaseES, where it reproduces the execution performance of complex slow queries with high fidelity.

0.37

Detecting DBMS Bugs by Constructing Equivalent Representations of Intermediate Query Results

cs.DB cs.SE ·2026-08-31 ·Xiaoxu Niu, Gong Chen, Jinfu Chen, Xiaoyuan Xie
code/data similarity 0.70
Abstract · 12 pages, 3 figures

Database Management Systems (DBMSs) support multiple SQL mechanisms for representing intermediate query results, including VIEWs, Common Table Expressions (CTEs), and Temporary Tables (TEMPTs). When these mechanisms are used to represent the same intermediate query result, the corresponding queries are expected to produce consistent results. However, we observe that such queries can return inconsistent results, indicating potential DBMS logic bugs. Existing approaches for detecting DBMS logic bugs have never explored result consistency across such equivalent representations. In this paper, we propose ERIQ, a novel testing approach for detecting DBMS logic bugs from the perspective of checking result consistency across Equivalent Representations of Intermediate Query Results. ERIQ constructs SQL variants using a VIEW, a CTE, or a TEMPT to represent the same intermediate query result, executes these variants, and compares their returned results. We evaluated ERIQ on four widely used open-source DBMSs: MySQL, MariaDB, Percona, and OceanBase. In total, ERIQ detected 64 bugs, 63 of which were confirmed by developers, and two have been fixed. Among the confirmed bugs, 54 were unique and previously unknown logic bugs, and one was a documentation issue.

0.24

Uncovering Systematic Blind Spots in Graph Database Testing through Integration and Process-Level Analysis

cs.SE ·2026-11-05 ·Tinghan Li, Aoyang Fang, Longbin Lai, Zifan Xie, Hongbo Zhao +3 more
similarity 0.65
Abstract

Graph databases are becoming foundational infrastructure for connected data, and a growing body of automated testing techniques has uncovered many bugs across mature graph database systems. Each technique checks a different aspect of correctness and exposes a different class of defect. Combining them is the natural next step, but each technique is judged by the real bugs it finds, not by useless reports or tests that can never fail, and that is what decides whether a developer can afford to fix the problems. To identify the remaining gaps, we integrated test capabilities from 8 testing methods and conducted an empirical study of their behavior, during which we found two blind spots. First, 12 hours of testing produce thousands of bug reports, far more than the number of genuine defects a mature database is likely to contain. Most reports are false positives caused by dialect or pattern-matching differences, and many of the remaining reports are duplicates. Second, analysis of process-level data reveals that many equivalence-based metamorphic test cases lose their ability to detect defects because of the optimization mechanism. Based on these findings, we summarize and realize these guidelines in GraphWeave, an integrated testing platform for graph databases. This includes a multi-agent system that analyzes the flood of bug reports to produce gating rules for issue filtering and adapter rules for dialect adaptation, and a plan-aware triage routes metamorphic pairs by plan, skipping those that share a plan on the same graph state and therefore cannot detect defects. We apply GraphWeave to NeuG, a recently open-sourced industrial graph database that has not been covered by prior graph database testing work. With modest manual effort, GraphWeave uncovers 19 previously unknown bugs, all of which have since been confirmed.

0.34

Agentic Evolution of SQL Test Generators

cs.SE ·2026-10-09 ·Suyang Zhong, Manuel Rigger
similarity 0.65
Abstract

Automated testing has uncovered many logic bugs in Database Management Systems (DBMSs). Existing tools are typically limited to a few SQL dialects and a subset of these dialects' features. Coding agents based on Large Language Models (LLMs) could encode the semantics of additional SQL features into reusable test generators, requiring no additional LLM calls during testing and thus enabling high efficiency and low cost. However, a key challenge is validating the added capabilities in the presence of potential hallucinations. In this paper, we propose Gungnir, an approach for evolving DBMS test generators using coding agents. Our core mechanism is witness replay: the coding agent provides a generation witness recording the database setup and semantic choices needed to generate a target statement. A validator replays these choices through the modified generator and checks that it reproduces the target statement. Historical witnesses serve as regression tests for subsequent modifications. We evaluated Gungnir on SQLite, DuckDB, and PostgreSQL and found 44 bugs confirmed by developers, of which 34 have been fixed. Beyond DBMSs, witness replay could generalize to other systems to guide agents in extending test generators.

0.25

Efficient Dialect-Agnostic SQL Equivalence Refutation

cs.DB ·2026-10-15 ·Junwen An, Manuel Rigger
publishedbenchmark similarity 0.65
Abstract

Identifying semantically inequivalent SQL queries is useful for many applications, such as evaluating Text-to-SQL and dialect translation systems. However, existing approaches often struggle with the diversity of SQL features and dialects, requiring significant manual effort and expertise to extend query feature support across Database Management Systems (DBMSs). To address these limitations, we present SQLWitness, an efficient, DBMS-agnostic query equivalence refutation engine. Our core idea is leveraging the DBMS execution engine to determine whether generated instances constitute counterexamples, thereby faithfully capturing the semantics of the SQL dialect and preventing false counterexamples. To efficiently generate test data, our approach explores a constrained data-generation space and uses an integrity-constraint-aware generation strategy. Furthermore, to guide time-efficient termination of this test-data generation process, we pair predicate-coverage signals with a statistical estimator. We extensively evaluated SQLWitness on six benchmark suites across four SQL dialects. SQLWitness refuted 5.34% to 88.4% more pairs than state-of-the-art baselines, and processed query pairs in 0.12 seconds on average, achieving a speedup from 2.1x to 16.4x over the baselines. Furthermore, in a case study, SQLWitness found 8.4% inconsistent translations within a dialect translation benchmark. We believe SQLWitness could be broadly adopted for many use cases.

0.35

Fix One, Miss Another: Leveraging Historical Bug Fixes to Uncover Residual Bugs in Database Systems

cs.SE ·2026-11-05 ·Yankai Rong, Zhaoyang Zhang, Shuang Liu, Wei Lu, Xiaoyong Du
similarity 0.62
Abstract · 20 pages, 9 figures, 3 tables

Database management systems (DBMSs) implement SQL constructs through grammar-driven dispatch handlers. Although handlers for syntactic siblings implement different SQL constructs, they often share a common implementation pattern, including argument extraction, type checking, and permission validation. Consequently, a bug fix that hardens the handler exercised by a historical bug may leave the corresponding logic in sibling handlers vulnerable to the same semantic flaw. We refer to such defects as sibling bugs. Despite the structural similarity among sibling handlers, existing bug-finding techniques rarely exploit historical fixes to systematically uncover these related defects. We present Reaper, a novel approach for detecting sibling bugs guided by historical bug fixes. Reaper first analyzes a bug fix to infer a trigger profile, which characterizes the conditions under which the defect is exposed, and a validation profile, which captures the semantic evidence needed to distinguish buggy behavior from correct behavior. It then projects the trigger profile onto syntactic siblings through grammar-guided test mutation, generating tests that exercise corresponding semantic paths in sibling handlers. Finally, Reaper validates suspicious behaviors using the inferred validation profile through a staged validation process, reducing false positives caused by complex DBMS behavior. We evaluate Reaper on four widely used DBMSs, including MariaDB, MySQL, OceanBase, and CockroachDB. Reaper uncovers 61 previously unknown bugs, of which 57 have been confirmed by developers and 13 have been fixed at the time of writing. These results demonstrate that historical bug fixes contain actionable semantic knowledge that can be systematically transferred across syntactic siblings to uncover otherwise overlooked defects.

0.29

PLSQLBench: Benchmarking LLM Systems for Executable Procedural Database Programming

cs.CL ·2026-08-16 ·Marianne Menglin Liu, Leonid Boytsov, Daniel W. Peterson, Pramuditha Perera, Rongguang Wang +11 more
top venuebenchmarkcode/data similarity 0.61
Abstract

We present PLSQLBench, to our knowledge the first benchmark for evaluating whether LLMs can write executable PL/SQL programs, with correctness measured through execution-based tests. Existing LLM evaluations largely target general-purpose code generation or declarative text-to-SQL, leaving procedural database programming underexplored. PLSQLBench contains 2,865 instances: 2,594 single-turn tasks and 271 multi-turn conversations spanning 978 turns. The benchmark combines complex schema-grounded tasks over enterprise-style Spider 2 databases, simpler schema-grounded tasks derived from Spider, and MBPP-derived procedural problems, covering varying levels of database grounding and procedural complexity. Experiments with eight LLMs reveal recurring difficulties in schema grounding, PL/SQL dialect fidelity, procedural control flow, exception handling, and cross-turn consistency. Tool-augmented LLM agents improve performance on several schema-grounded evaluations, although substantial gaps remain. These results highlight procedural database programming capabilities not directly assessed by conventional code generation or text-to-SQL benchmarks. Our code is available at https://github.com/oracle-samples/plsqlbench.

0.84