CS Paper Radar

Fix One, Miss Another: Leveraging Historical Bug Fixes to Uncover Residual Bugs in Database Systems

Yankai Rong, Zhaoyang Zhang, Shuang Liu, Wei Lu, Xiaoyong Du

cs.SE ·submitted 2026-11-05 09:48 UTC · v1

Abstract

Database management systems (DBMSs) implement SQL constructs through grammar-driven dispatch handlers. Although handlers for syntactic siblings implement different SQL constructs, they often share a common implementation pattern, including argument extraction, type checking, and permission validation. Consequently, a bug fix that hardens the handler exercised by a historical bug may leave the corresponding logic in sibling handlers vulnerable to the same semantic flaw. We refer to such defects as sibling bugs. Despite the structural similarity among sibling handlers, existing bug-finding techniques rarely exploit historical fixes to systematically uncover these related defects. We present Reaper, a novel approach for detecting sibling bugs guided by historical bug fixes. Reaper first analyzes a bug fix to infer a trigger profile, which characterizes the conditions under which the defect is exposed, and a validation profile, which captures the semantic evidence needed to distinguish buggy behavior from correct behavior. It then projects the trigger profile onto syntactic siblings through grammar-guided test mutation, generating tests that exercise corresponding semantic paths in sibling handlers. Finally, Reaper validates suspicious behaviors using the inferred validation profile through a staged validation process, reducing false positives caused by complex DBMS behavior. We evaluate Reaper on four widely used DBMSs, including MariaDB, MySQL, OceanBase, and CockroachDB. Reaper uncovers 61 previously unknown bugs, of which 57 have been confirmed by developers and 13 have been fixed at the time of writing. These results demonstrate that historical bug fixes contain actionable semantic knowledge that can be systematically transferred across syntactic siblings to uncover otherwise overlooked defects.

Comment
20 pages, 9 figures, 3 tables

Closest papers

Nearest neighbours by meaning across the whole index.

The History Is the Detector: Executing CVE Patch History, End-to-End

cs.CR cs.AIcs.SE ·2026-09-04 ·Qiushi Wu, Kevin Eykholt, Youngja Park, Xiaokui Shu, Dhilung Kirat +2 more
similarity 0.67
Abstract

Public vulnerability databases collect rich information about known software flaws, including their weakness types, affected components, and related patches. Fixing commits provide the exact code changes that removed these flaws. While these records capture why the original code was unsafe, they are documented mainly for human inspection rather than automated reuse. Consequently, the same unsafe conditions may still exist elsewhere in code without a known advisory, leaving much of this detection knowledge unused. We present BUGSTONE-E2E, a framework that transforms vulnerability history into executable detection rules and validates their findings. First, BUGSTONE-E2E mines reusable rules from verified fixing commits, capturing scan anchors, fix semantics, and CVE provenance and organizing them by CWE and language. Second, detection follows a funnel-shaped pipeline: early stages process a large pool of candidates using lightweight analysis, while later stages apply increasingly capable and expensive models to a shrinking set of targets. Specifically, BUGSTONE-E2E first enumerates call sites matching rule anchors using Tree-sitter, then removes benign sites using lightweight heuristics without LLM calls. Next, LLM-based agents inspect the remaining candidates guided by the rule. Following this inspection, the system re-triages surviving candidates and builds runtime verifications, then generates scope-checked patches validated via two-sided differential tests. Using 19,325 high-severity CVEs from 2022 to 2026, BUGSTONE-E2E identifies 2,710 fixing commits and constructs 1,033 detection rules across 56 CWE families, packaged into 172 skills. When applied across 14 programs, it produced runtime evidence for 644 findings. These results demonstrate that CVE history can be turned into an executable workflow, transforming past vulnerabilities into reproducible detection and repair.

0.20

Post-SQLFix: Property-Guided Repair of LLM-Generated SQL with Runtime Verification

cs.SE ·2026-10-30 ·Zhihao Guo, Zonghan Wu, Yilei Shao, Haizhou Du, Nan Tang +2 more
similarity 0.66
Abstract

Large Language Models (LLMs) for Text-to-SQL frequently generate queries with syntax errors, invalid identifier references, or violations of SQL scoping and typing rules. Repairing these errors is a program correctness problem, yet existing approaches rely on Execution Feedback (EF), whose symptom-level, dialect-specific error messages force the LLM to speculate on root causes. We identify an Arbitration Paradox in this paradigm: the system relies on unverified models both to propose repairs and to make the final judgment on whether those repairs satisfy the required properties. We resolve this paradox by reformulating Text-to-SQL repair as property-guided program repair. Our framework, Post-SQLFix, canonicalizes SQL into a dialect-aware Canonical Query Structure (CQS) and validates it through a hierarchical static analyzer with three stratified property layers (Syntactic, Context-Free Structural, Context-Sensitive Semantic). The analyzer produces precise, schema-aware violation diagnostics to guide the LLM's repair proposals. A Verify-Before-Apply Temporal Monitor then accepts only repairs that reduce the violation count, do not reintroduce resolved violations, and satisfy the operator contracts. These runtime invariants are bounded by the declared rule family and operator contracts, while a finite retry budget ensures halting. On three established benchmarks (BIRD, Spider, Mini-Dev) spanning over 3,000 evaluation instances, three SQL dialects (SQLite, MySQL, PostgreSQL), and six model configurations, Post-SQLFix resolves 77.8% of the execution errors that EF leaves unrepaired, while requiring less than half as many repair iterations and 30% fewer tokens. Crucially, accepted transitions under the monitor introduce no statically detectable structural regressions in our evaluation, translating checked repair fidelity into absolute accuracy improvements of up to +11.6 percentage points.

0.20

SWE-at-Scale: Scalable Issue Synthesis from Historical Bug Fixes

cs.SE ·2026-10-15 ·Silin Chen, Zihan Zhang, Xiaodong Gu, Jiale Huang, Yuling Shi +5 more
code/datacommunity pick similarity 0.66
Abstract · Our code and models are available at https://github.com/cslsolow/SWE-at-scale and https://huggingface.co/SWE-at-Scale

As coding agents become increasingly capable, robust evaluation requires a broader and more diverse supply of verifiable software engineering (SWE) tasks. However, real-world tasks are constrained by the natural frequency of issue reporting and the substantial cost of manual curation. Furthermore, existing synthetic approaches rely primarily on the pretrained knowledge of large models coupled with heuristic prompting, often failing to capture the authentic, complex mechanics of software failures. To exponentially scale SWE task production, we introduce SWE-at-Scale, a framework that automatically synthesizes realistic issues by reverse-engineering real-world software fixes. Our key insight is that historical fixes encapsulate deep mechanistic knowledge of bug formation: inverting a fix transforms repair supervision into bug-induction supervision grounded in actual development scenarios. SWE-at-Scale transfers these learned bug patterns to novel code contexts, repeatedly sampling diverse bug typologies to construct executable SWE tasks. Extensive experiments show that SWE-at-Scale generates more diverse and realistic SWE tasks than current synthesis baselines while posing greater challenges to state-of-the-art coding agents. Notably, SWE-at-Scale can generate distinct bug types from a single code context, with bug diversity scaling robustly even up to 200 generations per file. Overall, SWE-at-Scale turns finite real-world fixes into reusable supervision for scalable synthesis of realistic, diverse, and challenging SWE tasks.

0.59

Detecting DBMS Bugs by Constructing Equivalent Representations of Intermediate Query Results

cs.DB cs.SE ·2026-08-31 ·Xiaoxu Niu, Gong Chen, Jinfu Chen, Xiaoyuan Xie
code/data similarity 0.66
Abstract · 12 pages, 3 figures

Database Management Systems (DBMSs) support multiple SQL mechanisms for representing intermediate query results, including VIEWs, Common Table Expressions (CTEs), and Temporary Tables (TEMPTs). When these mechanisms are used to represent the same intermediate query result, the corresponding queries are expected to produce consistent results. However, we observe that such queries can return inconsistent results, indicating potential DBMS logic bugs. Existing approaches for detecting DBMS logic bugs have never explored result consistency across such equivalent representations. In this paper, we propose ERIQ, a novel testing approach for detecting DBMS logic bugs from the perspective of checking result consistency across Equivalent Representations of Intermediate Query Results. ERIQ constructs SQL variants using a VIEW, a CTE, or a TEMPT to represent the same intermediate query result, executes these variants, and compares their returned results. We evaluated ERIQ on four widely used open-source DBMSs: MySQL, MariaDB, Percona, and OceanBase. In total, ERIQ detected 64 bugs, 63 of which were confirmed by developers, and two have been fixed. Among the confirmed bugs, 54 were unique and previously unknown logic bugs, and one was a documentation issue.

0.24

TyPatch: Transforming Patches into Typestate Rules for Kernel Bug Detection

cs.SE cs.AIcs.CRcs.OS ·2026-09-12 ·Ruoyu Wang, Tuo Li, Jia Li
similarity 0.66
Abstract · 15 pages, 7 figures, 6 tables

Historical Linux kernel patches capture defect knowledge that applies beyond their original repair sites. Recent work has shown that large language models (LLMs) can generate static-analysis checkers from historical patches and use them to uncover new kernel bugs. However, complete-checker generation requires the model both to recover the defect semantics expressed by a patch and to implement sophisticated program-analysis machinery, including object tracking, alias analysis, path-state maintenance, and interprocedural propagation. Coupling these responsibilities in a single end-to-end code-generation task can turn a simple defect rule into an unstable and expensive analyzer-implementation problem. To address this problem, we present TyPatch, which decouples patch-specific defect semantics from analyzer implementation. An LLM translates each patch into a typestate rule specifying its tracked object, actions, guards, transitions, and violations. A shared backend then executes these rules, binding their actions to program events, tracking object identity across aliases, propagating typestate along program paths, and producing reports for all rules. On Linux v6.16, TyPatch finds 559 distinct bugs, 121 of which have been confirmed by kernel developers. In a matched 38-patch comparison with the state-of-the-art complete-checker construction workflow, TyPatch uses 88.3-90.1% fewer generation tokens, while its initial report pools achieve 3.42-14.95$\times$ the precision of those produced by that workflow.

0.11

Detecting Incorrect LLM Explanations for Automated Program Repair

cs.SE ·2026-10-28 ·Lam Nguyen Tung, Chenxi Gu, Xiaoning Du, Haoye Tian, Aldeida Aleti +1 more
similarity 0.64
Abstract

LLM-generated explanations help developers understand automated program repair (APR) rationale and assess patch quality. Incorrect explanations can mislead developers about a repair's reliability, making their detection important. However, manually inspecting the growing volume of generated repairs is impractical, motivating automated detection. Existing methods mainly target incorrect answers from general question-answering LLMs, often by sampling multiple responses and detecting contradictions. Applied to APR, they face two limitations: (1) they may depend on external verifiers that are unavailable or lack APR-specific reasoning; and (2) sampling strategies such as paraphrasing or re-prompting the same input often yield superficial, redundant responses that repeat the original error rather than expose it. We propose MetaReJ, the first method designed to detect incorrect APR explanations using metamorphic testing. Inspired by debate and viva examinations, MetaReJ challenges explanations through an argumentative strategy without requiring external verifiers. It applies novel metamorphic relations (MRs) to recursively question the LLM under equivalent and non-equivalent repair scenarios and detect contradictions. To obtain diverse samples, MetaReJ transforms the buggy program into variants that preserve or alter the underlying bug, then asks the LLM how to repair each variant. Contradictions among responses are encoded as MR violations, which indicate that the original explanation is incorrect. We evaluate MetaReJ on a manually labelled dataset of 1,756 explanations from multiple LLM-based APR systems. MetaReJ achieves an MCC of up to 0.43, outperforming prior methods by up to 333.3% in MCC.

0.19

Correct Verdicts, Flawed Reasoning: Structured Auditing of LLM-based Vulnerability Reasoning

cs.CR cs.LGcs.SE ·2026-10-05 ·Boyue Caroline Hu, Kaivalya Ahir, Ronghao Ni, Limin Jia
benchmark similarity 0.64
Abstract

Large Language Models (LLMs) are increasingly deployed for automated software vulnerability analysis. Binary classification alone is insufficient; practitioners need explanations to triage bugs and engineer patches. Standard practice relies on Chain-of-Thought (CoT) prompting, but free-form reasoning allows models to obscure logical leaps, hallucinated execution steps, and internal inconsistencies behind plausible prose. Our manual audit reveals that approximately 60% of correct vulnerability verdicts are accompanied by fabricated or unverifiable claims, and free-form explanations allow reasoning errors to evade LLM-as-a-judge evaluation. We present Vulnerability Explanation Reasoning Auditor (VERA), an automated framework for auditing LLM vulnerability reasoning. Rather than accepting free-form text, VERA asks models to output a Structured Reasoning Record (SRR) encoding tracked pointers, memory operations, and state transitions in machine-readable fields. A multi-stage judge audits each SRR against eight reasoning failure modes using deterministic checks, with LLM calls reserved for semantic interpretation. The standardized SRR schema also enables automated mutation testing to benchmark judges at scale without human annotation. Our evaluation shows reasoning flaws occur in correct verdicts just as frequently as incorrect ones, and VERA exposes 87% of reasoning errors that free-form LLM-as-judge systematically miss.

0.42

WitnessGym: Benchmarking Coding Agents on the Construction of Bug Witnesses

cs.SE cs.AI ·2026-09-29 ·Haomin Qi, Xiangzhe Xu, Yiming Huang, Jingbo Shang, Chengpeng Wang
benchmarkcode/data similarity 0.64
Abstract · 30 pages, 8 figures, and 12 tables, including appendices

Bug validation asks a coding agent to produce an executable witness for a reported bug. The witness combines a concrete input with a testing harness and exposes faulty behavior during execution. Such evidence makes audit findings actionable, yet benchmark evaluation is difficult when cases reuse public historical bugs and witnesses or require manual construction. We present WitnessGym, an automated framework for constructing bug-validation benchmarks through bug injection. It injects bugs into test-reached paths of real projects, rebuilds each project, and retains cases exposed by a construction-time witness. Bug specifications and execution adapters allow extension to additional bug types and languages. Bug-preserving transformations vary the surrounding structure while preserving the witness behavior. Based on real-world Java projects with test suites, WitnessGym automatically constructs 1,300 benchmark cases. The injected patches resemble historical bug patches and are difficult for the two evaluated models to distinguish in blinded comparisons. We evaluate four coding agent frameworks in six framework/model pairings across bug types, execution contexts, and transformation depths. Witness construction remains difficult even when the bug pattern is known. Our framework, benchmark cases, and evaluation scripts are available.

0.56