CS Paper Radar

A Testing Method for Detector Error Models in Quantum Error Correction Compilers

Junjie Luo, Yasunari Suzuki

cs.SE ·submitted 2026-11-16 04:13 UTC · v1

Abstract

A quantum error correction (QEC) compiler produces, from a code and a noise model, both the physical circuit to execute and a detector error model (DEM). The DEM lists the probability of each error mechanism and the measurements it flips. Decoders consume the DEM without verification, so a defect that describes the noise incorrectly calibrates the decoder against wrong rates and silently degrades the logical error rate. Existing techniques for testing quantum software judge correctness by crashes or by statistics of sampled outputs. DEM defects that only distort rates lie below what sampling can resolve at the error rates where QEC operates. We present DEMVal, a testing method that validates each model the compiler emits against a reconstructed reference, with rates obtained from outside the compiler under test. It compares the two models exactly, so its sensitivity is independent of the physical error rate. The reference can be built in three ways that jointly cover the pipeline from source program to emitted model. Applied to NVIDIA's CUDA-Q QEC, Riverlane's Deltakit, and the deq component of Microsoft's QDK-EC, DEMVal found six defects, all confirmed by the developers and fixed. Sampling could have exposed only the four defects that drop, merge, or misplace mechanisms, and exposing all four requires millions of samples per experiment. Even analytically optimal moment tests miss the two rate-level defects, while exact model comparison detects all six with no samples. To our knowledge, DEMVal is the first systematic testing method for the error-model semantics of QEC compilers.

Closest papers

Nearest neighbours by meaning across the whole index.

Noise-Resilient Testing of Quantum Programs Based on Output Dominance

cs.SE ·2026-10-03 ·Yuechen Li, Yige Yang, Tao Yue
similarity 0.81
Abstract

Quantum software testing (QST) is a central quality-assurance activity for checking whether quantum programs conform to their specifications. In the noisy intermediate-scale quantum (NISQ) era, gate, decoherence, and measurement errors distort observed outputs, making it difficult to distinguish program defects from noise. The existing noise-aware QST literature typically mitigates or models noise before applying a test oracle. However, these strategies can incur substantial cost and depend on calibration data, model training, or backend-specific information, limiting their applicability. We present Qomino, a dominance-guided QST approach for judging tests from noisy samples without a separate noise-estimation or mitigation phase. The approach combines decomposition-aware inverse construction, comparison of the expected outcome with the strongest observed competitor, and bounded retries for inconclusive decisions. We evaluate Qomino on 30 controlled buggy variants from six Qiskit programs and six simulated noisy backends, comparing it with six baselines spanning statistical, quantum-specific, and learning-based methods. Qomino achieves 85.65% accuracy over all six programs and 99.95% in the restricted two-program evaluation, significantly outperforming the corresponding baselines. Across the evaluated programs and backends under the stated settings, Qomino is faster than the compared quantum-specific and learning-based methods. Ablation and retry analyses support the contributions of its main components.

0.09

Are We Testing the Wrong Quantum Bugs? A Study of Fault Representation Gaps in Quantum Software Testing

cs.SE cs.PL ·2026-10-15 ·Asif Mohammed Saad, Ahsanul Ameen Sabit, Jonah Brandenburgh, Jiyuan Wang, Qian Zhang +1 more
similarity 0.78
Abstract

Quantum software relies on compilers, intermediate representations, and execution frameworks that transform programs before execution. Faults in these components can change the behavior of otherwise correct quantum programs. Yet quantum testing techniques are often evaluated using mutation-generated faults and curated bug benchmarks. We investigate how well the faults represented by these artifacts reflect those encountered in modern quantum software. We audit four quantum mutation-testing tools and three curated bug benchmarks, and collect 410 verified bugs from ten quantum software repositories over two years. We classify faults from all three sources into a common taxonomy of 56 bug families and compare their representation across mutation operators, curated benchmarks, and recent real-world bugs. The mutation tools support 20 bug families, and curated benchmarks add 15 more. Yet only 56 of the 410 real-world bugs (13.7%) belong to families represented by either source; the remaining 354 (86.3%) belong to families absent from both. The largest unrepresented families concern compiler transformations (66 bugs), type handling (53), and input validation (47). Unlike local gate or statement changes, these faults often depend on intermediate representations, compiler pipelines, input types, or dependency versions. Existing mutation operators primarily modify circuits and program constructs, whereas many real-world faults arise in the compilers, representations, and framework interfaces that process them. Addressing this gap requires looking beyond circuit-level faults toward testing semantic preservation across compiler transformations, correctness at representation boundaries, and interactions between software components. Our study provides empirical evidence to guide fault models and benchmarks that capture these mechanisms and preserve the contexts needed to reproduce them.

0.10

QDiff: Differential Testing of Quantum Software Stacks

cs.SE quant-ph ·2026-11-05 ·Jiyuan Wang, Qian Zhang, Guoqing Harry Xu, Miryung Kim
top venuehighly cited similarity 0.77
Abstract

Over the past few years, several quantum software stacks (QSS) have been developed in response to rapid hardware advances in quantum computing. A QSS includes a quantum programming language, an optimizing compiler that translates a quantum algorithm written in a high-level language into quantum gate instructions, a quantum simulator that emulates these instructions on a classical device, and a software controller that sends analog signals to a very expensive quantum hardware based on quantum circuits. In comparison to traditional compilers and architecture simulators, QSSes are difficult to tests due to the probabilistic nature of results, the lack of clear hardware specifications, and quantum programming complexity. This work devises a novel differential testing approach for QSSes, named QDIFF with three major innovations: (1) We generate input programs to be tested via semantics-preserving, source to source transformation to explore program variants. (2) We speed up differential testing by filtering out quantum circuits that are not worthwhile to execute on quantum hardware by analyzing static characteristics such as a circuit depth, 2 qubit gate operations, gate error rates, and T1 relaxation time. (3) We design an extensible equivalence checking mechanism via distribution comparison functions such as Kolmogorov Smirnov test and cross entropy

0.69

Testing and Verification of Quantum Compilers through Assurance Contracts and Evidence

cs.SE cs.ET ·2026-09-24 ·Furqan Nasir, Muhammad Arif Shah, Iftikhar Alam
similarity 0.76
Abstract · 38 pages, 3 figures

Quantum compiler assurance requires a semantic relation that matches the intended use and observations that can expose violations at the delivered interface. This critical integrative survey compares verified transformations, equivalence checking, differential and metamorphic testing, property-based testing, and evidence for assurance across revisions. A source catalogue and focused extraction matrix distinguish formal guarantees, reported empirical findings, analytical deductions, and proposed practice. A recorded coverage update and source-level selection decisions make the review auditable within its declared scope. Four documented failure cases connect conditional applicability, parameter association, termination, and phase conventions to method selection. A worked example involving layout metadata, parameter bindings, and ancillary qubits demonstrates why validating a circuit under a reconstructed interpretation can leave the delivered interface unchecked. An illustrative revision scenario then separates fault detection from change attribution. The resulting guidance specifies applicability, complementary observations, and treatment of inconclusive outcomes. Published bug counts, mutation results, and processing benchmarks support different claims and do not establish a common effectiveness ranking. The proposed regression architecture remains unevaluated as a complete system; its practical value requires controlled comparisons on independently adjudicated compiler-change events.

0.19

Noise-Aware Testing of NISQ Quantum Programs: Self-Baseline Oracle with a Controlled False-Positive Rate

cs.SE quant-ph ·2026-11-16 ·Mir Mohammad Yousuf, Shabir Ahmad Sofi
similarity 0.76
Abstract · 53 pages, 7 figures

Testing quantum programs on Noisy Intermediate-Scale Quantum (NISQ) devices faces a test-oracle problem: hardware noise and finite-shot sampling make the runs of a correct program vary as much as those of a faulty one, so oracles with a fixed acceptance threshold produce false positives that depend on the program and the backend. We present a noise-aware testing framework built on self-baseline calibration: each run of the program under test is compared with repeated executions of a trusted reference program on the same backend, and the acceptance threshold is the $(1-α)$ quantile of the finite-shot null distribution of the Jensen-Shannon divergence, so the oracle's false-positive rate is a level set in advance. A noncentral chi-square model predicts detection power, a homogeneity test and a two-run sentinel detect drift, a mutation engine labels every mutant exactly within a three-class noise-aware taxonomy, and zero-noise extrapolation attributes rejections to faults or noise. In simulation on nine core benchmarks, 32 QASMBench programs, ten real bugs, seven noise regimes and three device snapshots, the oracle's false-positive rate is 0.0011 at $α= 10^{-3}$ over 9,000 correct runs, against 36-74% for fixed thresholds, 8.9% for a QOIN-style filter and 10.8% for a globally calibrated threshold, while attaining the highest F1 on non-equivalent faults (0.941). On an IBM Heron device, using about 14 minutes of QPU time, it rejected none of 82 correct runs and detected 120 of 121 faults and semantic mutants, and a two-run sentinel reduced drift-induced false positives from 27.4% to 1.8%.

0.14

ConQEC: Concolic Testing for Quantum Error-Correcting Code Implementations

cs.SE quant-ph ·2026-10-22 ·Zihao Li, Ji Guan, Mingsheng Ying
top venue similarity 0.76
Abstract · 12 pages. Accepted at the 49th IEEE/ACM International Conference on Software Engineering (ICSE 2027)

Quantum error-correcting code (QECC) implementations are hybrid software-control systems: measured syndromes drive decoder dispatch, recovery actions, and logical checks. Existing QECC verification techniques can prove formal obligations, and general quantum testing can expose final failures, but developers still lack a scalable testing workflow that generates concrete faults and explains where a QECC repair path first diverges. This paper presents ConQEC, to our knowledge the first concolic testing framework for QECC implementations. By combining concrete (Pauli) fault replay with symbolic repair-path constraints, ConQEC generates tests that expose not only final logical failures but also fault--syndrome--decoder--recovery (FSDR) diagnostics: the injected fault, observed syndrome, decoder decision, recovery action, oracle result, and earliest divergent stage. Its symbolic encoding of Pauli faults, syndrome predicates, decoder and recovery relations, and oracle checks enables this diagnostic workflow to scale beyond exponentially increasing amplitude-level quantum-state reasoning. We implemented ConQEC and evaluated it on generated source-level QECC families and public QECC artifacts with up to thousands of qubits. ConQEC detects the seeded bug in every experimental case and reports a replay-valid, fault-localizing FSDR witness for each.

0.31

QTestBench: A Test Framework for Evaluating LLM-Generated Quantum Code

cs.LG cs.CLquant-ph ·2026-10-09 ·Meriem Terki, Nouhaila Innan, Minghao Shao, Muhammad Kashif, Alberto Marchisio +1 more
top venuebenchmark similarity 0.75
Abstract · Accepted at the 2026 IEEE 2nd International Conference on Quantum Artificial Intelligence (QAI), 2026

Large language models (LLMs) have shown strong results in classical code generation and are increasingly used to write the unit tests that safeguard that code. Whether they can do the same for quantum software is less clear, because quantum programs are probabilistic, depend on numerical tolerances, and are judged by domain-specific checkers, so a well-formed test can still disagree with the benchmark's verdict. Existing quantum benchmarks measure whether a model produces a correct solution, but not whether its tests can distinguish correct implementations from faulty ones. We introduce QTestBench, a framework that evaluates LLM-generated quantum test suites against independent benchmark checkers, never treating the tests themselves as ground truth. For each task, QTestBench prompts models to produce syntactic, semantic, and behavioral tests, injects candidate solutions at runtime, and compares each test verdict with the checker's label under three validation cohorts, each answering a distinct diagnostic question. We benchmark five LLMs on eight PennyLane QHack challenges and ten Qiskit HumanEval tasks. Generated suites almost always execute, but discrimination lags behind: the best generator reaches a test quality of 0.71 on QHack and 0.34 on Qiskit HumanEval, so executability alone overstates test quality. Oracle-verified synthetic mutants stabilize bug-detection estimates when natural faulty solutions are scarce, as on three of ten Qiskit tasks that yield none. QTestBench offers a reproducible, cross-framework benchmark for the trustworthiness of LLM-generated quantum tests.

0.27

Retromorphic Testing of Quantum Compiler Passes

cs.SE quant-ph ·2026-10-08 ·Mushahid Khan, Olivia Di Matteo
similarity 0.75
Abstract

Quantum compilers play a critical role in transforming high-level quantum programs into optimized, hardware-compatible circuits. However, verifying the correctness of compiler passes remains challenging, as determining the expected output of large, deeply entangled quantum circuits is computationally intractable. This challenge is further amplified when compiler passes modify already complex circuit structures, making manual validation of transformed circuits impractical. In this work, we perform a systematic analysis of unit tests for quantum compiler passes in four quantum programming frameworks (PennyLane, Qiskit, Cirq, and pytket). Our findings indicate validation is dominated by program-content and program-metric assertions, and test circuits are generally small and shallow. Motivated by these observations, we introduce a testing methodology for automated validation of quantum compiler passes based on retromorphic testing and principles from the Hadamard test. This methodology analyzes a compiler pass, test circuit, and expected pass behavior to verify semantic preservation and intended structural modifications. We implement our methods in a framework, RetroQ, and apply it to compiler passes in PennyLane and Qiskit. Experimental evaluation reproduced several existing bugs as well as uncovered previously undetected defects, such as flawed symbolic parameter handling, incorrect commutation logic, failure to recognize self-adjointness of gates, and runtime crashes. These findings highlight the need for compiler-pass-specific testing methodologies to improve the reliability of the evolving quantum software stack.

0.05