CS Paper Radar

Executable Specifications as First-Class Independent Verification and Validation Artifacts for Quantum Information and Machine Learning

Ragib Shahariar Ayon, Shibbir Ahmed

cs.SE ·submitted 2026-11-05 01:36 UTC · v1
publishedposition

Abstract

Executable specifications make software assumptions explicit and checkable, yet their role in quantum information and machine learning (QIML) remains underexplored. We argue that executable specifications should be treated as first-class artifacts for independent verification and validation (IV&V) of QIML systems. Static analysis, testing, simulation, and runtime checking can provide evidence about QIML behavior, while Design by Contract shows that selected quantum-state, measurement, and statistical properties can be expressed as checkable conditions. However, adapting existing methods for specification inference and assessment from conventional and machine-learning software to QIML raises several challenges, notably limited evidence for deriving generalizable specifications, cross-stage and probabilistic behavior, repository-scale reasoning across hybrid workflows, and weak behavioral oracles. We characterize these challenges and identify research directions for QIML corpora, specification semantics, repository-level analysis, and specification assessment.

Comment
4 pages, position paper, published at the Second AAAI Symposium on Quantum Information & Machine Learning (QIML)

Closest papers

Nearest neighbours by meaning across the whole index.

Testing and Verification of Quantum Compilers through Assurance Contracts and Evidence

cs.SE cs.ET ·2026-09-24 ·Furqan Nasir, Muhammad Arif Shah, Iftikhar Alam
similarity 0.71
Abstract · 38 pages, 3 figures

Quantum compiler assurance requires a semantic relation that matches the intended use and observations that can expose violations at the delivered interface. This critical integrative survey compares verified transformations, equivalence checking, differential and metamorphic testing, property-based testing, and evidence for assurance across revisions. A source catalogue and focused extraction matrix distinguish formal guarantees, reported empirical findings, analytical deductions, and proposed practice. A recorded coverage update and source-level selection decisions make the review auditable within its declared scope. Four documented failure cases connect conditional applicability, parameter association, termination, and phase conventions to method selection. A worked example involving layout metadata, parameter bindings, and ancillary qubits demonstrates why validating a circuit under a reconstructed interpretation can leave the delivered interface unchecked. An illustrative revision scenario then separates fault detection from change attribution. The resulting guidance specifies applicability, complementary observations, and treatment of inconclusive outcomes. Published bug counts, mutation results, and processing benchmarks support different claims and do not establish a common effectiveness ranking. The proposed regression architecture remains unevaluated as a complete system; its practical value requires controlled comparisons on independently adjudicated compiler-change events.

0.19

QisMC: A Model Checker for QISKIT Program Debugging

cs.LO quant-ph ·2026-08-25 ·Aochu Dai, Mingsheng Ying
benchmark similarity 0.71
Abstract

We present QisMC, the first quantum model checker dedicated to debugging Qiskit programs. On the theoretical side, we introduce the notion of quantum-classical transition system and a quantum computation tree logic (qCTL) grounded in Birkhoff-von Neumann logic for modeling the behaviors and specifying the properties of Qiskit programs, respectively. On the implementation side, QisMC provides an end-to-end framework encompassing transition system generation, logical formula representation, and model checking algorithms, and it efficiently performs image computation based on decision diagrams. These methods and design principles give QisMC advantages over previous Qiskit program debuggers, including a unified property specification language, a fully automated and exhaustive verification process, and the ability to generate counterexamples. Extensive illustrative examples and benchmark evaluations demonstrate the practicality, efficiency, and scalability of QisMC in verifying realistic quantum programs.

0.11

QEDIA: Quantum Program Verification in Lean

cs.PL quant-ph ·2026-10-13 ·Yuanjie, Ren, Zhicheng Zhang, Junyi Liu, Jinzheng Li +2 more
highly cited similarity 0.69
Abstract

Practical quantum programs interleave quantum and classical computation: classical data shape the quantum operations that are performed, and measurement outcomes shape the classical computation that follows. Formal verification of such programs in a proof assistant requires mechanizing a program logic that handles this interplay consistently at every level, from the mathematics of quantum mechanics, through program syntax and semantics, to specifications and proof rules. Such mechanizations, along with the program proofs built on them, typically require substantial manual effort. We present QEDIA, a framework for verifying quantum programs with classical variables in Lean 4. At its foundation is an open library that formalizes the quantum Hoare logic QHL+ across all these levels and proves the logic sound for both partial and total correctness. We construct the library and the program proofs, under human supervision, using an LLM-based prover agent, whose general-purpose core we extend for quantum program verification. We illustrate QEDIA with verified programs including the quantum Fourier transform, phase estimation, repeat-until-success circuits, and Hamiltonian simulation.

0.28

Noise-Resilient Testing of Quantum Programs Based on Output Dominance

cs.SE ·2026-10-03 ·Yuechen Li, Yige Yang, Tao Yue
similarity 0.67
Abstract

Quantum software testing (QST) is a central quality-assurance activity for checking whether quantum programs conform to their specifications. In the noisy intermediate-scale quantum (NISQ) era, gate, decoherence, and measurement errors distort observed outputs, making it difficult to distinguish program defects from noise. The existing noise-aware QST literature typically mitigates or models noise before applying a test oracle. However, these strategies can incur substantial cost and depend on calibration data, model training, or backend-specific information, limiting their applicability. We present Qomino, a dominance-guided QST approach for judging tests from noisy samples without a separate noise-estimation or mitigation phase. The approach combines decomposition-aware inverse construction, comparison of the expected outcome with the strongest observed competitor, and bounded retries for inconclusive decisions. We evaluate Qomino on 30 controlled buggy variants from six Qiskit programs and six simulated noisy backends, comparing it with six baselines spanning statistical, quantum-specific, and learning-based methods. Qomino achieves 85.65% accuracy over all six programs and 99.95% in the restricted two-program evaluation, significantly outperforming the corresponding baselines. Across the evaluated programs and backends under the stated settings, Qomino is faster than the compared quantum-specific and learning-based methods. Ablation and retry analyses support the contributions of its main components.

0.09

QTestBench: A Test Framework for Evaluating LLM-Generated Quantum Code

cs.LG cs.CLquant-ph ·2026-10-09 ·Meriem Terki, Nouhaila Innan, Minghao Shao, Muhammad Kashif, Alberto Marchisio +1 more
top venuebenchmark similarity 0.66
Abstract · Accepted at the 2026 IEEE 2nd International Conference on Quantum Artificial Intelligence (QAI), 2026

Large language models (LLMs) have shown strong results in classical code generation and are increasingly used to write the unit tests that safeguard that code. Whether they can do the same for quantum software is less clear, because quantum programs are probabilistic, depend on numerical tolerances, and are judged by domain-specific checkers, so a well-formed test can still disagree with the benchmark's verdict. Existing quantum benchmarks measure whether a model produces a correct solution, but not whether its tests can distinguish correct implementations from faulty ones. We introduce QTestBench, a framework that evaluates LLM-generated quantum test suites against independent benchmark checkers, never treating the tests themselves as ground truth. For each task, QTestBench prompts models to produce syntactic, semantic, and behavioral tests, injects candidate solutions at runtime, and compares each test verdict with the checker's label under three validation cohorts, each answering a distinct diagnostic question. We benchmark five LLMs on eight PennyLane QHack challenges and ten Qiskit HumanEval tasks. Generated suites almost always execute, but discrimination lags behind: the best generator reaches a test quality of 0.71 on QHack and 0.34 on Qiskit HumanEval, so executability alone overstates test quality. Oracle-verified synthetic mutants stabilize bug-detection estimates when natural faulty solutions are scarce, as on three of ten Qiskit tasks that yield none. QTestBench offers a reproducible, cross-framework benchmark for the trustworthiness of LLM-generated quantum tests.

0.27

QuSema: Detecting Silent Bugs in Quantum Libraries via Quantum-knowledge-enhanced Agents

cs.SE cs.AIquant-ph ·2026-10-07 ·Yujin Song, Kaining Zhang, Qixin Zhang, Shuai Wang, Pingchuan Ma +1 more
benchmark similarity 0.66
Abstract

Quantum libraries are now critical infrastructure for quantum algorithm development, yet their correctness remains difficult to test. Existing testing techniques mainly rely on failure-based or comparison-based oracles, exposing bugs only when executions fail, violate runtime checks, or disagree with another implementation. Their applicability is limited when suitable execution-based oracles are unavailable, leaving some silent bugs undetected. Such missed bugs can produce incorrect results that propagate into experimental conclusions, simulation studies, and algorithmic designs. Here we present QuSema, an autonomous testing agent for finding silent bugs in quantum libraries. QuSema uses constraints from quantum semantics and documentation as a source-level semantic oracle to assess whether implementation logic can produce invalid outputs from valid inputs. It operates through an agentic loop that repeatedly inspects library API documentation and source code, reasons about the intended behavior of quantum operations, identifies potential semantic deviations, and validates them by generating executable tests through library APIs. Guided by quantum-domain reasoning, QuSema turns high-level behavioral mismatches into concrete, user-triggerable bug reports, enabling it to uncover non-crash defects. We implement QuSema for Qiskit and PennyLane. On a benchmark of 20 historical silent bugs, QuSema achieves higher mean bug relocation counts than Claude Code and Codex, with the DeepSeek configuration costing less than Claude Code. QuSema also discovers 40 previously unknown bugs confirmed by the developers, including 30 silent bugs.

0.04

CC4Q Workbench: From Quantum Repositories to Analyzable Code Comments

cs.SE ·2026-10-23 ·Zenghui Zhou, Yuechen Li, Zheng Zheng
code/data similarity 0.65
Abstract

Quantum software development kits (SDKs) expose high-level APIs whose use requires an understanding of underlying mathematical principles, circuit semantics, and hardware properties. Code comments can provide natural-language or structural explanations that are not explicit in API calls. However, studying comments in a new quantum SDK still requires researchers to coordinate separate steps for comment extraction, quantum-specific classification, and manual verification, resulting in repeated integration effort. To address this problem, we present CC4Q Workbench, an open-source workbench that integrates these steps into a reusable workflow. Given a Python repository at a specified revision, CC4Q Workbench automatically extracts code-comment pairs, segments comments into sentence-level units, and applies two trained classifiers to identify quantum-specific content and assign quantum-knowledge categories. Its web interface supports exploration of bundled datasets, inspection of code snippets, corpus construction from new repositories, manual annotation with model predictions hidden during binary labeling, and classifier fine-tuning using exported annotations. The release provides aggregate statistics for six SDK datasets comprising 39,593 sentence-level code comment units (SCCUs), of which 17,517 are browsable in the web interface. Applied to three additional application-layer Qiskit repositories outside the original CC4Q dataset, the pipeline ran without code changes and produced 11,517 SCCUs. Manual verification of 379 SCCUs showed 85.0% agreement between human annotations and the binary classifier's predictions.

0.20

Irene: Equivalence Checking of Hybrid Quantum Programs via Structure-Preserving Symbolic Reduction

cs.PL cs.LOcs.SE ·2026-09-28 ·Jingyu Ke, Jingyang Li, Guoqiang Li
benchmark similarity 0.64
Abstract · 21 pages, 5 figures, 3 tables, 1 algorithm

Equivalence checking is essential for validating compiler transformations of hybrid quantum programs, which combine quantum operations, measurements, and classical control. Measurement-dependent control limits unitary reasoning, while dependencies between classical outcomes and quantum operations can enlarge intermediate symbolic states. We present Irene, an equivalence-checking framework for bounded hybrid quantum programs based on structure-preserving symbolic reduction. The framework progressively simplifies equivalence obligations through three levels of reasoning. At the gate level, algebraic identities simplify unitary regions. At the hybrid path-sum (HPS) level, reduced symbolic execution states are represented as typed graphs, whose isomorphism certifies equivalence. Remaining obligations are handled by density kernels that characterize transformations of input density operators into observable outputs, allowing comparison even when internal measurement histories differ. Residual coefficient differences are encoded as SMT queries. A common set of symbolic reductions supports HPS and density-kernel reasoning by preserving factored Boolean and arithmetic expressions, eliminating reducible dependencies before expanding residual sums. We evaluate Irene against five equivalence checkers on 1,982 program pairs from seven benchmark suites. Irene solves 1,584 pairs (79.92%), compared with 57.52% for MQT QCEC, the baseline with the highest aggregate coverage, with a mean end-to-end time of 3.93 seconds per solved pair. Applied as an equivalence-checking oracle, Irene also identifies 15 previously unknown bugs in quantum compilers, including Qiskit, Cirq, and PennyLane.

0.22