CS Paper Radar

A Reproducibility Protocol for Cross-Implementation Evaluation of Post-Quantum ACVP Test Vectors

Christopher M. Frost

cs.CR cs.SE ·submitted 2026-08-13 21:29 UTC · v1

Abstract

Independent implementations of a cryptographic standard should reproduce the same known-answer results, yet agreement is meaningful only when the corpus, revisions, public interfaces, exclusions, and evidence are precisely stated. This study defines a product-neutral reproducibility protocol for three public implementations of NIST ML-KEM against a pinned public Automated Cryptographic Validation Protocol corpus. Protocol v2 freezes provider-specific capabilities, applies one validation-error taxonomy symmetrically, preserves every selected case, and separates bytes, validation verdicts, unsupported operations, and adapter errors. The source-built experiment evaluated @noble/post-quantum 0.7.0, liboqs 0.16.0, and Go 1.26.4. Across three repetitions, the required Cartesian product comprised 2,160 base records: all 1,650 declared executable evaluations matched the NIST oracle, and all 510 unsupported records matched Go's predeclared capability boundary. Pairwise agreement was complete on every executable overlap: 720 of 720 noble-liboqs records and 210 of 210 records for each Go pairing. A separate keyGen-ek-projection diagnostic matched all 150 Go encapsulation-key projections without counting them as full key generation. Three frozen controls independently exercised byte comparison, verdict comparison, and malformed-response error separation; each produced its exact predeclared outcome. No base failure, adapter error, or status instability occurred. The evidence establishes bounded author-run repeatability and exposes a practical standards gap: public ML-KEM packages provide materially different deterministic and validation-test surfaces. Independent external reproduction remains unobserved. The results do not establish certification, exhaustive correctness, side-channel resistance, secure integration, or production assurance.

Comment
16 pages, 4 figures, 9 tables, 28 references. Research artifact: https://doi.org/10.5281/zenodo.21910571

Closest papers

Nearest neighbours by meaning across the whole index.

Testing and Verification of Quantum Compilers through Assurance Contracts and Evidence

cs.SE cs.ET ·2026-09-24 ·Furqan Nasir, Muhammad Arif Shah, Iftikhar Alam
similarity 0.66
Abstract · 38 pages, 3 figures

Quantum compiler assurance requires a semantic relation that matches the intended use and observations that can expose violations at the delivered interface. This critical integrative survey compares verified transformations, equivalence checking, differential and metamorphic testing, property-based testing, and evidence for assurance across revisions. A source catalogue and focused extraction matrix distinguish formal guarantees, reported empirical findings, analytical deductions, and proposed practice. A recorded coverage update and source-level selection decisions make the review auditable within its declared scope. Four documented failure cases connect conditional applicability, parameter association, termination, and phase conventions to method selection. A worked example involving layout metadata, parameter bindings, and ancillary qubits demonstrates why validating a circuit under a reconstructed interpretation can leave the delivered interface unchecked. An illustrative revision scenario then separates fault detection from change attribution. The resulting guidance specifies applicability, complementary observations, and treatment of inconclusive outcomes. Published bug counts, mutation results, and processing benchmarks support different claims and do not establish a common effectiveness ranking. The proposed regression architecture remains unevaluated as a complete system; its practical value requires controlled comparisons on independently adjudicated compiler-change events.

0.19

Capability-Gated Conformance Testing of Quantum Error-Correction Decoder Libraries

cs.AR quant-ph ·2026-09-07 ·Jiachen Shen, Hui Zhong
similarity 0.64
Abstract

A quantum error correction decoder is a library other people's results depend on, judged in one dominant way. Sample errors, decode, and count wrong logical observables. We ask what else can be checked there. Our conformance contract needs no oracle. One check asks that a returned correction explain the syndrome in the caller's index space. The other hands a decoder one instance under two presentations differing only in bookkeeping, where two feasible corrections of different weight prove the heavier is not minimum-weight. Verdicts are gated on what each library declares, so a firing contradicts a published guarantee. Nine configurations from five public libraries give three results. Documentation answers 4 of 54 capability questions. Bounded-distance correctness, the property callers most depend on, has a direct declaration yield of 0.0%, though its hypotheses hold in 62.1% of cases. Presentation sensitivity is real but shallow. One solver moved to a 26% heavier correction under a different numbering, which reaches the logical class at most once in twenty thousand shots. Established evaluation misses corruptions that preserve logical parity, while one summation over the caller's weights catches every one we injected. All 639 certificates ship as bundles a standalone verifier re-derives from first principles.

0.29

Understanding Post-Quantum Cryptography Adoption: Practices, Challenges, and Evolution in Open-Source Software

cs.SE cs.CR ·2026-10-27 ·Souhardya Saha Dip, Moshood A. Fakorede, Krishna Upadhyay, Umar Farooq
code/data similarity 0.64
Abstract · 12 pages, 5 figures, 6 tables

Post-quantum cryptography (PQC) standards establish algorithms for quantum-safe systems, but empirical evidence about how software projects implement and maintain PQC support remains limited. We investigate this implementation and maintenance work by mining 114,361 commits and 5,713 issues across 179 open-source repositories. We use human annotation and validated LLM classification to identify PQC algorithms in commits, classify their adoption status and deployment mode, and identify technical challenges in issue discussions. The commit classification for PQC algorithms reveals that 90% of commits concern maintenance of existing support. From the issue discussions, we construct an 11-category taxonomy of recurring technical challenges. Build, packaging, and dependency failures, along with functional defects, account for approximately half of categorized issues. To examine how PQC support evolves, we use time-series analysis and first implementations of individual algorithms within repositories around NIST standardization milestones. After the 2024 finalization, standardized algorithm names appeared in more than 80% of the PQC development changes analyzed, although earlier names such as Kyber continued to appear. Development activity increased over the observation period. Our milestone analysis, however, did not find a significant increase in first implementations. Migration plans should therefore account for sustained maintenance, prioritize build reliability and integration testing, and track both standardized and predecessor algorithm names. We make the dataset, taxonomy, annotations, and analysis pipeline publicly available.

0.44

Static Detection of Post-Quantum Cryptographic Algorithms in Stripped Binaries for Digital Forensic Examination and Migration Assurance

cs.CR ·2026-08-25 ·Muhammad Shaheer Bin Junaid
similarity 0.63
Abstract

Identifying which post-quantum algorithm a compiled binary implements is a core problem in binary analysis, and it becomes acute once the binary has been stripped, statically linked, and optimised, since the symbols, library dependencies, and runtime behaviour that conventional discovery tools rely on are then gone. This paper presents Kestrel, a static analysis method that identifies an algorithm from the number-theoretic transform constant tables its arithmetic depends on, and applies it to the standardised lattice schemes ML-KEM and ML-DSA, for which no prior method could confirm, from a shipped binary alone, that a quantum-vulnerable algorithm had been replaced by its approved successor. The fingerprints Kestrel derives from public scheme parameters are localised by a normalisation-and-multiset-matching procedure; the false-positive probability is established analytically. Across four independent implementation lineages and all build transformations, including compiler-level obfuscation, Kestrel achieved recall of 128 of 128 with zero false positives. Applied to 6,224 binaries on a production Linux system, it disclosed twelve uncatalogued programs containing ML-KEM, among them the OpenSSH key-exchange program and the container-management stack, where post-quantum code had entered production through the language runtime without the awareness of the projects distributing them. Kestrel distinguishes genuine implementations from advertised claims not backed by the underlying code, attributes each detection to its originating codebase, and, in a forensic disk-image trial, recovered a detection from unallocated space after the deleted binary could no longer be reconstructed. It thus provides a practical basis for cryptographic migration assurance, compliance verification, software supply-chain inspection, and post-quantum forensic examination.

0.23

What Output-Equivalence Oracles Miss: An Empirical Study of Equivalence-Invisible Bug Fixes in Quantum Transpilers

cs.SE quant-ph ·2026-09-12 ·Furqan Nasir, Arif Shah, Iftikhar Alam
similarity 0.63
Abstract · 22 pages, 4 figures, 6 tables

Quantum compilers face the test oracle problem, judged by an output-equivalence oracle: the compiled circuit must compute the same unitary as the original, modulo global phase and a qubit-layout permutation. This oracle, by construction, checks only that semantic map, not the circuit's own layout, permutation, or phase records: a defect there, or in a fixed-seed run's determinism, can pass unseen though the record is public. This empirical software engineering study of quantum transpiler correctness uses repository mining to measure how often this happens in real merged compiler fixes: a systematically identified corpus of Qiskit transpiler bug-fixes, classified by an independently dual-coded, source-validated fault-manifestation taxonomy. Nineteen of 68 fixes (28%, 95% Wilson CI 19-40%) repair faults invisible to this equivalence screen, even one augmented with compilation-validity, circuit-quality, and performance checks, and an extended 104-fix corpus over a wider window holds at the same rate with a tighter interval (29/104, 27.9%, CI 20-37%). A conservative floor remains even restricted to the one unconditionally equivalence-invisible channel (a corrupted layout or permutation record): 10 of 68 fixes (15%, CI 8-25%) beneath the 28% headline. The gap is not Qiskit-specific: it replicates in tket (7/21, 33%), with Cirq smaller but consistent. We detected no systematic differences on five inexpensive PR-level characteristics (19 vs 49, underpowered alone; the extended 29-vs-75 corpus tightens every interval toward zero). This class dominates the invisible set, concentrating at representation-boundary crossings. We release the corpus, codebook, and coding artifacts for quantum software testing research. Here we only measure it.

0.52

QTestBench: A Test Framework for Evaluating LLM-Generated Quantum Code

cs.LG cs.CLquant-ph ·2026-10-09 ·Meriem Terki, Nouhaila Innan, Minghao Shao, Muhammad Kashif, Alberto Marchisio +1 more
top venuebenchmark similarity 0.62
Abstract · Accepted at the 2026 IEEE 2nd International Conference on Quantum Artificial Intelligence (QAI), 2026

Large language models (LLMs) have shown strong results in classical code generation and are increasingly used to write the unit tests that safeguard that code. Whether they can do the same for quantum software is less clear, because quantum programs are probabilistic, depend on numerical tolerances, and are judged by domain-specific checkers, so a well-formed test can still disagree with the benchmark's verdict. Existing quantum benchmarks measure whether a model produces a correct solution, but not whether its tests can distinguish correct implementations from faulty ones. We introduce QTestBench, a framework that evaluates LLM-generated quantum test suites against independent benchmark checkers, never treating the tests themselves as ground truth. For each task, QTestBench prompts models to produce syntactic, semantic, and behavioral tests, injects candidate solutions at runtime, and compares each test verdict with the checker's label under three validation cohorts, each answering a distinct diagnostic question. We benchmark five LLMs on eight PennyLane QHack challenges and ten Qiskit HumanEval tasks. Generated suites almost always execute, but discrimination lags behind: the best generator reaches a test quality of 0.71 on QHack and 0.34 on Qiskit HumanEval, so executability alone overstates test quality. Oracle-verified synthetic mutants stabilize bug-detection estimates when natural faulty solutions are scarce, as on three of ten Qiskit tasks that yield none. QTestBench offers a reproducible, cross-framework benchmark for the trustworthiness of LLM-generated quantum tests.

0.27

FVQKD: A Formal Verification of Quantum Key Distribution Security in Rocq

cs.CR quant-phcs.LO ·2026-10-22 ·Ruixuan Deng, Zhengfeng Ji, Jiayu Zhang, Li Zhou
code/data similarity 0.62
Abstract · 87 pages: 12 pages of main text, 4 pages of references, and 71 pages of appendices. Artifact: https://github.com/fvqkd/FVQKD

Quantum key distribution (QKD), whose security is derived from quantum mechanics rather than computational assumptions, is moving into standardization and commercial deployment. Yet its security guarantees rest on technically delicate proofs checked by hand. Formal verification closes this gap, establishing a two-factor assurance: unconditional security from physical principles, and machine-checked proofs against reasoning errors. We present FVQKD, a Rocq library for machine-checking QKD security proofs. Its first result is a complete, end-to-end machine-checked proof of composable, finite-key security for BB84. The mechanization reconstructs the complete proof infrastructure from first principles, including the smooth entropy framework, the entropic uncertainty relation, and the Quantum Leftover Hash Lemma. It uncovers necessary side conditions that the published argument leaves unstated, one of which reveals a missing proof branch, and it grounds the information-theoretic bound in a formal semantics of protocol execution, which the pen-and-paper analysis lacks. The proof is entirely admit-free, relying only on the Rocq kernel and standard foundational axioms. To our knowledge, this is the first end-to-end formal verification of a QKD security proof. Built on MathComp, MathComp-Analysis, and CoqQ, the development provides quantum-information infrastructure designed for reuse across broader QKD variants and quantum cryptography applications.

0.33

QuSema: Detecting Silent Bugs in Quantum Libraries via Quantum-knowledge-enhanced Agents

cs.SE cs.AIquant-ph ·2026-10-07 ·Yujin Song, Kaining Zhang, Qixin Zhang, Shuai Wang, Pingchuan Ma +1 more
benchmark similarity 0.62
Abstract

Quantum libraries are now critical infrastructure for quantum algorithm development, yet their correctness remains difficult to test. Existing testing techniques mainly rely on failure-based or comparison-based oracles, exposing bugs only when executions fail, violate runtime checks, or disagree with another implementation. Their applicability is limited when suitable execution-based oracles are unavailable, leaving some silent bugs undetected. Such missed bugs can produce incorrect results that propagate into experimental conclusions, simulation studies, and algorithmic designs. Here we present QuSema, an autonomous testing agent for finding silent bugs in quantum libraries. QuSema uses constraints from quantum semantics and documentation as a source-level semantic oracle to assess whether implementation logic can produce invalid outputs from valid inputs. It operates through an agentic loop that repeatedly inspects library API documentation and source code, reasons about the intended behavior of quantum operations, identifies potential semantic deviations, and validates them by generating executable tests through library APIs. Guided by quantum-domain reasoning, QuSema turns high-level behavioral mismatches into concrete, user-triggerable bug reports, enabling it to uncover non-crash defects. We implement QuSema for Qiskit and PennyLane. On a benchmark of 20 historical silent bugs, QuSema achieves higher mean bug relocation counts than Claude Code and Codex, with the DeepSeek configuration costing less than Claude Code. QuSema also discovers 40 previously unknown bugs confirmed by the developers, including 30 silent bugs.

0.04