CS Paper Radar

CC4Q Workbench: From Quantum Repositories to Analyzable Code Comments

Zenghui Zhou, Yuechen Li, Zheng Zheng

cs.SE ·submitted 2026-10-23 13:59 UTC · v1
code/data

Abstract

Quantum software development kits (SDKs) expose high-level APIs whose use requires an understanding of underlying mathematical principles, circuit semantics, and hardware properties. Code comments can provide natural-language or structural explanations that are not explicit in API calls. However, studying comments in a new quantum SDK still requires researchers to coordinate separate steps for comment extraction, quantum-specific classification, and manual verification, resulting in repeated integration effort. To address this problem, we present CC4Q Workbench, an open-source workbench that integrates these steps into a reusable workflow. Given a Python repository at a specified revision, CC4Q Workbench automatically extracts code-comment pairs, segments comments into sentence-level units, and applies two trained classifiers to identify quantum-specific content and assign quantum-knowledge categories. Its web interface supports exploration of bundled datasets, inspection of code snippets, corpus construction from new repositories, manual annotation with model predictions hidden during binary labeling, and classifier fine-tuning using exported annotations. The release provides aggregate statistics for six SDK datasets comprising 39,593 sentence-level code comment units (SCCUs), of which 17,517 are browsable in the web interface. Applied to three additional application-layer Qiskit repositories outside the original CC4Q dataset, the pipeline ran without code changes and produced 11,517 SCCUs. Manual verification of 379 SCCUs showed 85.0% agreement between human annotations and the binary classifier's predictions.

Closest papers

Nearest neighbours by meaning across the whole index.

From Pattern Detection to Composition Analysis in Quantum Software

cs.SE quant-ph ·2026-08-11 ·Neilson Carlos Leite Ramalho, Erico Augusto da Silva, Anthony Accioly, Higor Amario de Souza, Marcos Lordello Chaim
code/data similarity 0.72
Abstract

Quantum software patterns provide high-level abstractions for building quantum programs, but there is still little empirical evidence on how they are adopted in practice. In prior work, we extended an existing quantum-pattern atlas into a 61-pattern catalog, created a knowledge base that links framework components to those patterns, and built a tool that mines pattern implementations from open-source code. We applied this tool on 80 projects and find that all 23 patterns occur in practice. In this work, we extend the tool with two additional matching channels and a vocabulary expansion step, and execute a quantitative evaluation of its accuracy on Qrisp, a framework not present in the knowledge base, reaching a micro-F1 of 0.712 against 0.449 without the expansion step. We then construct composition graphs that record calls among the high-level framework components associated with patterns and store them in a graph database. We use these graphs to examine how pattern implementations are assembled inside each framework, why patterns co-occur, and how much of a pattern's detection count comes from components called directly by developers rather than introduced through internal framework calls. We release qpa, an open-source mining pipeline, together with the knowledge base, which maps 286 framework components across five sources to the pattern catalog, maintained with the support of an LLM ensemble that classifies newly extracted components, and the resulting pattern usage dataset, to support reproducible studies on the adoption and evolution of quantum patterns.

0.20

Qlippy: A Retrieval-Augmented GenAI Assistant for Reproducible Quantum Workflows and Experiment Tracking

cs.AI quant-ph ·2026-09-04 ·Mahee Gamage, Vlad Stirbu
similarity 0.72
Abstract · Accepted for publication in the QGenAI Workshop at IEEE QCE 2026

Quantum software development is iterative and error-prone. Noisy hardware and repeated re-execution make experiment tracking, provenance, and reproducibility essential, yet these practices are hard to adopt because of tooling complexity and the specialized knowledge they demand. General-purpose language models can help but tend to hallucinate and lack grounding in domain-specific tooling. We present Qlippy, a retrieval-augmented GenAI assistant embedded in the development environment that grounds its responses in a curated corpus of quantum-software-engineering knowledge. Qlippy explains reproducibility and provenance concepts in context and augments existing Qiskit programs with MLflow-based experiment tracking aligned to the QProv schema. By separating knowledge from model parameters, grounding gives explicit control over the scope and provenance of the assistant's responses and reduces reliance on model scale, which points toward low-cost, privacy-preserving local deployment.

0.18

Large Language Models for Quantum Code Translation: An Exploratory Study

cs.SE quant-ph ·2026-10-26 ·Xiyuan Li, Yuechen Li, Jianjun Zhao
similarity 0.71
Abstract

Quantum software development is challenged by quantum-mechanical semantics and fragmented software development kits (SDKs), which complicate cross-SDK code translation and execution. In this paper, we propose an LLM-based approach for translating executable programs between high-level quantum SDKs. We develop a quantum-aware translation workflow whose prompt design and validation reflect quantum semantics and accommodate the version evolution of quantum ecosystems. We conduct a preliminary evaluation of 4 LLM configurations on 43 tasks, translating Qiskit programs to PennyLane and Q#. LLMs achieve a zero-shot Pass@1 of up to 72.33%, while one-shot prompting further improves the translation effectiveness. These results suggest potential for quantum code translation by LLMs.

0.23

QuSema: Detecting Silent Bugs in Quantum Libraries via Quantum-knowledge-enhanced Agents

cs.SE cs.AIquant-ph ·2026-10-07 ·Yujin Song, Kaining Zhang, Qixin Zhang, Shuai Wang, Pingchuan Ma +1 more
benchmark similarity 0.69
Abstract

Quantum libraries are now critical infrastructure for quantum algorithm development, yet their correctness remains difficult to test. Existing testing techniques mainly rely on failure-based or comparison-based oracles, exposing bugs only when executions fail, violate runtime checks, or disagree with another implementation. Their applicability is limited when suitable execution-based oracles are unavailable, leaving some silent bugs undetected. Such missed bugs can produce incorrect results that propagate into experimental conclusions, simulation studies, and algorithmic designs. Here we present QuSema, an autonomous testing agent for finding silent bugs in quantum libraries. QuSema uses constraints from quantum semantics and documentation as a source-level semantic oracle to assess whether implementation logic can produce invalid outputs from valid inputs. It operates through an agentic loop that repeatedly inspects library API documentation and source code, reasons about the intended behavior of quantum operations, identifies potential semantic deviations, and validates them by generating executable tests through library APIs. Guided by quantum-domain reasoning, QuSema turns high-level behavioral mismatches into concrete, user-triggerable bug reports, enabling it to uncover non-crash defects. We implement QuSema for Qiskit and PennyLane. On a benchmark of 20 historical silent bugs, QuSema achieves higher mean bug relocation counts than Claude Code and Codex, with the DeepSeek configuration costing less than Claude Code. QuSema also discovers 40 previously unknown bugs confirmed by the developers, including 30 silent bugs.

0.04

QTestBench: A Test Framework for Evaluating LLM-Generated Quantum Code

cs.LG cs.CLquant-ph ·2026-10-09 ·Meriem Terki, Nouhaila Innan, Minghao Shao, Muhammad Kashif, Alberto Marchisio +1 more
top venuebenchmark similarity 0.67
Abstract · Accepted at the 2026 IEEE 2nd International Conference on Quantum Artificial Intelligence (QAI), 2026

Large language models (LLMs) have shown strong results in classical code generation and are increasingly used to write the unit tests that safeguard that code. Whether they can do the same for quantum software is less clear, because quantum programs are probabilistic, depend on numerical tolerances, and are judged by domain-specific checkers, so a well-formed test can still disagree with the benchmark's verdict. Existing quantum benchmarks measure whether a model produces a correct solution, but not whether its tests can distinguish correct implementations from faulty ones. We introduce QTestBench, a framework that evaluates LLM-generated quantum test suites against independent benchmark checkers, never treating the tests themselves as ground truth. For each task, QTestBench prompts models to produce syntactic, semantic, and behavioral tests, injects candidate solutions at runtime, and compares each test verdict with the checker's label under three validation cohorts, each answering a distinct diagnostic question. We benchmark five LLMs on eight PennyLane QHack challenges and ten Qiskit HumanEval tasks. Generated suites almost always execute, but discrimination lags behind: the best generator reaches a test quality of 0.71 on QHack and 0.34 on Qiskit HumanEval, so executability alone overstates test quality. Oracle-verified synthetic mutants stabilize bug-detection estimates when natural faulty solutions are scarce, as on three of ten Qiskit tasks that yield none. QTestBench offers a reproducible, cross-framework benchmark for the trustworthiness of LLM-generated quantum tests.

0.27

HyQDB: LLM-Assisted Debugging for Hybrid Quantum Workflows

cs.SE ·2026-09-23 ·Charlie Campbell, Hao Mark Chen, Shuang Liang, Hongxiang Fan
benchmark similarity 0.67
Abstract

Hybrid quantum program failures frequently occur silently, yet existing debugging tools provide limited support for detecting and repairing them. These faults dominate the failures reported by domain experts, yet existing tools evaluate on public data that under-represents this failure mode. Our key insight is that faults divide into two classes that require different strategies: mechanical faults, which allow deterministic analysis, and conceptual faults, which require reconstructing the program's intent. To address this challenge, we present HyQDB, a tiered agent that injects deterministic hardware, physics and optimization evidence into the LLM repair process. When no evidence is detected, the agent treats this silence as a signal to escalate to a second intent-reconstruction tier, that infers the program's behaviour and reconciles it with the implementation. To evaluate HyQDB, we introduce QFaultBench, a benchmark built from an expert-derived fault taxonomy that represents the true failure modes of hybrid programs. On a held-out set of human-authored programs, HyQDB raises repair success over a standard LLM from 45% to 75%. We show that the escalation gate is crucial, giving a 20% improvement in mechanical fault repair accuracy.

0.12

PQC-Corpus: A Dataset of Linked Development Histories for Post-Quantum Cryptography

cs.CR cs.SE ·2026-11-07 ·Souhardya Saha Dip, Moshood A. Fakorede, Umar Farooq
datasetcode/data similarity 0.66
Abstract · 5 pages, 4 figures, 2 tables

As post-quantum cryptography (PQC) standards evolve, developers must adapt implementations, tests, and protocols. Studying this work requires connecting code changes to problem reports and developer reviews. We present PQC-Corpus, a dataset of linked development histories from 179 public GitHub repositories, spanning cryptographic libraries and general-purpose projects. It connects issues and closing pull requests to commits and full patches, with comments and reviews. We combine Git histories with GitHub threads, reconstruct 23 pull-request commit lists truncated at 250 entries, and check file integrity, schemas, and links. Two annotators independently assessed sampled commits and issues ($κ=0.86$ and $0.93$), labeling about seven in ten commits and two in three issues PQC-related within these samples. We provide full and filtered views with removal ledgers, and quantify how filtering reduces links from issues to commits. A patch-free view supports inspection before downloading patches, while traversal scripts follow issues to code changes. We make the dataset and collector publicly available for studies of specification changes, hybrid interoperability, cryptographic review, and migration assistance.

0.19

Benchmarking the Quantum Leap: Auditing LLM Evaluation for Quantum Code Generation

cs.SE quant-ph ·2026-10-19 ·Ethan Seefried, Romit Ghosh, Sharmin Afrose, Nicholas Stapleton, Vicente Layton Ortega +2 more
benchmark similarity 0.65
Abstract · 27 pages, 9 Figures, 14 Tables

Large language models (LLMs) can generate quantum programs, but reliable evaluation remains difficult because benchmarks differ in task composition, scoring, and failure handling. We evaluate 15 hosted frontier, open-weight, quantum-specialized, and retrieval-augmented models on five quantum code-generation suites (530 tasks), plus 10 models on our new QuantumBenchEval (QBE) benchmark (96 tasks across six topics). We report bootstrap confidence intervals, multiplicity-adjusted paired comparisons, and audits of output validity and evaluator behavior. QiskitHumanEval-Standard approaches saturation (98.7% pass@1), while its Hard variant retains headroom (63.8%). None of the five top-two comparisons across primary suites remains significant after multiple-comparison correction. Evaluator design substantially changes scores: removing checks for unspecified quantities raises GPT-6 Astra's QBE pass@1 from 0% to 94.1% (T1) and from 1.2% to 92.9% (T6), without changing model outputs. We identify 17 QuanBench-117 tasks with shared failures linked to benchmark-owned imports. Controlled probes also reveal unexecuted unit tests, leakage of canonical functions into candidate execution, and reference imports that block otherwise valid candidates. These defects can produce both false failures and false passes, so historical QuanBench results require revalidation under an isolated evaluator. Overall, leaderboard scores alone do not adequately characterize progress in quantum code generation. We release our harness, per-sample results, and audit artifacts and recommend validating evaluators, isolating candidate code, and reporting uncertainty and output validity.

0.30