CS Paper Radar

GarageWorks: A Shared Testbench Abstraction for Software Simulation and FPGA Testing

Kazutomo Yoshii

cs.AR ·submitted 2026-10-11 21:14 UTC · v1

Abstract

GarageWorks is a lightweight FPGA testing framework that enables the same cocotb testbench to be used for both software RTL simulation and FPGA-accelerated hardware simulation. A shared abstraction hides backend-specific interfaces, while synthesizable Advanced eXtensible Interface (AXI) dispatchers provide design-under-test (DUT)-specific control and test operations on the FPGA. GarageWorks is intended for rapid bare-metal testing of standalone designs and on-chip processing logic for scientific sensors, rather than full SoC-level prototyping. The framework has been validated on AMD Alveo V80 FPGAs with several internal designs and provides initial support for AI-assisted generation of DUT-specific test components.

Closest papers

Nearest neighbours by meaning across the whole index.

vTen: Tensor-Centric Verification Framework for Domain-Specific Accelerators

cs.DC cs.AR ·2026-10-06 ·Chanmin Baek, Keehyuk Lee, Mincheol Cha, Somi Hong, Xuan Truong Nguyen +1 more
published similarity 0.62
Abstract · 7 pages, 8 figures, 1 table. Accepted at the 63rd ACM/IEEE Design Automation Conference (DAC '26), Long Beach, CA, USA

The semantic gap between tensor-centric software models and signal-level hardware testbenches creates significant productivity bottlenecks in verifying Domain-Specific Accelerators (DSAs). Existing frameworks like Cocotb suffer from prohibitive synchronization overheads due to fine-grained interactions. To address this, we propose vTen, a data-centric framework that strictly decouples verification intent from execution mechanics. By leveraging a declarative DSL and kernel-granular batching, vTen minimizes host-simulator interaction frequency. Evaluation on a production-scale 3D U-Net accelerator demonstrates that vTen achieves a 2x performance improvement in simulation latency and a 60.3% reduction in code complexity compared to Cocotb.

0.36

Naviq: An Open-Source System-on-Chip Simulator for Rapid Prototyping, Verification, and Design Space Exploration

cs.AR ·2026-11-01 ·Scott Smith, Lukas Zscherpel, Alyssa Huang, Marissa Lanz, Deming Chen
publishedcode/data similarity 0.61
Abstract · 9 pages, 10 figures. This work is accepted to ASPDAC'27

Modern FPGA SoCs are no longer just collections of accelerators on top of a programmable fabric. They combine programmable logic with hardened processors, NoCs, and components that dominate end-to-end performance. Evaluating individual pieces is insufficient; system performance depends on the interplay of SoC components. However, current FPGA SoC simulations provide limited support for this style of evaluation. Vendor simulations capture these interactions but are prohibitively slow for iterative design-space exploration. At the same time, generic software models overly abstract the hardware components that determine real behavior, resulting in less accurate simulations. Fast, easy-to-use simulators that accurately model real SoCs are critical for system debugging and design space exploration. We present Naviq, an open-source, cycle-level software simulator for FPGA SoCs, tuned to AMD Versal-class architectures. Naviq bridges accurate but slow vendor simulation and fast software modeling: it captures key behaviors of modern commercial FPGA SoCs while being fast enough for debugging and design-space exploration. Naviq models hardened SoC infrastructure in C++ and integrates accelerators through Verilator, allowing designers to evaluate custom hardware in a full-system environment. At the center of Naviq is an AXI4-compliant NoC model that captures transaction ordering, packetization, routing, backpressure, and memory interactions. Naviq provides CPU and memory integration, protocol checking, checkpointing, tracing, and performance-analysis tools for debugging and design-space exploration. Experimental results show Naviq closely reproduces the bandwidth and latency of Vivado's RTL NoC model while accelerating the evaluated simulations by 81x-182x. We present five case studies demonstrating Naviq's capabilities. Naviq enables practical full-system evaluation of modern FPGA SoC designs.

0.38

FPGAgent: An LLM-Assisted Framework for Autonomous HLS Code Generation and Verification in FPGA Environments

cs.SE ·2026-08-23 ·Tianyu Wang, Wenjie Wang, Jianguo Yao, Haibing Guan, Xijun Li
benchmark similarity 0.60
Abstract

Large language models (LLMs) have shown substantial promise for high-level synthesis (HLS) code generation, but most existing approaches validate only simulation or synthesis results. Because of timing and place-and-route constraints, \emph{HLS code that passes simulation and synthesis may still fail to produce deployable, runnable designs on real FPGA platforms}. Moreover, the lack of public benchmarks has limited many evaluations to small, self-curated test suites. We propose FPGAgent, a multi-agent framework tailored to real FPGA environments for autonomous HLS coding with end-to-end executability validation. To the best of our knowledge, FPGAgent is \emph{the first task-specification-to-executable HLS generation framework experimentally validated on a well-established benchmark}. Given a natural-language task specification, FPGAgent injects HLS-specific knowledge and employs evolutionary search to iteratively derive reliable HLS kernel implementations. It then generates a C++ validation program to verify functional correctness, diagnoses potential defects, and guides targeted repairs. Finally, it synthesizes host code for compilation and board-level execution on FPGA hardware. We comprehensively evaluate FPGAgent with five established LLMs on HLS-Eval, a benchmark containing 78 tasks across multiple domains, and verify board-level executability on a real FPGA platform. Compared with existing baselines, FPGAgent improves the synthesizable rate by 16.9% on average, executability by 26.7%, and functional correctness by 30.6%. These results show that FPGAgent substantially improves the practical usability of LLM-based HLS generation and demonstrates the value of end-to-end validation.

0.49

VHDL-REPOBENCH: A Repository-Level Benchmark for Evaluating Large Language Models on VHDL Design Generation

cs.PL cs.ARcs.CL ·2026-10-04 ·Prashanth Vijayaraghavan, Akul Malhotra, Ashutosh Jadhav, Ehsan Degan, Vandana Mukherjee
publishedbenchmarkcode/data similarity 0.59
Abstract · 7 pages

Large Language Models (LLMs) are increasingly applied in hardware design automation, demonstrating strong potential in generating and understanding hardware description languages. However, most existing benchmarks focus on Verilog, with limited evaluation of VHDL, which remains widely used in industry and academia for FPGA and safety-critical systems. To address this gap, we introduce VHDL-REPOBENCH, a large-scale, cross-file, repository-level benchmark for assessing LLM capabilities on realistic VHDL design generation and analysis tasks. VHDL-REPOBENCH curates ~100 open-source VHDL repositories, encompassing ~2.5k VHDL files and ~500 testbenches, and provides structured problem statements, module stubs, and self-verifying testbenches. The benchmark enables comprehensive evaluation across syntax, semantic correctness, hierarchical reasoning, cross-file dependency resolution, and functional verification. We evaluate several state-of-the-art models, including GPT-4o, Llama-3-70B, Qwen2.5-72B, CodeLlama-70B, and multi-step reasoning approaches such as Reflexion and CoDes. Results reveal that while current LLMs achieve moderate line- and block-level accuracy, substantial challenges remain in multi-file reasoning, hierarchical design understanding, and specification-to-module generation. VHDL-REPOBENCH represents the first large-scale VHDL-focused benchmark and provides a valuable resource for the hardware design community to evaluate, compare, and advance LLM capabilities for practical VHDL development.

0.38

A RISC-V Simulator for the Tenstorrent Tensix Architecture

cs.DC cs.AR ·2026-11-03 ·Nick Brown, Daniyal Arshad
publishedcode/data similarity 0.59
Abstract · Author accepted version of paper in the RISC-V for HPC SC26 workshop

RISC-V based AI accelerators have become popular and not only for AI and ML, but also as vehicles for exploring HPC workloads more generally. Tenstorrent are one example, and whilst they document their Tensix architecture in detail and develop their software stack openly, running code has historically required a physical card, and their recently released simulator targets hardware equivalence rather than introspection. In this paper we present Wolfpine, a community developed RISC-V simulator for the Tensix architecture, supporting both the Wormhole and Blackhole generations and integrating seamlessly with the TT-Metal software stack. Constructed entirely from publicly available documentation, open source software and measurements on physical hardware, Wolfpine is written in pure Python for readability and ease of modification. An opt-in, provenance-tracked cost model adds timing fidelity incrementally, and typed event streams, per-component tracing and timeline export enable programmers to identify and diagnose performance bottlenecks that the hardware itself hides.

0.38

SoK: ARCUS: On the Efficiency and Efficacy of Hardware Fuzzing

cs.CR cs.SE ·2026-08-25 ·Alenkruth Krishnan Murali, Raghul Saravanan, Sai Manoj P D, Ashish Venkat
survey similarity 0.58
Abstract

This work presents a comprehensive analysis of contemporary hardware fuzzing techniques applied across three major abstraction layers: Instruction Set Architecture (ISA), microarchitecture, and Register-Transfer Level (RTL). Our study examines key factors including input stimulus quality, mutation strategies, feedback mechanisms, target platforms, reference models, and achieved coverage. We find challenges, goals, and design trade-offs vary significantly across abstraction layers. We further identify several unmet needs in current hardware fuzzing practices, such as intelligent input generation, reliable and scalable golden reference models, expressive feedback channels, and cross-layer integration. Building on these insights, we outline future research directions, including hybrid fuzzing frameworks, AI-assisted test generation, scalable reference models, standardized evaluation metrics and benchmarks, and human-in-the-loop automation for guided exploration and analysis. Together, they aim to unlock efficient, reliable, and comprehensive hardware verification solutions.

0.32

EmbedGym: Letting Coding Agents Develop and Debug Embedded Firmware Without Physical Hardware

cs.SE cs.AI ·2026-11-16 ·Minghui Zhao, Yiming Li, Sophie Zhu, Maxwell Khor, Lauren Ouyang +3 more
similarity 0.57
Abstract

Modern coding agents perform well on general software benchmarks because they can run code and unit tests in a closed loop. Embedded firmware development, however, breaks this loop: a program that compiles can still fail on real hardware due to pin polarity, bus protocol, timing, or networking bugs that an agent at a text terminal cannot observe. Existing embedded coding agents either skip hardware execution, drive GPIO-only simulators with multi-second delays, or test on physical boards in a static environment. In this paper, we present EmbedGym, an interactive virtual hardware environment that lets coding agents develop and debug embedded firmware without physical hardware. EmbedGym uses instruction-level microcontroller unit (MCU) emulators to execute the same binary that would be flashed onto a physical board, connects them to 41 peripheral models, an analog circuit simulator, and virtual WiFi/Bluetooth Low Energy (BLE) endpoints, and exposes an interface for agents to change sensor inputs, press buttons, and inspect hardware states. On top of it, we build EmbedGym-Bench, which consists of 598 tasks across 6 MCU-framework pairs graded by hidden timed test vectors, using a five-stage multi-agent workflow that synthesizes peripheral models and tasks from datasheets and driver libraries. We expose EmbedGym to a coding agent at three levels and evaluate eight models over more than 14K trajectories. Even when compilation succeeds 99.7% of the time, 19.9% of compiled firmwares fail at runtime (up to 73% on individual targets). In contrast, interactive hardware testing in EmbedGym increases pass rates across models by +4.9% to +14.7% (up to +31% on individual targets). Our results also show that an agent with EmbedGym outperforms prior embedded agent frameworks, and lightweight models with EmbedGym match frontier models without it at 30-70x lower cost.

0.28

Testyard Benchmarks: a Dynamic RISC-V-based Benchmark Framework including DfT insertion, Structural and Functional Test Flows

cs.AR ·2026-11-10 ·Francesco Angione, Nicola di Gruttola Giardino, Gabriele Filipponi, Giusy Iaria, Irith Pomeranz +1 more
benchmarkcode/data similarity 0.57
Abstract

The main challenge of modern computing manufacturing is to produce silicon products capable of operating under the worst possible conditions, both in data centers and under stringent functional safety constraints. These requirements impose exceptionally high quality standards on silicon vendors, while testing costs keep escalating as transistors continue to shrink. Evaluating new testing strategies, however, presents a dual challenge for researchers. Standard academic benchmarks, such as the ITC-99 suite, remain valuable for fast test prototyping, but they lack the structural complexity of modern VLSI devices. Assessing the true computational effort of academic methodologies on industrial scale designs is equally difficult without accessible and scalable models. A further obstacle is the absence of a unified framework able to abstract the commands imposed by each EDA tool and the configurations tied to each technology library, forcing researchers to spend time adapting and debugging fragile scripts. This work addresses both needs by proposing the Testyard framework, built on top of the Chipyard framework, to derive a set of open-source, industrial scale benchmarks. It also extends the Hammer VLSI flow abstraction framework with hardware testing capabilities. Three new functionalities are introduced into Hammer: Design-for-Testability (DfT) insertion, ATPG, and functional tests fault simulation. Each functionality is driven by high-level descriptions, while the framework translates these inputs into the commands required by each EDA tool. The extended framework is applied to 13 representative SoC configurations from the Chipyard ecosystem, reporting the first comprehensive structural and functional test results across three technology libraries and multiple fault models, including stuck-at, transition delay, and small delay faults.

0.58