CS Paper Radar

5uGUE: Agentic Hardware-in-the-Loop Fuzzing for 5G Devices and beyond

Guoqiang Hua, Sakshi Udeshi, Matheus Eduardo Garbelini, Daisuke Mashima, Sudipta Chattopadhyay

cs.NI ·submitted 2026-10-21 03:45 UTC · v2

Abstract

In this paper, we present 5uGUE, a large language model (LLM)-driven hardware-in-the-loop agent, which enables the wireless fuzzing of 5G-enabled devices (such as mobile phones). 5uGUE combines multi-step reasoning with tool-assisted interaction and integrates into the fuzzing pipeline by leveraging specialized tools for internet search, 5G protocol analysis, memory management, and test-case execution. The agent iteratively generates test cases, executes them on targets, collects runtime feedback, and adaptively refines the fuzzing strategy. We evaluate 5uGUE on five commercial off-the-shelf (COTS) 5G devices across six LLMs. The best performing runs for 5uGUE trigger 167 crashes in total (141 novel and 31 unique) within 60 interaction turns (49 to 133 minutes total runtime). In contrast, the best runs for the state-of-the-art baseline (5GHOUL) discovers only 13 crashes in total (10 novel and 6 unique) within 150 minutes. Overall, by integrating LLM-driven agents, 5uGUE demonstrates significantly better crash discovery performance (12.8x total, 5.2x unique improvement). Notably, our experiments identified 46 unique crashes, 44 of which were confirmed by the vendors but had been previously reported. More importantly, MediaTek confirmed two previously unknown, high-severity vulnerabilities, one of which was assigned CVE-2026-20525. Finally, to demonstrate that 5uGUE is agnostic to wireless protocols, we also extend it to Bluetooth Classic with only minimal code changes. In our evaluations, 5uGUE finds an average of 6.4 novel crashes and 1.6 unique crashes. In a nutshell, 5uGUE highlights that LLM agents are capable to operate effectively beyond software-in-the-loop settings, extending autonomous vulnerability discovery to the proprietary, hardware-bound systems that affect billions of end-user devices across multiple wireless protocols.

Closest papers

Nearest neighbours by meaning across the whole index.

SoK: ARCUS: On the Efficiency and Efficacy of Hardware Fuzzing

cs.CR cs.SE ·2026-08-25 ·Alenkruth Krishnan Murali, Raghul Saravanan, Sai Manoj P D, Ashish Venkat
survey similarity 0.65
Abstract

This work presents a comprehensive analysis of contemporary hardware fuzzing techniques applied across three major abstraction layers: Instruction Set Architecture (ISA), microarchitecture, and Register-Transfer Level (RTL). Our study examines key factors including input stimulus quality, mutation strategies, feedback mechanisms, target platforms, reference models, and achieved coverage. We find challenges, goals, and design trade-offs vary significantly across abstraction layers. We further identify several unmet needs in current hardware fuzzing practices, such as intelligent input generation, reliable and scalable golden reference models, expressive feedback channels, and cross-layer integration. Building on these insights, we outline future research directions, including hybrid fuzzing frameworks, AI-assisted test generation, scalable reference models, standardized evaluation metrics and benchmarks, and human-in-the-loop automation for guided exploration and analysis. Together, they aim to unlock efficient, reliable, and comprehensive hardware verification solutions.

0.33

SoK: You Find What You Seek: Rethinking Oracles, Guidance, and Input Generation in Hardware Fuzzing

cs.CR cs.AR ·2026-09-23 ·G Abarajithan, Zhenghua Ma, Cristian Tirelli, Andres Meza, Francesco Restuccia +2 more
survey similarity 0.62
Abstract · 13 pages

Hardware fuzzing is an active area in security verification research, yet its industrial adoption remains in its early stages. This SoK examines which lessons from software fuzzing carry over to hardware and where unique approaches are needed. By analyzing 52 fuzzers across RTL/IP, CPU, NoC, and SoC designs, we introduce an analytical framework that frames verification as a bounded search. This search is defined by its objective, oracle, guidance, input generation, target abstraction, and budget. Consequently, a campaign only uncovers failures it can effectively reach, recognize, and prioritize before exhausting its resources. We distinguish two roles for hardware fuzzing: (1) augmenting constrained-random verification (CRV) via feedback-guided coverage and (2) directed adversarial testing based on threat models and security specifications. Through our framework, we identify what each campaign can observe and generate, providing a basis for assessing the evidence behind reported results. Our analysis suggests that mainstream adoption of hardware fuzzing will require reusable interfaces, target-specific verification assets, reproducible evaluations, and transparent reporting of cost and user effort.

0.09

Closed-loop evaluation of LLM agents for embedded software development

cs.SE cs.AIcs.ARcs.LG ·2026-10-08 ·Jorge García-Carrasco, Sergio García-Carrasco, Alejandro Maté, Juan Trujillo
publishedbenchmarkcode/data similarity 0.62
Abstract · Published in Journal of Systems Architecture 179 (2026) 103937. 23 pages, 4 figures, 5 tables. Code and artifacts: https://github.com/jgcarr…

Large language models (LLMs) are increasingly deployed as coding agents that edit files, run builds and tests, inspect execution results, and repair software iteratively. Embedded firmware is a demanding target because correctness depends on closed-loop behavior under sensing, timing, and safety constraints, not only on static source quality. Yet embedded-agent evaluation remains limited and often emphasizes one-shot synthesis or offline correctness. We present a benchmark for closed-loop evaluation of embedded coding agents. Each task provides a plain-text engineering description, constrained workspace, and visible build-and-runtime surface. The agent must translate requirements into implementation and self-verification steps, then iterate until the required device behavior is achieved. The suite contains five embedded-control tasks and four feedback scenarios: one-shot generation, realistic self-verification, CI-style red/green feedback, and oracle-style detailed feedback. The implementation targets simulated ESP32 firmware for reproducibility. We evaluate seven GPT-family and Qwen-family configurations across five tasks and four scenarios, with three repetitions per condition for 420 runs. gpt-5.4 has the highest pass rate among evaluated configurations but does not saturate the benchmark; qwen3.5-27B is the strongest observed local model; and smaller local models degrade sharply in pass rate and search efficiency. These results suggest that capable local embedded coding agents are emerging.

0.69

LoopHIL: Looping Hardware into LLM-assisted Agents for Wireless Embedded Systems

cs.NI cs.AIcs.ARcs.SE ·2026-10-16 ·Dhairya Shah, Spanddhana Sara, Savitha Viswanadh Kandala, Ambuj Varshney
similarity 0.61
Abstract · 8 pages, 8 figures, 4 tables. Under review

LLM agents are widely used for software development and debugging, but their adoption in embedded systems, especially for wireless tasks, remains limited. This is because embedded development requires hardware-in-the-loop testing. Agents cannot directly observe the spectrum, probe pins, or measure power, so they cannot verify wireless link functionality; currently, human engineers perform this verification. We systematically evaluate whether LLM agents can establish wireless links when provided with hardware access. We introduce LoopHIL, a framework that lets agents control a multi-device RF bench, monitor it with a spectrum analyzer, logic analyzer, and energy meter, and validate links against a verification ladder, all within code-enforced safety limits. Across four increasingly complex tasks, from generating a carrier to backscatter decoding by a commercial receiver, and using six models from two vendors, hardware-in-the-loop increases task completion rates from 54% to 79% and from 17% to 83% on the most challenging task. It also halves task time and output tokens. With debugging tools, agents decrease tag power consumption by up to 96%. However, small models often report unverified claims, so independent hardware checks remain necessary.

0.56

EngineAgent: Agentizing On-Device LLM Inference Engines with Inference Stack Hierarchy Awareness and Hardware Abilities

cs.OS cs.AIcs.CLcs.SE ·2026-11-10 ·Zhengxiang Huang, Yujie Sun, Shengheng Chen, Zhaode Wang, Chaoyue Niu +4 more
similarity 0.61
Abstract

Rapidly evolving LLM architectures and heterogeneous mobile hardware make supporting new models in on-device inference engines a recurring and costly system challenge. Each model may require coordinated changes across conversion, graph construction, operators, and hardware backends. Existing coding agents often miss these cross-layer dependencies and struggle to diagnose failures across host--device execution paths. We present EngineAgent, an agentic system for automating on-device LLM model enablement. Its key insight is to separate reusable engine and hardware knowledge from model-specific implementation. Before enablement, EngineAgent constructs a hierarchy-aware engine index that captures engine abstraction layers and cross-layer dependencies, and pre-validates backend-specific build, execution, and debugging workflows using already supported models. For each new model, a builder uses this knowledge to coordinate cross-layer changes, while an auditor checks implementation completeness and drives iterative debugging through device-side validation. Across 12 tasks spanning three mobile LLMs and four engine--backend pairs on CPUs, GPUs, and NPUs, EngineAgent achieves 75% end-to-end success with either of two underlying LLMs, versus 33.3% for the strongest coding-agent baseline. Success requires compilation, on-device execution, and output-logit agreement with reference implementations. EngineAgent also reduces average per-task time and model-API cost relative to baselines using the same underlying LLMs.

0.21

GI-RTL-Fuzz: Grammar-Informed RTL Generation with LLMs for End-to-End Differential Simulator Fuzzing

cs.AR cs.CR ·2026-10-28 ·Ziyue Zheng, Zhi Qu, Yangdi Lyu
similarity 0.60
Abstract

Register-Transfer Level (RTL) simulators are fundamental to modern IC design workflows, yet like any complex software, they frequently contain critical bugs that compromise simulation accuracy. To systematically stress-test these tools, we introduce GI-RTL-Fuzz, a grammar-informed fuzzing framework that leverages Large Language Models (LLMs) to generate high-quality RTL test cases. Our key innovation is guiding the LLM's generation process using Backus-Naur Form (BNF) grammar coverage information, enabling the exploration of complex and rarely-tested language features. By performing differential testing against different RTL simulators, our framework uncovered a significant number of previously unknown inconsistencies in both the linting process and simulation results of major Verilog simulators.

0.13

How Do Hardware Faults Manifest Differently in LLM Agents?

cs.MA ·2026-10-08 ·Aiden Rubey, Zachary Coalson, Cong Chen, Jean-Philippe Monteuuis, Jonathan Petit +1 more
top venue similarity 0.60
Abstract · Accepted to NeurIPS 2026 MLforSystems and AI4Good Workshops; The first two authors contributed equally

We present the first systematic study of the resilience of large language model (LLM) agents to random hardware-induced faults. Prior work on hardware fault resilience in LLMs focuses on model-level outcomes, such as incorrect outputs or generation failures. Yet LLM agents repeatedly interact with tools and environments, creating loops that can change how hardware faults manifest. We conduct an extensive evaluation across three agent benchmarks, comprising 6,814 agent executions: 2,400+ clean runs and 4,300+ fault-injected runs with independently sampled random two-bit weight faults. We trace how these faults propagate through agent execution. Unlike prior studies, we find that agentic execution transforms model-level corruption into distinct system-level outcomes: (1) agents can recover from faulty model outputs and still complete tasks correctly; (2) this recovery increases computational and execution costs; (3) a few faults cause subtle but semantically meaningful errors while the agent otherwise behaves normally; and (4) for some models and scenarios, faults can alter security and alignment behavior.

0.36

POZZER: A Power Side Channel-guided Fuzzer for Black-Box Embedded Systems

cs.CR ·2026-09-20 ·Pouya Narimani, Kseniia Rogova, Addison Crump, Martin Mohl, Meng Wang +3 more
similarity 0.59
Abstract

Firmware fuzzing is an effective technique for discovering vulnerabilities in embedded systems. However, existing coverage-guided firmware fuzzers typically obtain feedback through firmware instrumentation, hardware debug interfaces, or firmware rehosting, which requires access to the firmware source code or binary image. Such requirements are often infeasible for off-the-shelf embedded devices, where firmware binaries are inaccessible, unrehostable, immodifiable, or undebuggable, necessitating fuzzing under black-box conditions. In this paper, we present POZZER, a power side-channel-guided fuzzer for black-box embedded systems. POZZER uses power traces as feedback to identify previously unseen behavior via an incrementally constructed graph-based representation of observed executions, guiding the fuzzer toward unexplored execution paths. Its non-profiling design requires neither prior firmware knowledge nor a clone device, extracting meaningful feedback from a single power trace per execution while remaining robust to measurement noise. We evaluate POZZER on 15 firmware targets across two platforms and two real-world commercial embedded devices. Across the resulting target-platform combinations, POZZER outperforms a blind fuzzer under the same time budget in 26 out of 30 target-platform combinations. Furthermore, POZZER discovers two previously unknown vulnerabilities in one of the commercial devices, both confirmed by the vendor, demonstrating its potential for identifying vulnerabilities in black-box embedded systems.

0.18