CS Paper Radar

Heuristic-Driven Resource Optimization for Decoupled Memory Channels on FPGAs

Arnab A. Purkayastha, Shobhit Aggarwal, Jay Tharwani

cs.AR cs.DC ·submitted 2026-10-18 20:52 UTC · v1
published

Abstract

Determining OpenCL channel depths for Decoupled Access-Execute (DAE) architectures in High-Level Synthesis (HLS) is critical for balancing performance and hardware efficiency. Conventional static analysis tools often over-provision these memory channels to ensure performance safety, leading to significant resource stranding. This paper presents an expansion of an LLVM-based static analysis framework, originally proposed for automated DAE generation, by introducing a hardware-aware refinement layer. We identify a counter-intuitive Performance Inversion Paradox linked to baseline memory stalls, where high-stall, grid-based kernels reach peak performance at minimal channel depths and suffer up to a 61% throughput degradation when depths are increased. We propose a Heuristic Refinement Layer that optimizes OpenCL channel depths based on this correlation between baseline stalls and performance saturation. Experimental results demonstrate that our expanded framework successfully identifies the Minimal Peak Performance point across various benchmarks. By avoiding the performance traps of standard over-provisioning, we achieve a 2.52x improvement in Resource Efficiency for an illustrative HotSpot application. This approach maintains peak execution speedup while ensuring the leanest possible hardware profile, outperforming conservative tool-based static predictions.

Comment
6 pages. Accepted version of a paper published in the 2026 IEEE International Conference on Interdisciplinary Approaches in Technology and Management for Social Innovation (IATMSI)
Journal ref
2026 IEEE International Conference on Interdisciplinary Approaches in Technology and Management for Social Innovation (IATMSI), Gwalior, India, 2026, pp. 1-6
DOI
10.1109/IATMSI68868.2026.11465435

Closest papers

Nearest neighbours by meaning across the whole index.

VIPER: Architecture-Aware Performance Modeling for Processing-in-Memory Design-Space Exploration

cs.AR ·2026-08-24 ·Haoran Geng, Tomas Sousa Pereira, Xiaoyang Lu, Xian-He Sun, Michael Niemier +1 more
code/data similarity 0.68
Abstract · 14 pages, 12 figures. Source code available at https://github.com/Notre-Dame-HW-SW-Codesign-Lab/VIPER

Processing-in-Memory (PIM) promises to reduce data movement overhead by executing computation in or near memory, but its realized application speedup remains highly design-dependent. Non-offloadable host execution, host-PIM transfers, limited PIM capacity, and device programming latency can limit end-to-end speedup, making fast early-stage design-space exploration (DSE) essential. However, existing PIM evaluation methods remain limited: circuit- and device-level tools cannot capture these end-to-end PIM performance factors, while cycle-accurate simulation is too slow for iterative DSE. To address this gap, we present VIPER, a unified, lightweight, and architecture-aware performance evaluation framework for PIM DSE. VIPER profiles host execution once and combines the measured host behavior with a PIM-aware analytical engine that sweeps PIM-side parameters across candidate designs. It supports both Processing Near Memory (PNM) and Processing Using Memory (PUM) under task-offloading and data-triggered execution by capturing host-PIM transfer, array access, in-memory computation, device programming latency, and capacity-induced partitioning, providing rapid architecture-aware performance estimates for iterative DSE without repeated cycle-accurate simulation. We validate VIPER against a commercial UPMEM system and more than 400 cycle-accurate gem5 configurations. VIPER predicts the UPMEM offloading decision and break-even region a priori, and, with a refined transfer model, captures the measured peak-and-rolloff behavior with 12\% mean speedup error across the DPU sweep (6\% up to the 256-DPU peak). Against gem5, VIPER achieves less than 10\% error while reducing evaluation time from hours to under one minute. Case studies of UPMEM, ReRAM/FeFET crossbars, and IMCRYPTO show that architecture-aware DSE reveals key performance trade-offs that device-level evaluation misses.

0.24

Randomized Caches in Hardware: Challenges and Trade-offs

cs.AR cs.CR ·2026-10-30 ·Xinrui Wang, Pascal Kosak, Christof Paar, Daniel Genkin, Yuval Yarom +1 more
similarity 0.68
Abstract

Randomized caches protect against side-channel attacks by obfuscating the mapping of memory addresses to cache locations. Existing proposals typically describe architectural behavior, leaving practical hardware implementation challenges largely unexplored. We implement four prominent designs, CEASER-S, ScatterCache, SassCache, and MIRAGE, and integrate them as the last-level cache (LLC) of a multi-core out-of-order RISC-V system-on-chip (SoC). We identify common and design-specific implementation challenges and develop practical solutions. We evaluate complete FPGA implementations by running SPEC CPU 2017 rate and GAPBS workloads to completion, providing, to our knowledge, the first comparative evaluation of these designs on a common full-system hardware platform. Pipelining the index derivation function (IDF) alone is insufficient: in three of the four variants, other components become the frequency bottleneck, reducing frequency by up to 35% relative to the baseline set-associative cache. Redesigning the critical paths while preserving mappings, rekeying, and victim-selection policies brings the three affected variants to no more than 10% below the baseline frequency, at the cost of added latency or reduced concurrency. Increased LLC access latency, rather than additional misses, dominates IPC degradation, reducing instructions per cycle by up to 5% for SPEC and 11% for GAPBS. Hardware overhead depends on the design and implementation technology: FPGA lookup-table usage increases by 10-35%, while ASIC area increases by 1-14%.

0.22

FlashOffloading: Low-Latency KV Cache Offloading for Long-Context LLM Serving

cs.AI ·2026-11-05 ·Xuan Wang, Zhuanhao Wu, Tingzhu Fang, Xushi Li, Sicheng Zhang +15 more
code/data similarity 0.67
Abstract · 13 pages including references,13 figures

In large language model (LLM) serving systems, the key-value (KV) cache grows linearly with sequence length and concurrency, so long-context inference easily hits the high-bandwidth memory (HBM) capacity wall and results in low concurrency. Offloading the KV cache to host dynamic random-access memory (DRAM) frees HBM space and raises concurrency, but the additional cache management and host-to-device (H2D) load overhead inflates inference latency, higher concurrency fails to translate into effective throughput. We present FlashOffloading, a host-DRAM-centric inference architecture that fully offloads the KV cache in both the prefill and decode phases. FlashOffloading builds rolling HBM cache for prefill, a hot cache for decode, a reverse sparse index with zero extra miss-lookup overhead, and a compute--load overlapped kernel, optimizing KV offloading latency across the entire pipeline and delivering throughput gains under a time-per-output-token (TPOT) service-level objective (SLO). In experiments deploying GLM-5.2 with multi-token prediction (MTP3) speculative decoding on an Ascend910C prefill--decode (PD)-disaggregated platform, FlashOffloading improves prefill throughput by up to 1.38\(\times\) and decode throughput by up to 1.91\(\times\) under a 30 ms TPOT constraint. FlashOffloading is integrated into an open-source serving framework: https://github.com/vllm-project/vllm-ascend/tree/releases/v0.30.0.

0.31

Dissecting How Die Scaling Breaks GPU Fine-grained Scheduling

cs.AR cs.DC ·2026-09-21 ·Xiaoze Fan, Jianhao Wang, Weihao Cui, Han Zhao, Zhuobin Huang +6 more
similarity 0.67
Abstract

Modern GPUs are no longer physically symmetric. Die scaling leads to both manufacturing-driven floorsweeping and cache and memory partitioning. The former creates chip-specific compute topologies, while the latter causes non-uniform memory access. These asymmetries are substantial. Topology-oblivious compute unit allocation can lead to up to 1.33x performance variation, while remote accesses increase HBM latency by up to 67% and nearly double L2 latency. However, these asymmetries are hidden behind the GPU's logical resource abstractions and can vary across chips. We develop lightweight characterization methods to uncover per-chip compute topology and memory affinity. We then use the discovered information to make existing fine-grained scheduling asymmetry-aware, considering not only how many resources are allocated but also which physical resources are assigned. Across full-GPU kernel execution, intra-application multiplexing, and inter-application co-location, asymmetry-aware scheduling improves mainstream kernels by up to 1.22x, multiplexed LLM inference by up to 14.3%, and avoids up to 1.33x performance variation.

0.38

Toki: Profiling HBM Performance on FPGA Systems with RISC-V Soft Cores and PCIe Host DMA Traffic

cs.AR ·2026-09-22 ·Andrea Galimberti, Andrea Motta, Gianni Antichi, Davide Zoni
publishedcode/data similarity 0.67
Abstract · Presented at the 63rd ACM/IEEE Design Automation Conference (DAC '26), July 26-29, 2026, Long Beach, CA, USA; 7 pages, 7 figures, 2 tables

Programmable RISC-V soft cores are becoming more widespread in data-center scenarios, making it crucial to design efficient systems that deploy them on FPGA chips with HBM memory. Toki, released as open source, is the first hardware-software framework that enables profiling the performance of HBM on FPGA accelerator cards by jointly considering (i) the execution of workloads on RISC-V soft cores instantiated on the FPGA and (ii) the injection of memory traffic from the host system via DMA over PCIe, providing insights that cannot be obtained with synthetic traffic generators alone. Extensive experiments target an AMD Alveo U55C card, deploying up to 60 RISC-V compute cores and stressing its HBM2 memory through real-world applications and microbenchmarks with user-defined access patterns. Results showcase how Toki can effectively profile the impact on HBM performance of the compute cores' organization, the workload's memory access patterns, data locality, contention over the memory controllers, and host traffic.

0.36

Chimera: A Configurable LLM Decode Accelerator Architecture with Compute-Memory-Floorplan Co-Design for HBM-Equipped Multi-Die FPGAs

cs.AR ·2026-11-14 ·Zhiheng Chen, Yifan Zhang, Yu-An Chou, Ye Qiao, Sitao Huang
benchmark similarity 0.66
Abstract

GPUs are widely used for LLM inference, but single-request decoding is memory-bound, leaving their compute underutilized while they draw substantial power. HBM-equipped FPGAs are well suited to LLM decoding: their customizable datapaths use memory bandwidth more energy-efficiently than GPUs. However, high memory bandwidth alone does not ensure high decode performance: exploiting it requires parallel compute and memory interfaces, whose resource costs and placement on the chip can limit the throughput. We present Chimera, a configurable LLM decode accelerator architecture that co-designs compute, memory access, and floorplan. Specifically, HBM Proxy Ports extend HBM access to compute lanes on every super logic region (SLR) die, and tiered expert storage holds mixture-of-experts weights beyond the HBM capacity. In addition, a unified configuration analysis estimates per-token latency and per-SLR resource use and guides the design of each configuration within resource budgets. On an AMD Alveo U280, we build ten designs for eight dense, mixture-of-experts, hybrid linear/standard-attention, and ternary LLMs, up to an 80B-parameter mixture-of-experts LLM on four memory tiers. Chimera's designs achieve a geometric mean of 1.6$\times$ the token throughput of prior state-of-the-art FPGA implementations and 3.0$\times$ and 1.5$\times$ the energy efficiency of the RTX 3090 and A100 GPUs, setting a new benchmark for LLM decoding on HBM-equipped FPGAs.

0.23

HPC-Bench: A Comprehensive Benchmark for High Performance Computing Codes

cs.PF ·2026-11-02 ·Bowen Cui, Junyu Yin, Tejas Ramesh, Leo Lim, Oscar Hernandez +1 more
publishedbenchmarkcode/data similarity 0.66
Abstract · Accepted at NeurIPS 2026 Evaluations & Datasets Track (poster). 31 pages, 13 figures, 19 tables

Large Language Models (LLMs) have shown strong capabilities in code generation and reasoning, motivating extensive benchmarks for evaluating functional correctness. However, performance optimization in High-Performance Computing (HPC) poses fundamentally different challenges that are not captured by existing evaluations: HPC performance depends on low-level code, intricate memory access patterns, and computational kernels embedded within sophisticated execution contexts. We introduce HPC-Bench, a performance-grounded benchmark for evaluating LLMs as optimizers of computational kernels in realistic HPC workloads, including 114 workloads across 16 HPC computational motifs. HPC-Bench provides a fully automated, end-to-end evaluation pipeline that combines program-level correctness verification with two correctness-gated metrics, Fast@k and Speedup@k, to characterize reliability and speedup under a fixed sampling budget. Our evaluation of three representative LLMs across serial CPU, OpenMP, and CUDA settings shows that stricter speedup requirements substantially reduce the probability of finding a correct and fast optimization. HPC-Bench is available at https://github.com/Deep-Learning-Profiling-Tools/HPC-Bench.

0.48

Reconstructing Fine-grained Memory Reuse Structure in GPU Kernels

cs.DC cs.PFcs.PL ·2026-11-10 ·Yanbo Zhao, Yueming Hao, Zecheng Li, Mao Lin, Shuyin Jiao +3 more
similarity 0.66
Abstract

Optimizing GPU data movement requires knowing which instructions access the same data and across which execution scopes reuse occurs. Existing profilers can attribute memory behavior to individual instructions but do not expose the reuse relations among them, while aliasing, dynamic indexing, and runtime parameters prevent static analysis from establishing these relations in general. Cycle-level simulation can reconstruct such relations, but at costs prohibitive for production workloads. We introduce two abstractions for representing this structure: Topological Distance, which labels each dynamic memory access with the smallest execution scope (thread, warp, block, or grid) containing its most recent reuse, and PCDepGraph, which organizes these labels into a PC-level graph of the kernel's memory reuse structure. Defined over logical execution scopes rather than transient hardware state, the representation remains stable across runs and GPU targets while directly indicating the sharing scope that candidate optimization mechanisms must cover. Our tool, cuVein, reconstructs PCDepGraph directly from optimized binaries, without sampling, at generally under 10x overhead. Across HPC and AI workloads on NVIDIA RTX 4090 and H100, optimizations guided by PCDepGraph eliminate up to 90.18% of memory instructions and yield up to 2.84x kernel speedup; reuse-scope distributions remain consistent across platforms, supporting that the representation captures program structure rather than transient hardware behavior.

0.22