CS Paper Radar

Efficient Reconfigurable Architectures for Deep Learning

Endri Taka

cs.AR ·submitted 2026-10-31 00:18 UTC · v1
hot direction

Abstract

The growing computational demands of deep learning (DL) have driven AI-optimized reconfigurable devices, such as the AMD Versal AI Engines (AIEs), the Intel Stratix 10 NX Tensor Blocks (TBs), and the AMD Ryzen AI NPUs. Their heterogeneous resources and novel dataflow models make efficient exploitation nontrivial, and FPGAs lack native support for structured sparsity. This dissertation optimizes dense general matrix multiplication (GEMM), the dominant DL kernel, on these devices, proposes FPGA architectural enhancements for structured sparsity, and extends the techniques to AMD Ryzen AI NPUs. First, MaxEVA optimizes GEMM on Versal AIEs, achieving up to 2.19$\times$ higher throughput and 20.4% higher energy efficiency than the state of the art. It is then extended to arbitrary matrix sizes, and a systematic framework is proposed for the Stratix 10 NX TBs. Both frameworks reach state-of-the-art int8 throughput of up to 77 and 68 TOPS, and energy efficiency of up to 0.94 and 1.35 TOPS/W, on Versal and Stratix, respectively. Next, Systolic Sparse Tensor slices (SSTs) are proposed as in-fabric hard blocks supporting dense, 2:4, 1:4, and a new 1:3 sparsity level. SST-based GEMM accelerators achieve up to 5$\times$ higher frequency and 10.9$\times$ lower area than conventional FPGA logic, and on sparse ViT and CNN models SSTs deliver up to 3.52$\times$ speedup over dense in-fabric acceleration with up to 13.3% area increase. Finally, a common systematic methodology is proposed to optimize GEMM across the XDNA and XDNA2 AMD NPUs, reaching state-of-the-art performance of up to 6.76 and 38.05 TOPS (int8) and 3.14 and 14.71 TOPS (bf16), respectively.

Comment
Ph.D. dissertation, Department of Electrical and Computer Engineering, The University of Texas at Austin, 2026

Closest papers

Nearest neighbours by meaning across the whole index.

FAME: An FPGA-Based Platform for Approximate Multipliers Evaluation with Pattern-Guided DNN Retraining

cs.AR cs.LGcs.PF ·2026-09-15 ·Rappy Saha, Nima Amirafshar, Jude Haris, Nima Taherinejad, José Cano
publishedcode/data similarity 0.73
Abstract · Accepted at the 38th IEEE/SBC International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD) 2026

Approximate multipliers can reduce hardware area and energy consumption in Deep Neural Network (DNN) inference; however, they introduce computational errors. Assessing the accuracy of numerous approximate multiplier designs across diverse DNN models and large-scale datasets remains challenging due to prohibitive evaluation times. This overhead primarily stems from the slow emulation of approximate multiplier behavior using look-up tables (LUTs) on CPU and GPU platforms. Moreover, the resulting accuracy degradation must be carefully quantified and, if necessary, mitigated (e.g., through retraining), further increasing the overall evaluation cost. To address these challenges, we propose FAME, an FPGA-based platform for evaluating approximate multipliers. The platform exploits the reconfigurable logic of Field-Programmable Gate Arrays (FPGAs) to implement approximate multipliers directly in hardware, eliminating the need for LUT-based emulation on CPU/GPU platforms and thereby enabling efficient DNN inference while significantly reducing evaluation time on large datasets. Furthermore, we introduce a pattern-guided DNN retraining technique to mitigate accuracy degradation induced by approximate multipliers. Specifically, retraining is guided by multiplier-specific patterns to effectively recover potential accuracy losses. We evaluate FAME using two DNN models, ResNet-18 and MobileNetV2, on the ImageNet dataset across 27 approximate multipliers. During inference, our approach achieves up to a 3.47x speedup in approximate multiplier evaluation compared to prior LUT-based emulation methods. Furthermore, the proposed retraining technique improves accuracy by up to 65.5% over existing retraining approaches for the evaluated multipliers. The code is publicly available at: https://github.com/gicLAB/FAME

0.28

A Cross-Domain Matrix Engine for Signal Processing and Deep Neural Net Inference

cs.AR cs.AI ·2026-11-02 ·Mehdi Safarpour
similarity 0.72
Abstract

Matrix multiplication dominates both PHY signal processing and Deep Neural Network (DNN) inference, yet these domains place opposing demands on the matrix engine. Signal processing requires low-latency execution of relatively small matrix operations, whereas DNN inference, particularly at large batch sizes, prioritizes high throughput and energy efficiency for large matrix multiplications. While offering efficient memory access profile, a Systolic Array (SA) adds fill and drain delays per matrix. Those can be amortized for large matrices but not for small matrices. Conversely, a tensor unit completes the kernel in few cycles but would require repeated reads of same operand tiles over large matrices. This makes SA too slow for signal processing and tensor too inefficient for AI. We show that this need not be a choice between systolic and tensor structuring of Multiply-Accumulate (MAC) units. We propose a single engine that operates primarily as a SA and, through low-cost reconfiguration, transforms into a tensor-style engine on a per-operation basis. MUXes create both the spatial reduction adder tree and temporal reduction accumulators from the existing MACs and registers within the systolic array. The proposed reconfigurable SA simulated using modified SCALESim simulator, verified on an Zynq 702 FPGA and synthesized using ASAP 7nm FinFET library. SCALE-Sim simulations shows up to 7.5x lower latency for small complex value PHY kernels. Even on DNN workloads, it achieves up to 1.65x speedups over vanilla SA execution with trivial increase of memory accesses, retaining the systolic array's memory access advantage. Synthesis results estimate reconfigurable fabric costs only 8% power and 13.6% area overheads.

0.19

AscSpMM: Efficient SpMM on Ascend NPUs via Pipelined Vector-Cube Collaboration

cs.AR ·2026-11-09 ·Fei Li, Haoxin Li, Kehong Liu, Yanzi Zhou, Weiguo Wu +3 more
code/data similarity 0.68
Abstract · 13 pages, 11 figures

Neural processing units (NPUs) provide high compute density and energy efficiency through specialized matrix engines and software-managed on-chip memory. However, their libraries primarily target dense computation. Sparse matrix-dense matrix multiplication (SpMM), a key kernel in graph analytics, sparse neural networks, and scientific computing, requires index-driven operand selection, indirect addressing, scattered memory accesses, and load balancing. These requirements conflict with the fixed shapes and contiguous layouts preferred by NPU matrix engines. GPU Tensor Core SpMM methods cannot be directly transferred to NPUs with decoupled vector and matrix engines. We present AscSpMM, which separates irregular operand preparation from regular matrix computation through a parameterized Vector-CSR representation. The vector engine decodes indices and gathers and reorganizes dense operands, while the Cube engine performs matrix multiply-accumulate operations. A double-slot ring buffer enables dependency-aware asynchronous pipelining, while operand reuse and runtime-adaptive tiling reduce cross-engine stalls. On an Ascend 910B3, AscSpMM achieves geometric-mean speedups of 2.36x over GEMM, 19.92x over an existing SpMM implementation, and 5.82x over cuSPARSE on an A100. It reduces matrix-input storage and peak HBM footprint by 48.24% and 22.98% on average, while remaining competitive with specialized GPU Tensor Core SpMM kernels. The code is available at https://github.com/LfLab-AI/Ascend-NPU-SpMM.

0.34

Model Compression and Hardware-Aware Acceleration for Deep Learning on FPGAs: A Co-Design Taxonomy and Comparative Analysis

cs.AR ·2026-08-21 ·Peter Forcha, H. Kajekusumadhar, Mbua Peter, Muhammed Kawser, Audrey Cyriell Mo +1 more
survey similarity 0.68
Abstract

Deploying deep neural networks on Field-Programmable Gate Arrays (FPGAs) requires joint reasoning about model compression and hardware acceleration, however the most comprehensive existing cross-platform treatment of this space, Deng et al.~\cite{deng2020model}, compared compression techniques against CPU, GPU, FPGA, and ASIC targets at the level of broad, qualitative trade-offs, and not specific FPGA resource consequences. This survey instead restricted the scope to FPGAs alone and organized 25 compression-hardware co-design case studies (2015--2026) into a five-category taxonomy defined by which FPGA resources each strategy primarily reshapes: DSP-eliminating, DSP-repurposing/mixed-precision, sparsity-exploiting, memory-hierarchy-driven, and toolchain/deployment-level. Normalizing these case studies along a common set of dimensions (compression ratio, accuracy change, throughput, energy efficiency, and DSP/LUT/BRAM utilization) surfaces a central, quantitative finding; of the 25 reviewed works, only \emph{one} reported a compression ratio and accuracy change measured against a single common baseline, and only \emph{two} reported energy efficiency normalized against a common GPU baseline, exposing a field-wide characterization gap that no individual toolchain (FINN, HLS4ML, Vitis AI, or DNNWeaver) resolves on its own. Building on this taxonomy and meta-analysis, we formalize six open challenges: toolchain fragmentation, accuracy--efficiency characterization, automated mixed-precision optimization, sparse computation reliability, persistent memory bottlenecks, and FPGA-based training. Each is paired with a concrete next step grounded in extending an existing, cited technique, not a general call for future work.

0.21

Enabling Memory-efficient Im2win Convolution with Multi-precision Support on GPU CUDA and Tensor Cores

cs.DC cs.CV ·2026-08-21 ·Xiang Fu, Jixiang Ma, Xinpeng Zhang, Peng Zhao, Shuai Lu +1 more
published similarity 0.68
Abstract · Accepted at the 2026 International Joint Conference on Neural Networks (IJCNN 2026). To appear in IEEE Xplore

Convolution is a principal computational bottleneck in deep neural networks, and its efficiency depends on tight integration between algorithms and GPU hardware. Existing GPU convolution methods suffer from large memory overhead, poor cache utilization, limited effectiveness across kernel sizes, or numerical instability. This work extends the im2win paradigm -- a universal, memory-efficient convolution method with contiguous memory access for all kernel sizes -- to run efficiently in full precision on CUDA cores and half precision on tensor cores. By introducing new kernel designs and optimizations such as zig-zag memory access and asynchronous data movement, im2win efficiently exploits hardware-accelerated half-precision matrix multiply-accumulate operations. Across twelve CNN benchmarks, im2win achieves up to 2.8x higher TFLOPS than its CUDA core implementation, 1.4x higher than cuDNN, and 6.4x higher than GEMM-based convolution with cuBLAS, while using as little as 53% and 35% of their memory, respectively. These results establish im2win as a unified, high-performance convolution framework for modern GPU architectures.

0.50

Adaptive Matrix Multiplication for Dynamic Shapes on Ascend NPUs

cs.AR ·2026-08-11 ·Yuhang Zhou, Jiang Peng, Qianyu Jiang, Zhibin Wang, Xinghui Tian +5 more
similarity 0.67
Abstract

Matrix Multiplication (MatMul) faces a "generalization crisis" driven by highly dynamic tensor shapes. This crisis is particularly acute on Ascend NPUs, where explicitly controlled architectures and strict physical constraints render existing GPU-centric optimizations ineffective. To resolve this, we propose AdaptCore, an adaptive framework for universally high-performance MatMul on Ascend NPUs. AdaptCore systematically decouples operator optimization into spatial tiling and instruction orchestration. It first maps dynamic shapes into a hardware-aware 2D tiling taxonomy to balance on-chip capacity limits and multi-core parallelism. Furthermore, it integrates a composable optimization library with a deterministic analytical performance model. By mathematically evaluating hardware state mutations, AdaptCore proactively selects and caches optimal implementations, enabling O(1) overhead runtime dispatching. Evaluations demonstrate that AdaptCore delivers a remarkable 1.85x mean speedup across 80,000 input shapes, and achieves up to a 1.48x acceleration in representative end-to-end models over the highly-tuned native vendor library (ACLNN).

0.04

Precision-Aware Variable Bit Processing Elements for Hardware-Efficient Systolic Array Designs

cs.AR cs.ETcs.LGeess.IV ·2026-08-23 ·Dantu Nandini Devi, Madhav Rao
published similarity 0.67
Abstract

Systolic arrays (SAs) have emerged as prominent hardware accelerators for matrix operations in deep learning, while floating point number formats enable precision control across computational domains. This research investigates approximate computing techniques for floating point (FP) multipliers in Weight Stationary Systolic Arrays, focusing on IEEE 754 (FP32), TensorFloat-32 (TF32), and Brain Floating point (BF16) formats. By integrating partial product matrix (PPM) column truncation with positive and negative compressors in the FP multiplier architecture, we optimize the trade-off between computational efficiency and accuracy. NSGA-II optimization algorithm was employed to explore the vast design space for evolving FP multiplier designs, towards achieving substantial hardware improvements while maintaining acceptable output quality. Substantial hardware benefits were observed in the FP multiplier designs across various applications, while preserving output quality. The FP approximated Processing Elements designed in the SA was found to offer comparable CNN accuracy for models trained on MNIST, F-MNIST, and CIFAR-10 dataset. The FP approximated SA designs that fall in the top 10 CNN performance offered substantial hardware gains in the range of 66% to 92% footprint savings, 60% to 93% of power benefits with 21% to 54% improvement in the delay when compared with the corresponding exact implementations mentioned in the literature for running the model trained on CIFAR-10 dataset. The TF32 and BF16 approximated SA designs also achieved substantial gains while maintaining comparable CNN accuracy. Our findings confirm that targeted approximation in FP multiplier design significantly improves the efficiency of hardware accelerators for error-tolerant applications, establishing an effective approach to hardware resource optimization in contemporary computing architectures.

0.27

Bit-Sparsity Aware Acceleration with Compact CSD Code on Generic Matrix Multiplication

cs.AR ·2026-11-11 ·Zixuan Zhu, Xiaolong Zhou, Chundong Wang, Li Tian, Zunkai Huang +1 more
top venuehighly cited similarity 0.67
Abstract · Accepted for publication in IEEE Transactions on Computers (TC)

The ever-increasing demand for matrix multiplication in artificial intelligence (AI) and generic computing emphasizes the necessity of efficient computing power accommodating both floating-point (FP) and quantized integer (QINT). While state-of-the-art bit-sparsity-aware acceleration techniques have demonstrated impressive performance and efficiency in neural networks through software-driven methods such as pruning and quantization, these approaches are not always feasible in typical generic computing scenarios. In this paper, we propose Bit-Cigma, a hardware-centric architecture that leverages bit-sparsity to accelerate generic matrix multiplication. Bit-Cigma features (1) CCSD encoding, an optimized on-chip sparsification technique based on canonical signed digit (CSD) representation; (2) segmented dot product, a multi-stage exponent matching technique for long FP vectors; and (3) the versatility to efficiently process both FP and QINT data types. CCSD encoding halves the cost of CSD encoding while achieving optimal bit-sparsity, and segmented dot product improves both accuracy and throughput. Bit-Cigma cores are implemented using 65 nm technology at 1 GHz, demonstrating substantial gains in performance and efficiency for both FP and QINT configurations. Compared to state-of-the-art Bitlet, Bit-Cigma achieves 3.2\texttimes{} performance, 6.1\texttimes{} area efficiency, and 15.3\texttimes{} energy efficiency when processing FP32 data while ensuring zero computing error.

0.68