CS Paper Radar

SparseForge: Beyond Spike Sparsity toward Hardware-Exploitable Neuron Pruning for Efficient SNN Acceleration

Kainan Wang, Sanhe Fu, Aili Wang

cs.AR ·submitted 2026-11-01 05:54 UTC · v1
publishedbenchmark

Abstract

Spiking Neural Networks (SNNs) provide an event-driven paradigm for energy-efficient edge computing. However, operation-level sparsity in SNNs does not necessarily eliminate the hardware costs associated with redundant output neurons, including neuron-state maintenance, partial-sum (psum) accesses, and on-chip storage requirements. This paper presents SparseForge, a hardware-aware sparse SNN acceleration framework that jointly optimizes neuron pruning, sparse dataflow, and accelerator microarchitecture. At the algorithm level, we introduce a gated leaky integrate-and-fire (G-LIF) neuron to learn differentiable neuron masks with binary spike semantics. A dual regularization scheme is further developed to align pruning with both synaptic-operation reduction and psum buffer capacity reduction. At the dataflow level, we design a sparse Column-Wise Product (CWP) dataflow with a variant of incremental spike encoding to improve weight-access regularity and reduce sparse-index overhead under dynamic spike activity. At the architecture level, we develop a mask-aware sparse processing unit (SPU) that supports compressed spike decoding, sparse implicit im2col conversion, neuron-mask-gated psum updates, and decoupled sparse execution. Experimental results across multiple SNN backbones and datasets show that SparseForge retains 1.75%--77.99% of neurons and 0.16%--62.36% of synaptic operations, with a configurable accuracy--efficiency trade-off. The buffer-capacity-aware regularization reduces the required psum buffer capacity by up to 96.18\%. FPGA implementations on a Xilinx ZCU9EG achieve 296.94--619.49 GOPS/W across the evaluated workloads. Across matched network and dataset settings, SparseForge provides 1.24--4.17 higher FPS and 1.50--7.18 higher GOPS/W than representative FPGA-based SNN accelerators, while reducing EDP by 1.70--19.13, with benchmark-dependent accuracy--efficiency trade-offs.

Comment
Author accepted manuscript. Published in IEEE Transactions on Circuits and Systems I: Regular Papers

Closest papers

Nearest neighbours by meaning across the whole index.

PSA: A Pruning-Friendly Sparse-Aware Accelerator for Efficient Sparse Convolution in SNNs

cs.AR ·2026-10-15 ·Sanhe Fu, Kainan Wang, Bo Wang, Aili Wang
top venue similarity 0.83
Abstract · Accepted to IEEE International Conference on Artificial Intelligence Circuits and Systems (AICAS 2026)

Spiking neural networks (SNNs) are promising for energy-efficient artificial intelligence due to their event-driven sparse computation, and neuron pruning can further reduce redundant operations. However, in practical hardware accelerators, spike sparsity and pruning-induced sparsity produce highly irregular sparse workloads that are difficult to execute efficiently. Existing approaches do not jointly support sparse-input execution, low-overhead sparse convolution lowering and pruning-friendly execution. This paper presents PSA, a Pruning-friendly Sparse-aware Accelerator for efficient sparse convolution in SNNs. PSA features a Sparse-Aware Merge-Accumulation (SAMA) dataflow that aligns sparse spike indices and converts irregular activations into continuous computation streams for efficient sparse convolution execution. It further introduces a Sparse Implicit Group-wise im2col (SIG-im2col) convolution lowering mechanism that converts sparse convolution into SpMM on-chip with low mapping cost and memory overhead. In addition, PSA integrates a lightweight pruning-friendly gating mechanism to directly support pruned SNNs. Results on ResNet-18/CIFAR-10 show that PSA achieves a 4.45x improvement in energy efficiency and an 18.52x reduction in energy-delay product (EDP) over the dense baseline.

0.41

BASC : Behavior-Aligned Quantization and Pruning for Low-Bit Spiking Neural Networks

cs.NE ·2026-08-12 ·Linliang Chen, Yan Zhong, Xin Liu, Sai Li, Wang Kang
similarity 0.75
Abstract

Spiking Neural Networks (SNNs) encode information through binary spikes and compute in an event-driven manner, offering an energy-efficient paradigm for machine intelligence. However, high-performance SNNs incur substantial memory and timestep-wise computation costs that hinder deployment on resource-constrained devices. Quantization and pruning provide complementary routes to reducing these costs, yet both make their decisions with local criteria that overlook temporal task feedback in quantization and inter-channel dependencies in pruning. Consequently, optimizing either criterion can still yield suboptimal compression performance. We refer to this discrepancy as criterion-behavior mismatch and propose Behavior-Aligned SNN Compression (BASC), a unified framework with two lightweight modules. For quantization, the scale is applied to synaptic current at every timestep and therefore shifts spike timing. Temporal-Behavior Scale Correction (TSC) makes the scale learnable under a temporal loss, allowing firing behavior to inform scale optimization. For pruning, channel importance depends on how channels jointly drive the membrane potential across the firing threshold. Boundary-Level Inter-Channel Correction (BIC) uses channelwise importance scores for initial selection and inter-channel information to re-evaluate only channels near the pruning threshold. Extensive experiments on static and neuromorphic benchmarks show that lower-bit BASC models match or outperform higher-bit baselines and retain this accuracy advantage after structured pruning, while further reducing model storage and synaptic operations.

0.04

Beyond Masked Sparsity: SNACK Enables Truly Sparse Neural Networks on GPU

cs.LG cs.DC ·2026-10-02 ·Jafar Badour, Maurice van Keulen, Elena Mocanu
top venuecode/data similarity 0.74
Abstract · Accepted at NeurIPS 2026. 23 pages, 9 figures. Code: https://github.com/JafarBadour/Snack-SNN

Deep neural networks continue to grow in parameter count, driving up training and inference cost on GPUs. Sparse neural networks and Dynamic Sparse Training (DST) promise to reduce these costs, but most implementations rely on binary masks over dense tensors and recover little of the theoretical compute, memory, or energy savings. We propose SNACK, a truly sparse GPU layer that stores and computes only non-zero connections. SNACK exposes a simple PyTorch API for restructuring connections and backpropagating gradients entirely in the sparse paradigm, and ships SNACK-COO, a custom COO-format SpMM CUDA kernel with a batch-to-Streaming-Multiprocessor mapping tuned for the small-batch, high-sparsity regime typical of large-model training and single-stream inference. At the kernel level, SNACK is up to 7x faster than the masked dense baseline (Dense+Mask) and competitive with cuSPARSE, Sputnik, and FlashSparse at 95% sparsity. At 90% sparsity, a single SNACK layer accelerates training by 8x and 3.7x, and inference by 4x and 2x, over Dense+Mask and fully dense layers, respectively, while using 72% less memory than dense and substantially less energy. End-to-end, SNACK reduces GPT-2 peak training memory by up to 40% and graph-style inference latency by 4.8x over Dense+Mask at 99% sparsity.

0.51

Low-Latency Activation-Regularized Sparse Neural Operators with Distillation Assistance Towards Real-Time Neuromorphic Virtual Sensing

cs.LG ·2026-08-25 ·William Howes, Farid Ahmed, Syed Bahauddin Alam
similarity 0.73
Abstract · 37 pages, 14 tables, 7 figures

Virtual sensing enables digital twins and safety-critical systems to reconstruct and forecast spatial-temporal physics in real time. However, conventional computational and data-driven methods often face challenges in generalization, latency, and energy efficiency for edge deployment. Neural operators offer a promising alternative but remain reliant on power-intensive hardware. Spiking neurons and neuromorphic computing can improve efficiency, yet surrogate-gradient training and multi-step spiking introduce convergence and latency challenges. We propose the Sparse-Activation-ReLU (SAR) layer, a single-step alternative that promotes activation sparsity without surrogate-gradient training while remaining compatible with event-based computing. Within a trunk-based NOMAD architecture, SAR achieves over a fivefold improvement in the combined Latency-Error-Energy (LEE) metric compared with Variable Spiking Neuron (VSN) and Leaky Integrate-and-Fire (LIF) implementations. We further analyze spiking entropy and feature usage and introduce synthetic knowledge distillation, reducing the LEE score by more than twofold. Finally, we improve VSN through a ReLU-based spiking loss and graph-neighbor thresholding. On the Heat Exchanger dataset, these approaches reduce L2 error by more than twofold and nearly sevenfold, respectively, while reducing spiking and spatial aggregation. Overall, the work presented is a step towards energy-efficient virtual sensing by providing an alternative framework that can be positioned towards neuromorphic or other edge device integration that can be a gold standard to compare latency, energy, and error performance for future efficient designs that are sparsity or brain-inspired spiking based.

0.38

Non-uniform Memory Partitioning For Low-Power Spiking Neural Networks

cs.AR ·2026-08-31 ·Simon Richter, Darío Fernández Khatiboun, Maryam Sadeghi, Milad Zamani, Farshad Moradi
top venuebenchmark similarity 0.73
Abstract · 5 pages, 3 figures, Publishes at IEEE AICAS 2025

Spiking Neural Networks (SNNs) naturally excel in processing temporally rich and sparse data. However, because of their time-stepped processing, memory access, specifically to synaptic weights stored in SRAM (static random-access memory), tends to dominate total power consumption. To address this issue, without incurring a large area overhead, we propose to leverage the greatly varying average firing rate of neurons in the network to efficiently allocate synaptic weights to an on-chip memory consisting of multiple non-uniformly sized memory banks. By assigning weights of frequently firing neurons to shallow, low-access cost memory and less actively accessed weights to deeper, high-density memories, the average power consumption of the synaptic weight memory is decreased without incurring a large area overhead. To benchmark our proposed architecture and find optimal configurations of memory arrangements, we perform an automatic exploration based on application requirements and hardware constraints. For memory designs synthesized in 28-nm CMOS technology, we show that our architecture can achieve a synaptic weight memory access power reduction of up to 61\% compared to a conventional design, with a 2.1$\times$ lower area overhead, as compared to a traditional uniformly partitioned memory bank that achieves a comparable reduction.

0.68

APEX: A Dual-Sparsity Accelerator for Precise and Efficient SNN Inference

cs.AR ·2026-08-19 ·Devgokul Bawa Venkatesh, Sreeram Radhakrishnan, Rajshekhar Rakshit, Gopalakrishnan Srinivasan
similarity 0.73
Abstract

Spiking Neural Networks (SNNs) have emerged as an energy-efficient alternative to Artificial Neural Networks (ANNs), leveraging sparse accumulate operations in the place of power-hungry multiply-and-accumulate operations. ANN-SNN conversion is a widely adopted approach to realize deep SNNs with accuracy comparable to that of ANNs. The Quantization-Clip-Floor-Shift (QCFS) activation minimizes conversion error, yet requires a large number of inference timesteps to match the source ANN accuracy on real-world vision datasets. PASCAL addresses this by proposing the Precise ANN-SNN Conversion Integrate-and-Fire (PASC-IF) neuron, which guarantees mathematical equivalence between the converted SNN and the source ANN, thereby achieving ANN-equivalent accuracy at significantly reduced timesteps. Despite this algorithmic advancement, the hardware implications of deploying the PASC-IF neuron remain unexplored. In this work, we present APEX, a dual-sparsity SNN inference accelerator that integrates the PASC-IF neuron into the LoAS hardware framework. The three-stage PASC-IF datapath is realized as a fully combinational circuit with no additional latency cost. APEX exploits dual sparsity in both input spikes and weights through a fully temporal-parallel dataflow, enabling efficient sparse computation and reduced memory traffic. Across all evaluated models, the PASC-IF neuron on average achieves up to 3% higher accuracy than the standard IF neuron, with a power overhead of only 1.3%-5.4%, an area overhead of 2.1%-2.7%, and 40% energy reduction for best accuracy configurations.

0.17

A Resource-Efficient and Precision-Scalable FPGA Accelerator for Spiking Neural Networks Using Minifloat Arithmetic

cs.NE cs.AR ·2026-10-23 ·Ali Mehrabi, Philip C. Jose
similarity 0.73
Abstract

Spiking neural networks (SNNs) are well suited to event-driven edge intelligence, but the arithmetic complexity, memory footprint, and power consumption associated with conventional floating-point implementations can limit their deployment on resource-constrained hardware. This paper presents a precision-scalable and resource-efficient SNN hardware accelerator architecture based on parameterized minifloat arithmetic. The proposed architecture implements recurrent leaky integrate-and-fire (LIF) neurons and leaky-integrator (LI) readout neurons using a unified minifloat datapath whose word length and mantissa width can be configured at synthesis time. A valid-driven arithmetic subsystem employs pipelined minifloat multiplication and addition, while time-multiplexed processing enables arithmetic resources to be shared across synaptic accumulation, neuron-state updates, recurrent connections, and output processing. This organization reduces replicated arithmetic hardware and provides a scalable design approach for different SNN sizes and precision requirements. The architecture supports E8M23, E5M10, E5M6, and E4M3 formats within the same RTL framework, allowing precision, memory capacity, timing, and power to be traded without modifying the underlying neuron architecture. FPGA implementation results demonstrate that reducing minifloat precision substantially decreases arithmetic-resource utilization and estimated dynamic power, while also reducing the theoretical storage requirement of synaptic parameters. A recurrent event-based classification network is used as a hardware validation case, demonstrating that substantial precision reduction can be achieved while retaining competitive inference accuracy. The results establish parameterized minifloat computation as an effective basis for scalable, low-resource, and energy-efficient FPGA SNN accelerators.

0.21

SpikeDuet: CPU-Accelerator Co-Execution of Sparse SNNs on Commodity MCUs

cs.AR ·2026-10-17 ·Muhammad Fahd Ibrahim Bhatti, Muhammad Ali Siddiqi
similarity 0.73
Abstract

Spiking neural networks (SNNs) expose sparse event activity, but commodity microcontrollers increasingly pair general-purpose processors with dense matrix-vector accelerators whose suitability for sparse inference is unclear. We investigate whether the on-chip Matrix Vector Processor (MVP) of the EFR32MG24 can accelerate SNN inference beyond optimized event-driven Cortex-M33 execution. Our automated snnTorch-to-C deployment flow combines hardware-calibrated engine selection, function-preserving neuron reordering that places co-active neurons adjacently for the accelerator, and asynchronous scheduling that overlaps CPU and MVP computation. On physical hardware, heterogeneous execution achieves 1.27--1.87$\times$ speedup and 11--34\% lower energy per inference than optimized sparse Cortex-M33 execution across five evaluated network configurations, while the smallest Braille network remains CPU-optimal. Reordering alone moves Braille-384's dominant hidden layer from the CPU to the MVP, and its benefit persists under asynchronous execution. These results show that activity locality and concurrency between CPU and MVP can jointly improve sparse SNN inference on commodity MCUs.

0.28