CS Paper Radar

Event-triggered Implicit Perturbation for Zeroth-Order Fine-Tuning of Spiking Transformers

Tengteng Lei, Prabodh Katti, Rashi Dutt, Houssem Sifaou, Tan Peng, Osvaldo Simeone, Kai Xu, Bipin Rajendran

cs.AR cs.LGcs.NE ·submitted 2026-08-21 15:32 UTC · v1

Abstract

Zeroth-order (ZO) optimization estimates gradients using only forward-pass evaluations, making it suitable for fine-tuning non-differentiable, event-driven spiking neural networks (SNNs). However, its deployment on in-memory computing (IMC) accelerators is constrained by the repeated read-modify-write (RMW) operations arising from explicit weight perturbation and the prohibitive hardware footprint of random number generators (RNGs) for statistically independent per-weight perturbations. To address these challenges, we propose an implicit-perturbation ZO (IPZO) architecture in which perturbation sums computed by an event-triggered perturbation generation unit (PGU) are combined with the weighted sums produced by the IMC array, eliminating perturbation-induced RMW operations while preserving weight-stationary execution of IMC. By exploiting spike sparsity, the PGU generates and accumulates perturbation contributions only for spike-activated weight rows, reducing the required row dimension of the RNG array. An address-driven XOR recombination scheme (PGU-XOR) is further introduced to mitigate the spatial correlations caused by direct RNG reuse (PGU-Reuse). The results show that (1) PGU-XOR matches software RNGs in accuracy on Spikingformer/CIFAR-10 (76.41% vs. 76.53%) and perplexity (PPL) on SpikeGPT/WikiText-2 (54.20 vs. 53.23), whereas PGU-Reuse degrades accuracy by 9.56 percentage points and increases PPL by 11.8; (2) implemented in a TSMC 16-nm CMOS technology, PGU-XOR incurs 40.3%-46.0% area and 15.2%-48.9% energy overhead per matrix-vector multiplication relative to PGU-Reuse, yet its faster convergence reduces the total perturbation energy to 0.51x that of PGU-Reuse at iso-accuracy; (3) IPZO reduces the perturbation energy to 0.46x-0.83x that of conventional explicit weight perturbation for a batch size of B=64 and T=4 time steps, with the advantage growing as BT decreases.

Closest papers

Nearest neighbours by meaning across the whole index.

Non-uniform Memory Partitioning For Low-Power Spiking Neural Networks

cs.AR ·2026-08-31 ·Simon Richter, Darío Fernández Khatiboun, Maryam Sadeghi, Milad Zamani, Farshad Moradi
top venuebenchmark similarity 0.68
Abstract · 5 pages, 3 figures, Publishes at IEEE AICAS 2025

Spiking Neural Networks (SNNs) naturally excel in processing temporally rich and sparse data. However, because of their time-stepped processing, memory access, specifically to synaptic weights stored in SRAM (static random-access memory), tends to dominate total power consumption. To address this issue, without incurring a large area overhead, we propose to leverage the greatly varying average firing rate of neurons in the network to efficiently allocate synaptic weights to an on-chip memory consisting of multiple non-uniformly sized memory banks. By assigning weights of frequently firing neurons to shallow, low-access cost memory and less actively accessed weights to deeper, high-density memories, the average power consumption of the synaptic weight memory is decreased without incurring a large area overhead. To benchmark our proposed architecture and find optimal configurations of memory arrangements, we perform an automatic exploration based on application requirements and hardware constraints. For memory designs synthesized in 28-nm CMOS technology, we show that our architecture can achieve a synaptic weight memory access power reduction of up to 61\% compared to a conventional design, with a 2.1$\times$ lower area overhead, as compared to a traditional uniformly partitioned memory bank that achieves a comparable reduction.

0.68

Neuromorphic Pseudo-Random Number Generators with a Low Power Hardware Implementation

cs.NE q-bio.NC ·2026-09-30 ·Jafar Shamsi, Navid Akbari, Sonia Sennik, Aaron Gruber, Wilten Nicola
similarity 0.65
Abstract · 33 pages, 3 figures

Pseudo-random number generation often requires trade-offs among quality, power consumption, and bandwidth to produce unpredictable sequences of numbers. The brain, on the other hand, efficiently generates unpredictable output complex network dynamics occurring in a high-dimensional state. This state, which is hypothesized to be chaotic, relies on the balance between excitation and inhibition. Here, we investigated if computational models of these chaotic balanced states can be harnessed for Neuromorphic Pseudo-Random Number Generators (NPRNGs) in low power hardware. We successfully constructed a balanced spiking neural network model consisting of leaky-integrate-and-fire neurons that could be readily implemented in low power FPGAs and used as a NPRNG. The prototyped NPRNG consumed 3.24 mW during operation and produced pseudo-random numbers at 120kbps. In both hardware and software instantiations, NPRNGs produce high-quality random numbers as validated by standard metrics for testing RNG quality.

0.21

SpikeQ: Spike-Guided Post-Training Quantization for Spiking Neural Networks

cs.NE cs.AIcs.CV ·2026-10-16 ·Kai Sun, Peibo Duan, Zhanglu Yan, Guowei Zhang, Yongsheng Huang +1 more
similarity 0.64
Abstract

On neuromorphic chips, spiking neural networks (SNNs) consume far less energy than artificial neural networks (ANNs), but the limited memory of these chips requires low-bit synaptic weights and membrane potentials. Existing post-training quantization (PTQ) methods provide them without retraining and, like those for ANNs, minimize numerical errors such as the mean squared error (MSE). In an SNN, however, a quantization error reaches later layers only through the spikes it flips. We find that errors of equal MSE can cause very different accuracy losses, whereas errors that flip the same fraction of spikes cause the same loss. We therefore propose SpikeQ, a spike-guided PTQ framework. For weights, we select the level of each weight by simulating the downstream neurons and minimizing their spike errors. For membrane potentials, most existing methods quantize the state after the reset, whereas neuromorphic chips compare the threshold with the membrane potential stored at a fixed bit width. We target this pre-fire quantization and build a reset-aware grid that places the threshold on a rounding boundary, so rounding never flips the spike at the current time step. We evaluate SpikeQ on seven benchmarks of image classification, semantic segmentation, and object detection. With 2-bit pre-fire membrane potentials, every existing grid collapses, whereas SpikeQ retains most of the full-precision accuracy. With both at 4 bits, the models stay within 2.59 points of full precision, shrink their memory footprint 7.8--7.9 times, and use 6.4--6.9 times less energy.

0.15

BPU-SpikeMM:Exact Matrix Multiplication via Spatio-Temporal Spike Encoding on Real Neuromorphic Hardware

cs.DC ·2026-11-13 ·Rongyuan Guo, Yufeng Zhang, Haotian Wang, Canming Yao, Yuxiang Huan +2 more
theory similarity 0.64
Abstract

Matrix multiplication is the dominant computational kernel in deep learning, yet its growing scale strains conventional von Neumann hardware in energy efficiency and bandwidth. Neuromorphic architectures offer a promising alternative, but mathematical equivalence between matrix multiplication and spiking computation does not guarantee direct execution on real neuromorphic hardware, as the specific hardware paradigm may still require additional data recovery, bit-weight reconstruction, or state readout after neuromorphic execution. This paper presents BPU-SpikeMM (Brain Processing Unit-based Spike Matrix Multiplication), an exact spatio-temporal spike encoding framework for matrix multiplication on real neuromorphic hardware. We prove the exact equivalence of spike-encoded matrix multiplication under the BPU execution paradigm, distinguish mathematical equivalence from native direct-readout equivalence on BPU hardware, and build a spatio-temporal offloading pipeline supporting multiple encoding schemes. Furthermore, we extend the framework to a multi-dimensional encoding design space and evaluate the applicability of different encodings on real BPU hardware in terms of correctness, temporal resource cost, spatial resource cost, and event-driven metrics. The results confirm native direct-readout equivalence across all evaluated encoding schemes and show that their applicability to BPU execution depends on joint trade-offs among temporal cost, spatial resource consumption, and event activity.

0.12

A Resource-Efficient and Precision-Scalable FPGA Accelerator for Spiking Neural Networks Using Minifloat Arithmetic

cs.NE cs.AR ·2026-10-23 ·Ali Mehrabi, Philip C. Jose
similarity 0.64
Abstract

Spiking neural networks (SNNs) are well suited to event-driven edge intelligence, but the arithmetic complexity, memory footprint, and power consumption associated with conventional floating-point implementations can limit their deployment on resource-constrained hardware. This paper presents a precision-scalable and resource-efficient SNN hardware accelerator architecture based on parameterized minifloat arithmetic. The proposed architecture implements recurrent leaky integrate-and-fire (LIF) neurons and leaky-integrator (LI) readout neurons using a unified minifloat datapath whose word length and mantissa width can be configured at synthesis time. A valid-driven arithmetic subsystem employs pipelined minifloat multiplication and addition, while time-multiplexed processing enables arithmetic resources to be shared across synaptic accumulation, neuron-state updates, recurrent connections, and output processing. This organization reduces replicated arithmetic hardware and provides a scalable design approach for different SNN sizes and precision requirements. The architecture supports E8M23, E5M10, E5M6, and E4M3 formats within the same RTL framework, allowing precision, memory capacity, timing, and power to be traded without modifying the underlying neuron architecture. FPGA implementation results demonstrate that reducing minifloat precision substantially decreases arithmetic-resource utilization and estimated dynamic power, while also reducing the theoretical storage requirement of synaptic parameters. A recurrent event-based classification network is used as a hardware validation case, demonstrating that substantial precision reduction can be achieved while retaining competitive inference accuracy. The results establish parameterized minifloat computation as an effective basis for scalable, low-resource, and energy-efficient FPGA SNN accelerators.

0.21

FlexSpIM: An Event-Based Digital Compute-In-Memory Accelerator with Flexible Operand Resolution and Layer-Wise Hybrid Stationarity

cs.AR ·2026-09-08 ·Nicolas Chauvaux, Adrian Kneip, Charlotte Frenkel
similarity 0.63
Abstract · 14 pages, 17 figures, 2 tables

Compute-in-memory (CIM) accelerators for spiking neural networks (SNNs) offer a promising solution for achieving $μ$s-level inference latency and ultra-low energy in edge vision applications. However, their limited flexibility at both circuit and system levels restricts their deployment across diverse workloads. This work introduces FlexSpIM, a digital CIM architecture supporting arbitrary operand resolution and shape within a unified storage for weights and neuron states (i.e., membrane potentials). These circuit-level capabilities enable a layer-level hybrid weight- and output-stationary dataflow, maximizing operand reuse and reducing costly on- and off-chip data movement during SNN execution. Measurement results from a fabricated FlexSpIM prototype in 40-nm CMOS demonstrate competitive 1-bit-normalized energy efficiency and higher throughput compared with prior fixed-precision digital CIM-based SNN accelerators, while providing bitwise resolution reconfiguration. Evaluated on the IBM DVS gesture dataset, FlexSpIM achieves 95.8% accuracy while enabling up to 45% energy and 52% latency reductions in large-scale systems compared with fixed stationarity approaches.

0.13

SparseForge: Beyond Spike Sparsity toward Hardware-Exploitable Neuron Pruning for Efficient SNN Acceleration

cs.AR ·2026-11-01 ·Kainan Wang, Sanhe Fu, Aili Wang
publishedbenchmark similarity 0.63
Abstract · Author accepted manuscript. Published in IEEE Transactions on Circuits and Systems I: Regular Papers

Spiking Neural Networks (SNNs) provide an event-driven paradigm for energy-efficient edge computing. However, operation-level sparsity in SNNs does not necessarily eliminate the hardware costs associated with redundant output neurons, including neuron-state maintenance, partial-sum (psum) accesses, and on-chip storage requirements. This paper presents SparseForge, a hardware-aware sparse SNN acceleration framework that jointly optimizes neuron pruning, sparse dataflow, and accelerator microarchitecture. At the algorithm level, we introduce a gated leaky integrate-and-fire (G-LIF) neuron to learn differentiable neuron masks with binary spike semantics. A dual regularization scheme is further developed to align pruning with both synaptic-operation reduction and psum buffer capacity reduction. At the dataflow level, we design a sparse Column-Wise Product (CWP) dataflow with a variant of incremental spike encoding to improve weight-access regularity and reduce sparse-index overhead under dynamic spike activity. At the architecture level, we develop a mask-aware sparse processing unit (SPU) that supports compressed spike decoding, sparse implicit im2col conversion, neuron-mask-gated psum updates, and decoupled sparse execution. Experimental results across multiple SNN backbones and datasets show that SparseForge retains 1.75%--77.99% of neurons and 0.16%--62.36% of synaptic operations, with a configurable accuracy--efficiency trade-off. The buffer-capacity-aware regularization reduces the required psum buffer capacity by up to 96.18\%. FPGA implementations on a Xilinx ZCU9EG achieve 296.94--619.49 GOPS/W across the evaluated workloads. Across matched network and dataset settings, SparseForge provides 1.24--4.17 higher FPS and 1.50--7.18 higher GOPS/W than representative FPGA-based SNN accelerators, while reducing EDP by 1.70--19.13, with benchmark-dependent accuracy--efficiency trade-offs.

0.23

PSA: A Pruning-Friendly Sparse-Aware Accelerator for Efficient Sparse Convolution in SNNs

cs.AR ·2026-10-15 ·Sanhe Fu, Kainan Wang, Bo Wang, Aili Wang
top venue similarity 0.63
Abstract · Accepted to IEEE International Conference on Artificial Intelligence Circuits and Systems (AICAS 2026)

Spiking neural networks (SNNs) are promising for energy-efficient artificial intelligence due to their event-driven sparse computation, and neuron pruning can further reduce redundant operations. However, in practical hardware accelerators, spike sparsity and pruning-induced sparsity produce highly irregular sparse workloads that are difficult to execute efficiently. Existing approaches do not jointly support sparse-input execution, low-overhead sparse convolution lowering and pruning-friendly execution. This paper presents PSA, a Pruning-friendly Sparse-aware Accelerator for efficient sparse convolution in SNNs. PSA features a Sparse-Aware Merge-Accumulation (SAMA) dataflow that aligns sparse spike indices and converts irregular activations into continuous computation streams for efficient sparse convolution execution. It further introduces a Sparse Implicit Group-wise im2col (SIG-im2col) convolution lowering mechanism that converts sparse convolution into SpMM on-chip with low mapping cost and memory overhead. In addition, PSA integrates a lightweight pruning-friendly gating mechanism to directly support pruned SNNs. Results on ResNet-18/CIFAR-10 show that PSA achieves a 4.45x improvement in energy efficiency and an 18.52x reduction in energy-delay product (EDP) over the dense baseline.

0.41