CS Paper Radar

Towards an Expressivity-Normalized Energy-Demand Comparison of ANNs and SNNs

Miriam Kranzlmüller, Pascal Esser, Gitta Kutyniok

cs.LG ·submitted 2026-08-30 15:56 UTC · v1

Abstract

Spiking neural networks (SNNs) are often regarded as energy-efficient alternatives to artificial neural networks (ANNs), yet their advantage depends critically on both network architecture and data properties. We develop an analytical framework to compare fully-connected ReLU ANNs and integrate-and-fire SNNs for time-series data with respect to their theoretical energy efficiency at matched expressive capacity. By relating an inference-energy model to theoretical bounds on representational expressivity, we derive an expressivity-normalized efficiency ratio and explicit thresholds in network width, spike sparsity, and ANN depth scaling. Our analysis characterizes the regimes in which event-driven computation offsets the temporal overhead of SNNs, providing capacity-aware principles for designing energy-efficient temporal networks. It shows that ANNs exceed SNNs in expressivity-normalized efficiency only in specific regimes.

Closest papers

Nearest neighbours by meaning across the whole index.

BASC : Behavior-Aligned Quantization and Pruning for Low-Bit Spiking Neural Networks

cs.NE ·2026-08-12 ·Linliang Chen, Yan Zhong, Xin Liu, Sai Li, Wang Kang
similarity 0.72
Abstract

Spiking Neural Networks (SNNs) encode information through binary spikes and compute in an event-driven manner, offering an energy-efficient paradigm for machine intelligence. However, high-performance SNNs incur substantial memory and timestep-wise computation costs that hinder deployment on resource-constrained devices. Quantization and pruning provide complementary routes to reducing these costs, yet both make their decisions with local criteria that overlook temporal task feedback in quantization and inter-channel dependencies in pruning. Consequently, optimizing either criterion can still yield suboptimal compression performance. We refer to this discrepancy as criterion-behavior mismatch and propose Behavior-Aligned SNN Compression (BASC), a unified framework with two lightweight modules. For quantization, the scale is applied to synaptic current at every timestep and therefore shifts spike timing. Temporal-Behavior Scale Correction (TSC) makes the scale learnable under a temporal loss, allowing firing behavior to inform scale optimization. For pruning, channel importance depends on how channels jointly drive the membrane potential across the firing threshold. Boundary-Level Inter-Channel Correction (BIC) uses channelwise importance scores for initial selection and inter-channel information to re-evaluate only channels near the pruning threshold. Extensive experiments on static and neuromorphic benchmarks show that lower-bit BASC models match or outperform higher-bit baselines and retain this accuracy advantage after structured pruning, while further reducing model storage and synaptic operations.

0.04

Exploring napping paradigm for Recurrent Spiking Neural Networks

cs.LG ·2026-09-12 ·Andreas Massey, Stefano Nichele, Aliaksandr Hubin
similarity 0.71
Abstract · Accepted at the 7th International Workshop on Active Inference (IWAI 2026). To appear in Springer Communications in Computer and Information…

Biological organisms minimize free energy by balancing two competing demands on their internal world model: it must be accurate enough to predict sensory input, yet simple enough to generalize beyond it. Two mechanisms regulate this balance offline: sleep reduces complexity through gradual synaptic downscaling, while stochastic noise attenuates precision, relaxing the constraint sensory input imposes on synaptic reorganization. Engineered Spiking Neural Networks (SNNs) leave this balance unaddressed, favoring instantaneous, noiseless weight normalization instead. This paper investigates the hypothesis that a biologically inspired micro-sleep paradigm, napping -- combining proportional weight scaling with continuous stochastic membrane activity -- can replicate the stability of normalization while shedding model complexity. We evaluate this in an unsupervised recurrent SNN trained via trace-based spike-timing-dependent plasticity (STDP) on Gabor-preprocessed MNIST. We tune napping across three regularization regimes by sweeping its duration and membrane noise level, then compare the best configuration against weight normalization. Across all three regimes, well-tuned napping matches the accuracy of normalization: accuracy peaks at brief durations and low noise, then declines monotonically as either grows. Clustering diverges, with the strongest geometric separation arising at longer durations and higher noise -- the two terms of free energy pulling apart, accuracy rewarding data fit and structure rewarding the simpler representation that gradual, noisy downscaling induces. This gain carries a simulation cost normalization avoids, so napping is most compelling where representational structure, rather than raw classification efficiency, is the priority.

0.22

SpikingVLA: Asynchronous Spiking Vision-Language-Action Models

cs.CL ·2026-10-07 ·Jingya Wang, Dehao Zhang, Shuai Wang, Malu Zhang, Yang Yang +1 more
similarity 0.70
Abstract

ANN-to-SNN conversion offers a practical route toward energy-efficient spiking Vision-Language-Action (VLA) models by bypassing the substantial cost of training large-scale SNNs from scratch. However, existing methods often require many timesteps to maintain competitive performance, resulting in substantial inference latency for real-time VLA deployment. To address this challenge, we introduce SpikingVLA, an ANN-to-SNN conversion framework that enables accurate and low-latency spiking VLA inference. Specifically, we propose a Dendritic Integrate-and-Fire (DIF) neuron that alleviates channel-wise activation outliers through dendritic mixing and adaptive somatic firing, enabling accurate ANN-to-SNN conversion with fewer timesteps. Building on DIF neurons, we further introduce an asynchronous execution mechanism that overlaps temporal computation across VLA components, reducing synchronization overhead and latency. Extensive experiments demonstrate that SpikingVLA achieves competitive navigation performance with substantially improved inference efficiency. Compared with existing spiking VLA methods, SpikingVLA improves SR and SPL by 11.9\% and 12.6\%, respectively, while reducing first-action latency by 11.2$\times$. These results establish SpikingVLA as a practical framework for deploying pretrained VLA models with high-performance and low-latency spiking inference.

0.28

Controlling the latency in temporal coded spiking models for neuromorphic datasets

cs.NE ·2026-10-14 ·Alban Gattepaille, Alexandre Muzy
theory similarity 0.69
Abstract

Spiking neural networks (SNNs) promise energy-efficient, event-driven computation on neuromorphic hardware, where response time, or latency, matters as much as energy for real-time applications. Time-to-first-spike (TTFS) coding with learnable synaptic delays is a natural way to exploit the timing of events. However, in state-of-the-art spiking neuron models, spike times emerge from nonlinear membrane dynamics and thresholds: no closed-form expression links network parameters to spike times, so latency can only be measured empirically, not specified or guaranteed at design time. Moreover, TTFS models are mostly evaluated on static datasets, which hides their latency on truly temporal data. Our hypothesis is that, if spike times are computed directly as closed-form functions of presynaptic spike times and synaptic delays, bypassing membrane potential and thresholding, latency becomes an explicit function of the learnable parameters that can be bounded, and therefore controlled, analytically. We build on this hypothesis with the Delay Neural Network (DeNN) in real time, which turns event streams into successive single-spike images, computes each spike time from standardised inter-spike intervals and synaptic delays, and links successive images through a long-term memory. We prove lower and upper bounds on the latency of every neuron of a layer, and show that bounding the synaptic parameters after training with a single hyperparameter places the latency anywhere between these bounds, without retraining. On a ten-class subset of the Google Speech Commands dataset, this post-training control exposes the accuracy--latency trade-off: a three-hidden-layer network reaches 94.25\% accuracy while deciding, on average, 477~ms after the end of the one-second input, and about 250~ms with confidence-based early decisions. Learning is also shown to concentrate the long-term memory on a few neurons. Latency thus becomes a design parameter of spiking models rather than a quantity measured after the fact, a step towards neuromorphic systems with real-time guarantees.

0.16

Contrastive Attention Mitigates Spectral Bias in Spiking Transformers

cs.AI ·2026-10-01 ·Xiaoli Liu, Malu Zhang, Yang Yang
similarity 0.69
Abstract · Spiking Neural Networks

Spiking Transformers merge the energy-efficiency of spiking neural networks (SNNs) with the representational power of self-attention, creating a promising architecture for high-performance, energy-efficient computation. However, a performance gap persists versus its counterparts in artificial neural networks (ANNs). Unlike prior works attributing this to binary activations, we reveal that both spiking neurons and spiking self-attention (SSA) act as low-pass filters through multiscale spectral analysis. This characteristic leads to the dissipation of high-frequency components. To address this issue, we propose the Spiking Contrastive Attention (SCA) paradigm, which draw inspiration from the edge-detection and differential sensing properties of biological visual system. By extracting contrast prototypes via global contrastive aggregation and applying local differential refinement, SCA effectively enhances high-frequency information. Extensive experiments show that SCA is a general module that consistently boosts Spiking Transformers across image classification, semantic segmentation, and event-based tracking. Furthermore, it achieves lower complexity, offering superior efficiency over original SSA. These results establish its potential as a fundamental building block for energy-efficient Spiking Transformers.

0.16

Large Language Models with At Most One Spike per Neuron

cs.NE cs.CL ·2026-09-04 ·Zhuoya Zhao, Parsa Omidi, Aref Jafari, Richard Naud
similarity 0.68
Abstract

Leveraging their inherent sparse event-driven computation, spiking neural networks (SNNs) offer a promising path toward energy-efficient large language models (LLMs). Time-to-first-spike (TTFS) coding generates at most one spike per neuron within a time window, yielding extremely low firing rates. However, conventional TTFS SNNs are restricted to specific structures, making it challenging to encode certain blocks in LLM -- such as layer normalization and matrix multiplication --using TTFS. To overcome this limitation, we introduce a reference-based strategy specifically to encode the four core LLM components: embedding layers, layer normalization, attention-related operations and dropout. We construct a fully TTFS-based SNN architecture and train it end-to-end. Experiments on modern LLMs like BERT and GPT-2 demonstrate that our approach achieves performance comparable to ANN counterparts on natural language understanding and common-sense reasoning, while a clear gap remains on language modeling perplexity. To the best of our knowledge, this is the first work to scale a spiking LLM to 1.5 billion parameters using TTFS coding. We also report an estimate of spike-related energy; this is a spike-count proxy under an established cost model rather than a measurement on neuromorphic hardware.

0.15

AI at a Turning Point: From ChatGPT through Neural Networks to Neuromorphic AI

cs.LG cs.NE ·2026-10-29 ·Volker Kempe
published similarity 0.68
Abstract · 88 pages, 20 figures

The tension between the established technologies of connectionist AI and their neuromorphic challengers is attracting growing attention, above all because of the enormous energy demand of the former and the low power consumption of the latter. Developing and running large AI models consumes gigawatt- to terawatt-hours of energy, while the human brain operates on a continuous 20 watts. Spiking neural networks (SNNs) counter this energy demand with a frugal approach and bring elements of self-organization into the methodological repertoire of AI: local learning instead of global error backpropagation, event-driven computing, and time as a carrier of information. They do not replace today's AI but extend it; in hybrid systems each approach can be deployed where its strengths lie. This monograph explains the field in a generally accessible yet mathematically rigorous way. Using the text-generation module of ChatGPT as an example, the principle, elegance, and computational cost of transformer-based AI are presented and compared with the coding, learning, and network-design methods of SNNs. Throughout, relevant mechanisms of the human nervous system serve as model and reference point. Topics include spike-timing-dependent plasticity, population and temporal coding, the Neural Engineering Framework, neuromorphic hardware from Loihi 2 to commercial edge processors, and the prospects for classical and neuromorphic AI to complement and extend each other.

0.28

Energy-efficient End-to-End Communication System with Spiking Neural Networks

cs.NI ·2026-11-03 ·Pooja Plavilayil Anil, Te-Chuan Chiu, Nguyen Van Huynh
similarity 0.68
Abstract · 6 pages, 5 figures, 1 table, conference

2030 is expected to witness the first commercial 6G systems. However, traditional modular communication architectures cannot meet the latency, optimisation, and integration requirements demanded by various sub-millisecond, data-intensive emerging 6G applications. End-to-end (E2E) communication addresses these limitations by replacing traditional processing blocks with neural networks. Nevertheless, conventional artificial neural networks (ANNs) in E2E systems use dense computation. This results in substantial energy consumption and computational resource requirements, which may not be feasible in resource constrained devices. This paper addresses the aforementioned challenge by developing a spiking neural network (SNN) encoder that targets lower encoder inference energy while maintaining communication reliability. In particular, the SNN encoder combines sparse spike activity, sinusoidal temporal modulation, and weighted temporal aggregation to map one-hot encoded messages to distinguishable transmit signals. The model is evaluated across antenna configurations and SNN timesteps to investigate the bit error rate (BER)-energy trade-off. The simulation results show that the SNN encoder achieves comparable BER under suitable timestep selections, while reducing the average estimated encoder energy by 68.93%, 52.20%, and 35.69% for SNN timesteps 2, 3, and 4, respectively, across five antenna configurations. These findings highlight the potential of the proposed SNN encoder for energy-efficient E2E communication systems.

0.15