CS Paper Radar

EM-SDTrack: Efficient Multi-Scale Spike-driven Tracking for Event Cameras

Yimeng Shan, Haodi Wu, Wei Zheng, Xiaowei Ye, Shuai Wang, Yu Liang, Wei Hua, Chenxiao Dou, Malu Zhang

cs.NE ·submitted 2026-11-17 12:46 UTC · v1

Abstract

Event-driven spiking neural networks (SNNs) match the sparse spatiotemporal output of event cameras, making spike-driven trackers highly attractive for neuromorphic edge deployment. However, events reveal a target only through their spatial distribution, whose extent varies more than tenfold within one sequence. Event tracking therefore demands multi-scale representation, which existing spike-driven trackers lack. Transferring multi-scale schemes from artificial neural networks either breaks spike-driven inference with floating-point multiplication or repeats backbone computation across input scales. To address this, we propose Spike-driven Multi-Scale Interaction (SDMI), which performs cross-scale interaction in a single spiking forward pass. Its core, Dyadic Spike Sampling (DSS), restricts sampling coordinates to dyadic fractions and thus requires only shifts and additions. The resulting M-SDTrack is fully spike-driven at inference. For tighter edge budgets, we further propose Polarity-Privileged Distillation (PPD), which distils M-SDTrack into EM-SDTrack. PPD treats the event counts cancelled by the voxel representation as privileged information. Its core constraint, Polarity-Guided Curvature Distillation (PGCD), aligns the curvature of the template-search relation under polarity-guided perturbations. On FE108, FELT and VisEvent, M-SDTrack achieves state-of-the-art accuracy with an AUC of 65.1, 41.7 and 42.0. EM-SDTrack retains 99% of the accuracy of its teacher M-SDTrack-Tiny with 27% of its energy and 18% of its parameters. On the Lynxi KA200 neuromorphic chip, it runs losslessly in real time at 116 FPS and 0.12 W.

Closest papers

Nearest neighbours by meaning across the whole index.

SpikeAD: Toward Multiplication-Free Spike-Controlled Neuromorphic Architecture for Energy-Efficient Small Object Detection

cs.NE ·2026-10-28 ·Zhifan Song, Haralampos-G. Stratigopoulos
code/data similarity 0.74
Abstract

Energy-constrained aerial platforms require accurate object detection under strict power and latency budgets, yet conventional inference remains bottlenecked by multiply-accumulate (MAC) operations. While spiking neural networks (SNNs) offer bio-inspired efficiency, many SNN detectors retain hybrid MACs in intermediate layers, or use spike-driven inference where multi-level spikes are interpreted as repeated binary events. We propose SpikeAD, a direct-trained spike-controlled neuromorphic aerial detector. In SpikeAD, sparse bounded integer spikes act strictly as multiplexer control signals for multiplier-free shift-add inference, preserving spike boundaries throughout a four-scale bidirectional feature pyramid. We introduce the Spike-Gated Multi-Scale (SGMS) block, performing receptive field expansion with activity-driven channel selection, implementable with adders and comparators. Evaluated on VisDrone and DOTA and across Nano, Small, and Medium variants, SpikeAD achieves up to 46.2% mAP@50 on VisDrone (+4.8% over SNN prior art), and generalizes to oriented bounding box (77.6% mAP@50 on DOTA) using the proposed SpikeOBB head. Under the Horowitz 45nm arithmetic energy model following standard methodology in SNN literature, SpikeAD achieves up to 18.9x estimated reduction relative to an equivalent MAC-based implementation. We propose the Spike Processing Core (SPC), an RTL compute primitive to validate hardware realizability for reconfigurable and scalable custom accelerator chip design on FPGA/ASIC, highlighting a promising path toward next-generation ultra-low-power neuromorphic processors for aerial object detection. Link to the open-source and open-weight GitHub repository will be added upon publication.

0.19

Learning While Inferring: Local and Parallel Learning for Edge SNNs across Sensing Modalities

cs.NE ·2026-10-02 ·Yanxun Zhang, Yifei Wang, Changze Lv, Jingwen Xu, Yiyang Lu +4 more
similarity 0.72
Abstract

Edge intelligence requires models to sense continuously in real time and to keep adapting on-device, all under tight compute, energy, and memory budgets. Although spiking neural networks (SNNs) enable efficient event-driven inference, standard surrogate-gradient backpropagation (BP) serializes updates and blocks ongoing inference. We investigate Bidirectional Spike-Based Distillation (BSD) as an on-device learning principle that lets edge SNNs learn while inferring. BSD couples a stimulus-driven forward network with an independent target-driven reverse network and aligns their intermediate representations through local objectives. Because the two pathways have disjoint computation graphs until alignment, forward inference, reverse inference, and stage-wise updates can run concurrently. On 25 benchmarks spanning the five sensing modalities of SOUL, BSD stays within 3.8 percentage points of matched BP baselines on average, while only its forward branch is needed at deployment. The learned representations also transfer well to few-shot class-incremental learning without replaying past data. By removing the dense floating-point backward chain, BSD reduces the projected training latency to $0.72\times$ and the estimated training energy to $0.36\times$ that of BP.

0.15

SpikingVLA: Asynchronous Spiking Vision-Language-Action Models

cs.CL ·2026-10-07 ·Jingya Wang, Dehao Zhang, Shuai Wang, Malu Zhang, Yang Yang +1 more
similarity 0.72
Abstract

ANN-to-SNN conversion offers a practical route toward energy-efficient spiking Vision-Language-Action (VLA) models by bypassing the substantial cost of training large-scale SNNs from scratch. However, existing methods often require many timesteps to maintain competitive performance, resulting in substantial inference latency for real-time VLA deployment. To address this challenge, we introduce SpikingVLA, an ANN-to-SNN conversion framework that enables accurate and low-latency spiking VLA inference. Specifically, we propose a Dendritic Integrate-and-Fire (DIF) neuron that alleviates channel-wise activation outliers through dendritic mixing and adaptive somatic firing, enabling accurate ANN-to-SNN conversion with fewer timesteps. Building on DIF neurons, we further introduce an asynchronous execution mechanism that overlaps temporal computation across VLA components, reducing synchronization overhead and latency. Extensive experiments demonstrate that SpikingVLA achieves competitive navigation performance with substantially improved inference efficiency. Compared with existing spiking VLA methods, SpikingVLA improves SR and SPL by 11.9\% and 12.6\%, respectively, while reducing first-action latency by 11.2$\times$. These results establish SpikingVLA as a practical framework for deploying pretrained VLA models with high-performance and low-latency spiking inference.

0.29

Mask IPL: Noise-Free Intrinsic Position Learning via Computation Graph Clipping for Event-Based Spike-Driven Tracking

cs.CV ·2026-09-16 ·Yimeng Shan, Malu Zhang
similarity 0.70
Abstract

Spiking Neural Networks (SNNs) match the event-driven nature of event cameras and naturally extract spatiotemporal features. These properties have motivated a series of recent studies on event-based tracking with SNNs. Intrinsic Position Learning (IPL) acquires strong position information without introducing additional parameters, making it a mainstream approach for position encoding in event-based spike-driven tracking. However, the mechanism behind its effectiveness lacks systematic theoretical analysis. Moreover, our analysis reveals that IPL introduces noise in both forward and backward propagation. The former increases inference error, while the latter prevents parameters from converging to better solutions. This paper presents a systematic analysis of IPL and demonstrates that its effectiveness stems from the synergy between IPL and multi-stage convolution. The zero blocks in the joint tensor act as zero padding for convolution, and the resulting boundary effect propagates layer by layer through multi-stage convolution. Every parameter update is therefore driven by a gradient that perceives the relative displacement between template and search frames. Positional encoding added after the convolutional stage cannot provide this information. We further propose a simple Computation Graph Clipping method that applies a validity mask determined by the layout to the operations of every layer, making invalid regions equivalent to zero padding in both forward and backward propagation. This eliminates the noise without introducing additional parameters and makes the actual gradient coincide with the ideal gradient. We name the improved method Mask IPL. Without increasing parameters or computational cost, Mask IPL improves the AUC of the Tiny-scale tracker on FE108, FELT, and VisEvent, and consistently improves the Base-scale tracker as well.

0.35

Benchmarking spiking neural networks across sensing modalities on edge devices

cs.NE ·2026-08-27 ·Xin Du, Di Yu, Changze Lv, Yuqi Zhang, Zhuo Chen +10 more
benchmarkcode/data similarity 0.69
Abstract

Edge computing systems need to support diverse sensing workloads under tight energy and memory constraints, thereby motivating deployment-aware model selection. Spiking neural networks (SNNs) are a promising alternative to conventional artificial neural networks (ANNs), yet systematic evidence for when and why they provide practical advantages remains limited. Here, we present a benchmark of SNNs across five sensing modalities and multiple edge devices, systematically evaluating spike encoding, neuron models, and network topologies under consistent training and deployment protocols. We find that SNN advantages are strongly modality-dependent: while SNNs achieve performance broadly comparable to ANNs across most workloads, wireless sensing emerges as a particularly favorable domain. Frequency-domain and feature-space analyses further explain this result by showing that spiking dynamics naturally align with the spectral-temporal structure of wireless signals. Our deployment analysis further shows that SNN advantages are not one-dimensional, with energy gains often accompanied by modality-dependent system costs. Finally, we provide an open-source framework for reproducible benchmarking and deployment profiling, offering a practical foundation for algorithm-software-hardware co-design on emerging edge and neuromorphic computing platforms.

0.55

Neuromorphic Decoding and Dynamical Image Reconstruction from Populations of Spiking Retinal Ganglion Cells

cs.ET cs.LGeess.SP ·2026-10-12 ·Ning Zhang, Arto Nurmikko
similarity 0.69
Abstract

Decoding visual stimuli from the spiking activity of retinal ganglion cells (RGCs) informs both our understanding of the retinal code and the design of visual prostheses. Natural vision is dynamic, yet state-of-the-art decoders mostly reconstruct single frames with artificial neural networks that process dense, continuous-valued signals rather than the retina's sparse, asynchronous spikes. We asked whether dynamic scenes can be reconstructed and tracked entirely within the spike domain, and at what computational cost. We built an end-to-end spiking neural network (SNN) whose neurons integrate the recorded RGC spikes over time. Populations of up to 1836 RGCs were recorded ex vivo from mouse retina on high-density microelectrode arrays while flashed images, moving bars, drifting gratings and randomly moving handwritten digits were presented. For moving digits, one shared spiking encoder fed two readouts, one reconstructing the image and one decoding the digit's position. Compute energy was estimated from counted synaptic operations. The SNN decoded what and where at once. It reconstructed the stimuli with a structural similarity (SSIM) of up to 0.93, refining the reconstruction as spikes accumulated, and tracked a randomly moving target with a mean error of 1.33 pixels on a $32\times32$ grid. Its fidelity nearly matched that of the best of 17 larger spiking and artificial decoders (5.9 million to 1.7 billion parameters, versus 5.4 million), and at a mean firing rate of 14.2% it needed an estimated $\sim$70-fold less compute energy than a dense artificial network of the same architecture. Dynamic visual scenes can thus be decoded from retinal population activity in the spike domain, with a compact network and at a compute cost compatible with implantable and wearable neural interfaces, supporting spike-native decoders for visual prostheses.

0.23

SDPAD: A Fully Spike-Driven Pipeline for End-to-End Autonomous Driving

cs.RO cs.AI ·2026-10-08 ·Chengjun Zhang, Yuhao Zhang, Jie Yang, Mohamad Sawan
benchmark similarity 0.69
Abstract

End-to-end autonomous driving demands trajectory planners that are both highly accurate and cheap enough for edge deployment. State-of-the-art artificial neural network (ANN) planners meet the accuracy requirement at the cost of heavy dense computation, while spiking neural networks (SNNs)---though promising orders-of-magnitude energy savings through sparse, event-driven arithmetic---still lag far behind in planning accuracy. We present \textbf{SDPAD}, a fully spike-driven end-to-end planning pipeline that closes this gap. SDPAD converts a pre-trained ANN perception stack into integer-spike form via quantized ANN2SNN conversion, lifts multi-view images into the bird's-eye-view (BEV) space with a spike-driven-max (SDM) depth distribution (Spike-3D-Lift), and plans through the Spike-QFormer, a spiking query transformer in which ego, agent, and map queries distilled from the BEV scene are fused by learnable waypoint queries via cross-attention, followed by deformable spike-cross-attention refinement. Every operation is gated by integer spikes and inference is a single feed-forward pass without temporal simulation loops. On the nuScenes open-loop benchmark, SDPAD achieves an average $L_2$ error of 0.40\,m and a collision rate of 0.12\%, on par with strong ANN planners while consuming 69.9\,mJ---less than 2\% of recent ANN baselines. In closed-loop evaluation on the NAVSIM navtest split, SDPAD reaches 86.3 PDMS, surpassing the previous SNN planner SAD by 4.3 points and matching mainstream ANN planners at a fraction of their energy. To our knowledge, SDPAD is the first fully spike-driven planner evaluated in end-to-end autonomous driving, demonstrating that SNNs can rival dense ANNs in complex driving tasks.

0.16

Multi-Depth Temporal Fusion for Feedforward, Locally Trained Spiking Neural Networks

cs.NE cs.CVcs.LG ·2026-09-29 ·Aidin Attar, Eleonora Cicciarella, Michele Rossi
code/data similarity 0.68
Abstract · 22 pages. Submitted to Neurocomputing. Code available at https://github.com/aidinattar/multi-depth-temporal-fusion-snn

We propose a new spiking neural network (SNN) design to process static images and event streams using time-to-first-spike (TTFS) latencies. Our key research question is which architectural choices best accommodate local and online learning in multi-layer convolutional SNNs. This question is addressed via an original framework combining residual-like connections with multi-depth feature aggregation and consensus. The full SNN pipeline features an early-vision front end, to convert raw visual data into sparse spike latencies, a four-layer convolutional backbone trained layerwise with unsupervised spike-timing-dependent plasticity (STDP), a deterministic Multi-Depth Temporal Fusion (MDTF) and a final classifier trained with reward-modulated spike-timing-dependent plasticity (R-STDP). Rather than replacing early features in deeper layers, the proposed MDTF preserves early temporal evidence, adding sparse residual events from intermediate layers, and incorporating deeper features only when they agree in time with earlier representations. The resulting architecture is experimentally validated across MNIST, Fashion-MNIST, CIFAR-10, and N-MNIST, delivering strong classification performance under a fully local learning regime. Selective multi-depth fusion significantly outperforms traditional STDP/R-STDP baselines on higher-variability visual tasks (achieving +18.2 pp on Fashion-MNIST and +29.2 pp on CIFAR-10). Furthermore, activity-budget analyses show that the network retains high accuracy even when removing a large fraction of late or weak spike events, confirming its high data efficiency and reduced event-processing requirements. The codebase is publicly available at github.com/aidinattar/multi-depth-temporal-fusion-snn.

0.38