CS Paper Radar

Scenario-aware Adaptive Neuromorphic Intelligence for multimodal sensor fusion

Kapil Bhardwaj, Tauseef Ejaz, Sayani Majumdar

eess.SY ·submitted 2026-11-05 11:10 UTC · v1

Abstract

Multisensory Physical AI systems operating in isolated conditions must make timely and reliable decisions within strict energy and hardware budgets. In this context, the present analytical study investigates the feasibility of reliable multimodal sensor fusion using a neuromorphic approach based on memristor-based Integrate-and-Fire neuron circuits driven by aggressively compressed representations of sensory information. Rather than relying on fixed processing and learning parameters, the proposed framework introduces scenario-dependent adaptability into individual stages of the neuromorphic decision-making pipeline, including sensor-data compression, nonlinear spike encoding, reliability assessment, and memristive learning-forgetting dynamics. By coupling multiple sensor inputs through adaptive memristive synapses, the framework investigates how reliable and correlated observations can strengthen learned associations, while weak or obsolete relationships gradually decay through nonlinear forgetting. The resulting neuronal responses are evaluated across ten representative driving scenarios derived from the NuScenes dataset by comparing their encoded output patterns with predefined synthetic ground-truth codes representing the expected actuator responses. The simulations demonstrate that scenario-adaptive activity-vector sizing achieves a 61.5% reduction in retained sensory information, while improving mean bit-wise agreement from approximately 55% to 61.2% compared with a fixed maximum-resolution configuration. These findings highlight the potential of introducing scenario-aware adaptability directly into the building blocks of a neuromorphic system, providing a foundation for future hardware implementations that could significantly transform the capabilities of Physical AI systems operating under stringent resource and environmental constraints.

Comment
29 pages, 17 figures

Closest papers

Nearest neighbours by meaning across the whole index.

Low-Latency Activation-Regularized Sparse Neural Operators with Distillation Assistance Towards Real-Time Neuromorphic Virtual Sensing

cs.LG ·2026-08-25 ·William Howes, Farid Ahmed, Syed Bahauddin Alam
similarity 0.66
Abstract · 37 pages, 14 tables, 7 figures

Virtual sensing enables digital twins and safety-critical systems to reconstruct and forecast spatial-temporal physics in real time. However, conventional computational and data-driven methods often face challenges in generalization, latency, and energy efficiency for edge deployment. Neural operators offer a promising alternative but remain reliant on power-intensive hardware. Spiking neurons and neuromorphic computing can improve efficiency, yet surrogate-gradient training and multi-step spiking introduce convergence and latency challenges. We propose the Sparse-Activation-ReLU (SAR) layer, a single-step alternative that promotes activation sparsity without surrogate-gradient training while remaining compatible with event-based computing. Within a trunk-based NOMAD architecture, SAR achieves over a fivefold improvement in the combined Latency-Error-Energy (LEE) metric compared with Variable Spiking Neuron (VSN) and Leaky Integrate-and-Fire (LIF) implementations. We further analyze spiking entropy and feature usage and introduce synthetic knowledge distillation, reducing the LEE score by more than twofold. Finally, we improve VSN through a ReLU-based spiking loss and graph-neighbor thresholding. On the Heat Exchanger dataset, these approaches reduce L2 error by more than twofold and nearly sevenfold, respectively, while reducing spiking and spatial aggregation. Overall, the work presented is a step towards energy-efficient virtual sensing by providing an alternative framework that can be positioned towards neuromorphic or other edge device integration that can be a gold standard to compare latency, energy, and error performance for future efficient designs that are sparsity or brain-inspired spiking based.

0.38

SpikeOcc: Towards Resilient Multimodal 3D Occupancy Perception via Spiking Experts

cs.CV ·2026-10-11 ·Yiding Sun, Zijie Xu, Hongqiang Lin, Jiajun Cao, Dongxu Zhang +5 more
similarity 0.65
Abstract

Multimodal 3D occupancy prediction underpins safe autonomous driving through comprehensive scene understanding. However, existing methods have largely pursued higher accuracy through sophisticated fusion, leaving resilience to missing sensors underexplored. In this paper, we investigate spiking neural networks (SNNs) as a means of building more resilient multimodal perception. Still, the discrete nature of spiking neurons makes optimization challenging. To overcome this difficulty, SpikeOcc is introduced with two complementary components, a Spiking Mixture-of-Experts (SMoE) and Sensor Availability Learning (SAL). Specifically, SMoE organizes RGB, LiDAR, and joint fusion into specialized pathways, enabling full-width unimodal representations while retaining the original multimodal features via a bounded residual. Beyond sensor dropout, SAL employs a staged curriculum with explicit routing supervision, progressively improving robustness under severe sensing conditions. Remarkably, SAL also yields notable gains for conventional models, highlighting its broader potential for resilient multimodal learning. Extensive experiments demonstrate that SpikeOcc maintains competitive full-sensor accuracy while markedly raising the performance floor under sensor outages. SpikeOcc illustrates the promise of neuromorphic computing, pointing toward more reliable and energy-efficient autonomous driving.

0.30

Sensing to Intelligence: Principles for Neuromorphic Circuits and Systems

cs.NE ·2026-09-19 ·Saptarshi Maiti, Chetan Singh Thakur
position similarity 0.64
Abstract

Neuromorphic engineering began with the idea that the physical behavior of a system could itself be used for computation, taking inspiration from the way nervous systems sense, adapt, and evolve in time. The field has since expanded far beyond its early analog circuits to include event-based sensors, spiking processors, emerging memory devices, mixed-signal systems, and large-scale neural accelerators. With this expansion, however, the meaning of neuromorphic has become increasingly broad. In this Perspective, we argue that neuromorphic engineering should be defined neither by resemblance to biological components nor by any particular device, signal representation, or substrate. Its potential lies in identifying computational principles in biological systems and translating them into the organization and dynamics of artificial machines. A system is truly neuromorphic when the invoked biological or physical principle plays a causal, design-relevant role in how information is represented, how state evolves, or what capability the complete system achieves. We develop this view across sensing, collective computation, memory, learning, and interaction. Adaptation, nonlinear dynamics, attractor structure, variability, and closed-loop action illustrate how computation can be embedded in a machine's evolving physical state. From these examples, we identify a set of design principles for neuromorphic systems. As neuromorphic engineering moves toward wider deployment, this perspective shifts the central question from how closely machines resemble nervous systems to what computation becomes possible when their principles are understood and deliberately translated into new machines.

0.23

Input-dependent Neuromodulation Enhances Mixed-Signal Neuromorphic Hardware

cs.NE cs.ET ·2026-10-30 ·Aashman Mehta, AbdelQader AlKilany, Dan F. M. Goodman, Giacomo Indiveri, Melika Payvand
similarity 0.61
Abstract · 5 pages, 3 figures. Submitted to IEEE ISCAS 2027

Gating mechanisms allow a fixed neural network to adapt its computation to the current input by dynamically modulating its effective parameters. In the brain, neuromodulation uses this principle to move neural circuits between dynamical regimes, and deep learning exploits input-dependent gating, e.g., in gated recurrent networks and self-attention. Input-dependent modulation has also recently been shown to improve spiking neural networks, with particularly strong benefits in small, parameter-limited ones. This makes neuromodulation especially promising for neuromorphic processors at the edge, where area, memory and energy budgets constrain network size, while circuit physics limits parameter precision. Here we demonstrate input-dependent, within-trial neuromodulation of a spiking recurrent neural network on a mixed-signal neuromorphic processor. An off-chip modulator reads the on-chip network's input and activity and adjusts its population-level gains while the chip's analog circuits compute with the physics of the silicon. Neuromodulation raises binary spoken-digit classification accuracy by $+17.3 \pm 10.4$ percentage points, tested across five hardware network instantiations.

0.23

Scaled Hippocampus-inspired Neural Networks on Neuromorphic Memristive Hardware

cs.NE cs.ET ·2026-09-14 ·Joseph A. Kilgore, Jeffrey D. Kopsick, Zahin Ahmed, Giorgio A. Ascoli, Gina C. Adam
similarity 0.60
Abstract

The hippocampus, a key brain region for learning and memory, exhibits rich structural diversity, sparse communication, and robust dynamics with incredible energy efficiency. It offers promising insights for novel computing capabilities, particularly when co-designed with emerging hardware technologies. In this work, we draw inspiration from the rodent CA3 hippocampal subregion to develop the first spiking neural network with neuronal diversity and biologically-realistic resting state dynamics demonstrated on memristor hardware. We propose a network downscaling methodology utilizing a 4-prong objective function and demonstrate a small-scale CA3-inspired network with 179 Izhikevich-modeled neurons, 3 neuronal types and 17,996 synapses with similar resting-state dynamics as the orders-of-magnitude larger full-scale network. The small-scale network is mapped to an FPGA/memristor platform using a greedy algorithm and 18,316 memristors. Benefiting from memristor noise, the hardware implementation shows continuous periodic behavior, outperforming simulated hardware. This work showcases the potential of biologically-realistic algorithms on emerging hardware for neuromorphic computing.

0.25

Towards bio-inspired visual navigation: a neuromorphic navigation paradigm for MAVs

cs.RO ·2026-10-15 ·Junjie Jiang, Ankang Zhou, Zheng Fang
similarity 0.60
Abstract

Biological visual navigation achieves remarkable robustness at extremely low power, whereas frame-based robotic systems relying on artificial neural networks remain limited in both respects, particularly for energy-constrained micro aerial vehicles (MAVs) operating in complex environments. Inspired by the event-driven, sparse, and asynchronous principles of biological visual systems, neuromorphic sensing and computation provide a promising alternative for robust and energy-efficient navigation. In this paper, we propose the first end-to-end event-driven neuromorphic paradigm for MAV visual navigation in complex environments by integrating neuromorphic vision, spiking neural networks, and neuromorphic processors. The proposed method first extracts the environmental representation from event streams using a spiking perceptual representation module. This representation is fed into a spiking navigation module, where guidance learning and population coding enable efficient policy learning with few time steps. The navigation module then outputs a waypoint converted into executable trajectory points. Experiments show that the proposed neuromorphic method achieves navigation performance comparable to ANN-based methods while markedly reducing power consumption. The spiking perceptual module consumes only 19.57% of the power of its ANN counterpart, while the spiking navigation module deployed on Intel Loihi consumes only 12.37% of that required by its ANN counterpart running on NVIDIA Orin NX. Beyond prior neuromorphic systems focused mainly on relatively simple navigation subtasks, we demonstrate a complete neuromorphic visual navigation system spanning simulation and real-world deployment. Our results establish the feasibility of the neuromorphic paradigm for autonomous navigation and provide a bio-inspired pathway toward robust and low-power MAV navigation.

0.25

SpikeMoE: Brain-Inspired Competitive Routing for Flexible Spiking Mixture-of-Experts

cs.AI ·2026-10-01 ·Xiaoli Liu, Yujie Liang, Jialin Li, Chenxiao Dou, Malu Zhang
similarity 0.60
Abstract

Spiking Neural Networks (SNNs) enable event-driven computation through biologically inspired dynamics at the neuronal scale, while Mixture-of-Experts (MoE) perform conditional computation through expert selection at the model scale. Integrating their strengths offers potential for flexible neural architectures. A key challenge, however, lies in designing an expert selection mechanism based on spiking activity. To address this, we introduce a spike-based k-WTA Router inspired by competition-inhibition observed in the hippocampal CA1 region. The router incorporates lateral inhibition and refractory period to select Top-K experts according to discrete spike counts. Building on this, we present SpikeMoE, a framework that integrates neuronal-scale spiking dynamics with model-scale expert selection. To address incomplete multisensory inputs in multimodal tasks, we further equip SpikeMoE with a two-stage missing-modality modeling module that combines empirical prototypes from an observed-modality pool with modality-specific learnable embeddings to construct missing-modality representations. Experiments on vision, language, and multimodal benchmarks demonstrate that SpikeMoE achieves state-of-the-art performance among the SNN baselines, matches or exceeds the performance of ANN counterparts, and maintains robustness across diverse missing-modality conditions. These results demonstrate a favorable trade-off between performance and energy efficiency, validating the integration of spiking dynamics with sparse expert computation and highlighting SpikeMoE as a promising approach to energy-efficient brain-inspired computing.

0.29

DVA-Neurons: Design and Verification of Adaptive LIF Neurons: From Single-Neuron Dynamics to Multi-Neuron Spiking Networks

cs.AI cs.AReess.SP ·2026-09-19 ·Thanh Pham, Riadul Islam
similarity 0.60
Abstract

Spiking Neural Networks (SNNs) offer a promising path toward ultra-low-power artificial intelligence inference by emulating the event-driven computation of biological neurons. However, two challenges limit their practical deployment. First, fixed-parameter Leaky Integrate-and-Fire (LIF) neurons lack the adaptation mechanisms observed in biology, where neurons modulate their excitability based on firing history. Second, scaling from single neurons to multi-neuron networks introduces challenges in synaptic weight distribution and inter-neuron spike routing that are absent in isolated designs. This paper addresses both issues through the extension, verification, and physical implementation of adaptive LIF neurons at three architectural scales. This work contributes: a 2nd-order neuron with two-stage synaptic filtering for richer temporal dynamics; a fully-connected 6-neuron spiking network with configurable weights (100 to 5) demonstrating weight-based inter-neuron communication; and a direct verification methodology enabling per-cycle observation of all internal states. All designs were synthesized targeting Selected Area Electron Diffraction (SAED) 14 nm Complementary Metal-Oxide-Semiconductor (CMOS) technology at 1 GHz and verified with Cocotb-based Python testbenches under pulsed current stimuli (amplitude 80, ISI=3). The results show that adaptation effectively modulates firing: 31% suppression in the 2nd-order neuron (25 vs.\ 36 spikes) and 31% reduction in postsynaptic firing in the network (18 vs.\ 26 spikes). Physically, the 2nd-order neuron costs 1.77x more area and 1.52x more power than the 1st-order baseline, while the 6-neuron network demonstrates near-linear scaling (5.7x area, 5.3x power). Seven verification bugs spanning testbench connectivity, fixed-point overflow, and Verilog expression-width semantics are documented.

0.21