CS Paper Radar

HippocamFLEX: FPGA Prototyping of Hippocampus-Inspired Architecture for Context-Aware Sequence Prediction

Bhuvanesh S. Kayarabettu, Hamed Poursiami, Keiland Cooper, Babak Shahbaba, Norbert Fortin, Maryam Parsa, Akhilesh Jaiswal

cs.AR ·submitted 2026-10-29 20:04 UTC · v1

Abstract

Unlike conventional architectures that treat memory as static pattern storage, the hippocampus organizes experience as ordered sequences and retrieves them through temporally compressed replay spanning prior, current, and anticipated states. This capacity of context-appropriate sequence prediction remains absent from conventional deep learning accelerators and existing neuromorphic fabrics. We present HippocamFLEX, a fully synthesizable digital implementation of these hippocampal computational principles, realized with SystemVerilog RTL on the AMD Zynq UltraScale+,XCZU9EG. The design maps the hippocampal trisynaptic pathway onto three pipeline stages: a Dentate Gyrus / Entorhinal Cortex (DG/EC) stage for hypervector proximity matching, a CA3 stage for context-gated recurrent sequence prediction with multi-phase temporal tagging, and a CA1 stage that integrates past, present, and predicted future events into a theta-sequence inspired output. While much brain-inspired hardware relies on analog mixed-signal design for energy efficiency, this work targets FPGA prototyping as a deliberate architectural choice: FPGAs offer rapid design-space exploration without tape-out cost, bit-exact determinism that simplifies verification. The design supports 64 events across 16 contexts and is verified on physical silicon across context-isolation, temporal-ordering, and mid-sequence sliding-window test cases and is inherently scalable to larger event and context sets.

Closest papers

Nearest neighbours by meaning across the whole index.

HippocamPEX: A Hippocampus-Inspired In-Silico Architecture for Context-Aware Sequence Prediction

cs.AR ·2026-10-30 ·Md Abdullah al Kaiser, Keiland W. Cooper, Rouhan Noor, Ankur Singh, Zhishun Yang +7 more
similarity 0.78
Abstract · 6 figures

The brain's hippocampus enables episodic memory - the ability to encode events within their context, organize them into sequences, and generate predictive representations to guide adaptive behavior. Modern computing architectures, however, are optimized for large-scale arithmetic and struggle with such context-dependent sequential prediction or "cognitive" tasks. To translate these capabilities into a mixed-signal architecture, we first derived a working model from the neuroscience literature distilling key hippocampal computations into engineering-friendly abstractions. We then leveraged converging advances in in-memory computing, neuromorphic computing, asynchronous circuits, and analog computing to design a mixed-signal hardware architecture implementing these computations, called HippocamPEX. Finally, we demonstrated that HippocamPEX reproduces characteristic representational dynamics of hippocampal subregions and performs robust context-specific sequence prediction through detailed activity and circuit simulations. By tightly coupling hardware design with neuroscience knowledge, this work establishes a pathway for artificial systems capable of rapid, adaptive, and context-aware decision-making, addressing a central gap between biological and conventional computing.

0.21

Scaled Hippocampus-inspired Neural Networks on Neuromorphic Memristive Hardware

cs.NE cs.ET ·2026-09-14 ·Joseph A. Kilgore, Jeffrey D. Kopsick, Zahin Ahmed, Giorgio A. Ascoli, Gina C. Adam
similarity 0.65
Abstract

The hippocampus, a key brain region for learning and memory, exhibits rich structural diversity, sparse communication, and robust dynamics with incredible energy efficiency. It offers promising insights for novel computing capabilities, particularly when co-designed with emerging hardware technologies. In this work, we draw inspiration from the rodent CA3 hippocampal subregion to develop the first spiking neural network with neuronal diversity and biologically-realistic resting state dynamics demonstrated on memristor hardware. We propose a network downscaling methodology utilizing a 4-prong objective function and demonstrate a small-scale CA3-inspired network with 179 Izhikevich-modeled neurons, 3 neuronal types and 17,996 synapses with similar resting-state dynamics as the orders-of-magnitude larger full-scale network. The small-scale network is mapped to an FPGA/memristor platform using a greedy algorithm and 18,316 memristors. Benefiting from memristor noise, the hardware implementation shows continuous periodic behavior, outperforming simulated hardware. This work showcases the potential of biologically-realistic algorithms on emerging hardware for neuromorphic computing.

0.25

Continuous Memory Machines

cs.AI ·2026-10-06 ·Ciaran Regan, Kai Arulkumaran, Luke Darlow, Stefania Druga, Sebastian Risi +1 more
publishedcode/data similarity 0.61
Abstract · NeurIPS 2026 Workshop: Personalized, Aligned, Long-Term Memory for AI Systems (PALM)

Recurrent neural networks typically compress information into a single vector-valued recurrent state, forcing short-term computation and long-term retention to share the same representation. Past extensions alleviate this bottleneck by increasing the memory capacity or separating timescales, but lack the combination of rapid neuron-level processing and longer-term retention found in biology. To that end, we introduce the Continuous Memory Machine (CMM), a recurrent architecture with matrix-valued short- and long-term memory states serving distinct functional roles. Building on the Continuous Thought Machine (CTM), the CMM's short-term memory tracks recent neural activity, with uniquely parameterized neuron-level models learning to use these activity patterns for computation. A persistent long-term memory stores information for later use, with a Transformer jointly updating both memory stores, providing an expressive bidirectional read--write mechanism such that each store can reorganize its own contents and both read from and write to the other. Across algorithmic, in-context learning, and recurrent reasoning tasks, the CMM outperforms a broad suite of baselines, exhibiting stronger generalization than prior memory-augmented networks while preserving the CTM's interpretable attention patterns. Code is available at https://github.com/SakanaAI/continuous-memory-machines.

0.43

Empirical Boundaries of Reservoir Computing for Few-Shot Sequence Pattern Tasks

cs.LG cs.AI ·2026-10-14 ·Mario Mallea, Angela Nebot, Franscisco Mugica
similarity 0.59
Abstract · 4th International Workshop on Reservoir Computing (RC 2026) 35th International Conference on Artificial Neural Networks (ICANN 2026)

Efficient sequence modeling remains a fundamental challenge in machine learning. Despite the success of Transformers, recurrent neural networks (RNNs) have recently re-emerged as efficient alternatives. Particularly, Echo State Networks (ESNs) offer a compelling framework of randomized RNNs that have been extensively studied for time-series forecasting and dynamical systems. This paper presents an empirical analysis of distributional, architectural, and optimization variants of ESNs evaluated on few-shot sequence-to-sequence (seq2seq) pattern tasks. These synthetic tasks utilize discrete symbol sequences, characterized by delays, inactive phases, and triggered markers to isolate and evaluate memory and pattern recognition capabilities. The minimality of these tasks is crucial to understand the strengths and limitations of machine learning models. Furthermore, when the input signal utilizes a one-hot encoding, ESN dynamics are governed strictly by their weight distributions, offering an ideal lens through which to study reservoir computing. Our findings map the performance boundaries of ESNs relative to state-of-the-art (SOTA) models. We show that preserving a linear state regime is vital for memory tasks, such as pattern completion or simple copy. We propose a data-dependent modulation mechanism for nonlinear input injection into the reservoir (G-ESN). G-ESN functions as a statistical filter capable of solving tasks that conventional ESNs cannot (the adding problem and bracket matching). Finally, for Deep ESNs, we evaluate a shared readout mechanism across layers that outperforms conventional concatenated readouts on structural tasks, including selective copy, sorting problem, and bracket matching.

0.13

Low-Latency Activation-Regularized Sparse Neural Operators with Distillation Assistance Towards Real-Time Neuromorphic Virtual Sensing

cs.LG ·2026-08-25 ·William Howes, Farid Ahmed, Syed Bahauddin Alam
similarity 0.59
Abstract · 37 pages, 14 tables, 7 figures

Virtual sensing enables digital twins and safety-critical systems to reconstruct and forecast spatial-temporal physics in real time. However, conventional computational and data-driven methods often face challenges in generalization, latency, and energy efficiency for edge deployment. Neural operators offer a promising alternative but remain reliant on power-intensive hardware. Spiking neurons and neuromorphic computing can improve efficiency, yet surrogate-gradient training and multi-step spiking introduce convergence and latency challenges. We propose the Sparse-Activation-ReLU (SAR) layer, a single-step alternative that promotes activation sparsity without surrogate-gradient training while remaining compatible with event-based computing. Within a trunk-based NOMAD architecture, SAR achieves over a fivefold improvement in the combined Latency-Error-Energy (LEE) metric compared with Variable Spiking Neuron (VSN) and Leaky Integrate-and-Fire (LIF) implementations. We further analyze spiking entropy and feature usage and introduce synthetic knowledge distillation, reducing the LEE score by more than twofold. Finally, we improve VSN through a ReLU-based spiking loss and graph-neighbor thresholding. On the Heat Exchanger dataset, these approaches reduce L2 error by more than twofold and nearly sevenfold, respectively, while reducing spiking and spatial aggregation. Overall, the work presented is a step towards energy-efficient virtual sensing by providing an alternative framework that can be positioned towards neuromorphic or other edge device integration that can be a gold standard to compare latency, energy, and error performance for future efficient designs that are sparsity or brain-inspired spiking based.

0.38

Beyond Low-Pass Dynamics: Frequency-Selective Spiking Reservoirs with Resonant Neurons

cs.AI ·2026-10-27 ·Aurora Micheli, Claudio Gallicchio, Nergis Tomen
top venuecode/data similarity 0.58
Abstract · Accepted at NeurIPS 2026. 30 pages (10 main text), 4 figures. Code: https://github.com/AuroraMicheli/HRF-Res

Spiking reservoir computing combines the training efficiency of fixed recurrent reservoirs with the energy efficiency of event-driven inference, where computation is triggered by sparse spikes rather than executed densely at every timestep. However, existing spiking reservoirs almost exclusively rely on Leaky Integrate-and-Fire (LIF) neurons, whose low-pass dynamics limit their ability to distinguish information encoded at different temporal frequencies. We introduce HRF-Res, a liquid state machine in which LIF neurons are replaced by a heterogeneous population of Harmonic Resonate-and-Fire (HRF) neurons. These neurons exhibit intrinsic oscillatory dynamics and selectively respond to input frequencies, enabling the reservoir to capture frequency-specific temporal patterns. The result is more discriminative representations and higher classification accuracy than LIF-based reservoirs, achieved without sacrificing the energy efficiency of spiking computation, with up to 19x lower energy consumption than an equivalent non-spiking oscillatory reservoir. Evaluated on ten benchmarks spanning time series, sequential images, audio spike trains, and neuromorphic vision, HRF-Res achieves state-of-the-art accuracy among spiking reservoir methods. These results demonstrate that introducing frequency-selective dynamics into spiking reservoirs improves representational quality and classification performance while preserving their inherent energy efficiency.

0.45

SpikeLite: Lightweight Spiking Neural Networks for Time-Series Forecasting

cs.LG cs.AI ·2026-09-28 ·Bang Hu, Changze Lv, Mingjie Li, Xiaoqing Zheng, Wei cao +1 more
similarity 0.58
Abstract

Spiking neural networks (SNNs) offer an energy-efficient paradigm for time-series forecasting through spike-driven computation. However, recent SNN forecasters often pursue higher accuracy through increasingly complex attention mechanisms, or specialized neuronal dynamics, weakening the lightweight motivation of SNNs. We introduce SpikeLite, a spiking forecasting framework built around two modules: a Frequency-Selective Spiking Encoder (FSSE) for frequency-sensitive temporal encoding and a Sparse Spiking Channel Attention (SSCA) module for selective cross-channel interaction. FSSE exploits the low-pass filtering behavior of LIF dynamics to reorganize each input sequence into frequency-sensitive components while collectively preserving the input at the decomposition stage. SSCA then learns a binary mask from encoded channel representations and uses it to selectively exchange information within spike-driven self-attention, retaining informative cross-channel interactions while suppressing redundant ones. When explicit channel interaction is unnecessary, SpikeLite uses the lighter FSSE-only channel-independent path. Experiments under the SeqSNN and SpikF protocols cover four standard multivariate and eight long-term forecasting benchmarks. SpikeLite achieves the best aggregate performance under both protocols, with an average $R^2$ of 0.790 and RSE of 0.440, and lowest average MSE/MAE of 0.343/0.345 in long-term forecasting. Moreover, evaluation on the ECL dataset shows that SpikeLite achieves the lowest reported energy consumption, further demonstrating its potential for energy-efficient time-series forecasting.

0.23

Large Language Models with At Most One Spike per Neuron

cs.NE cs.CL ·2026-09-04 ·Zhuoya Zhao, Parsa Omidi, Aref Jafari, Richard Naud
similarity 0.58
Abstract

Leveraging their inherent sparse event-driven computation, spiking neural networks (SNNs) offer a promising path toward energy-efficient large language models (LLMs). Time-to-first-spike (TTFS) coding generates at most one spike per neuron within a time window, yielding extremely low firing rates. However, conventional TTFS SNNs are restricted to specific structures, making it challenging to encode certain blocks in LLM -- such as layer normalization and matrix multiplication --using TTFS. To overcome this limitation, we introduce a reference-based strategy specifically to encode the four core LLM components: embedding layers, layer normalization, attention-related operations and dropout. We construct a fully TTFS-based SNN architecture and train it end-to-end. Experiments on modern LLMs like BERT and GPT-2 demonstrate that our approach achieves performance comparable to ANN counterparts on natural language understanding and common-sense reasoning, while a clear gap remains on language modeling perplexity. To the best of our knowledge, this is the first work to scale a spiking LLM to 1.5 billion parameters using TTFS coding. We also report an estimate of spike-related energy; this is a spike-count proxy under an established cost model rather than a measurement on neuromorphic hardware.

0.15