CS Paper Radar

Fractional-order hardware for neuromorphic computing: Is the order really the problem?

Christof Teuscher

cs.ET cs.NE ·submitted 2026-09-09 22:38 UTC · v1

Abstract

Does a neuromorphic system need a true power-law memory kernel, and if so, can anyone build one? Neuromorphic systems process signals spanning many timescales at once, from milliseconds to tens of seconds. Integer-order circuits buy each additional timescale with an additional state variable. Fractional-order dynamics offer a different bargain: one operator whose power-law kernel carries a continuum of timescales, tuned by one parameter, the order alpha. A fractional derivative is non-local, so evaluating it costs storage and arithmetic that grow with the retained history, where an integer-order derivative costs a constant. This review organizes the hardware literature around that cost. We derive the retained history needed to hold the truncation error below a tolerance epsilon, show that it scales as epsilon^(-1/alpha), and set beside it a second and independent limit on the direct form: in fixed point the weights themselves underflow, so word length caps the usable history however long the buffer is. The two limits move at very different rates with the order, and where they cross decides whether a word length can serve an order at all. We use both to sort published hardware into three strategies, note a fourth the numerical literature has developed and this hardware has not, and survey digital, analog and device work. Along the way we ask whether the field is worried about the right obstacle. It is not. Fabricated constant-phase devices already span the orders two groups identify as task-optimal, so the order gap has largely closed, leaving a residual gap near 0.1 and at the lower order describing cortical adaptation. What remains is a frequency-band gap of about three decades at the low end. That corner is not empty, since double-layer electrodes work there, but every device in it is discrete, and no integrable thin-film element has been characterized there.

Closest papers

Nearest neighbours by meaning across the whole index.

Fractional State Space Transition for Long Sequence Modeling

cs.LG cs.CL ·2026-09-28 ·Ivan Kobyzev, Abbas Ghaddar, Ali Nasiri-Sarvi, Lifeng Shang, Yufei Cui
top venue similarity 0.60
Abstract · NeurIPS 2026 (Oral)

State Space Models (SSMs) compress sequence history into a bounded recurrent state, making the resulting memory law a central architectural choice for long-context performance. Most modern SSMs rely on ODE-based dynamics that lead to exponential forgetting, limiting their ability to retain information over broad temporal ranges. We introduce FRAC, a selective SSM architecture derived from fractional dynamics that replaces this exponential decay with power-law long memory. To make fractional dynamics practical, FRAC approximates the heavy-tailed target kernel with a finite-state, log-spaced sum of exponential modes. This construction turns fractional memory into an efficient recurrent module with parallel training and prefill, while retaining bounded-state autoregressive decoding. Extensive experiments, including 1.3B-parameter language modeling, demonstrate that FRAC consistently improves long-context performance over state-of-the-art SSM baselines while staying competitive on short-context. These results show that fractional dynamics provide a practical and effective prior for long-context SSMs.

0.36

Towards Design of Efficient Neuromorphic Controllers

eess.SY ·2026-11-10 ·Koen Scheres, Erik Steur, Maurice Heemels
similarity 0.58
Abstract · Submitted to European Control Conference 2027

One of the major sources of inspiration for neuromorphic engineering is the central nervous system, which is orders of magnitude more efficient than classical digital clock-based computers. Neuromorphic control aims at recreating this efficiency in control systems, combined with (formal) stability and performance guarantees. In our previous work [1], we have established formal guarantees for the emulation of static output feedback controllers implemented using integrate-and-fire neurons. In this paper, we investigate the ``efficiency'' of the resulting controllers in terms of sparse spiking activity. By changing the network structure of our original implementation, foregoing the unit gain assumption made in our previous work, we will establish that the chosen architecture of the spiking controller significantly affects its sparsity and discuss the implications of these observations for the efficient implementation of spiking controllers.

0.22

Neuromorphic Pseudo-Random Number Generators with a Low Power Hardware Implementation

cs.NE q-bio.NC ·2026-09-30 ·Jafar Shamsi, Navid Akbari, Sonia Sennik, Aaron Gruber, Wilten Nicola
similarity 0.56
Abstract · 33 pages, 3 figures

Pseudo-random number generation often requires trade-offs among quality, power consumption, and bandwidth to produce unpredictable sequences of numbers. The brain, on the other hand, efficiently generates unpredictable output complex network dynamics occurring in a high-dimensional state. This state, which is hypothesized to be chaotic, relies on the balance between excitation and inhibition. Here, we investigated if computational models of these chaotic balanced states can be harnessed for Neuromorphic Pseudo-Random Number Generators (NPRNGs) in low power hardware. We successfully constructed a balanced spiking neural network model consisting of leaky-integrate-and-fire neurons that could be readily implemented in low power FPGAs and used as a NPRNG. The prototyped NPRNG consumed 3.24 mW during operation and produced pseudo-random numbers at 120kbps. In both hardware and software instantiations, NPRNGs produce high-quality random numbers as validated by standard metrics for testing RNG quality.

0.21

Neuromorphic architectures as numerical solvers for computational neuroscience

cs.AR q-bio.QM ·2026-08-28 ·Jakob Jordan, Ole Richter, Congyang Li, Mihai A. Petrovici, Rajit Manohar
similarity 0.56
Abstract · 23 pages, 7 figures

Neuromorphic computing is closely associated with spiking neuronal networks. However, an alternative class of so-called "rate-based" models arising from computational neuroscience and machine learning forgoes spiking interactions and instead relies on continuous coupling between neurons. Existing neuromorphic implementations designed around spike-based interactions are not well-suited for emulating such models. Here view the distributed simulation of these models as message-passing algorithms on parallel hardware. Leveraging prior art in numerical algorithms and distributed simulation, we outline steps that enable the design of efficient digital neuromorphic accelerators for non-spiking neuronal models. In particular, we show that multi-bit packets, rather than spikes, are the most efficient communication strategy in packet-switched networks and that compared to basic numerical integration methods, higher-order differential equation solvers decrease both computation and communication costs while achieving lower numerical error, but that these benefits are ultimately limited by arithmetic precision. Using our proposed design principles, we convert an existing neuromorphic architecture into a distributed numerical solver - a spikeless neuromorphic system - for continuously-coupled neuronal models. We thereby demonstrate that our theoretical considerations indeed translate into practical advantages, namely reduced energy consumption and delay.

0.29

Input-dependent Neuromodulation Enhances Mixed-Signal Neuromorphic Hardware

cs.NE cs.ET ·2026-10-30 ·Aashman Mehta, AbdelQader AlKilany, Dan F. M. Goodman, Giacomo Indiveri, Melika Payvand
similarity 0.55
Abstract · 5 pages, 3 figures. Submitted to IEEE ISCAS 2027

Gating mechanisms allow a fixed neural network to adapt its computation to the current input by dynamically modulating its effective parameters. In the brain, neuromodulation uses this principle to move neural circuits between dynamical regimes, and deep learning exploits input-dependent gating, e.g., in gated recurrent networks and self-attention. Input-dependent modulation has also recently been shown to improve spiking neural networks, with particularly strong benefits in small, parameter-limited ones. This makes neuromodulation especially promising for neuromorphic processors at the edge, where area, memory and energy budgets constrain network size, while circuit physics limits parameter precision. Here we demonstrate input-dependent, within-trial neuromodulation of a spiking recurrent neural network on a mixed-signal neuromorphic processor. An off-chip modulator reads the on-chip network's input and activity and adjusts its population-level gains while the chip's analog circuits compute with the physics of the silicon. Neuromodulation raises binary spoken-digit classification accuracy by $+17.3 \pm 10.4$ percentage points, tested across five hardware network instantiations.

0.23

Event-Driven Language Models with Sparse Neural Activity for Neuromorphic Hardware

cs.NE cs.LG ·2026-08-31 ·Simon Richter, Ruhai Lin, Jason Yik, Taylor Kergan, Rui-Jie Zhu +2 more
published similarity 0.55
Abstract · 8 pages, 4 figures, Accepted at IEEE MCSOC2026

Inference with transformer-based large language models (LLMs) is often limited by the memory-bound KV cache and quadratic attention cost. State-space models (SSMs) mitigate this through linear attention and fixed-size recurrent states, but their large dense linear projections remain computationally expensive even after quantization. We introduce a method that induces sparse neural activity in heavily quantized linear-attention models with minimal performance loss. Activations below a per-projection trainable threshold ($\pm Δ$) are nullified while preserving crucial outliers, achieving comparable performance to dense models with up to 4$\times$ fewer effective arithmetic operations. Targeting a multi-core, multi-chip neuromorphic platform, where event-driven execution converts unstructured sparsity into throughput at both the compute and communication levels, a capability GPU architectures fundamentally lack, we project up to 37$\times$ higher throughput and 16$\times$ lower power versus edge GPU inference of a comparable transformer-based model, and up to 5.4$\times$ improvements over the non-sparsified baseline. These results position sparse, quantized linear-attention models as a natural fit for deploying LLMs on event-driven multi-core platforms.

0.33

MIAO: Microscaling Intermediate Accumulation Optimization

cs.AR ·2026-11-15 ·Chloe Wong, Nicolas Rakela, Man Shi, Hansung Kim, Ruohan Richard Yan +2 more
similarity 0.55
Abstract · 18 pages, 7 figures, 7 tables

Microscaling (MX) quantization enables lower-precision large language model (LLM) inference with limited accuracy degradation, reducing memory requirement and multiplication cost. However, despite reduced operand precision, accumulation is often retained at high floating-point precisions, leaving a substantial hardware cost that operand quantization alone does not address. In particular, systolic arrays, as used in tensor accelerators such as Google TPU and AWS Trainium, distribute accumulation across distinct processing elements but conventionally use uniform accumulator precision throughout the array. This spatial organization offers an opportunity to reduce hardware cost by tailoring precision to individual accumulation stages. In this work, we present MIAO, a hardware-software co-design framework that jointly selects product precision and stage-dependent accumulator widths to minimize hardware area and energy cost under an end-to-end perplexity constraint. MIAO first calibrates MX block-scale shifts to reduce intermediate magnitudes while preserving operand quantization accuracy, then assigns narrower accumulators to earlier stages and progressively wider accumulators to later stages. To efficiently explore the resulting design space, MIAO combines hardware cost estimates derived from logic synthesis with a calibrated accuracy predictor to identify promising precision configurations, then validates the accuracy of promising candidates through bit-exact end-to-end LLM evaluation. Across MXFP8, MXFP6, and MXFP4, MIAO reduces processing element (PE) array area by up to 38.7% and energy by up to 36.1% relative to the all-BF16 accumulation baseline, with at most 1.60% additional perplexity degradation over operand-only MX quantization across six LLMs.

0.23

SymbolicLight V2: Hybrid Neuromorphic Architecture and Sparse Execution for Low-Energy Language Inference

cs.CL ·2026-09-09 ·Ting Liu
similarity 0.55
Abstract · 22 pages, 8 figures, 11 tables

SymbolicLight V2 combines sparse event computation with continuous-state processing in a hybrid neuromorphic language architecture. Extending V1's spike-gated dual paths, it adds graded signed events at further projections and softmax-free local attention. We implement the 194M-parameter model on an Alveo U50C FPGA using digital fixed-point arithmetic and on an ARM CPU using sparse integer execution. Across three same-checkpoint FPGA implementations at 175 MHz, active-row weight gathering and valid-state KV loading raise decode throughput from 474.6 to 643.2 tokens/s for a 32-token prefix and 128 outputs. Estimated gross card energy falls from 0.06087 to 0.04407 J per generated token, a 27.6% reduction. Complete-request energy, including prefill, falls by 24.4-27.7% across three prefix lengths. An independent idle split attributes 82.8% of gross card energy to loaded idle, explaining the benefit of shorter token latency. Against the recorded RTX 5090 compiled-FP32 baseline, integer FPGA execution uses 89.1% less estimated card energy during short-context decode; arithmetic precisions differ, and the GPU baseline is not the lowest-energy tested configuration. On four Cortex-A76 cores of a ROCK 5T, complete requests reach 65.4 tokens/s at 9.80 W and 0.151 J per generated token at the adapter's AC input. These results connect event sparsity to omitted computation and data movement. The mechanisms also support other dedicated V2 implementations: increasing throughput by a greater factor than active power lowers energy per generated token. Evaluation holds the deployed checkpoint fixed; its quality trails a same-budget dense control, so the results do not establish equal-quality efficiency.

0.28