CS Paper Radar

Contrastive local learning in large nonlinear analog circuits

Pragyan Pandey, Peter Wu, Deema Alyahya, Eric Y. Ma

cs.LG physics.app-ph ·submitted 2026-11-12 22:32 UTC · v1

Abstract

Machine learning in physical hardware promises to reduce training cost by colocating computation, memory and adaptation within device physics. Contrastive local learning networks (CLLNs) offer a compelling route: an electrical circuit relaxes to equilibrium in free and weakly clamped states, and each adaptive element updates from local voltage contrasts rather than backpropagated error signals. Yet whether nonlinear transistor implementations can scale beyond proof-of-concept circuits, and what design principles control that scaling, have remained unknown. Here we show that nonlinear analog circuits trained by contrastive local rules can learn larger machine-learning tasks and remain robust under hardware nonidealities. Using device-level circuit simulations, we systematically explore initialization, operating regime, loss-dependent clamping and training parameters, and scale CLLNs beyond ten thousand adaptive edges. The same local rule trains classifiers that reach the accuracy regime of conventional digital methods on Ionosphere and handwritten-digit benchmarks, and a network that autoregressively generates language over a 122-token vocabulary. These results establish the feasibility of local learning in large nonlinear analog circuits.

Comment
19 pages (13 main, 6 supplementary) and 11 figures (5 main, 6 supplementary)

Closest papers

Nearest neighbours by meaning across the whole index.

An all-transistor physical-learning transformer can be trained to perform in-context learning

cs.LG cond-mat.soft ·2026-10-08 ·Michael C. Abbott, Benjamin B. Machta
similarity 0.68
Abstract · 18 pages, 11 figures, plus appendices

Most systems trained by equilibrium propagation or coupled learning are in the same class as multi-layer perceptrons: while they may be universal function approximators, they struggle to learn associative tasks which are easy for transformers. Equilibrium propagation relies on a bidirectional flow of information through the physical system between input and output. The reverse flow is broken by more complex nonlinear systems, such as those performing multiplication which is needed to express dot-product attention. Here we explicitly restore this bidirectional flow by introducing extra circuit elements for this purpose into a double or dyadic circuit. We then use these ideas to construct an attention mechanism exploiting the exponential current law of sub-threshold MOSFET devices. We show that in simulations this circuit can be trained to perform a simple in-context learning task, despite using realistic reverse components which transmit only an approximate gradient. For this training to succeed we need cross-entropy loss (not mean squared error). We generate nudges appropriate for this loss using a novel floating circuit element, based on the same MOSFET sub-circuit used for softmax. We train the resistors in this circuit using the Lion optimiser, which can be implemented using one capacitor per parameter to store local memory of past gradient estimates.

0.26

Analog-Friendly Predictive Coding without Activation Derivatives

cs.LG cs.AIcs.NE ·2026-09-26 ·Francesco Innocenti
similarity 0.63
Abstract · 20 pages, 14 figures

Predictive coding (PC) is a local, energy-based alternative to backpropagation (BP) whose iterative inference dynamics make it attractive for implementation on analog hardware. However, standard nonlinear PC requires evaluating the derivative of the activation function during both inference and learning, which can be difficult to realise physically. Here, we introduce \textit{activation-matched Bregman PC}, replacing standard squared-error energies with Bregman divergences matched to the activation function. This formulation eliminates activation derivatives and, when combined with inference via mirror descent, yields local inference and learning rules requiring only weighted sums, local prediction errors, state integration, and the activation function. In digital experiments, Bregman PC performs comparably to standard PC and BP on classification and generative tasks, while preserving characteristic learning dynamics of PC and its convergence to BP under stable large-model parameterisations. These results provide a more analog-friendly formulation of nonlinear PC while retaining its key computational properties.

0.30

Local learning with imperfect analog synapses

cs.ET ·2026-10-20 ·Théo Dupuis, Adrien Renaudineau, Guillaume Bourgeois, Joel Minguet Lopez, Gabriele Navarro +4 more
similarity 0.62
Abstract

Analog in-memory computing delivers efficient neural network inference at the edge, but systems deployed in non-stationary environments may need to adapt in situ to locally acquired data. Backpropagation faces two structural obstacles here. Its global error transport lets stochastic write errors compound across layers, and its backward pass demands bidirectional, high-precision peripheral circuitry absent from inference-oriented crossbar arrays. We introduce Channel-wise Spatial Gradient learning (CwSG), a strictly local, forward-only rule that trains each convolutional layer by rewarding spatial variation in its feature maps. In simulations replaying the measured trajectories of 1,189 phase-change memory and 1,268 filamentary memristor devices, CwSG learns with single, unverified, sub-1 V pulses. It matches backpropagation on phase-change memory with 59-fold fewer pulses and outperforms it by 53 percentage points on highly variable filamentary memristors, with 8-bit forward converters and without transposed crossbar reads or network-wide activation storage. Analog training thus becomes a problem of credit assignment, in which algorithmic design rather than hardware compensation alone overcomes severe programming noise.

0.15

Fast simulation of nonlinear deep resistive networks for energy-based computation

cs.ET ·2026-09-07 ·Filip Osana, Julie Grollier, Damien Querlioz
similarity 0.62
Abstract

Deep resistive networks are electronic energy-based systems in which computation is performed by the steady-state voltages of nonlinear circuits. Nonlinear devices enable expressive input-output transformations, but make the circuit equilibria costly to compute during simulation and training. Recent coordinate-descent solvers have achieved large speedups over SPICE-class circuit simulators, but only for nonlinearities modeled as ideal-diode models. This mathematical simplification is not sufficient for practical analog circuits. Here we extend coordinate-descent simulation to realistic monotone nonlinearities, including Shockley diodes, antiparallel diode pairs, and piecewise-linear current-voltage characteristics. With neighboring voltages fixed, each node update remains a scalar Kirchhoff-law solve. Single-exponential characteristics admit closed-form Lambert-(W) updates, while more general monotone characteristics can be handled with scalar root-finding methods. Across networks with one to three hidden layers and hidden widths from 64 to 1024, the solver reproduces matched SPICE steady-state voltages with relative $L_1$ errors below $1.1\times10^{-4}$ for 90% of validation samples, while achieving speedups up to $(1.7\times10^3)$. We further train a $1568\times100\times20$ double-Shockley network on MNIST, reaching about 3.0% test error and reducing the per-epoch training time by roughly $4.4\times10^2$. Together, these results establish a practical route to the circuit-level design and training of large-scale analog energy-based systems incorporating realistic nonlinear devices.

0.27

Lock-in EP: An In-Situ Training Algorithm for Oscillatory Hardware

cs.LG ·2026-10-05 ·Sowjanya Tammali, Wilkie Olin-Ammentorp
similarity 0.61
Abstract

Analog hardware platforms offer the potential to reduce energy consumption over digital architectures, but in order to succeed, large-scale analog systems must also be able to operate with or recover from the variability of their components. Towards this goal, we derive and demonstrate the lock-in equilibrium propagation (LIEP) training method. LIEP provides local gradient information for each component in an oscillatory network without separate forward and backward sweeps, potentially allowing for in-situ learning capabilities on analog oscillatory hardware platforms. We demonstrate that LIEP can be used both for ab-initio training as well as recovering performance when pre-trained parameters are perturbed. We show that LIEP can be formulated as a three-factor update rule, and suggest that although the method is currently only validated on shallow networks, alternate architectures may allow it to extend to deep and large-scale networks addressing complex tasks.

0.21

NN-Based FeFET Compact Modeling for SPICE Simulation

cs.ET ·2026-11-04 ·Rodion Novkin, Simon Thomann, Hussam Amrouch
similarity 0.58
Abstract · 7 pages

Prototyping and early-stage circuit evaluation for emerging transistor technologies require fast and accurate compact models for SPICE simulations. Standard physics-based compact models, such as BSIM, offer high fidelity but demand extensive domain expertise and time-consuming equation formulation for every new physical effect, significantly slowing technology development. Neural network (NN)-based models offer a scalable alternative approach, but they rely heavily on smooth numerical solver data with explicit high-order derivative information. However, measurement data suffers from complex noise patterns, making derivative extraction challenging and complicating NN-based transistor modeling. This work presents a derivative-free NN training framework employing spectral normalization to naturally smooth derivative behavior and filter measurement noise. Using drain current measurements from a fabricated 28 nm ferroelectric device, we design an NN-based compact model that captures the full spectrum of polarization amplitudes and supports dynamic polarization switching in transient simulations. To demonstrate its capability, we deploy the model in a commercial SPICE software to simulate write and read operations for large-scale memory arrays with up to 512x512 size, including parasitic effects. Our approach bridges raw measurements with NN-based SPICE-compatible transistor modeling, significantly reducing model development time and enabling rapid circuit-level prototyping for emerging devices.

0.11

PRISM-UDE: Physics-Regularized Iterative Symbolic Modeling of 3nm FinFETs via Universal Differential Equation

cs.ET cs.LG ·2026-08-14 ·Pranavanath Balamurali, Prathamesh Dinesh Joshi, Raj Abhijit Dandekar, Rajat Dandekar, Sreedath Panat
benchmark similarity 0.58
Abstract

Compact transistor models are the mathematical backbone of circuit simulation. However, at advanced nodes such as 3nm, transport physics becomes too complex for traditional hand-derived equations to capture accurately. Purely data-driven neural surrogates, on the other hand, are numerically unstable inside circuit solvers and offer no physical insight into their own predictions. We introduce PRISM-UDE (Physics-Regularized Iterative Symbolic Modeling via Universal Differential Equations), a framework that embeds a small neural network inside a physics-based transistor model, using the network only to learn the transport behavior that the analytical baseline misses, rather than replacing the physics altogether. Once trained, this neural correction is distilled into a single, interpretable closed-form expression via symbolic regression, making the final model fully analytical and simulator-ready. Applied to a 3nm FinFET benchmark dataset, PRISM-UDE reduces prediction error more than sixfold (70.33% to 11.01%) relative to the standard physics-only baseline. The distilled expression preserves this accuracy almost exactly while eliminating the neural network entirely. We further validate the extracted expression directly inside a SPICE circuit simulator, confirming stable, physically consistent behavior under both static bias sweeps and dynamic switching conditions.

0.04

Input-dependent Neuromodulation Enhances Mixed-Signal Neuromorphic Hardware

cs.NE cs.ET ·2026-10-30 ·Aashman Mehta, AbdelQader AlKilany, Dan F. M. Goodman, Giacomo Indiveri, Melika Payvand
similarity 0.58
Abstract · 5 pages, 3 figures. Submitted to IEEE ISCAS 2027

Gating mechanisms allow a fixed neural network to adapt its computation to the current input by dynamically modulating its effective parameters. In the brain, neuromodulation uses this principle to move neural circuits between dynamical regimes, and deep learning exploits input-dependent gating, e.g., in gated recurrent networks and self-attention. Input-dependent modulation has also recently been shown to improve spiking neural networks, with particularly strong benefits in small, parameter-limited ones. This makes neuromodulation especially promising for neuromorphic processors at the edge, where area, memory and energy budgets constrain network size, while circuit physics limits parameter precision. Here we demonstrate input-dependent, within-trial neuromodulation of a spiking recurrent neural network on a mixed-signal neuromorphic processor. An off-chip modulator reads the on-chip network's input and activity and adjusts its population-level gains while the chip's analog circuits compute with the physics of the silicon. Neuromodulation raises binary spoken-digit classification accuracy by $+17.3 \pm 10.4$ percentage points, tested across five hardware network instantiations.

0.23