CS Paper Radar

Fast simulation of nonlinear deep resistive networks for energy-based computation

Filip Osana, Julie Grollier, Damien Querlioz

cs.ET ·submitted 2026-09-07 11:22 UTC · v1

Abstract

Deep resistive networks are electronic energy-based systems in which computation is performed by the steady-state voltages of nonlinear circuits. Nonlinear devices enable expressive input-output transformations, but make the circuit equilibria costly to compute during simulation and training. Recent coordinate-descent solvers have achieved large speedups over SPICE-class circuit simulators, but only for nonlinearities modeled as ideal-diode models. This mathematical simplification is not sufficient for practical analog circuits. Here we extend coordinate-descent simulation to realistic monotone nonlinearities, including Shockley diodes, antiparallel diode pairs, and piecewise-linear current-voltage characteristics. With neighboring voltages fixed, each node update remains a scalar Kirchhoff-law solve. Single-exponential characteristics admit closed-form Lambert-(W) updates, while more general monotone characteristics can be handled with scalar root-finding methods. Across networks with one to three hidden layers and hidden widths from 64 to 1024, the solver reproduces matched SPICE steady-state voltages with relative $L_1$ errors below $1.1\times10^{-4}$ for 90% of validation samples, while achieving speedups up to $(1.7\times10^3)$. We further train a $1568\times100\times20$ double-Shockley network on MNIST, reaching about 3.0% test error and reducing the per-epoch training time by roughly $4.4\times10^2$. Together, these results establish a practical route to the circuit-level design and training of large-scale analog energy-based systems incorporating realistic nonlinear devices.

Closest papers

Nearest neighbours by meaning across the whole index.

Contrastive local learning in large nonlinear analog circuits

cs.LG physics.app-ph ·2026-11-12 ·Pragyan Pandey, Peter Wu, Deema Alyahya, Eric Y. Ma
similarity 0.62
Abstract · 19 pages (13 main, 6 supplementary) and 11 figures (5 main, 6 supplementary)

Machine learning in physical hardware promises to reduce training cost by colocating computation, memory and adaptation within device physics. Contrastive local learning networks (CLLNs) offer a compelling route: an electrical circuit relaxes to equilibrium in free and weakly clamped states, and each adaptive element updates from local voltage contrasts rather than backpropagated error signals. Yet whether nonlinear transistor implementations can scale beyond proof-of-concept circuits, and what design principles control that scaling, have remained unknown. Here we show that nonlinear analog circuits trained by contrastive local rules can learn larger machine-learning tasks and remain robust under hardware nonidealities. Using device-level circuit simulations, we systematically explore initialization, operating regime, loss-dependent clamping and training parameters, and scale CLLNs beyond ten thousand adaptive edges. The same local rule trains classifiers that reach the accuracy regime of conventional digital methods on Ionosphere and handwritten-digit benchmarks, and a network that autoregressively generates language over a 122-token vocabulary. These results establish the feasibility of local learning in large nonlinear analog circuits.

0.20

NN-Based Transistor Modeling for Ultra-Fast SPICE Simulation and Standard Cell Library Characterization

cs.ET cs.LG ·2026-10-22 ·Rodion Novkin, Hussam Amrouch
benchmark similarity 0.58
Abstract · 11 pages

Neural network (NN)-based transistor models have proven to be a promising solution to accelerate device modeling. Although these models demonstrated remarkable speedup in circuit simulations, they were not rigorously tested in complex tasks within standard EDA tool flows for chip design, which require very high transistor model precision to provide accurate results and converge properly. To investigate their full potential and promise, we develop NN-based transistor models that accurately predict the drain current and various charges for a 3nm nanosheet, and employ the models for a standard cell library characterization. This enables us to evaluate the accuracy and speedup under a very wide range of SPICE simulations, benchmarking our models against an industry-standard implementation. Since standard cell library characterization involves thousands of SPICE simulations, it serves as a great benchmark for transistor model quality and consistency. To further evaluate model accuracy, we also perform analog- and digital-level simulations for various complex circuits. Our experiments reveal that NN-based models are a valid alternative to existing transistor compact models, capable of producing accurate delay and power estimates with a much higher speed and convergence time compared to the conventional Verilog-A-based industry-standard (BSIM-CMG) transistor compact model. Our NNs achieve sub-0.11% and 0.76% errors for delay and total power evaluation on a wide range of circuits, while taking up to 8 times less time during the standard cell library characterization. Additionally, we provide comprehensive information and guidance on NN accuracy, unveiling the relationship between NN size and its effect on circuit simulation precision.

0.35

PINND: Physics Informed Neural Networks for Solving the Drift-Diffusion Equations

cs.LG ·2026-10-23 ·Luke Ragan, Matteo G. C. Alasio, Enrico Bellotti, Luca Dal Negro
similarity 0.58
Abstract

The drift-diffusion equations are the fundamental building block for the modeling, design, and optimization of modern semiconductor devices. Solutions of the drift-diffusion equations rarely give way to analytical forms, and exhibit strong nonlinearity, with the solutions ranging over several orders of magnitude. These attributes have led to research into numerical solvers, and while many commercial Technology Computer Aided Design (TCAD) packages now exist, they may require long simulation times, exhibit sensitivity to initial guesses, and rely on mesh generation. Further, these solvers are not well suited for inverse problems where the design parameters of a device are retrieved from its desired functional characteristics. To address all these challenges, this work demonstrates the usage of Physics Informed Neural Networks for the Drift-diffusion (PINND) equations with errors comparable to traditional simulation software using advanced training techniques to solve problems beyond the well-behaved equilibrium regime. For this study, we specifically choose an abrupt PN-junction in both one and two dimensions that features extreme gradients of the carrier concentrations at the Ohmic contacts and an abrupt discontinuity in the doping profile. Using the proposed PINN architecture, we are able to retrieve accurate results for all observables under all bias conditions. Additionally, we apply the PINN solver towards the inverse problem of retrieving doping profiles directly from desired electric field distributions or electrostatic potentials under non-zero applied biases. Our work establishes a powerful PINNs-based approach for semiconductor device modeling with the ability to solve inverse problems under non-equilibrium conditions for microelectronics research and discovery.

0.27

An all-transistor physical-learning transformer can be trained to perform in-context learning

cs.LG cond-mat.soft ·2026-10-08 ·Michael C. Abbott, Benjamin B. Machta
similarity 0.57
Abstract · 18 pages, 11 figures, plus appendices

Most systems trained by equilibrium propagation or coupled learning are in the same class as multi-layer perceptrons: while they may be universal function approximators, they struggle to learn associative tasks which are easy for transformers. Equilibrium propagation relies on a bidirectional flow of information through the physical system between input and output. The reverse flow is broken by more complex nonlinear systems, such as those performing multiplication which is needed to express dot-product attention. Here we explicitly restore this bidirectional flow by introducing extra circuit elements for this purpose into a double or dyadic circuit. We then use these ideas to construct an attention mechanism exploiting the exponential current law of sub-threshold MOSFET devices. We show that in simulations this circuit can be trained to perform a simple in-context learning task, despite using realistic reverse components which transmit only an approximate gradient. For this training to succeed we need cross-entropy loss (not mean squared error). We generate nudges appropriate for this loss using a novel floating circuit element, based on the same MOSFET sub-circuit used for softmax. We train the resistors in this circuit using the Lion optimiser, which can be implemented using one capacitor per parameter to store local memory of past gradient estimates.

0.26

A Novel Algorithm to Simulate Conductance-based Neuronal Networks

eess.SY ·2026-11-06 ·Amir Shahhosseini, Rodolphe Sepulchre
theory similarity 0.57
Abstract

Conductance-based models are central to neurophysiology and neuromorphic engineering, yet their simulation scales poorly to large networks. We study and analyze an iterative solver that operates in signal space. Each iteration reduces to $N$ decoupled scalar linear time-varying differential equations, one per neuron, which are solved in parallel and in closed form. The solver admits two complementary interpretations: a variable metric forward-backward splitting method, rooted in operator theory, and a waveform relaxation quasi-Newton method, rooted in circuit simulation. We prove uniform and superlinear convergence, with a bound that reflects the conductance structure of the dynamics. The algorithm is illustrated on the classical Hodgkin--Huxley neuron and on networks of up to 5000 neurons.

0.28

Analog-Friendly Predictive Coding without Activation Derivatives

cs.LG cs.AIcs.NE ·2026-09-26 ·Francesco Innocenti
similarity 0.54
Abstract · 20 pages, 14 figures

Predictive coding (PC) is a local, energy-based alternative to backpropagation (BP) whose iterative inference dynamics make it attractive for implementation on analog hardware. However, standard nonlinear PC requires evaluating the derivative of the activation function during both inference and learning, which can be difficult to realise physically. Here, we introduce \textit{activation-matched Bregman PC}, replacing standard squared-error energies with Bregman divergences matched to the activation function. This formulation eliminates activation derivatives and, when combined with inference via mirror descent, yields local inference and learning rules requiring only weighted sums, local prediction errors, state integration, and the activation function. In digital experiments, Bregman PC performs comparably to standard PC and BP on classification and generative tasks, while preserving characteristic learning dynamics of PC and its convergence to BP under stable large-model parameterisations. These results provide a more analog-friendly formulation of nonlinear PC while retaining its key computational properties.

0.30

SPA: Structure-Preserving Approximation of Gated Activations under FHE

cs.CR ·2026-10-23 ·Rui Li, Wenyuan Wu, Weijie Miao
similarity 0.53
Abstract

Nonlinear activation evaluation remains a major bottleneck in neural inference under fully homomorphic encryption (FHE). FHE supports efficient addition and multiplication, but lacks native operations for exponential functions, reciprocals, and value-dependent selection. High-degree polynomial approximations on fixed intervals are therefore commonly used to replace nonlinear activations. They can achieve small errors but often incur substantial multiplicative depth and latency. We observe that several widely used nonlinear activations share the algebraic structure $A(x)=xg(x)$ with $g(-x)=1-g(x)$. This class includes GELU, SiLU, fixed-parameter Swish, and the scalar nonlinear branches of GEGLU/SwiGLU. We propose \SPA{} (\emph{Structure-Preserving Approximation}) to exploit this shared structure. \SPA{} retains the exact odd component $x/2$ and approximates only the even residual as a polynomial in $x^2$, yielding a comparison-free arithmetic kernel. This construction removes redundant odd degrees of freedom and enables lower-cost polynomial evaluation at a given error target. Plaintext experiments with Gemma-4-E2B and Llama-3.2-3B show that SPA largely preserves downstream task performance without weight updates. Fixed-interval CKKS benchmarks show favorable error--latency trade-offs. On $[-5,5]$, \SPA{}-balanced achieves a $3.25\times$ speedup over MOAI (ICLR'~26) for erf-GELU with comparable error. On the same interval, \SPA{}-accurate achieves $79.22\%$ lower root mean squared error (RMSE) and a $10.40\times$ speedup over Euston (S\&P'~26) for QuickGELU. For SiLU, \SPA{}-accurate achieves $19.02\%$ lower RMSE and a $2.06\times$ speedup over THOR (CCS'~25).

0.24

NN-Based FeFET Compact Modeling for SPICE Simulation

cs.ET ·2026-11-04 ·Rodion Novkin, Simon Thomann, Hussam Amrouch
similarity 0.53
Abstract · 7 pages

Prototyping and early-stage circuit evaluation for emerging transistor technologies require fast and accurate compact models for SPICE simulations. Standard physics-based compact models, such as BSIM, offer high fidelity but demand extensive domain expertise and time-consuming equation formulation for every new physical effect, significantly slowing technology development. Neural network (NN)-based models offer a scalable alternative approach, but they rely heavily on smooth numerical solver data with explicit high-order derivative information. However, measurement data suffers from complex noise patterns, making derivative extraction challenging and complicating NN-based transistor modeling. This work presents a derivative-free NN training framework employing spectral normalization to naturally smooth derivative behavior and filter measurement noise. Using drain current measurements from a fabricated 28 nm ferroelectric device, we design an NN-based compact model that captures the full spectrum of polarization amplitudes and supports dynamic polarization switching in transient simulations. To demonstrate its capability, we deploy the model in a commercial SPICE software to simulate write and read operations for large-scale memory arrays with up to 512x512 size, including parasitic effects. Our approach bridges raw measurements with NN-based SPICE-compatible transistor modeling, significantly reducing model development time and enabling rapid circuit-level prototyping for emerging devices.

0.11