CS Paper Radar

Low-Rank Kolmogorov-Arnold Generator Flows for Interpretable Operator Learning

Minjong Cheon

cs.LG physics.comp-ph ·submitted 2026-10-09 05:37 UTC · v1

Abstract

Neural operators that learn physical laws from data should generalize beyond their training distribution and expose the learned law in closed form. The Kolmogorov-Arnold neural operator achieves both for position-dependent dynamics by learning a dense pseudo-differential symbol over all positions and frequencies, at a cost that grows with the square of the resolution and with closed forms that must be read out by hand. This dense symbol is unnecessary: a finite-order variable-coefficient differential operator has a symbol whose separation rank is bounded by the number of its coefficient functions, independently of the resolution, and the symbols that this network learns on its own benchmarks have rank two. We introduce the Kolmogorov-Arnold flow (KAFlow), which learns the generator of a physical operator as a low-rank symbol built from univariate Kolmogorov-Arnold functions, fixes the gauge of these functions so that every term has a unique place, integrates the dynamics with exact exponential integrators, and distills its closed form automatically. On the synthetic benchmarks of the Kolmogorov-Arnold neural operator, KAFlow lowers the out-of-distribution loss by 3.4 to 11 times and recovers every coefficient to within 10^-4 without symbolic templates, reaching the precision floor of the data. On long-horizon quantum dynamics, it lowers the state infidelity by up to 6 orders of magnitude, and by 30 times when trained only on position and momentum measurements. KAFlow also learns a variable-order fractional operator 15 times more accurately than existing operator networks and discovers nonlocal, fractional and non-polynomial dispersion relations on which sparse regression fails.

Closest papers

Nearest neighbours by meaning across the whole index.

Tractable Inference with Kolmogorov-Arnold Networks

cs.LG physics.comp-phmath.NA ·2026-10-21 ·Juan Diego Toscano, Sascha Ranftl, Zhiwei Gao, Li-Lian Wang, George Em Karniadakis
similarity 0.63
Abstract

Machine-learning surrogates in computational science often require not only accurate predictions but also expectations, variances, conserved integrals, and normalization constants. Conventional neural networks, including Kolmogorov--Arnold networks (KANs), generally evaluate these quantities through Monte Carlo sampling or multidimensional quadrature, making repeated inference costly, particularly in high dimensions. We introduce KANs with Tractable Operators for Uncertainty and Constraints (TOUCKANs), whose finite-degree polynomial outer functions yield a decomposable sum--product representation. Under factorizing input measures, expectations and higher moments then reduce algebraically to deterministic recursions over one-dimensional leaf integrals. Independent and shared inner-block variants form a parameter-efficiency hierarchy, and a universal-approximation proof for its most restricted member extends to the complete class. The same deterministic operators allow conserved integrals and normalization to be imposed throughout training. Across uniform and Gaussian benchmarks through 50 dimensions, deterministic mean and variance errors remained below the median fixed-budget Monte Carlo errors in every case. TOUCKANs also conserved Cahn--Hilliard mass to machine precision and imposed normalization and orthogonality through deterministic moments, enabling joint computation of the lowest four Schrödinger eigenpairs in two, three, and five dimensions. TOUCKANs therefore connect KAN approximation with tractable probabilistic circuits, making statistical operators and integral constraints directly available within an expressive neural surrogate.

0.20

Unrolled gradients in disguise: bridging interpolation-based and Jacobian regularization for stable neural dynamics

cs.LG math.NA ·2026-11-02 ·Maya Janvier, Etienne Meunier
top venue similarity 0.62
Abstract · Accepted at NeurIPS 2026

Hybrid models and neural differential equations (NDEs) are becoming increasingly important for modelling physical systems; however, they often encounter stability and accuracy issues during long-term integration. Training on unrolled trajectories is known to limit these divergences but quickly becomes too computationally expensive due to the need to compute gradients over an iterative process, making this method impractical for large-scale physical systems (e.g. weather and climate forecasting). Two popular paradigms for improving the stability of learned dynamics are to train on interpolated or noisy samples or to regularize the Jacobian of the learned representation. In this work, we demonstrate that these paradigms can be understood under a common lens and that they both correspond to an approximation of the gradients obtained in an unrolled learning strategy. Building on this understanding, we derive a principled and complete approach to regularize training and demonstrate its impact on the stability of learned dynamical systems.

0.39

HYDRA: Hyperbolic Dynamic Representation Architecture for Kolmogorov-Arnold Networks

cs.LG cs.AI ·2026-08-12 ·Zhao Su, Yuxin Xia, Haoran Li, Jun Shen, Qi Zhu +2 more
benchmark similarity 0.62
Abstract

Kolmogorov-Arnold Networks (KANs) enhance nonlinear function approximation by replacing scalar weights with learnable univariate functions. However, assigning an independent function to every connection results in substantial parameter redundancy, limiting their scalability and efficiency. To reduce this redundancy, we introduce \textbf{HY}perbolic \textbf{D}ynamic \textbf{R}epresentation \textbf{A}rchitecture (HYDRA), a parameter-efficient hyperbolic extension of KAN that combines spline-based functional learning with representations in the Poincaré ball. HYDRA maps vector-valued inputs into a bounded hyperbolic latent space, performs KAN-style updates in tangent space, and employs a low-rank prototype block to share functional transformations across hidden dimensions. The resulting hyperbolic representations provide a structured radial coordinate for interpretation, while radius control improves training stability by preventing boundary saturation. Extensive experiments across eight benchmark datasets demonstrate that HYDRA consistently achieves competitive or superior predictive performance while improving parameter efficiency and representation interpretability.

0.04

FHKAN: Fractional Hermite Kolmogorov-Arnold Networks for Data-Driven and Physics-Informed Learning

cs.LG ·2026-10-18 ·Arsha Farnoush, Amirhossein Sadr, Aida Pakniyat, Dara Rahmati, Saeid Gorgin +1 more
similarity 0.62
Abstract

Kolmogorov-Arnold Networks (KANs) replace fixed scalar weights by learnable univariate maps, yet most spectral KANs still rely on Chebyshev polynomials that live on a bounded interval and therefore require a saturating squashing of the pre-activation. We introduce the Fractional Hermite Kolmogorov-Arnold Network (FHKAN), a neural architecture that equips KANs with an adaptive, trainable fractional-order generalized Hermite basis. With a nonnegative shape parameter, the underlying generalized Hermite functions are orthonormal on the entire real line and uniformly bounded under a Gaussian envelope, so they act directly on unbounded pre-activations without any saturating map. We derive closed-form derivatives of the generalized, fractional, and scaled basis away from the origin and determine its order of differentiability at the origin, where the fractional power map creates a singular point. On synthetic regression, FHKAN attains the lowest test error at every network depth among nine fixed activations and the spline-based KAN. On image classification, image denoising, and sentiment analysis, the best-performing fractional Hermite block yields the highest accuracy or reconstruction quality. On physics-informed problems, FHKAN reaches mean absolute errors of order 10^-6 to 10^-5 on the Lane-Emden equation, residuals of order 10^-4 on the Burgers equation, and absolute errors below 10^-4 on a fractional delay differential equation.

0.21

SINO: Scale-Invariant Neural Operator

cs.LG ·2026-09-29 ·Kaichen Ouyang, Chenglei Yu, Chuanrui Wang, Tailin Wu
code/data similarity 0.61
Abstract · 38 pages, 9 figures

In scientific machine learning, physical fields governed by partial differential equations exhibit low-rank structure and scale invariance. When solving equations on coarse grids, missing information leads to the closure problem: modeling unresolved physics to recover lost dynamics. Although closure terms depend on grid resolution, they represent scale-invariant physical laws. A model truly learning physics should capture these mechanisms with low-rank parameterization rather than memorizing grid-specific patterns. Inspired by this, we propose the Scale-Invariant Neural Operator (SINO), which learns on normalized physical scales via a dual-branch architecture operating in spectral and spatial domains. SINO uses bottleneck MLPs to generate continuous convolution kernels, embedding an explicit low-rank inductive bias that concentrates more than 95 percent of variance in 2-3 modes, as validated by PCA across benchmarks, while drastically reducing parameters. This principled design yields 38 times steeper scaling law exponents than FNO, demonstrating superior parameter efficiency. We compare SINO with traditional models (U-Net, DeepONet), Transformer models (Transolver, Oformer, GK-Transformer), and frequency-domain models (FNO, AMFNO, UFNO) on closure problems spanning externally forced Burgers turbulence, decaying Burgers turbulence, KS turbulence, Kolmogorov-forced NS turbulence, and decaying NS turbulence. Experiments show SINO achieves 1.5-38 times error reduction and 2-23 times parameter efficiency over baselines, with superior scaling laws reflecting exceptional data efficiency from principled low-rank design. Code is available at https://github.com/AI4Science-WestlakeU/SINO.

0.43

Data-driven Koopman mode approximation: A neural power iteration algorithm

eess.SY cs.LGmath.OC ·2026-08-27 ·Guillaume O. Berger, Raphaël M. Jungers
theory similarity 0.60
Abstract

This paper proposes a novel data-driven algorithm to approximate the dominant eigenfunctions (aka.~modes) of the Koopman operator of nonlinear dynamical systems using neural networks. The relevance of learning the dominant Koopman modes is to approximate nonlinear dynamics by linear ones in a lifted space, thereby enabling simplified control and analysis. To fight the curse of dimensionality arising from using expressive templates (here neural networks) for the mode approximation, the proposed method leverages a power-iteration scheme that directly learns the dominant Koopman modes without explicitly constructing the projection of the Koopman operator on the template of functions. Our approach connects to other approaches in the literature that avoid the curse of dimensionality by learning small dictionaries of functions, but differs from them in that we do not require ``anti-collapse mechanisms'' to ensure that the learned dictionary is expressive enough to approximate the Koopman operator since our power-iteration scheme is designed to converge toward the dominant modes of the projected Koopman operator. The approach is fully data-driven, requiring only sampled state transitions. Theoretical guarantees are provided, showing convergence under increasing sample size and network width (in connection with the neural tangent kernel theorem). Numerical experiments demonstrate that the method achieves accurate and smooth approximations of dominant modes while avoiding the limitations of traditional techniques such as extended dynamic mode decomposition.

0.24

SW-KAN: Kolmogorov-Arnold Networks with Stieltjes-Wigert q-Orthogonal Polynomials

cs.LG cs.AI ·2026-09-03 ·Amirhosein Azarpour, Seyyed Moein Kazemi
code/data similarity 0.60
Abstract · 22 pages, Code and pretrained models available at: https://github.com/amirhoseinazarpour/SW-KAN

Kolmogorov-Arnold Networks (KANs) represent a paradigmatic shift in deep learning by replacing fixed node activations with learnable univariate functions on edges, offering enhanced interpretability and parameter efficiency. While recent polynomial-based KAN variants have addressed the computational overhead of original B-spline implementations, they introduce a fundamental yet underexplored challenge: the domain mismatch between unbounded real-valued inputs and the bounded or semi-infinite support of orthogonal polynomial bases. To address this limitation, we propose the Stieltjes-Wigert Kolmogorov-Arnold Network (SW-KAN), a novel architecture that employs Stieltjes-Wigert q-orthogonal polynomials defined on the semi-infinite domain (0, infinity). We introduce a smooth exponential-of-tanh mapping that stably bridges the domain gap while preserving well-conditioned gradients, and leverage a numerically stable three-term recurrence that evaluates polynomial expansions in O(N) operations without special-function calls. Through comprehensive experiments spanning image classification and continuous function approximation, we demonstrate that SW-KAN achieves superior accuracy-efficiency trade-offs across diverse tasks. The log-normal weight structure and learnable q-parameter of Stieltjes-Wigert polynomials provide a distinct inductive bias that enables robust performance under resource-constrained conditions, including reduced feature dimensionality and limited training data. The proposed architecture not only outperforms established polynomial KAN baselines on standard benchmarks but also exhibits strong representational capacity for approximating complex multivariate functions with remarkably few parameters, making it a compelling alternative for efficient function approximation and classification in resource-constrained settings.

0.43

Stabilizing Neural Emulation of Spatiotemporal Physical Dynamics through Denoising Koopman Latent Dynamics

cs.CE ·2026-11-03 ·Chenxu Wu, Sixing Chen, Junming Hou, Jiahan Huang, Jianqiao Zeng +3 more
hot direction similarity 0.60
Abstract · 21 pages,12 figures

Koopman operator theory provides a principled framework for reduced-order modeling of nonlinear spatiotemporal dynamics through linear evolution in an observable space. In practice, however, learned finite-dimensional representations are typically approximately invariant, causing unresolved dynamics and accumulated closure errors. Recent approaches, inspired by the Mori-Zwanzig formalism, address these effects through deterministic memory or closure modeling, yet robust extrapolation beyond the training horizon remains challenging. Here, we propose KoopFlow, an efficient prediction-correction framework based on the key insight that finite-dimensional Koopman error accumulation can be reinterpreted as a progressive distributional degradation process. Accordingly, KoopFlow treats Koopman predictions as noisy intermediate estimates, which are then transported toward the target latent distribution through a generative model, thereby mitigating accumulated errors while retaining computational efficiency and temporal flexibility. Moreover, we reveal a non-monotonic relationship between Koopman spectral coherence and predictive accuracy. Experiments across diverse nonlinear dynamical systems demonstrate improved long-horizon prediction accuracy and stability over representative baselines, particularly beyond the training horizon.

0.27