CS Paper Radar

Kolmogorov-Arnold Networks: Curse of Dimensionality and Structural Bias

Yunhao Ni, Yuxin Guo, Jie Luo, Wenjun Wu, Lei Huang

cs.LG ·submitted 2026-10-15 08:21 UTC · v1
theory

Abstract

Kolmogorov-Arnold networks (KANs) have a dimension-independent approximation exponent when the target admits a sufficiently smooth representation by univariate edge functions. We ask whether this favorable scaling belongs to the architecture or to the assumed target class. For arbitrary-depth standard spline-KANs, we prove a worst-case lower bound on isotropic Sobolev balls: below a dimension-independent accuracy threshold, the required number of spline coefficients grows at least exponentially with input dimension. Existing composition-matched MLP results, by contrast, show that favorable exponents on low-arity targets are not exclusive to KANs. We also revisit empirical comparisons. On four scaling-law targets, spline-KAN approximants constructed from known decompositions transfer exactly to MLPs with the cubic ReLU activation $σ(x)=(\max\{0,x\})^3$, although their full dense parameter counts are larger. In controlled ridge-profile scans, KANs have a strong advantage when each nonlinear component depends on one input coordinate. When a component couples multiple input coordinates, this advantage may narrow sharply. At tight budgets, interacting targets often favor MLPs. In the random-label test on MNIST, ReLU-MLPs fit the randomized training labels more efficiently than spline-only KANs. Together, these results locate favorable scaling in target structure and a compatible inductive bias, not a general KAN-specific escape from the curse of dimensionality.

Comment
25 pages

Closest papers

Nearest neighbours by meaning across the whole index.

Geometry-Constrained Kolmogorov-Arnold Networks: Learning Edge Geometry via Banach Duality

cs.LG stat.ML ·2026-08-26 ·K S Sesh Kumar
top venuecode/data similarity 0.67
Abstract · Accepted at NeurIPS 2026 Code: https://github.com/seshkumar/geometric-kan-neurips2026

Kolmogorov-Arnold Networks (KANs) learn univariate functions on edges, making the choice of edge response central to their design. Existing variants use basis expansions such as splines or polynomials, whose functional form is chosen before training. We introduce geometry-constrained KANs, which instead learn the shape of power-law edge responses through an exponent inspired by Banach duality maps. Banach-KAN combines this adaptive power-law response with a bounded hyperbolic tangent (tanh) component. We relate these responses to convex potentials and bound edge sensitivity on bounded input ranges. Across symbolic regression benchmarks, Banach-KAN is competitive with fixed-basis models, particularly with limited clean data. Learned power-law responses remain competitive under measurement noise, while splines are stronger with plentiful clean data. Fixed-exponent and branch-removal controls on a core subset support learning the exponent and retaining both Banach components. Learned exponents describe the shapes of the fitted edge responses.

0.43

Neural scaling laws and evolution of learnable activation functions of Kolmogorov-Arnold networks

cs.LG ·2026-10-01 ·Tilen Cadez, Sanghoon Lee, Kyoung-Min Kim
similarity 0.67
Abstract · 13 pages, 10 figures. Supplementary Notes will be provided in the published version

Kolmogorov-Arnold Networks (KANs) represent a compelling alternative to traditional Multi-Layer Perceptron (MLP)-based neural networks. By employing activation functions as learnable elements, KANs offer superior interpretability, making them suited for scientific domains. In this work, we investigate the neural scaling laws of KANs and the structural evolution of their learnable activation functions under dataset expansion. Specifically, we evaluate the scaling behavior of three KAN variants---BSRBF-KAN, Gottlieb-KAN, and Faster-KAN---across standard image classification benchmarks (MNIST and Fashion-MNIST) and a specialized scientific regression task (magnetic parameter estimation from domain images of moiré magnetic textures). Our results demonstrate that the test loss ${\cal L}$ exhibits a broken neural scaling law (BNSL) behavior as a function of the dataset size $N_D$. After passing through a random-guess regime, the loss follows architecture- and task-dependent scaling behavior. The loss crosses from a faster- to a slower-scaling branch, ${\cal L}\propto N_D^{-α}$ and ${\cal L}\propto N_D^{-β}$ with $α>β$ for image classification tasks. The exponents $α$ and $β$ depend strongly on both the specific network architecture and the dataset-size regime, ranging from 0.4 to 1.5 and from 0.06 to 0.6, respectively. For the magnetic parameter-regression task, the loss follows a single scaling law with its exponent ranging from 1.28 to 2.59. Additionally, we provide a structural analysis of how activation functions refine their complexity as data volume increases, finding that dataset expansion drives a transition from simple linear-like approximations toward stable, interpretable symbolic forms. These findings provide a quantitative roadmap for the efficient application of KANs while managing the trade-off between model expressivity and computational overhead.

0.41

HYDRA: Hyperbolic Dynamic Representation Architecture for Kolmogorov-Arnold Networks

cs.LG cs.AI ·2026-08-12 ·Zhao Su, Yuxin Xia, Haoran Li, Jun Shen, Qi Zhu +2 more
benchmark similarity 0.66
Abstract

Kolmogorov-Arnold Networks (KANs) enhance nonlinear function approximation by replacing scalar weights with learnable univariate functions. However, assigning an independent function to every connection results in substantial parameter redundancy, limiting their scalability and efficiency. To reduce this redundancy, we introduce \textbf{HY}perbolic \textbf{D}ynamic \textbf{R}epresentation \textbf{A}rchitecture (HYDRA), a parameter-efficient hyperbolic extension of KAN that combines spline-based functional learning with representations in the Poincaré ball. HYDRA maps vector-valued inputs into a bounded hyperbolic latent space, performs KAN-style updates in tangent space, and employs a low-rank prototype block to share functional transformations across hidden dimensions. The resulting hyperbolic representations provide a structured radial coordinate for interpretation, while radius control improves training stability by preventing boundary saturation. Extensive experiments across eight benchmark datasets demonstrate that HYDRA consistently achieves competitive or superior predictive performance while improving parameter efficiency and representation interpretability.

0.04

Kolmogorov-Arnold Classifier Systems as Universal Approximators

cs.LG cs.NE ·2026-09-29 ·Hiroki Shiraishi, Hisao Ishibuchi, Masaya Nakata
top venuetheorycode/data similarity 0.62
Abstract

As the input dimension $n$ grows, rule-based machine learning, such as Learning Classifier Systems (LCSs), faces a fundamental scalability bottleneck for function approximation: both rule count and parameter count grow exponentially with $n$. Traditional LCSs partition the $n$-dimensional input space directly, requiring $\mathcal{O}(m^n)$ rules for adequate coverage, where $m$ is the per-variable resolution. This article breaks from this paradigm by reorganizing rules dimension-wise, guided by the Kolmogorov-Arnold representation theorem: any continuous $n$-dimensional function can be expressed as a finite superposition of one-dimensional functions. The proposed Kolmogorov-Arnold Classifier System (KACS) decomposes the target function into one-dimensional subproblems and assigns a dedicated ruleset to each, reducing the worst-case rule count from $\mathcal{O}(m^n)$ to $\mathcal{O}(mn^2)$ and replacing $n$-dimensional local models with one-dimensional models requiring only two parameters per rule, independent of $n$. We also provide the first constructive proof that an LCS, namely KACS, is a universal approximator for continuous functions on compact domains. Evaluated against a direct $n$-dimensional input space partitioning approach under otherwise identical conditions, KACS achieves competitive accuracy in many settings while using only 2\% to 40\% of the parameters. Our implementation is available at https://github.com/YNU-NakataLab/KACS.

0.58

Shallow neural network approximation in mixed Sobolev spaces

cs.LG math.NA ·2026-09-04 ·Yuwen Li, Guozhi Zhang
theory similarity 0.62
Abstract · 40 pages, 2 figures

We investigate the best $L_2$ approximation of mixed Sobolev spaces by shallow neural networks with $n$ neurons and general activation functions. We first establish an activation-independent Fourier-block principle: if an activation has univariate approximation order $ρ$ in the sense of the Fourier-block property, then the global approximation rate has algebraic order $\min\{α,ρ\}$ for target functions of mixed smoothness $α$, up to explicit logarithmic factors. To verify this property for concrete activations, we introduce a structured univariate approximation condition that implies the Fourier-block property with explicit parameters. For $\mathrm{ReLU}^k$, a matching algebraic lower bound identifies $\min\{α,k+1\}$ as the optimal algebraic approximation exponent in any dimension, up to logarithmic factors in the upper bound. The framework also yields the exponent $\min\{α,k+1\}$ for cardinal B-splines and soft-$\mathrm{ReLU}^k$, and the full mixed-smoothness exponent $α$ for ELU and cosine activations, again up to logarithmic~factors.

0.19

FHKAN: Fractional Hermite Kolmogorov-Arnold Networks for Data-Driven and Physics-Informed Learning

cs.LG ·2026-10-18 ·Arsha Farnoush, Amirhossein Sadr, Aida Pakniyat, Dara Rahmati, Saeid Gorgin +1 more
similarity 0.62
Abstract

Kolmogorov-Arnold Networks (KANs) replace fixed scalar weights by learnable univariate maps, yet most spectral KANs still rely on Chebyshev polynomials that live on a bounded interval and therefore require a saturating squashing of the pre-activation. We introduce the Fractional Hermite Kolmogorov-Arnold Network (FHKAN), a neural architecture that equips KANs with an adaptive, trainable fractional-order generalized Hermite basis. With a nonnegative shape parameter, the underlying generalized Hermite functions are orthonormal on the entire real line and uniformly bounded under a Gaussian envelope, so they act directly on unbounded pre-activations without any saturating map. We derive closed-form derivatives of the generalized, fractional, and scaled basis away from the origin and determine its order of differentiability at the origin, where the fractional power map creates a singular point. On synthetic regression, FHKAN attains the lowest test error at every network depth among nine fixed activations and the spline-based KAN. On image classification, image denoising, and sentiment analysis, the best-performing fractional Hermite block yields the highest accuracy or reconstruction quality. On physics-informed problems, FHKAN reaches mean absolute errors of order 10^-6 to 10^-5 on the Lane-Emden equation, residuals of order 10^-4 on the Burgers equation, and absolute errors below 10^-4 on a fractional delay differential equation.

0.21

SW-KAN: Kolmogorov-Arnold Networks with Stieltjes-Wigert q-Orthogonal Polynomials

cs.LG cs.AI ·2026-09-03 ·Amirhosein Azarpour, Seyyed Moein Kazemi
code/data similarity 0.62
Abstract · 22 pages, Code and pretrained models available at: https://github.com/amirhoseinazarpour/SW-KAN

Kolmogorov-Arnold Networks (KANs) represent a paradigmatic shift in deep learning by replacing fixed node activations with learnable univariate functions on edges, offering enhanced interpretability and parameter efficiency. While recent polynomial-based KAN variants have addressed the computational overhead of original B-spline implementations, they introduce a fundamental yet underexplored challenge: the domain mismatch between unbounded real-valued inputs and the bounded or semi-infinite support of orthogonal polynomial bases. To address this limitation, we propose the Stieltjes-Wigert Kolmogorov-Arnold Network (SW-KAN), a novel architecture that employs Stieltjes-Wigert q-orthogonal polynomials defined on the semi-infinite domain (0, infinity). We introduce a smooth exponential-of-tanh mapping that stably bridges the domain gap while preserving well-conditioned gradients, and leverage a numerically stable three-term recurrence that evaluates polynomial expansions in O(N) operations without special-function calls. Through comprehensive experiments spanning image classification and continuous function approximation, we demonstrate that SW-KAN achieves superior accuracy-efficiency trade-offs across diverse tasks. The log-normal weight structure and learnable q-parameter of Stieltjes-Wigert polynomials provide a distinct inductive bias that enables robust performance under resource-constrained conditions, including reduced feature dimensionality and limited training data. The proposed architecture not only outperforms established polynomial KAN baselines on standard benchmarks but also exhibits strong representational capacity for approximating complex multivariate functions with remarkably few parameters, making it a compelling alternative for efficient function approximation and classification in resource-constrained settings.

0.43

Approximating Smooth Functionals with ReLU Networks

cs.LG stat.ML ·2026-09-14 ·Shuhao Jiao
theory similarity 0.61
Abstract

We study the uniform approximation of smooth scalar-valued functionals on an infinite-dimensional separable Hilbert space by ReLU neural networks. A key feature in deep learning for functional data is the varying importance of different coordinates/dimensions. Representing the functional input in a basis expansion, we quantify the importance of each coordinate through both the magnitude of its corresponding basis score and the directional sensitivity of the target functional. Our analysis combines coordinate truncation, anisotropic partitioning, local Taylor approximation, and ReLU network realization, while allowing unrestricted interactions among the retained coordinates. We establish a general nonasymptotic upper bound for the uniform approximation error and a complementary pseudo-dimension-based lower bound for the worst-case approximation error. Under generalized exponential coordinate decay $w_ds_d\asymp\exp(-cd^ρ)$, with $ρ>0$, the upper and lower bounds match at the leading order, which is stretched-exponential in the logarithm of the network size budget, and thus yield the nearly optimal approximation rate. This is the first work to characterize neural network approximation error for infinite-dimensional functional inputs explicitly through the joint dimensional decay of coordinate magnitudes and directional sensitivities.

0.15