CS Paper Radar

HYDRA: Hyperbolic Dynamic Representation Architecture for Kolmogorov-Arnold Networks

Zhao Su, Yuxin Xia, Haoran Li, Jun Shen, Qi Zhu, Qingguo Zhou, Binbin Yong

cs.LG cs.AI ·submitted 2026-08-12 15:48 UTC · v1
benchmark

Abstract

Kolmogorov-Arnold Networks (KANs) enhance nonlinear function approximation by replacing scalar weights with learnable univariate functions. However, assigning an independent function to every connection results in substantial parameter redundancy, limiting their scalability and efficiency. To reduce this redundancy, we introduce \textbf{HY}perbolic \textbf{D}ynamic \textbf{R}epresentation \textbf{A}rchitecture (HYDRA), a parameter-efficient hyperbolic extension of KAN that combines spline-based functional learning with representations in the Poincaré ball. HYDRA maps vector-valued inputs into a bounded hyperbolic latent space, performs KAN-style updates in tangent space, and employs a low-rank prototype block to share functional transformations across hidden dimensions. The resulting hyperbolic representations provide a structured radial coordinate for interpretation, while radius control improves training stability by preventing boundary saturation. Extensive experiments across eight benchmark datasets demonstrate that HYDRA consistently achieves competitive or superior predictive performance while improving parameter efficiency and representation interpretability.

Closest papers

Nearest neighbours by meaning across the whole index.

Geometry-Constrained Kolmogorov-Arnold Networks: Learning Edge Geometry via Banach Duality

cs.LG stat.ML ·2026-08-26 ·K S Sesh Kumar
top venuecode/data similarity 0.68
Abstract · Accepted at NeurIPS 2026 Code: https://github.com/seshkumar/geometric-kan-neurips2026

Kolmogorov-Arnold Networks (KANs) learn univariate functions on edges, making the choice of edge response central to their design. Existing variants use basis expansions such as splines or polynomials, whose functional form is chosen before training. We introduce geometry-constrained KANs, which instead learn the shape of power-law edge responses through an exponent inspired by Banach duality maps. Banach-KAN combines this adaptive power-law response with a bounded hyperbolic tangent (tanh) component. We relate these responses to convex potentials and bound edge sensitivity on bounded input ranges. Across symbolic regression benchmarks, Banach-KAN is competitive with fixed-basis models, particularly with limited clean data. Learned power-law responses remain competitive under measurement noise, while splines are stronger with plentiful clean data. Fixed-exponent and branch-removal controls on a core subset support learning the exponent and retaining both Banach components. Learned exponents describe the shapes of the fitted edge responses.

0.43

SW-KAN: Kolmogorov-Arnold Networks with Stieltjes-Wigert q-Orthogonal Polynomials

cs.LG cs.AI ·2026-09-03 ·Amirhosein Azarpour, Seyyed Moein Kazemi
code/data similarity 0.67
Abstract · 22 pages, Code and pretrained models available at: https://github.com/amirhoseinazarpour/SW-KAN

Kolmogorov-Arnold Networks (KANs) represent a paradigmatic shift in deep learning by replacing fixed node activations with learnable univariate functions on edges, offering enhanced interpretability and parameter efficiency. While recent polynomial-based KAN variants have addressed the computational overhead of original B-spline implementations, they introduce a fundamental yet underexplored challenge: the domain mismatch between unbounded real-valued inputs and the bounded or semi-infinite support of orthogonal polynomial bases. To address this limitation, we propose the Stieltjes-Wigert Kolmogorov-Arnold Network (SW-KAN), a novel architecture that employs Stieltjes-Wigert q-orthogonal polynomials defined on the semi-infinite domain (0, infinity). We introduce a smooth exponential-of-tanh mapping that stably bridges the domain gap while preserving well-conditioned gradients, and leverage a numerically stable three-term recurrence that evaluates polynomial expansions in O(N) operations without special-function calls. Through comprehensive experiments spanning image classification and continuous function approximation, we demonstrate that SW-KAN achieves superior accuracy-efficiency trade-offs across diverse tasks. The log-normal weight structure and learnable q-parameter of Stieltjes-Wigert polynomials provide a distinct inductive bias that enables robust performance under resource-constrained conditions, including reduced feature dimensionality and limited training data. The proposed architecture not only outperforms established polynomial KAN baselines on standard benchmarks but also exhibits strong representational capacity for approximating complex multivariate functions with remarkably few parameters, making it a compelling alternative for efficient function approximation and classification in resource-constrained settings.

0.43

Kolmogorov-Arnold Networks: Curse of Dimensionality and Structural Bias

cs.LG ·2026-10-15 ·Yunhao Ni, Yuxin Guo, Jie Luo, Wenjun Wu, Lei Huang
theory similarity 0.66
Abstract · 25 pages

Kolmogorov-Arnold networks (KANs) have a dimension-independent approximation exponent when the target admits a sufficiently smooth representation by univariate edge functions. We ask whether this favorable scaling belongs to the architecture or to the assumed target class. For arbitrary-depth standard spline-KANs, we prove a worst-case lower bound on isotropic Sobolev balls: below a dimension-independent accuracy threshold, the required number of spline coefficients grows at least exponentially with input dimension. Existing composition-matched MLP results, by contrast, show that favorable exponents on low-arity targets are not exclusive to KANs. We also revisit empirical comparisons. On four scaling-law targets, spline-KAN approximants constructed from known decompositions transfer exactly to MLPs with the cubic ReLU activation $σ(x)=(\max\{0,x\})^3$, although their full dense parameter counts are larger. In controlled ridge-profile scans, KANs have a strong advantage when each nonlinear component depends on one input coordinate. When a component couples multiple input coordinates, this advantage may narrow sharply. At tight budgets, interacting targets often favor MLPs. In the random-label test on MNIST, ReLU-MLPs fit the randomized training labels more efficiently than spline-only KANs. Together, these results locate favorable scaling in target structure and a compatible inductive bias, not a general KAN-specific escape from the curse of dimensionality.

0.29

Edge-Centric Adaptive Neural Activation via Parametric Tanh Functions

cs.LG cs.NE ·2026-10-25 ·Andreas Falkenberg
similarity 0.63
Abstract · 11 pages, 4 figures

Within the broader landscape of artificial intelligence architectures, this paper introduces a compact, edge-centric neural framework that resolves the computational bottlenecks of modern edge-activated paradigms like Kolmogorov-Arnold Networks (KANs) while retaining their expressiveness over traditional Multi-Layer Perceptrons (MLPs). While KANs demonstrate remarkable representational capacity by placing learnable univariate functions on connection edges rather than hidden nodes, their reliance on non-local B-splines introduces severe optimization hurdles, high memory overhead, and poor hardware scaling. To bridge this gap, we propose the Parametric Tanh Connection Function, a smooth, bounded architectural primitive defined as $a_{\tanh}(x;δ,α,β,γ) = δ+ α\tanh(βx + γ)$. This formulation unifies offset, amplitude, local slope, and horizontal displacement into a lightweight, 4-parameter differentiable mapping optimized edge-by-edge via backpropagation. Evaluated on dense architectures using MNIST and CIFAR-10, our parametric tanh connection model achieves significant absolute accuracy gains over traditional affine baselines-reaching 95.63% $\pm$ 0.08% on MNIST and 44.74% $\pm$ 0.18% on CIFAR-10. Crucially, it matches or exceeds the expressiveness of unconstrained 7-parameter exponential connection baselines while requiring up to 85% less training runtime. By replacing complex splines or hyper-flexible exponents with a bounded, analytically smooth hyperbolic tangent kernel, this work delivers a highly scalable, hardware-friendly compromise for heterogeneous connection-level deep learning.

0.22

KANLab: TensorFlow Playground for Kolmogorov-Arnold Networks

cs.LG ·2026-10-14 ·Lukas Niehaus, Robin Rawiel, Ulf Krumnack
code/data similarity 0.63
Abstract · preprint

KANLab is a browser-based visualization tool designed to interactively teach the core concepts of Kolmogorov-Arnold Networks (KAN), forked from the widely successful TensorFlow Playground. To maximize educational clarity, KANLab omits residual connections and learnable scaling parameters found in modern KAN implementations, ensuring that displayed B-spline activation curves directly correspond to their underlying trainable control points. Users can inspect and modify activation functions, initialization schemes and observe gradient descent correcting manual disruptions. Furthermore, user can explore phenomena such as activation variance, interactive pruning, and re\-/training within a compact 2D machine learning setting. KANLab is open-source and available at https://kanlab.app/

0.43

Low-Rank Kolmogorov-Arnold Generator Flows for Interpretable Operator Learning

cs.LG physics.comp-ph ·2026-10-09 ·Minjong Cheon
similarity 0.62
Abstract

Neural operators that learn physical laws from data should generalize beyond their training distribution and expose the learned law in closed form. The Kolmogorov-Arnold neural operator achieves both for position-dependent dynamics by learning a dense pseudo-differential symbol over all positions and frequencies, at a cost that grows with the square of the resolution and with closed forms that must be read out by hand. This dense symbol is unnecessary: a finite-order variable-coefficient differential operator has a symbol whose separation rank is bounded by the number of its coefficient functions, independently of the resolution, and the symbols that this network learns on its own benchmarks have rank two. We introduce the Kolmogorov-Arnold flow (KAFlow), which learns the generator of a physical operator as a low-rank symbol built from univariate Kolmogorov-Arnold functions, fixes the gauge of these functions so that every term has a unique place, integrates the dynamics with exact exponential integrators, and distills its closed form automatically. On the synthetic benchmarks of the Kolmogorov-Arnold neural operator, KAFlow lowers the out-of-distribution loss by 3.4 to 11 times and recovers every coefficient to within 10^-4 without symbolic templates, reaching the precision floor of the data. On long-horizon quantum dynamics, it lowers the state infidelity by up to 6 orders of magnitude, and by 30 times when trained only on position and momentum measurements. KAFlow also learns a variable-order fractional operator 15 times more accurately than existing operator networks and discovers nonlocal, fractional and non-polynomial dispersion relations on which sparse regression fails.

0.21

FHKAN: Fractional Hermite Kolmogorov-Arnold Networks for Data-Driven and Physics-Informed Learning

cs.LG ·2026-10-18 ·Arsha Farnoush, Amirhossein Sadr, Aida Pakniyat, Dara Rahmati, Saeid Gorgin +1 more
similarity 0.61
Abstract

Kolmogorov-Arnold Networks (KANs) replace fixed scalar weights by learnable univariate maps, yet most spectral KANs still rely on Chebyshev polynomials that live on a bounded interval and therefore require a saturating squashing of the pre-activation. We introduce the Fractional Hermite Kolmogorov-Arnold Network (FHKAN), a neural architecture that equips KANs with an adaptive, trainable fractional-order generalized Hermite basis. With a nonnegative shape parameter, the underlying generalized Hermite functions are orthonormal on the entire real line and uniformly bounded under a Gaussian envelope, so they act directly on unbounded pre-activations without any saturating map. We derive closed-form derivatives of the generalized, fractional, and scaled basis away from the origin and determine its order of differentiability at the origin, where the fractional power map creates a singular point. On synthetic regression, FHKAN attains the lowest test error at every network depth among nine fixed activations and the spline-based KAN. On image classification, image denoising, and sentiment analysis, the best-performing fractional Hermite block yields the highest accuracy or reconstruction quality. On physics-informed problems, FHKAN reaches mean absolute errors of order 10^-6 to 10^-5 on the Lane-Emden equation, residuals of order 10^-4 on the Burgers equation, and absolute errors below 10^-4 on a fractional delay differential equation.

0.21

Temporal Geometry of Deep Networks: Hyperbolic Representations of Training Dynamics for Intrinsic Explainability

cs.LG cs.AIcs.NEphysics.data-an ·2026-10-02 ·Ambarish Moharil
top venue similarity 0.59
Abstract · Published as a main conference paper at ICLR 2026. 10 Main Pages, 22 pages of supplementary material

Intrinsic explainability remains a challenging problem, particularly in contexts where multilayer perceptrons (MLPs) require dynamic re-training within an optimization environment. This paper investigates how MLPs and their training dynamics can be represented and studied in non-Euclidean spaces; our representation features the Poincaré model of hyperbolic geometry. We aim to capture the geometric evolution of their weighted topology and self-organization over time. Instead of restricting the analysis to single checkpoints---as per established measure-based explainability methods---we construct temporal \textit{parameter graphs}, i.e., snapshots over time $T$ steps of the optimization/training process for MLPs. This reflects the view that neural networks encode information not only in their weights but also in the trajectory traced during training. Drawing on the idea that many complex networks admit embeddings in hidden metric spaces where distances correspond to connection likelihood, we present a geometric and temporal graph-based metalearning framework for obtaining dynamic hyperbolic representations of the underlying neural parameter graphs. Our model embeds temporal parameter graphs in the Poincaré model ball, and learns from them while maintaining equivariance to within-snapshot neuron permutations and invariance to permutations of past snapshots. In doing so, the approach preserves functional equivalence over time and recovers the latent evolving geometry of the network. Experiments on regression and classification tasks with trained MLPs show strong meta-network performance, accompanied by hyperbolic temporal representations. This reveals how the network structure emerges over time under specific training environments, thus providing insights into the network's self-organization.

0.55