CS Paper Radar

Edge-Centric Adaptive Neural Activation via Parametric Tanh Functions

Andreas Falkenberg

cs.LG cs.NE ·submitted 2026-10-25 08:39 UTC · v1

Abstract

Within the broader landscape of artificial intelligence architectures, this paper introduces a compact, edge-centric neural framework that resolves the computational bottlenecks of modern edge-activated paradigms like Kolmogorov-Arnold Networks (KANs) while retaining their expressiveness over traditional Multi-Layer Perceptrons (MLPs). While KANs demonstrate remarkable representational capacity by placing learnable univariate functions on connection edges rather than hidden nodes, their reliance on non-local B-splines introduces severe optimization hurdles, high memory overhead, and poor hardware scaling. To bridge this gap, we propose the Parametric Tanh Connection Function, a smooth, bounded architectural primitive defined as $a_{\tanh}(x;δ,α,β,γ) = δ+ α\tanh(βx + γ)$. This formulation unifies offset, amplitude, local slope, and horizontal displacement into a lightweight, 4-parameter differentiable mapping optimized edge-by-edge via backpropagation. Evaluated on dense architectures using MNIST and CIFAR-10, our parametric tanh connection model achieves significant absolute accuracy gains over traditional affine baselines-reaching 95.63% $\pm$ 0.08% on MNIST and 44.74% $\pm$ 0.18% on CIFAR-10. Crucially, it matches or exceeds the expressiveness of unconstrained 7-parameter exponential connection baselines while requiring up to 85% less training runtime. By replacing complex splines or hyper-flexible exponents with a bounded, analytically smooth hyperbolic tangent kernel, this work delivers a highly scalable, hardware-friendly compromise for heterogeneous connection-level deep learning.

Comment
11 pages, 4 figures

Closest papers

Nearest neighbours by meaning across the whole index.

SW-KAN: Kolmogorov-Arnold Networks with Stieltjes-Wigert q-Orthogonal Polynomials

cs.LG cs.AI ·2026-09-03 ·Amirhosein Azarpour, Seyyed Moein Kazemi
code/data similarity 0.67
Abstract · 22 pages, Code and pretrained models available at: https://github.com/amirhoseinazarpour/SW-KAN

Kolmogorov-Arnold Networks (KANs) represent a paradigmatic shift in deep learning by replacing fixed node activations with learnable univariate functions on edges, offering enhanced interpretability and parameter efficiency. While recent polynomial-based KAN variants have addressed the computational overhead of original B-spline implementations, they introduce a fundamental yet underexplored challenge: the domain mismatch between unbounded real-valued inputs and the bounded or semi-infinite support of orthogonal polynomial bases. To address this limitation, we propose the Stieltjes-Wigert Kolmogorov-Arnold Network (SW-KAN), a novel architecture that employs Stieltjes-Wigert q-orthogonal polynomials defined on the semi-infinite domain (0, infinity). We introduce a smooth exponential-of-tanh mapping that stably bridges the domain gap while preserving well-conditioned gradients, and leverage a numerically stable three-term recurrence that evaluates polynomial expansions in O(N) operations without special-function calls. Through comprehensive experiments spanning image classification and continuous function approximation, we demonstrate that SW-KAN achieves superior accuracy-efficiency trade-offs across diverse tasks. The log-normal weight structure and learnable q-parameter of Stieltjes-Wigert polynomials provide a distinct inductive bias that enables robust performance under resource-constrained conditions, including reduced feature dimensionality and limited training data. The proposed architecture not only outperforms established polynomial KAN baselines on standard benchmarks but also exhibits strong representational capacity for approximating complex multivariate functions with remarkably few parameters, making it a compelling alternative for efficient function approximation and classification in resource-constrained settings.

0.43

HYDRA: Hyperbolic Dynamic Representation Architecture for Kolmogorov-Arnold Networks

cs.LG cs.AI ·2026-08-12 ·Zhao Su, Yuxin Xia, Haoran Li, Jun Shen, Qi Zhu +2 more
benchmark similarity 0.63
Abstract

Kolmogorov-Arnold Networks (KANs) enhance nonlinear function approximation by replacing scalar weights with learnable univariate functions. However, assigning an independent function to every connection results in substantial parameter redundancy, limiting their scalability and efficiency. To reduce this redundancy, we introduce \textbf{HY}perbolic \textbf{D}ynamic \textbf{R}epresentation \textbf{A}rchitecture (HYDRA), a parameter-efficient hyperbolic extension of KAN that combines spline-based functional learning with representations in the Poincaré ball. HYDRA maps vector-valued inputs into a bounded hyperbolic latent space, performs KAN-style updates in tangent space, and employs a low-rank prototype block to share functional transformations across hidden dimensions. The resulting hyperbolic representations provide a structured radial coordinate for interpretation, while radius control improves training stability by preventing boundary saturation. Extensive experiments across eight benchmark datasets demonstrate that HYDRA consistently achieves competitive or superior predictive performance while improving parameter efficiency and representation interpretability.

0.04

Geometry-Constrained Kolmogorov-Arnold Networks: Learning Edge Geometry via Banach Duality

cs.LG stat.ML ·2026-08-26 ·K S Sesh Kumar
top venuecode/data similarity 0.62
Abstract · Accepted at NeurIPS 2026 Code: https://github.com/seshkumar/geometric-kan-neurips2026

Kolmogorov-Arnold Networks (KANs) learn univariate functions on edges, making the choice of edge response central to their design. Existing variants use basis expansions such as splines or polynomials, whose functional form is chosen before training. We introduce geometry-constrained KANs, which instead learn the shape of power-law edge responses through an exponent inspired by Banach duality maps. Banach-KAN combines this adaptive power-law response with a bounded hyperbolic tangent (tanh) component. We relate these responses to convex potentials and bound edge sensitivity on bounded input ranges. Across symbolic regression benchmarks, Banach-KAN is competitive with fixed-basis models, particularly with limited clean data. Learned power-law responses remain competitive under measurement noise, while splines are stronger with plentiful clean data. Fixed-exponent and branch-removal controls on a core subset support learning the exponent and retaining both Banach components. Learned exponents describe the shapes of the fitted edge responses.

0.43

Hybrid Parallel Bessel KAN Model: a Kolmogorov Arnold Network: integrated framework for developing a novel architecture to reconstruct the fused-basis approach via a parameter free quadratic interaction

cs.LG ·2026-10-13 ·Mintu Pathak, Meshv Patel, Mohendra Roy, Jwngsar Brahma
similarity 0.59
Abstract

The Kolmogorov-Arnold Network (KAN) challenges traditional MLPs by placing learnable activation functions on network edges. However, scaling KANs to spatial data exposes limitations in default B-Splines. While hybrid architectures mix multiple basis families on a single edge to resolve this, monolithic superposition causes geometric overlap, parameter explosion, and severe numerical ill-conditioning ($κ(G) \approx 10^{16}$).To overcome this, we propose Parallel BesselKAN, which isolates B-Spline and low-order Bessel branches into parallel processing pipelines integrated via a parameter-free quadratic interaction. This decoupled architecture acts as an efficient internal ensemble, bypassing monolithic stability bottlenecks. It compresses the parameter footprint by $35\%$ while achieving $97.99\%$ accuracy on MNIST. Although standard MLPs retain a hardware speed advantage for pure vision tasks, Parallel BesselKAN provides a robust, parameter-efficient framework stabilizing mixed-basis networks and paving the way for scalable spatial adaptations like Convolutional KANs.

0.32

KANLab: TensorFlow Playground for Kolmogorov-Arnold Networks

cs.LG ·2026-10-14 ·Lukas Niehaus, Robin Rawiel, Ulf Krumnack
code/data similarity 0.58
Abstract · preprint

KANLab is a browser-based visualization tool designed to interactively teach the core concepts of Kolmogorov-Arnold Networks (KAN), forked from the widely successful TensorFlow Playground. To maximize educational clarity, KANLab omits residual connections and learnable scaling parameters found in modern KAN implementations, ensuring that displayed B-spline activation curves directly correspond to their underlying trainable control points. Users can inspect and modify activation functions, initialization schemes and observe gradient descent correcting manual disruptions. Furthermore, user can explore phenomena such as activation variance, interactive pruning, and re\-/training within a compact 2D machine learning setting. KANLab is open-source and available at https://kanlab.app/

0.43

TorchCurves: Differentiable Parametric Curves

cs.LG stat.ML ·2026-11-10 ·Alex Shtoff
code/data similarity 0.56
Abstract · Software paper for a library I've been working on for more than a year

We present TorchCurves, a PyTorch library that treats vector-valued parametric curves as learnable primitives. Its central operation maps batches of scalar inputs to points on curves represented in B-spline or Legendre polynomial bases, with differentiation through both inputs and coefficients. Curve coefficients can be optimized jointly with the surrounding model or produced by another network. The library provides trainable modules, reusable basis modules, and a functional interface, supporting applications ranging from continuous feature embeddings to Kolmogorov-Arnold networks and functional regression. Its implementation combines classical evaluation algorithms with vectorized PyTorch operations, emphasizing readability and maintainability without custom hardware kernels. Timing experiments compare curve evaluation against NumPy and SciPy and KAN training against EfficientKAN across a range of problem sizes. The results illustrate where this generality remains computationally practical and where overhead is substantial. Library is available on GitHub at https://github.com/alexshtf/torchcurves.

0.71

Evaluating the Effectiveness of SechKAN on 1D Data

cs.LG cs.AI ·2026-09-22 ·Hoang-Thang Ta
theorycode/data similarity 0.56
Abstract · 13 pages

The connection between the Kolmogorov-Arnold representation theorem (KART) and neural network design has led to the development of Kolmogorov-Arnold Networks (KANs), with applications ranging from STEM problems to AI tasks. In this paper, we investigate the effectiveness of a KAN variant, SechKAN, which relies on hyperbolic secant (sech) functions as basis functions, with a 1D projection to reduce the number of parameters to a level comparable to MLPs. We evaluate SechKAN on three 1D classification datasets: UCI Human Activity Recognition (UCI HAR), ElectricDevices, and Crop, and compare it with several effective networks, including EfficientKAN, MLP, CNN1D, ResNet1D, and DSCNN1D, using approximately comparable parameter budgets. The results indicate that SechKAN achieves competitive performance across the three datasets, with particularly strong performance on Crop. Ablation studies further show that grid size and normalization affect performance, suggesting that SechKAN's effectiveness depends on the dataset and architectural choices. Our source code and experimental implementation are publicly available at: https://github.com/hoangthangta/SechKAN_1D.

0.40

SH-WRNN: Implicit Spherical Harmonics Weight Field Routing Neural Networks for Asymmetric Edge Intelligence

cs.LG ·2026-09-13 ·Zhibin Jiao, Xiangjing An
code/data similarity 0.56
Abstract

Deep learning architectures remain rigidly built upon traditional fully connected layers. While networks scale up, few challenge this foundational root. In this work, we reshape this paradigm by transforming the core synapse weight matrix from static, discrete parameters into a differentiable, continuous field governed by spherical harmonics functions. We introduce the Implicit Spherical Harmonics Weight Field Routing Neural Network (SH-WRNN), which constrains weight matrices within a continuous parametric field instead of optimizing millions of localized discrete weights. When retrieving the weight matrix of the current layer, connection parameters are localized using latitude and longitude on a rectangular plane mapped from the continuous field. The latitudinal coordinate is specified by activated neurons from the previous layer, while the longitudinal coordinate is determined by keys generated from previous layer activations via matrix multiplication. By evaluating intersections on this map, the network dynamically extracts its connection weights on-the-fly. Empirical validation on MNIST demonstrates that under compact configurations of (32, 10, 10) and (32, 3, 10), SH-WRNN achieves robust accuracies of 91.05% and 81.54% within a single training epoch. Furthermore, we propose an asymmetric Surface Baking scheme. Upon convergence, the continuous weight field is baked once into a static parametric surface. By eliminating analytical spherical harmonics calculations during inference and reducing dynamic matrix extraction to high-speed localized memory slicing, this scheme achieves asymmetric algorithmic acceleration with negligible accuracy degradation. This paradigm shift bypasses GPU memory-bandwidth monopolies, opening a novel path to reshape the advantages of CPU computing. Code is available at https://github.com/jzb1111/SphericalHarmonyRoutedNeuralNetWork.

0.47