CS Paper Radar

Diversity of EML-type operators

Andrzej Odrzywołek

cs.SC cs.LGmath.LO ·submitted 2026-09-10 08:14 UTC · v1

Abstract

The discovery of the EML operator, sufficient to evaluate the standard explicit purely transcendental elementary functions, has led to considerable interest and discussion across multiple scientific disciplines. However, most authors have focused on the binary EML itself, while numerous similar variants with slightly different properties are now known. This article attempts to close this gap by enumerating and classifying them. We also take this opportunity to clarify common misconceptions related to the EML operator. The principal goal, symbolic regression within an architecture as close as possible to proven neural networks which combine matrix multiplication with a single univariate non-linear activation function, remains beyond reach. Instead, we propose a Möbius layer, with rational functions replacing matrix operations, and showcase the recently discovered activation function eml(x,1/x), which allows exp(x) and ln(x) to be recovered separately, and hence all elementary functions to be evaluated within a rational generalization of the neural network.

Comment
25 pages, 2 figures, see also the TNG Big Techday conference recording at https://youtu.be/8942GJdrCYI?si=Q4XO0ZlQRK9JDAvK. Wolfram Mathematica implementation of a Goldstern-type single operator in the Appendix. Follow-up to arXiv:2603.21852

Closest papers

Nearest neighbours by meaning across the whole index.

SPA: Structure-Preserving Approximation of Gated Activations under FHE

cs.CR ·2026-10-23 ·Rui Li, Wenyuan Wu, Weijie Miao
similarity 0.63
Abstract

Nonlinear activation evaluation remains a major bottleneck in neural inference under fully homomorphic encryption (FHE). FHE supports efficient addition and multiplication, but lacks native operations for exponential functions, reciprocals, and value-dependent selection. High-degree polynomial approximations on fixed intervals are therefore commonly used to replace nonlinear activations. They can achieve small errors but often incur substantial multiplicative depth and latency. We observe that several widely used nonlinear activations share the algebraic structure $A(x)=xg(x)$ with $g(-x)=1-g(x)$. This class includes GELU, SiLU, fixed-parameter Swish, and the scalar nonlinear branches of GEGLU/SwiGLU. We propose \SPA{} (\emph{Structure-Preserving Approximation}) to exploit this shared structure. \SPA{} retains the exact odd component $x/2$ and approximates only the even residual as a polynomial in $x^2$, yielding a comparison-free arithmetic kernel. This construction removes redundant odd degrees of freedom and enables lower-cost polynomial evaluation at a given error target. Plaintext experiments with Gemma-4-E2B and Llama-3.2-3B show that SPA largely preserves downstream task performance without weight updates. Fixed-interval CKKS benchmarks show favorable error--latency trade-offs. On $[-5,5]$, \SPA{}-balanced achieves a $3.25\times$ speedup over MOAI (ICLR'~26) for erf-GELU with comparable error. On the same interval, \SPA{}-accurate achieves $79.22\%$ lower root mean squared error (RMSE) and a $10.40\times$ speedup over Euston (S\&P'~26) for QuickGELU. For SiLU, \SPA{}-accurate achieves $19.02\%$ lower RMSE and a $2.06\times$ speedup over THOR (CCS'~25).

0.24

MorphingNet: Symbolic Regression Where the Network Is the Formula

cs.LG ·2026-10-14 ·Jincheng Gao, Jiahe Yi, Shuo Sun
similarity 0.59
Abstract · 15 pages, 1 figure, 9 tables

We introduce MorphingNet, a network for symbolic regression (SR) whose units change their function continuously. Instead of searching over expressions built from a fixed operator library, MorphingNet trains the form of each function as a parameter, so SR becomes one continuous optimization problem. Its units are either pole-free rational functions or complex exponential--logarithm functions, which move between exponentials, sinusoids, damped oscillations, power laws and logarithms as their parameters change. The network is a readable formula at every step of training, and no operator library is needed to train it; a structure-recovery step afterwards rewrites the trained formula into short elementary expressions. Training is dense linear algebra, so it is GPU-friendly and scales to large data. MorphingNet achieves state-of-the-art accuracy on SRBench, recovers most generating formulas on synthetic benchmarks, and on large data is up to two orders of magnitude faster than the strongest search method. By turning the search for a formula into the training of a network, MorphingNet offers a new paradigm for SR and for scientific discovery from data.

0.24

Constant-Depth Rational-Weight GNNs and their Connection to Schanuel's Conjecture

cs.LG cs.LO ·2026-10-12 ·Daumantas Kojelis, Carsten Lutz
theory similarity 0.55
Abstract

A landmark result states that the expressive power of graph neural networks (GNNs) is bounded from above by the 1-dimensional Weisfeiler-Leman (1-WL) test. We ask whether a constant-depth GNN can match the expressive power of 1-WL while using only rational weights. We prove that this is the case when Schanuel's conjecture, a famous open problem from transcendental number theory, is taken to be true. Our GNN construction works even with integer weights, and more generally with a broad range of rational weights. As the activation function, it may use logistic sigmoid, hyperbolic tangent, or related functions. It is uniform across all graph sizes. We also prove a partial converse: if a GNN with exponentiation, hyperbolic sine or hyperbolic cosine as activation matches the expressive power of the 1-WL test in a slightly more expressive setting where graphs may contain certain auxiliary vertices, then this implies Schanuel's conjecture restricted to the numbers that arise in computations of the GNN. We also conduct experiments showing that the constructed GNNs indeed match the 1-WL test.

0.22

Riordan Arrays as Structured Operators for Artificial Intelligence

cs.LG math.CO ·2026-11-11 ·Dennis Davenport, Dewayne A. Dixon, Moussa Doumbia
similarity 0.55
Abstract

Riordan arrays have traditionally been studied as infinite lower-triangular matrices that encode generating-function transformations and combinatorial relationships. This note proposes a complementary direction: using Riordan arrays as structured operators in artificial intelligence and data analysis. For a data sequence with generating function $A(z)$, a Riordan array $(g(z),f(z))$ produces the transformation \[ A(z)\longmapsto g(z)A(f(z)). \] This yields a parameterized family of causal linear operators that combine convolution, substitution, and multiscale mixing. The resulting structure may support parameter-efficient neural-network layers, variable-length sequence models, interpretable recurrent systems, invertible architectures, and models for hierarchical or recursively generated data. We explain what the infinite-dimensional viewpoint does and does not provide, outline several possible architectures, identify theoretical and numerical challenges, and propose an initial experimental program. The aim is not to claim that Riordan arrays are appropriate for all data, but to identify settings in which their algebraic structure supplies a useful inductive bias.

0.29

SMILE: Bridging Continuous Optimization and Discrete Symbolic Recovery

cs.LG ·2026-09-04 ·Mansooreh Montazerin, Antonio Ortega, Ajitesh Srivastava
top venue similarity 0.54
Abstract · Accepted to NeurIPS 2026

Symbolic regression (SR) discovers closed-form mathematical expressions from data, offering interpretability beyond black-box models. Existing methods suffer from slow convergence in combinatorial search spaces and lack mechanisms to exploit compositional structure in the data. We introduce SMILE (Sine, Multiplication, Identity, Logarithm, Exponential), a hybrid framework that unifies continuous gradient-based optimization with discrete symbolic recovery through three stages: structural analysis of the data to identify the compositional hierarchy of the target expression, continuous optimization to learn parameters of a network that encodes the target expression using interpretable activations, and symbolic recovery through structured pruning, coefficient optimization, and rounding. This final stage distills the learned network into a compact expression with exact symbolic constants. We evaluate SMILE on SRBench across ground-truth and black-box datasets, with ablation studies validating each component. SMILE achieves the highest symbolic solution rate at the largest noise levels, demonstrating strong robustness where competing methods degrade substantially. It consistently lies on the Pareto front of accuracy versus complexity, recovering significantly simpler expressions in a fraction of the time required by the competing methods.

0.25

The Emergent Symbolic Structure of Artificial Neural Networks

cs.CL cs.AI ·2026-08-30 ·R. Thomas McCoy, Paul Soulos, Tal Linzen, Paul Smolensky
similarity 0.54
Abstract · 30 pages, plus 29 pages of references and appendices

Modern systems in artificial intelligence (AI) somehow excel in domains for which they seem poorly suited. Intelligence has traditionally been modeled as operating over structured combinations of symbols, such as logical formulas. However, the strongest modern AI systems are based on neural networks, which instead represent information in continuous vectors. Vectors seem inadequate for capturing the structure of language, logic, and other cognitive domains, yet neural networks achieve impressive performance in these areas. How do they do it? In this work, we propose a potential answer: Despite appearances, perhaps the internal representations of neural networks implicitly realize symbolic structure. In support of this hypothesis, we show that the vector representations of a variety of neural networks can be closely approximated with symbolic structures: we can replace the network's entire representation-generating process with a closed-form equation instantiating a symbolic structure, and the network's behavior remains largely unchanged. This finding holds for both small-scale neural networks trained to manipulate lists as well as large language models (LLMs) operating in four domains that are central in symbolic traditions: arithmetic, logic, computer code, and language. Further, our symbolic approximation allows us to modify an LLM's behavior in targeted ways via precise interventions on its internal representations, showing that the LLM's behavior is reliant on the symbolic structures we have identified. This work provides a potential way to reconcile longstanding symbolic conceptions of intelligence with the vector-based nature of modern AI.

0.45

Łukasiewicz Neural Networks Extended: Residual Architectures and Crystallization Strategies for Interpretable Rule Extraction

cs.LG cs.AI ·2026-09-26 ·Carlos Leandro
theory similarity 0.54
Abstract · 21 pages

A feed-forward neural network whose weights are integers and whose activation is the truncated identity implements, neuron by neuron, the connectives of Łukasiewicz many-valued logic. This exact correspondence --- established theoretically by Castro and Trillas and developed into a training algorithm by Leandro --- enables \emph{symbolic knowledge extraction}: training produces not a black-box model but a logical formula. Two obstacles have limited the approach to shallow architectures and small datasets: crystallization (forcing weights to integers) succeeds only probabilistically under the original Levenberg--Marquardt training scheme, and the theoretical guarantees break down as networks grow deeper. This paper addresses both obstacles. First, we prove that \emph{residual connections} (skip connections of the kind used in ResNets) extend Łukasiewicz neural networks to arbitrary depth while preserving the symbolic correspondence \emph{at merge neurons} by construction: merge neurons in a Łukasiewicz residual block automatically satisfy the neuron-classification proposition, regardless of the inner layer weights; inner-layer neurons are trained toward representability by the crystallization strategy. Second, we analyse three crystallization strategies --- Levenberg--Marquardt (corrected), straight-through estimation (STE), and proximal regularization --- characterizing their theoretical guarantees, failure modes, and interpretability trade-offs.

0.21

LieDiscover: Adaptive Symbolic Library Construction for Explicit Open-form Symmetry Discovery

cs.SC cs.LG ·2026-09-27 ·Xinxin Li, Jianming Ma, Xingyu Cui, Da Li, Juan Zhang +1 more
similarity 0.53
Abstract

Discovering underlying symmetries from data has emerged as a crucial challenge in scientific discovery. Existing data-driven methods for symmetry discovery fail to determine the exact number and mathematical form of unknown infinitesimal generators. Recent explicit methods represent generators using a predefined function library and identify them through algebraic optimization, but they often struggle to capture complex symmetries involving high-order polynomials or transcendental functions. To address this limitation, we formulate symmetry discovery as a joint optimization problem over the function library and coefficients. We propose a novel framework that leverages an encoder-decoder architecture to dynamically generate symbolic expressions and expand the library. This generation process is optimized via reinforcement learning, which accelerates the exploration of the symbolic search space through step-wise rewards. Experiments demonstrate that LieDiscover can successfully uncover open-form infinitesimal generators involving high-order polynomials or transcendental functions, which remain intractable for existing methods. The discovered symmetries also improve performance in downstream PDE solving and discovery tasks.

0.27