CS Paper Radar

CNet: A Complex-Valued Deep Learning Framework with Wirtinger Autodifferentiation and FFT--Hadamard Convolution

Marcel Crasmaru

cs.LG cs.MSmath.OC ·submitted 2026-10-06 15:57 UTC · v3

Abstract

CNet is a C++/CUDA framework for deep complex-valued neural networks (CVNNs), trained by gradient descent with explicit Wirtinger (CR-calculus) derivatives. Complex models are underexplored yet natural where data is intrinsically complex -- RF/IQ, MRI k-space, radar/SAR, audio spectra -- and phase carries information real networks discard. CNet is physics-native: a network is a cascade of complex (often unitary) operations on an amplitude vector, and classification is a Born-rule measurement p_k = |z_k|^2 / ||z||^2 rather than a softmax over real logits. Every layer ships a CPU reference and a finite-difference-verified CUDA kernel; the static, batch-cloned computation graph captures cleanly into CUDA Graphs, outperforming eager and compiled torch.complex64 on an H100. We present three studies. (1) A fully complex causal language model, built on a new causal (triangular-masked) Fourier mixer with an O(N log N) Bluestein evaluation of the forward pass and its Wirtinger adjoint, matches a parameter-matched real causal FNet on character-level tiny-shakespeare in under half the steps. We introduce Born-rule attention, a content-based causal mixer whose weights are the quantum-measurement probabilities |<Q_k, K_j>|^2 -- to our knowledge the first classical, complex-valued attention to replace softmax with the squared-magnitude Born score; with rotary embeddings and per-token complex normalization it reaches 1.52 nats/char, matching softmax self-attention, and transfers unchanged to byte-level (1.31 nats/char) and sub-word WikiText-103. (2) On radio-modulation classification (RML2016.10a/RML22) we trace the pooled-power accuracy ceiling to discarded temporal structure and recover it with a learnable cyclostationary conjugate-lag operator. (3) A bottleneck analysis of the Fourier phase problem in coherent-diffraction imaging isolates where complex-valued networks still need new operators.

Closest papers

Nearest neighbours by meaning across the whole index.

CrossRAFT: Cross-Domain Complex-Valued Feature Extraction for Ultrasound Motion Estimation

eess.SP ·2026-09-08 ·Yang Leng, Yuchen Tang, Kai-Hang Yiu, Yik-Chung Wu, Wei-Ning Lee
similarity 0.60
Abstract

Accurate multi-dimensional motion estimation is fundamental to broad biomedical ultrasound applications, primarily performed using real-valued radio-frequency (RF), analytic, and in-phase and quadrature (IQ) signals. Real-valued RF and analytic signals offer high subsample accuracy but are computationally intensive. Regardless of the signal domain used, traditional algorithms suffer from interframe decorrelation, resolution tradeoffs, and a lack of a carrier and low sampling in the lateral direction. Emerging deep learning (DL) models demonstrate improved performance and faster inference. However, existing architectures are real-valued and disrupt inherent phase coupling crucial for accurate motion estimation by treating complex components as independent channels. To address this, we propose CrossRAFT, an end-to-end trainable network designed for subsample motion estimation directly from complex-valued ultrasound signals. Built on the RAFT architecture, CrossRAFT features complex-valued encoders and a custom correlation module to explicitly extract and use phase information. We evaluated CrossRAFT across real RF, analytic, and IQ signal presentations on simulated phantoms, synthetic echocardiographic, and private in vivo echocardiographic datasets. Experimental results show that CrossRAFT estimates displacements accurately under complex tissue motion and in the challenging lateral direction. Compared to real RF-based RAFT, analytic-based CrossRAFT reduces vertical and horizontal displacement errors by 69% and 51% on the synthetic echocardiographic data, respectively. These findings demonstrate that leveraging phase information via complex-valued neural networks provides a powerful framework for ultrasound motion estimation, showing potential for clinical elastography and functional imaging.

0.30

Complex-valued Phase-Coherent Transformers

cs.LG ·2026-09-18 ·Leona Hioki
similarity 0.59
Abstract

Complex-valued Transformers have inherited softmax attention over the raw complex inner product. Outside natively complex domains this standard form stays near chance, and no complex attention had been shown to correct it. We show that the match must be a scaled cosine score: L2-normalise queries and keys, so the score reads their cosine similarity and ignores their magnitudes, and hold that score at order-one scale. With this the same models train on four diagnostic tasks under two different gates; without the normalisation they stay at chance on ListOps and Needle under both gates and fall far below on the other two, and a normalised score placed at too small a scale fails as well. The resulting family of phase-coherent Transformers (\PCT) matches or exceeds the strongest real-valued baseline across long-range memory, positional retrieval, hierarchical reasoning, frequency-domain classification and physical complex signals; it shows no degradation up to depth 20; and its loss decreases log-linearly over a 61-fold range of parameters. A member of the family, complex screening combined with a phase-coherent recurrence, is the first genuinely complex-valued neural network to solve Path-X, with 91.6% of its trainable parameters complex-valued against 38.2% for S4. We record these as signs of generalisation not previously seen in complex-valued neural networks.

0.47

Beyond Trained Models: Compiling GNNs for a Sound Explainer Benchmark

cs.LG cs.AI ·2026-10-02 ·Steve Azzolin, Francesco Paolo Nerini, Stefano Teso, Francesco Bonchi, Bruno Lepri +2 more
benchmark similarity 0.58
Abstract · Preprint

Explainers for Graph Neural Networks (GNNs) are commonly evaluated by their plausibility, i.e., how well their explanations recover a predefined ground truth, such as a motif planted in the data. This protocol implicitly assumes that a GNN trained on such data relies on the intended motif. Although prior work has questioned this assumption, plausibility remains widespread. First, we show that the assumption is violated on several widely used benchmarks, where, e.g., degree statistics alone suffice to solve the task. Then, we remove this confounder by replacing training with compilation. We achieve this by introducing $\mathsf{Gracr}$, the first compiler translating graded modal logic formulas into GNN weights, yielding models that replicate the behaviour of the corresponding formulas. Since the behaviour of the model is now known by construction, we can define its ground truth explanation formally and compute it exactly. Building on this, we introduce $\mathsf{Gracr}\mathsf{Bench}$, a benchmark of compiled GNNs for the evaluation of explainers against this exact ground truth. Experiments on eleven explainers across six tasks show its effectiveness for fine-grained diagnostic evaluation: notably, we discover that most explainers are not robust to indirect influences or alternative implementations of the same formula. These results position $\mathsf{Gracr}\mathsf{Bench}$ as a novel, rigorous evaluation setting for graph post-hoc explainability.

0.20

AURA: Angular Update Rate Adaptation for training complex-valued neural networks

cs.LG math.NA ·2026-09-22 ·Enrico Ballini, Allan Peter Engsig-Karup, Tito Andriollo
similarity 0.57
Abstract

Complex-valued neural networks (CVNNs) are increasingly adopted for complex-valued data; however, they are often trained with first-order optimizers inherited from the real-valued case. The efficiency of these methods depends largely on the step size, and their step-size rules ignore the angular information available in the complex plane. We address step-size adaptation in the complex domain by introducing AURA (Angular Update Rate Adaptation), a per-parameter step-size adaptation that can be added on top of any first-order optimizer, and removed from it, without altering its update direction. AURA measures the agreement between consecutive updates of each complex parameter, in length, alignment, and sense of rotation, and enlarges the step when they are consistent and reduces it when they are not. It requires no additional gradient evaluations and only inexpensive vector operations per step. We combine AURA with Adam and Muon and compare the resulting methods with well-known first-order optimizers on four test cases of increasing complexity, ranging from the approximation of scalar complex functions to physics-informed training. Fully connected neural networks are used throughout this work. All hyperparameters other than the step size are held fixed across test cases; for one case, we also tune the hyperparameters of each optimizer under the same budget. Our empirical tests show that AURA improves the convergence of its base optimizer in most cases with a small per-step overhead, and we identify the conditions under which it fails to do so.

0.26

ECA-BLS: An Efficient Complex-Augmented Broad Learning System

cs.LG ·2026-08-30 ·A. Rahaman, A. Quadir, M. Sajid, M. Akhtar, M. Tanveer
publishedbenchmark similarity 0.56
Abstract

Broad Learning System (BLS) is an efficient alternative to deep architectures due to its fast training, analytical learning, and strong generalization under limited data. However, existing BLS variants are confined to real-valued representations, restricting their ability to capture nonlinear interactions and second-order statistical dependencies inherent in real-world data. Notably, no prior BLS model fully exploits the complete second-order statistics that naturally emerge when data are embedded in the complex domain. To address this limitation, this paper introduces the first complex augmented Broad Learning System (CA-BLS), which transforms real-valued inputs into phase-encoded complex representations and adopts widely linear modeling to jointly leverage covariance and pseudo-covariance information via complex conjugate augmentation. This enables effective modeling of latent nonlinearities, coherence structures, and second-order dependencies inaccessible to conventional BLS formulations. To mitigate the additional computational cost of complex augmentation, an Efficient Complex Augmented BLS (ECA-BLS) is further developed, reformulating CA-BLS entirely in the real domain while preserving its exact decision function, achieving up to 75\% fewer multiplications and over 60\% fewer additions. A rigorous theoretical analysis proves the mathematical equivalence between CA-BLS and ECA-BLS, ensuring zero theoretical loss. Extensive experiments on 26 benchmark datasets from the UCI and KEEL repositories demonstrate that ECA-BLS consistently outperforms classical BLS and recent state-of-the-art randomized neural networks in accuracy, average rank, and statistical significance, establishing augmented second-order modeling as a critical and previously missing dimension of BLS research.

0.36

Learning nonlinear interactions and cross-feature couplings with product-unit residual networks

cs.LG cs.NE ·2026-11-13 ·Ziyuan Li, Uwe Jaekel, Babette Dellen
benchmarkhot direction similarity 0.56
Abstract

Understanding nonlinear feature interactions is crucial in science and engineering, yet standard multilayer perceptrons (MLPs) often capture such interactions only implicitly, leading to entangled representations that can impair robustness and interpretability. We investigate complex-valued product-unit residual networks (CV-PURe) that integrate multiplicative product units with residual connections to explicitly model cross-feature couplings while stabilizing optimization. We conduct a systematic evaluation on an interaction-driven synthetic benchmark and real-world datasets and evaluate prediction together with input-level interaction recovery under a reproducible protocol. Across five data splits and five training seeds, CV-PURe achieves the lowest mean RMSE on four of six scientific benchmarks, reducing Friedman error by 53.9% relative to MLP. Its Friedman interaction maps assign 60.7% of total strength to the known interacting pair, compared with 31.0% for MLP. On a synthetic task containing third- and fourth-order products and 20 distractors, CV-PURe reduces RMSE from 0.1376 to 0.0797 relative to a complex-valued MLP and recovers all nine true pairwise projections in every training/sampling repetition. Bike Sharing experiments connect improved prediction to the contribution of temporal, calendar, and seasonal context through paired ablations of both PURe variants. Parkinsons Telemonitoring extends the context analysis to repeated measurements. Noise and learning-curve evaluations demonstrate sustained predictive gains under perturbed inputs and limited supervision, while inference measurements quantify computational cost. This article extends our ICCS 2026 paper, Modeling Nonlinear Feature Interactions with Product-Unit Residual Networks (doi:10.1007/978-3-032-29921-5_6).

0.29

The Tangled Web They Weave: Exploring Neural Representations with Directed Simplicial Complexes

cs.LG ·2026-10-31 ·Michael Bleher
published similarity 0.55
Abstract · 52 pages, 16 figures. Accepted at NeurReps 2026 (Symmetry and Geometry in Neural Representations)

Understanding how artificial neural networks (ANNs) represent and combine features internally is a central challenge in mechanistic interpretability. Existing neural circuit-discovery methods represent neuron interactions as pairwise edges in a graph. But distributed neural representations consist of large groups of neurons that are inherently linked by higher-order causal dependencies: suppressing a subgroup of neurons in concert may shift the activation of other neurons in the feature representation. We propose a method to construct filtered directed simplicial complexes (DSCs) that give a topological representation of such dependencies, built from the activation profiles of an ANN on a given set of inputs. Our approach treats ANNs as structural causal models and uses counterfactual ablations to test for causal links between groups of internal neurons within specific input contexts. As a proof of concept, we apply the approach to a pair of small multilayer perceptron (MLP) architectures in a synthetic two-dimensional quadrant classification task. In these examples, DSCs recover the designed feature pathways and coordinated class transitions, and the complexes of trained networks are higher-dimensional than those of their handcrafted references. As validation, we compare our method with graph-based circuit-discovery baselines and find that their 1-skeletons agree within the relations that a weight graph can represent. These results support DSCs as context-dependent descriptions of multi-neuron computation and may thus provide a novel, orthogonal framework to study the phenomenon of distributed feature representations and polysemanticity.

0.33

Harnessing Magnitude-Only and Complex Measurements for Improved Dynamic MRI Reconstruction with Learned Priors

eess.IV cs.AIcs.CVcs.LG+1 ·2026-08-18 ·Mahdi Saberi, Yaşar Utku Alçalar, Merve Gülle, Chetan Shenoy, Mehmet Akçakaya
top venue similarity 0.55
Abstract · To appear in IEEE TBME

MRI reconstruction methods for undersampled k-space data naturally utilize complex-valued measurements. Parallel developments in sparse phase retrieval have shown that magnitude-only measurements may provide complementary information for signal recovery. However, their use in MRI reconstruction remains largely unexplored, due to lack of practical settings where informative magnitude measurements can be obtained without additional scan time. In this work, we investigate the use of auxiliary k-space magnitude information for accelerated steady-state dynamic MRI reconstruction, and demonstrate strong consistency of k-space magnitudes across time-frames. Building on this observation, we propose $\mathbb{C}+\text{Mag}$, a magnitude-informed physics-driven deep learning reconstruction method. The proposed method employs an ADMM-based unrolling framework with a novel magnitude-aware data-fidelity formulation, where quadratically smoothed optimization and momentum-based updates are introduced to address the non-differentiability and non-convexity of the magnitude constraints. Experiments on retrospectively undersampled cine MRI and phase-contrast flow MRI datasets, as well as prospectively undersampled real-time cine MRI acquisitions, demonstrate improved artifact suppression, sharper anatomical recovery, and better preservation of phase information compared to conventional PD-DL methods, which is further supported through blinded expert reader evaluations.

0.43