CS Paper Radar

Karhunen-Loève Transforms of Autoregressive Windows Based on Matched Symmetry Groups

Mitchell A. Thornton

eess.SP cs.ITmath.NA ·submitted 2026-11-08 21:11 UTC · v1
theoryunusual

Abstract

The Karhunen-Loève transform of a window of a stationary autoregressive process of any order $p$ is described through its poles, extending the Algebraic Diversity framework, which selects a transform by the matched group. For simple roots every basis vector is a sum of at most $p$ sinusoids, at the frequencies where the spectrum equals the eigenvalue, plus modes decaying from the ends. The window covariance differs from a member of the class diagonalized by the type-II discrete cosine transform by a closed-form matrix of rank $2p$, exactly when no pole is zero, so the eigenvalues interlace spectrum samples on the cosine grid, and the coding-gain loss of the cosine transform is an exact function of that matrix with computable bounds: for the $8\times8$ Markov block of image coding at pole $0.95$ it is below 1.2 percent. By Kronecker's theorem the rank is at most $2\max(p,q)$ for rational spectra; a moving-average window departs in its Hankel corners, and an integrated (ARIMA) window departs by low rank from the type-VII sine class, exact for the random walk. The exact symmetry is generically the reflection and contains a wreath product for interleaved windows; the dihedral group of the even extension is matched approximately, the dominant pole's sign choosing cosine or sine. The two groups give the transform a fast form: a parity fold, half-length cosine cores and a cascade of at most $p$ first-order Cauchy stages, exactly $p$ for a minimal AR($p$) model. The first-order case recovers results obtained by other means.

Comment
v1: 38 pages, 7 figures

Closest papers

Nearest neighbours by meaning across the whole index.

On the Real Part of Butterfly Factorizations, with Application to Fast Transforms

eess.SP ·2026-10-05 ·Ma{ë}l Chaumette, R{é}mi Gribonval, Elisa Riccietti
similarity 0.48
Abstract

Fast transforms such as the Discrete Cosine Transform can be expressed as the real part of a product of complex-valued structured sparse matrices known as Kronecker-sparse factors. More generally, products of such structured sparse matrices appear in square dyadic butterfly and monarch matrices, which are known to represent many fast transforms efficiently. Yet, it has remained unclear whether taking the real or the imaginary part of such products preserves this structure. In this letter, we give a condition on the factors to ensure that the structure is preserved. This condition is satisfied in the case of the Discrete Cosine Transform, the Discrete Sine Transform and the Discrete Hartley Transform, allowing us to provide a new representation of these transforms as products of real-valued Kronecker-sparse factors, which opens new avenues for efficient GPU implementations using low-precision numerical format.

0.22

Frequency Moments Beyond Equality: Streaming Cosine Density Moments

cs.DS ·2026-09-07 ·Qin Zhang
theory similarity 0.48
Abstract · 44 pages

For a stream of nonzero vectors $x_1,\ldots,x_n\in\mathbb{R}^d$, let $u_i=x_i/\|x_i\|_2$. We define the cosine density of the $i$-th stream element by $D_i:=\sum_{j\in[n]}\langle u_i,u_j\rangle$ and study the density moments $M_p:=\sum_{i\in[n]}D_i^p$ in both the signed- and nonnegative-cosine regimes. These quantities are similarity-aware analogues of classical frequency moments: replacing cosine similarity by equality (that is, $D_i = \sum_{j\in[n]} \mathbf{1}\{u_j = u_i\}$) gives $M_p=F_{p+1}$ and, in particular, $M_{-1}=F_0$, the number of distinct elements. We give one-pass streaming algorithms and lower bounds whose dependence on $d$ is tight up to logarithmic factors for $p=-1$, $p=1$, and even $p \ge 2$ (the latter under signed cosine), and whose exponents differ by less than $1/2$ for odd $p \ge 3$ under nonnegative cosine. Our results thus extend several fundamental statistics from the classical data stream literature to cosine similarity, a widely used measure for comparing vector embeddings in modern AI systems. The main challenge in proving a space lower bound for nonnegative cosine is to eliminate unwanted contributions without relying on pairs of opposite vectors. We address this through a construction that we call \emph{equal-sum moment isolation}: two insertion-only prefixes have the same cardinality and vector sum, and a finite-difference comparison cancels the baseline shared by all their vectors while isolating the desired higher-order signal. This proof framework may be useful for other insertion-only streaming lower bounds, where direct cancellation is not possible.

0.45

Quantum algorithms for orthogonal polynomial transforms

cs.DS quant-ph ·2026-10-02 ·Anupam Prakash, Shree Hari Sureshbabu, Dylan Herman, Shouvanik Chakrabarti
similarity 0.47
Abstract

A quantum orthogonal polynomial transform (QOPT) is an algorithmic primitive that maps a superposition of standard basis states $\sum_k α_{k} \ket{k}$ coherently to a basis of normalized univariate polynomials orthogonal with respect to a probability measure $μ(x)$. We provide new discrete and continuous QOPTs for polynomial families in the Askey scheme extending the results for the quantum Hermite transform (Jain et al., STOC'26). Our efficient QOPT algorithms require time $O(\text{polylog}(N, 1/ε))$, where $N$ is the grid size and $ε$ is the error, for the discrete Charlier, Meixner and Krawtchouk transforms and for the integer-order Laguerre transform in the continuous setting. The efficient QOPTs are obtained by uncovering the links between Askey scheme polynomials and Gaussian quantum optical gates and developing a compilation framework for $SU(2)$ and $SU(1,1)$ optical gates on Cartesian grids extending the framework developed by Iyer et al. (arXiv:2602.15180). Further, we reduce the continuous Jacobi transform to the discrete Hahn transform and provide an $O\!\left(N\operatorname{polylog}((N+α+β+1)/ε)\right)$-time Hahn transform, a quadratic speedup over the naive implementation. This is based on a more efficient compilation of the Clebsch--Gordan transform for coupling $\mathrm{SU}(2)$ representations with spins $(j_{1}, j_{2})$. Finally, we develop a new framework for Laguerre transforms for all orders $ν>0$ by fast-forwarding the corresponding radial oscillator using a 3-term chirp decomposition and an efficient algorithm for the Quantum Hankel Transform on a logarithmic grid.

0.23

A Note on Scaling in Randomly Rotated Quantization and Its Connection to the CDEF +1 Pythagorean Relation

cs.IT cs.LG ·2026-09-08 ·Uri Erez
similarity 0.47
Abstract

Quantization schemes based on randomized rotations have recently received renewed attention, including the roles of MMSE and unbiased reconstruction scalings. In this note, we point out the connection to classical results in statistical signal processing and communication theory. Specifically, the two reconstruction scales used in the EDEN line of work admit a natural interpretation as finite-dimensional, realization-dependent counterparts of the Wiener and unbiased coefficients in the classical CDEF formulation. At finite blocklength, the CDEF +1 relation holds pointwise for each rotation realization as an exact geometric (Pythagorean) identity, but does not hold after averaging the distortions over the rotation. The classical SNR relation $\sf{SNR}_{\rm MMSE}=\sf{SNR}_{\rm MMSE,U}+1$ is recovered as $d\to\infty$: once the overall scale is handled separately, the empirical coordinate statistics of a randomly rotated vector approach their i.i.d. Gaussian counterparts, and the rotation-dependent quantities concentrate. Importantly, EDEN goes beyond this classical correspondence: for every finite $d$, its Haar-rotation formulation guarantees exact conditional unbiasedness, a stronger property than the second-order notion of unbiasedness in CDEF. We further comment on two distinct roles random rotations play in quantization: one is approximate Gaussianization of the coordinates; the other is decorrelation of reconstruction errors across quantization branches.

0.23

How Much Does Random Rotation Help GPTQ? Sharp Worst-Case Rate Gaps in the High-Rate Regime

cs.IT cs.LG ·2026-11-01 ·Runguo Li
similarity 0.47
Abstract · 28 pages, 3 figures

GPTQ is Babai's nearest-plane algorithm on the lattice of the Cholesky factor of the layer's input second moment $Σ$. In the high-rate, entropy-coded regime it needs $\tfrac12\log_2(\AM/\GM)$ of its Cholesky pivots more bits than WaterSIC, which is within $0.255$ bit of the rate-distortion limit for every $Σ$; random rotations close most of this gap in practice, and Ordentlich and Polyanskiy asked for the worst-case gap between rotated GPTQ and WaterSIC. We show that after a Haar rotation the $k$-th pivot converges almost surely, uniformly in $k\geδn$ and over spectra $ν$ of bounded condition number, to $1/S_ν(-k/n)$, a classical formula from the theory of Lyapunov exponents, and use it to answer the question. Over spectra with condition number $κ$ the worst rotated gap is $\tfrac12\log_2κ-\log_2\lnκ+\tfrac12-\tfrac1{\ln2}+o(1)$ bits, attained only by two-point spectra, just $\tfrac12\log_2\lnκ+O(1)$ bits below the worst case without rotation, so rotation cannot replace waterfilling in general. It nearly can on real layers: rotation cuts the cost of an $η$-fraction of outlier eigenvalues of size $μ$ from about $ημ$ to $η^2μ/2$ to leading order, but cannot remove spectral spread on a logarithmic scale; for non-Gaussian weights it also adds a weight-side rate cost, which we characterize exactly under a standard high-rate assumption. On $96$ Hessians of Qwen3 and Llama-3.1 the rotated gap has median $0.04$ bit and maximum $0.36$ bit, against $2$--$5$ bits in the worst case for the same condition numbers, and the limit matches it to within $0.009$ bit.

0.33

Non-asymptotic Analysis of Expected Reconstruction Risk for Trigonometric Polynomial Models

cs.IT ·2026-09-24 ·Hang Xu, Yi Shen
similarity 0.46
Abstract

We investigate the expected reconstruction risk of trigonometric polynomial models under different sampling schemes. Through numerical experiments, we observe that when the sampling nodes $\{t_l\}_{l=1}^m$ are i.i.d. random variables uniformly distributed over $[0,1)$, the associated structured random matrix $\pmb{A} \in \mathbb{C}^{m \times N}$ with $A_{l,k} = e^{2π\mathrm{i} kt_l}, k \in Γ= \{-q, \dots, q\}, N = 2q+1$ frequently becomes nearly singular or severely ill-conditioned. As a consequence, the expected reconstruction risk exhibits divergent behavior. In contrast, when the sampling nodes $t_l$ are either equidistant points or small random perturbations of an equidistant grid, the expected reconstruction risk undergoes a sharp phase transition at the interpolation threshold $m=N$. To better understand the underlying mechanisms behind these different phenomena, we characterize the expected reconstruction risk through the spectral quantity $\sum_{i=1}^{r} \frac{1}{σ_i^2(\pmb{A})}$, where $σ_i(\pmb{A})$ denotes the singular values of the sampling matrix. Based on this spectral representation, we theoretically prove that the expected reconstruction risk diverges under uniformly distributed random sampling. Furthermore, we derive an explicit formula for the expected reconstruction risk in the equidistant sampling case and establish upper and lower bounds for the expected reconstruction risk under jittered sampling.

0.28

Zeros of the Spectrogram of White Noise with Hermite Windows

eess.SP math.PR ·2026-10-09 ·Charaf Bellourak, Sylvain Meignen, Kévin Polisano, Mickaël Nahon
theory similarity 0.45
Abstract

We study the zeros of the spectrogram of complex white noise analyzed with Hermite windows of order n, extending the classical Gaussian-window correspondence with Gaussian Analytic Functions to the polyanalytic setting. Using the polyanalytic Bargmann transform, these zeros split into two families according to the sign of the local Jacobian, giving rise to a natural notion of charge, and we show that these zeros are almost surely simple, isolated, and form a translation and rotation-invariant point process. We derive closed-form expressions for the one-point intensity, decomposed by charge, and also for the unsigned, signed, and hybrid two-point intensities. Besides, we prove that unlike the total repulsion observed with a Gaussian window, the Hermite window of order n generates a pair correlation function that converges at the origin to a strictly positive constant depending on n. Finally, we propose a numerical method for detecting spectrogram zeros via the winding number around candidate local minima.

0.22

Inference and Uncertainty Quantification for Streaming $r$-PCA

cs.LG stat.ML ·2026-08-18 ·Haoshu Xu, Hongzhe Li
theory similarity 0.44
Abstract

We address two open questions in streaming PCA via Oja's algorithm: sharp operator-norm convergence for general rank under sub-Gaussian data, and distributional inference for the resulting subspace estimator. Existing convergence analyses, even in the rank-one case, either assume bounded data or leave non-vanishing remainder terms that prevent adaptation to a polynomially vanishing tail spectrum, while existing distributional results are confined to the rank-one case. Our convergence theory removes these remainder terms and yields a sharp rate. In the dense-tail spiked covariance regime, this rate matches the minimax rate up to logarithmic factors. More generally, we prove a matching lower bound, up to logarithmic factors, across both dense-tail and sparse-tail regimes under a mild nondegeneracy condition. The analysis yields a linearization of Oja's iterates, which in turn enables a high-dimensional Gaussian approximation for the general-rank subspace estimation error with an explicit limiting covariance. We also establish a row-wise Gaussian approximation over convex sets for the aligned difference, recovering prior rank-one results as special cases. For practical inference, we develop an online multiplier bootstrap algorithm and prove its consistency. Beyond streaming PCA, our techniques contribute to Gaussian approximation and bootstrap inference for nonconvex stochastic approximation.

0.42