CS Paper Radar

Frequency Moments Beyond Equality: Streaming Cosine Density Moments

Qin Zhang

cs.DS ·submitted 2026-09-07 01:48 UTC · v2
theory

Abstract

For a stream of nonzero vectors $x_1,\ldots,x_n\in\mathbb{R}^d$, let $u_i=x_i/\|x_i\|_2$. We define the cosine density of the $i$-th stream element by $D_i:=\sum_{j\in[n]}\langle u_i,u_j\rangle$ and study the density moments $M_p:=\sum_{i\in[n]}D_i^p$ in both the signed- and nonnegative-cosine regimes. These quantities are similarity-aware analogues of classical frequency moments: replacing cosine similarity by equality (that is, $D_i = \sum_{j\in[n]} \mathbf{1}\{u_j = u_i\}$) gives $M_p=F_{p+1}$ and, in particular, $M_{-1}=F_0$, the number of distinct elements. We give one-pass streaming algorithms and lower bounds whose dependence on $d$ is tight up to logarithmic factors for $p=-1$, $p=1$, and even $p \ge 2$ (the latter under signed cosine), and whose exponents differ by less than $1/2$ for odd $p \ge 3$ under nonnegative cosine. Our results thus extend several fundamental statistics from the classical data stream literature to cosine similarity, a widely used measure for comparing vector embeddings in modern AI systems. The main challenge in proving a space lower bound for nonnegative cosine is to eliminate unwanted contributions without relying on pairs of opposite vectors. We address this through a construction that we call \emph{equal-sum moment isolation}: two insertion-only prefixes have the same cardinality and vector sum, and a finite-difference comparison cancels the baseline shared by all their vectors while isolating the desired higher-order signal. This proof framework may be useful for other insertion-only streaming lower bounds, where direct cancellation is not possible.

Comment
44 pages

Closest papers

Nearest neighbours by meaning across the whole index.

Moments of crosscorrelation demerit factors of binary sequences

cs.IT cs.DMeess.SPmath.CO+1 ·2026-09-04 ·Daniel J. Katz, Harmony M. Vargas
unusual similarity 0.57
Abstract · 32 pages

Families of sequences with low mutual aperiodic crosscorrelation assist the design of systems for multi-user asynchronous communications and multiple-input multiple-output radar. The crosscorrelation demerit factor of a pair of sequences is the sum of the squared magnitudes of their crosscorrelation values at every shift when the sequences are normalized to unit Euclidean norm, and the merit factor is the reciprocal of the demerit factor. For each positive integer $\ell$, we endow the $2^{2 \ell}$ pairs of binary sequences of length $\ell$ with uniform probability measure and study the distribution of their crosscorrelation demerit factors. Sarwate showed that the mean value is always $1$ regardless of length $\ell$. We develop a method for finding an exact formula for the $p$th central moment (for any positive integer $p$) as a function of $\ell$. Formulae for the variance and third central moment ($p=2$ and $3$) are then obtained by hand calculations, while the fourth through sixth central moments are obtained by computer-assisted calculations. Our theory also shows that all the central moments must be strictly positive for $p\geq 2$ and $\ell \geq 3$.

0.31

A Note on Scaling in Randomly Rotated Quantization and Its Connection to the CDEF +1 Pythagorean Relation

cs.IT cs.LG ·2026-09-08 ·Uri Erez
similarity 0.50
Abstract

Quantization schemes based on randomized rotations have recently received renewed attention, including the roles of MMSE and unbiased reconstruction scalings. In this note, we point out the connection to classical results in statistical signal processing and communication theory. Specifically, the two reconstruction scales used in the EDEN line of work admit a natural interpretation as finite-dimensional, realization-dependent counterparts of the Wiener and unbiased coefficients in the classical CDEF formulation. At finite blocklength, the CDEF +1 relation holds pointwise for each rotation realization as an exact geometric (Pythagorean) identity, but does not hold after averaging the distortions over the rotation. The classical SNR relation $\sf{SNR}_{\rm MMSE}=\sf{SNR}_{\rm MMSE,U}+1$ is recovered as $d\to\infty$: once the overall scale is handled separately, the empirical coordinate statistics of a randomly rotated vector approach their i.i.d. Gaussian counterparts, and the rotation-dependent quantities concentrate. Importantly, EDEN goes beyond this classical correspondence: for every finite $d$, its Haar-rotation formulation guarantees exact conditional unbiasedness, a stronger property than the second-order notion of unbiasedness in CDEF. We further comment on two distinct roles random rotations play in quantization: one is approximate Gaussianization of the coordinates; the other is decorrelation of reconstruction errors across quantization branches.

0.23

Optimal Passes and Perfect Sampling for Similarity Graph Statistics

cs.DS ·2026-09-17 ·Qin Zhang
hot direction similarity 0.49
Abstract · 32 pages

We study statistical estimation on implicit weighted similarity graphs presented as node-arrival streams. Previous work~\cite{LZ26b} obtained constant-pass, sublinear-space algorithms for several basic statistics of such graphs, including the diversity index $\mathsf{DI}=\sum_i d_i^{-1}$ and the degree moments $M_p=\sum_i d_i^p$ for $p>0$, together with their associated sampling problems $L_{\mathsf{DI}}$ and $L_{M_p}$. We settle three questions left unresolved by the previous work. First, we show that any two-pass streaming algorithm that $1.1$-approximates $\mathsf{DI}$ requires $Ω(n)$ bits of space; combined with the three-pass $\tilde{O}(\sqrt n)$-space algorithm of~\cite{LZ26b}, this establishes a sharp transition between two and three passes for $\mathsf{DI}$. Second and third, for every fixed $c>0$ we give a three-pass $n^{-c}$-perfect $L_{\mathsf{DI}}$-sampler using $O_c(\sqrt n\log n)$ words and a two-pass $n^{-c}$-perfect $L_{M_p}$-sampler using $O_{p,c}(n^{1-1/(p+1)}\log n)$ words. Matching lower bounds up to polylogarithmic factors show that both the number of passes and the polynomial dependence on $n$ are optimal.

0.43

Streaming Algorithms for Gaussian Kernel Density Statistics

cs.DS ·2026-09-09 ·Qin Zhang
similarity 0.49
Abstract · 37 pages

Motivated by data produced by generative systems, \cite{LZ26b} formulates similarity-aware statistics via a weighted similarity graph, replacing equality with similarity in classical frequency-based statistics. Although this framework captures semantic relationships between nonidentical items, under general similarity functions even coarse one-pass approximation can require linear space. We therefore ask whether the geometric structure present in natural vector similarities can overcome this barrier. We answer this question affirmatively for the Gaussian kernel. For fixed-dimensional Euclidean vector streams, we study similarity-aware analogues of classical frequency statistics, including the number of distinct elements and frequency moments, through the diversity index and Gaussian density moments. We give one-pass sublinear-space approximation algorithms that exploit the geometric and analytic properties of the Gaussian kernel, and complement them with lower bounds. Our results show that geometric structure can fundamentally change the streaming complexity of similarity-aware statistical analysis.

0.20

Statistics of Similarity Graphs in Node-Arrival Streams

cs.DS ·2026-09-03 ·Kaiwen Liu, Qin Zhang
highly cited similarity 0.48
Abstract · 27 pages

In this paper, we study several statistical problems on similarity graphs in the node-arrival streaming model, including degree moments, diversity index, degree-moment sampling, and diversity sampling. We develop constant-pass, sublinear-space streaming algorithms for these problems and establish space lower bounds that nearly match the upper bounds in their dependence on the stream length.

0.53

Karhunen-Loève Transforms of Autoregressive Windows Based on Matched Symmetry Groups

eess.SP cs.ITmath.NA ·2026-11-08 ·Mitchell A. Thornton
theoryunusual similarity 0.48
Abstract · v1: 38 pages, 7 figures

The Karhunen-Loève transform of a window of a stationary autoregressive process of any order $p$ is described through its poles, extending the Algebraic Diversity framework, which selects a transform by the matched group. For simple roots every basis vector is a sum of at most $p$ sinusoids, at the frequencies where the spectrum equals the eigenvalue, plus modes decaying from the ends. The window covariance differs from a member of the class diagonalized by the type-II discrete cosine transform by a closed-form matrix of rank $2p$, exactly when no pole is zero, so the eigenvalues interlace spectrum samples on the cosine grid, and the coding-gain loss of the cosine transform is an exact function of that matrix with computable bounds: for the $8\times8$ Markov block of image coding at pole $0.95$ it is below 1.2 percent. By Kronecker's theorem the rank is at most $2\max(p,q)$ for rational spectra; a moving-average window departs in its Hankel corners, and an integrated (ARIMA) window departs by low rank from the type-VII sine class, exact for the random walk. The exact symmetry is generically the reflection and contains a wreath product for interleaved windows; the dihedral group of the even extension is matched approximately, the dominant pole's sign choosing cosine or sine. The two groups give the transform a fast form: a parity fold, half-length cosine cores and a cascade of at most $p$ first-order Cauchy stages, exactly $p$ for a minimal AR($p$) model. The first-order case recovers results obtained by other means.

0.33

Phase Transitions in Heavy-Tailed Mean Estimation under $\ell_p$ Norms

cs.LG math.ST ·2026-11-15 ·Ishaq Aden-Ali, Yeshwanth Cherapanamjeri, Mikael Møller Høgsgaard, Kasper Green Larsen, Nikita Zhivotovskiy
top venuebenchmarktheory similarity 0.47
Abstract · Accepted at NeurIPS 2026

We study the problem of estimating the mean of a random vector in $\mathbb R^d$ under $\ell_p$ norms, assuming only the existence of a covariance matrix. In the Euclidean norm, the sample mean is already optimal in expectation, and robust estimators are usually needed only to obtain high-probability tails. Our results show that this picture changes for $\ell_p$ norms with $p>2$: robustification is already necessary even for the expected risk. We prove that the minimax risk exhibits a sharp phase transition, controlled by the interaction between the sample size $n$, the dimension $d$, and the norm parameter $p$. In the large-sample regime, the classical Gaussian rate remains achievable. In the complementary regime, heavy-tailed distributions create a genuinely non-Gaussian obstruction. Furthermore we show that the simple estimator: coordinate-wise median of means whose number of buckets chosen according to the geometry of the loss, optimally tracks this phase transition in its expected risk minimax. We also show that natural projection-based median-of-means estimators for general norms, despite achieving the optimal confidence trade-off relative to the sample-mean benchmark of Lugosi and Mendelson (Probab. Theory Relat. Fields, 2019), can be polynomially suboptimal for $\ell_p$ norms even in the constant-probability regime.

0.31

The Exact Approximation Ratio of Uniformly Rotated Coordinate-wise Median in the Euclidean Plane

cs.CE cs.GT ·2026-09-16 ·Song Zichen
theory similarity 0.47
Abstract

Uniformly rotated coordinate-wise median chooses a random orthonormal coordinate system, takes a median in each coordinate, and maps the resulting point back to the Euclidean plane. We determine its exact worst-case expected approximation ratio when social cost is the $L_p$ norm of the agents' Euclidean distances and $1<p<2$. The ratio is \(\frac{2^{2-1/p}}π\int_0^{π/2}(\cos^pθ+\sin^pθ)^{1/p}\ddθ\). This expression was previously established as a lower bound by Chan, Lin, and Wang; our contribution is the matching upper bound. The proof establishes a strengthened coordinate-wise median inequality relative to an arbitrary reference point. Its right-hand side is linear in a sum of direction-dependent norms, which permits direct averaging over rotations without the loss incurred by passing through a $p$th-moment bound. We give all auxiliary inequalities and a self-contained proof of tightness using the established two-cluster-and-outlier construction. The upper bound holds for every finite profile and every measurable choice within the coordinate median intervals, while odd-size profiles suffice for the matching lower bound. The result characterizes this fixed mechanism, rather than the optimal approximation ratio among all randomized strategyproof mechanisms.

0.26