CS Paper Radar

Spectral-Domain Information Geometry of Stationary Covariance Models: Gaussian and Elliptical Extensions

Shu Yang, Dongying Li, Wenxian Yu

eess.SP math.ST ·submitted 2026-11-09 10:56 UTC · v1

Abstract

This paper develops an information-geometric framework for stationary covariance structures. We begin with the infinite-dimensional setting, where stationary covariance structures can be characterized without finite-window boundary effects. By formulating the covariance structure as a bilinear form on a suitable class of test functions, we establish its spectral representation and show how finite-dimensional Toeplitz covariance matrices arise as restrictions to nested observation subspaces. For zero-mean proper complex Gaussian wide-sense stationary (WSS) processes, we show that the normalized finite-dimensional Fisher information converges to a spectral-domain metric. In logarithmic spectral coordinates, this metric becomes the standard $L^2$ inner product, yielding an explicit Fisher--Rao distance that is also recovered as the asymptotic normalized affine-invariant Riemannian distance between Toeplitz covariance matrices. We then extend the framework to elliptical observation models by separating global covariance scale from shape. The shape Fisher information is orthogonal to radial nuisance directions and is characterized by a dimension- and radial-model-dependent coefficient. We derive a condition under which this coefficient converges to its Gaussian value as the observation dimension tends to infinity. The resulting limiting spectral-shape geometry is the centered $L^2$ geometry of log-spectra and is invariant to positive rescaling. Numerical experiments illustrate the theoretical convergence results and the recovery of the Gaussian shape coefficient.

Comment
26 pages, 7 figures

Closest papers

Nearest neighbours by meaning across the whole index.

On the parametric and semiparametric Fisher information matrix for non-zero mean stationary spherical invariant random processes

eess.SP math.ST ·2026-09-17 ·Jean-Pierre Delmas, Habti Abeida, Stefano Fortunati
similarity 0.72
Abstract · a main file (with 4 figures) and supplementary materials

The classical Whittle formula provides a closed-form expression for the asymptotic Fisher information matrix (FIM) rate of multidimensional, real-valued, zero-mean, purely nondeterministic stationary Gaussian processes (GPs), expressed in terms of their parameterized spectra within a maximum-likelihood framework. However, the Gaussian assumption underlying this result restricts its applicability to many real-world signals exhibiting heavy-tailed or non-Gaussian behavior. In this paper, we extend Whittle's result to multidimensional, real-valued stationary compound Gaussian processes (CGPs) with arbitrary mean, under a unified framework encompassing fully known, parameterized, and completely unknown density generators. Building upon the Slepian-Bangs formula for $n$ consecutive observations and extending it to the semiparametric setting, we leverage the asymptotic properties of block Toeplitz matrices to obtain a closed-form spectral-domain expression for the asymptotic FIM rate. The resulting expression, common to all density generator families, generalizes the standard zero-mean Gaussian formula by incorporating a distribution-dependent contribution associated with the nonzero mean, and an additional covariance term that is invariant to the choice of non-Gaussian distribution. This extension enables efficient and theoretically grounded performance analysis for heavy-tailed signal processing applications.

0.41

Asymptotic Frequency-Domain FIM for multidimensional stationary and cyclostationary compound-Gaussian processes: Application to DOA estimation

eess.SP ·2026-11-02 ·Jean-Pierre Delmas, Habti Abeida
similarity 0.65
Abstract

The classical Whittle formula provides a closed-form expression for the asymptotic Fisher information matrix (FIM) of stationary, zero-mean Gaussian processes (GPs) with finite-dimensional parameterizations, expressed in terms of their power spectra. While this result was recently extended to nonzero-mean, multidimensional, real-valued stationary compound-Gaussian processes (CGPs), its applicability remains limited. In this paper, we significantly broaden this theoretical framework with two primary contributions. First, we extend the Whittle formula to purely cyclostationary real-valued CGPs with parameterized cyclic spectra, and further generalize it to circular and noncircular complex-valued multidimensional stationary CGPs, encompassing both purely and almost cyclostationary CGPs. Second, we apply these novel Whittle-type formulas to evaluate the impact of source spectral characteristics on the Cramer-Rao bound (CRB) for direction-of-arrival (DOA) estimation, particularly in scenarios involving frequency offsets.

0.12

Conditional Fisher-Information Central Limit Theorems under Log-Concavity with Information-Theoretic Consequences

cs.IT math.PR ·2026-09-07 ·Tong Ye, Liu-Quan Yao
benchmarktheory similarity 0.62
Abstract · 47 pages

We establish conditional central limit theorems in Fisher information under log-concavity in every fixed dimension. For conditionally centered normalized sums, after whitening by the averaged conditional covariance, the averaged conditional Fisher information converges to the dimension if and only if it is finite at one convolution level. The scalar criterion follows as the one-dimensional case; we also provide an independent scalar proof based on a second-order continuity theorem for Fisher production on Gaussian-smoothed, tail-controlled classes. For the original sums, the averaged Fisher information matrix converges in operator norm to the inverse averaged conditional covariance. Consequently, the conditional relative Fisher information with respect to the limiting Gaussian law vanishes, and the Gaussian logarithmic Sobolev inequality yields convergence in conditional relative entropy and conditional entropy. We give two operational consequences. For any fixed finite-constellation low-power input, the first-order conditional mutual-information slope converges to the Gaussian-noise benchmark. For Gaussian signaling at any fixed signal covariance, the mutual-information gap from that benchmark is bounded by the conditional relative Fisher deficit and hence vanishes asymptotically.

0.38

Dimension-Free Private Mean Estimation is Impossible Without the Covariance

cs.LG cs.DS ·2026-11-04 ·Renaud Gaucher, Ephraim Linder, Adam Smith, Jonathan Ullman, Lydia Zakynthinou
similarity 0.59
Abstract

We study the limits of differentially private Gaussian mean estimation when the covariance matrix is unknown. Prior work shows that when the covariance $Σ$ is known, the optimal sample complexity can depend only on its spectrum, rather than on the ambient dimension $d$. We establish new lower bounds for Gaussian mean estimation with Euclidean error which show that private estimators cannot achieve similar guarantees when the covariance is unknown. First, when the sample size $n$ is smaller than the effective dimension $\mathrm{trace}(Σ)/\|Σ\|_{\text{op}}$, the optimal error when the covariance is unknown can be larger than the optimal error when it is known by a factor of $\sqrt{d/n}$. Second, when the sample size is larger than the effective dimension, even for diagonal covariance matrices with effective dimension bounded by $\log d$, the optimal error when the covariance is unknown can be larger than the optimal error when it is known by a factor of $\sqrt{d}/n^{3/2}$. Together, these results show that in the unknown covariance setting, exploiting anisotropic structure under differential privacy can require a dimension-dependent error or sample complexity.

0.21

Sharp Asymptotic Theory of Maximum Likelihood Estimation for Gaussian Processes with an RBF Kernel

cs.LG math.STmath.PR ·2026-10-07 ·Ameer Qaqish, Sudipto Banerjee, Didong Li
hot direction similarity 0.59
Abstract

Gaussian processes (GPs) are widely used across machine learning, spatial statistics, time-series analysis, optimization, Bayesian statistics, and scientific applications. A central component of a GP model is its kernel, which is typically specified through a parametric family. Among the most widely used choices is the radial basis function (RBF), also known as the squared exponential or Gaussian kernel, owing to its simple form, smoothness, and flexibility. In practice, the kernel parameters are routinely estimated by the maximum likelihood estimators (MLEs), as implemented by standard GP software. Despite this widespread use, the asymptotic behavior of the MLEs remains poorly understood under fixed-domain asymptotics, even for the RBF kernel. The main difficulty arises from the increasingly strong dependence among densely sampled observations and the nonlinear dependence of the covariance matrix on the kernel parameters. In this paper, we address this gap by providing, to the best of our knowledge, the first complete asymptotic characterization of the joint MLE of the spatial variance, lengthscale, and nugget variance under fixed-domain asymptotics. We establish consistency, derive convergence rates for all three parameters, prove joint asymptotic normality, and show that these rates are minimax optimal.

0.56

Efficient Estimation of High Information Projections using Nearest Neighbours

cs.LG stat.ML ·2026-08-26 ·David P. Hofmeyr
similarity 0.59
Abstract

An intuitive method for dimensionality reduction is proposed, which is highly effective for finding interesting projections of multivariate data. Following similar intuitive motivation to a number of existing techniques, the proposed method is based on enhancing the nearest neighbour relationships in the data. The proposed projection arises from the spectral decomposition of a matrix designed to encode the local covariance structure in the data, where the local covariance at a point is captured by pairs of its nearest neighbours. We show that under standard regularity conditions this matrix is a consistent estimator of the so-called ``Density Information Matrix'' (DIM); a non-parametric analogue of the Fisher Information Matrix. Spectral decompositions of DIMs have been shown to be connected with the important problems of Independent Components Analysis and, in the supervised context, Sufficient Dimension Reduction. However, existing estimators of the DIM are computationally expensive to compute and only target the DIM of a surrogate density, which is proportional to the square of the true underlying density. In addition, we go on to explore the practical utility of our method in aiding the downstream tasks of cluster analysis and outlier detection.

0.22

Directional Statistics in Signal Processing: A Geometry Driven Overview of Models, Learning, and Applications

eess.SP stat.MEstat.ML ·2026-11-03 ·Anders S. Olsen, Kanti V. Mardia, Dimitri Van De Ville, Morten Mørup
survey similarity 0.58
Abstract

Many signal-processing objects are defined by periodicity, normalization, sign or global-phase ambiguity, basis invariance, or positive-semidefinite constraints and therefore cannot be modeled faithfully in ordinary Euclidean space. This paper develops a geometry driven framework for directional statistics in signal processing, connecting signal representations to their natural manifolds or quotient spaces and to probability models compatible with their invariances. After reviewing instances of directional statistics in the signal processing literature, we review circular, toroidal, spherical, projective, Stiefel, Grassmann, and covariance-valued distributions, emphasizing their statistical assumptions, computational requirements, and dimensional limitations relevant to signal processing. We describe various modeling and estimation strategies, including analytical and gradient-based maximum likelihood, structured low-rank parameterizations, mixtures and hidden Markov models (HMMs), score-based and contrastive estimation, and geometry-aware K-means. To exemplify the consequences of geometry-aware modeling, we study multivariate phase coherence in synthetic data and functional neuroimaging, comparing toroidal phase vectors to projective, Grassmann, and positive-semidefinite representations. We also study speaker separation in a car environment, showing that directional representations outperform a standard Euclidean model. These examples demonstrate that when the geometric representation, statistical distribution, inference method, and computational regime are considered together, they have the potential to lead to improved models across a broad range of signal processing applications.

0.33

Bregman geometry in regular exponential families: dual representations and relative smoothness

cs.LG stat.MLmath.OC ·2026-11-13 ·Mame Diarra Fall, Éric Barat, Nicolas Dobigeon
similarity 0.56
Abstract

Regular exponential-family distributions are ubiquitous in statistics due to their appealing and convenient properties. Notably, they can be expressed in terms of Bregman divergences, revealing fundamental connections between statistics, convex analysis, and information geometry. However, this geometric structure has not been systematically exploited to characterize these distributions with respect to their expectation parametrization. Building on a projective characterization, we establish a general construction of equivalent dual Bregman representations, together with normalization and uniqueness results. These representations express log-densities as ordinary Bregman divergences in dual coordinate systems, providing geometric characterizations of the parameter space. So-called $β$- and $α$-scaled dual representations emerge as particular instances of this framework. We further derive dual Bregman representations of conjugate priors and establish that they preserve conjugacy. Applications to classical distributions illustrate the geometric structures underlying seemingly different statistical models. Moreover, we show that these representations provide natural reference geometries for establishing relative smoothness of the primal Bregman representation of the distribution, with explicit bounds that are sharp in the scalar and separable settings. An illustrative application to the Poisson distribution sheds new light on several works in the literature, providing complementary geometric justifications for reference geometries while deriving existing algorithms.

0.27