CS Paper Radar

Robust Covariance Estimation for Entry-wise Contaminated and Partially Observed Data via Expectation Maximization Algorithm

Matys Savidan, Mohammed Nabil El Korso

cs.LG stat.ML ·submitted 2026-10-15 11:31 UTC · v1

Abstract

Robust covariance estimation under entry-wise contamination and missing data is challenging in various signal processing applications. Under the Student-$t$ model, observation-level reweighting induces a uniform attenuation across samples, which can lead to cumulative information loss even when most coordinates remain informative. The alternative Student-$t^*$ model enables coordinate-wise robustness, but its E-step is not available in closed form. We extend inference for the Student-$t^*$ model to incomplete observations through a stochastic approximation EM (SAEM) algorithm operating on a reduced latent space: stochastic updates of the coordinate-wise scales are combined with Rao-Blackwellized Gaussian conditional moments, which avoids explicit sampling of latent Gaussian variables while preserving a likelihood-based framework. Experiments on synthetic and real sensor data show a lower Fisher-Rao error than Gaussian, Student-$t$, imputation-based and detect-then-estimate estimators at moderate to high outlier rates under diffuse contamination.

Closest papers

Nearest neighbours by meaning across the whole index.

SAUSS: Stochastic Approximation with Unbiased Simulated Scores for Limited Dependent Variable Models

cs.LG stat.MEecon.EM ·2026-08-26 ·Sokbae Lee, Yuan Liao, Myung Hwan Seo, Youngki Shin
theory similarity 0.60
Abstract

Simulated maximum likelihood is widely used in latent-variable models where likelihood evaluation requires intractable integrals or probabilities. With a fixed number of simulation draws, its estimating equation is generally biased. We develop Stochastic Approximation with Unbiased Simulated Scores (SAUSS), combining score simulation with stochastic gradient descent. The Hajivassiliou--McFadden identity, an instance of Fisher's identity, expresses the likelihood score as the conditional expectation of the complete-data score given the observed data. Exact conditional draws yield an unbiased simulated score for any fixed number of draws. The estimator averages stochastic gradient iterates computed using mini-batches of fixed size regardless of sample size, avoiding repeated full-sample likelihood evaluations. Under local regularity and initialization conditions, we establish asymptotic normality and a functional central limit theorem for the partial-sum process of the iterates under multi-pass sampling from an observed sample. These results support plug-in and random-scaling confidence intervals that remain asymptotically valid when algorithmic uncertainty is non-negligible relative to sampling uncertainty. We develop multinomial probit as a leading application. In simulations and two applications, SAUSS returns estimates in every reported run, including cases where the simulated maximum likelihood implementation returns none. When both return estimates, SAUSS gives comparable estimates in less time.

0.21

Moment-Accurate Gaussian Mixtures for Constant-Step Stochastic Approximation

cs.LG stat.ML ·2026-10-04 ·Xiaoli Li, Wei Biao Wu
similarity 0.59
Abstract

Local Gaussian models of constant-step learning predict output variability and expected losses, but weak convergence alone does not justify these moment predictions. We establish moment-accurate Gaussian mixtures by matching stationary energy with local Ornstein--Uhlenbeck limits, ruling out quadratic tail mass invisible to weak convergence. For step size $a$, the second-order Wasserstein error is $o(\sqrt a)$, uniformly over invariant laws, using each law's actual root weights. The assumptions combine confinement, descent, finitely many hyperbolic equilibria and root continuity with finite-variance innovations. The result yields observable covariances, expected objective gaps and first-order mean shifts, while allowing singular covariances, compatible saddles and weights without a limit. For additive noise given by a fixed invertible transform of independent standardized Student $t_3$ coordinates, symmetry gives an order-sharp $\sqrt a$ smooth-test bound. Numerical transport calculations demonstrate the value of root-specific covariances; controlled SGD studies assess observable predictions across step sizes, batch sizes and model geometries.

0.29

Input Bias Correction in Gaussian Process Regression

cs.RO cs.LG ·2026-10-22 ·Muzaffar Qureshi, Tochukwu E. Ogri, Trivikram Satharasi, Kyle Volle, Rushikesh Kamalapurkar
top venue similarity 0.58
Abstract · Accepted for publication in IEEE Robotics and Automation Letters (RA-L) (Oct, 2026)

In traditional Gaussian process (GP) regression, the inputs are typically assumed to be known, and uncertainty enters only through the measurements. When scalar fields are mapped using GP regression with measurements obtained from a mobile sensor, the inputs correspond to the estimated pose of the mobile sensor. The goal in this letter is to understand how a recorded GP, trained on erroneous inputs (e.g., pose estimates from dead-reckoning), can be corrected in a numerically efficient way when better estimates of the inputs become available. In the developed bias-aware GP (Ba-GP) framework, the recorded GP posterior-mean gradient is used to propagate estimated input bias and noise into mean and covariance corrections. The resulting additive covariance structure is then exploited to develop numerically efficient first-order correction algorithms. A simulation study and an indoor light-intensity mapping experiment show that Ba-GP improves predictive accuracy and uncertainty calibration relative to noisy-input GP baselines, with an order-of-magnitude reduction in computational cost relative to full GP retraining.

0.33

Adaptive Robust Confidence Sets in High Dimensions

cs.DS math.STcs.LGstat.ME+1 ·2026-10-11 ·Ilias Diakonikolas, Chao Gao, Daniel M. Kane, Yuetian Luo
theory similarity 0.58
Abstract

We study adaptive confidence sets for mean vectors of multivariate Gaussians with identity covariance under Huber contamination when the contamination proportion is unknown. We show that an information-theoretically optimal adaptive confidence set can be constructed by simultaneously quantifying uncertainty across all one-dimensional projections. The resulting confidence set achieves the optimal diameter and reveals a substantial adaptation cost compared with the minimax estimation error. Although statistically optimal, this construction is computationally infeasible. We establish a Statistical Query (SQ) lower bound showing that achieving diameter $O(m^{-1/2})$ requires either exponentially many queries or query tolerance $d^{-Ω(m)}$. Thus, in contrast to robust mean estimation, adaptation to the unknown contamination proportion incurs both a statistical cost and a computational barrier within the SQ framework. Finally, we give a low-degree moment-matching algorithm that constructs an adaptive confidence set using $d^{O(m)}$ samples and time. Its diameter bound contains an additional $O(m^{-1/2})$ term beyond the information-theoretic rate, matching the dependence on $m$ in the SQ lower bound. Together, these results characterize the statistical limits of adaptive robust confidence sets under Huber contamination and quantify a computational barrier within the SQ framework.

0.36

Assumption-lean logistic regression with missing covariates

cs.IT stat.MLcs.LGmath.ST+1 ·2026-10-05 ·Jyotishka Ray Choudhury, Kabir Aladin Verchand, Richard J. Samworth, Ashwin Pananjady
similarity 0.58
Abstract

Missing covariates are frequently encountered in supervised learning problems, and classical methods for estimation using such data use carefully chosen imputation schemes for missing data, or likelihood approximations that lead to nonconvex $M$-estimation problems. These methods and their relatives are suitable for scenarios in which the covariate distribution is known, and more broadly, have enjoyed tremendous success in linear models. But even in basic nonlinear problems such as logistic regression in moderate dimensions, such methods can experience drastic failure modes when the covariate distribution is unknown. Motivated by the need for reliable alternatives, we consider the problem of parameter estimation in logistic regression with missing covariates. Crucially, we operate in the assumption-lean setting where the covariate distribution is unknown (but bounded). We design a stochastic approximation method that is based on $Z$-estimation with a novel monotone operator, and establish that our algorithm is computationally efficient and achieves provable signal recovery at parametric rates under the hypothesis that covariates are missing completely at random. Our theory sharply characterizes the $\ell_2^2$ risk of the estimator in terms of the missingness profile, accommodating heterogeneous observation probabilities. Importantly, it shows that our method always outperforms the de facto ``complete-case'' estimator that ignores observations with any missing data. Even in the setting with homogeneous missingness (in which each covariate is observed independently with probability $q$), our bounds exhibit intricate and nonstandard dependence on $q$ that can yield significant improvements over using only complete cases. We complement our upper bounds with new information-theoretic lower bounds that show that this intricate dependence on $q$ is fundamental in a minimax sense.

0.25

Latent Score-Based Bayesian Cramér-Rao Bound Estimation for High-Dimensional Imaging Systems

cs.IT stat.MLcs.LGmath-ph+1 ·2026-10-02 ·Evan Scope Crafts, Thomas Wynn, Seonyeong Park, Mark Anastasio, Umberto Villa
similarity 0.57
Abstract

We propose a data-driven framework for estimating the Bayesian Cramér-Rao bound (CRB) in high-dimensional imaging systems with complex, analytically intractable priors. Direct CRB computation is challenging in this setting due to the need to model the prior score and to form and invert the Bayesian Fisher information matrix in very high dimensions. To address these issues, we first reformulate the inverse problem in the latent space of a pre-trained variational autoencoder, thereby dramatically reducing the dimensionality of the bound estimation problem while preserving the spatial structure of the images. The Bayesian CRB is formed in this latent space and mapped back to the native parameter space using a change-of-variables formula. Second, to learn the latent prior score, we introduce a new sliced score matching objective defined in a Bochner space endowed with an $H^1(Ω)$ Sobolev norm in space. This "Bochner-space sliced score matching" objective is consistent with standard sliced score matching, but penalizes errors in the spatial gradients of the score, suppressing high-frequency artifacts that otherwise contaminate the resulting CRB estimates. We validate the approach on a stylized quantitative photoacoustic computed tomography (qPACT) breast imaging problem with over one million unknown parameters, using a foundation-model autoencoder derived from Stable Diffusion. The proposed method yields stable, artifact-free Bayesian CRB estimates that reflect the highly non-Gaussian structure of the learned prior and reveal the substantial impact of the prior on the relative performance of competing qPACT design schemes.

0.28

Online Covariance Estimation for Averaged Stochastic Gradient Descent on Riemannian Manifolds

cs.IT stat.MLcs.LGmath.OC+1 ·2026-10-24 ·Jingrong Guan, Liwei Jiang, Sen Na
similarity 0.57
Abstract · 60 pqges, 7 figures, 2 tables

We study online inference for averaged Riemannian stochastic gradient descent (RSGD) when the unknown parameter lies on a manifold. Although the averaged RSGD iterate is asymptotically normal, practical inference requires estimating the unknown covariance operator in its limiting distribution. Existing online batch-means methods cannot be applied directly because a general Riemannian manifold lacks an addition operation, and the tangent space containing the Riemannian gradient varies across iterations. Through vector transport, we construct overlapping and non-overlapping covariance estimators that adaptively combine information from successive batches while accounting for the changing tangent spaces. The proposed estimators are online, require neither Riemannian Hessian computations nor a prespecified terminal sample size, and attain the same convergence rates as their Euclidean counterparts (Zhu et al., 2023). Together with asymptotic normality, these estimators enable online statistical inference for RSGD on manifolds. We illustrate the proposed framework through a detailed study of streaming principal component analysis on Grassmann manifolds.

0.14

Variational Outlier-Robust Gaussian Process Regression with Generative Modeling

cs.LG ·2026-08-17 ·Arslan Majal, Aamir Hussain Chughtai
similarity 0.56
Abstract · 5 pages, 1 figure

Outliers can substantially distort Gaussian process regression (GPR) due to its conventional Gaussian observation likelihood, leading to inaccurate model learning and prediction. To address this limitation, this article introduces a generative GPR model that captures observation-specific contamination and adaptively mitigates the influence of outliers. Subsequently, a variational generalized expectation-maximization procedure is used to learn the latent variables and GPR model parameters. Experiments on synthetic and real datasets under different contamination settings demonstrate that the proposed method remains competitive with-and in several cases outperforms-robust GPR baselines in prediction accuracy. Moreover, the proposed method shares the cubic computational scaling of the compared GPR methods.

0.23