CS Paper Radar

Information-Based Calibration of Uncertainty Quantification in Product-of-Experts Gaussian Process Models

Yean Hoon Ong, Paolo Barucca, Wei Pan, Jun Wang

cs.LG ·submitted 2026-08-29 16:13 UTC · v1
top venue

Abstract

Gaussian process (GP) regression with a single global GP (GP-glo) incurs cubic computational cost, limiting scalability to large datasets. Product-of-experts GP models (GP-pro), which combine local GP models to capture global correlations, alleviate this computational burden. However, training local experts on disjoint data subsets can lead to overestimated posterior variances. We propose GP-pro-c, a product-of-experts GP model that calibrates these variances using an information-based method. The method exploits the monotonicity and submodularity of information gain in GPs to define a calibration ratio that reduces the posterior variance of individual local GP models. We evaluate GP-pro-c using negative log-likelihood (NLL), root mean squared error (RMSE), and expected normalised calibration error (ENCE). Experiments on four synthetic functions and six regression datasets show that GP-pro-c achieves average reductions of 2.3% in NLL and 12.0% in ENCE compared with the uncalibrated GP-pro model. The proposed method mitigates posterior variance overestimation while maintaining predictive accuracy and reducing computational complexity. GP-pro-c provides a promising approach for uncertainty estimation in scalable GP models and may serve as a useful surrogate model for Bayesian optimisation with high-dimensional and large-scale data.

Comment
Published in the Journal of Artificial Intelligence Research, Volume 86 (2026)
Journal ref
Journal of Artificial Intelligence Research, Vol. 86 (2026)
DOI
10.1613/jair.1.20374

Closest papers

Nearest neighbours by meaning across the whole index.

Bayesian Optimisation Using Product-of-Experts Gaussian Process Models with Uncertainty Calibration

cs.LG ·2026-09-14 ·Yean Hoon Ong
similarity 0.80
Abstract

Bayesian optimisation (BO) typically relies on a single global Gaussian process (GP) model as its surrogate model. However, GP regression has cubic computational complexity in the number of training data points, limiting its applicability to large-scale optimisation problems. The product-of-experts Gaussian process model with uncertainty calibration (GP-pro-c) mitigates this limitation by combining multiple local GP experts, enabling improved uncertainty quantification, reduced computational cost, and preservation of global correlations. Despite these desirable properties, the use of GP-pro-c in BO has not been thoroughly studied. This paper introduces BO-pro-c, a Bayesian optimisation algorithm that uses GP-pro-c as its surrogate model, and evaluate its performance across a diverse range of BO settings. Experimental results suggest that BO-pro-c maintains competitive optimisation performance while achieving a 0.9% reduction in simple regret and a 39.4% reduction in computational overhead relative to a BO algorithm based on a single global GP model.

0.40

Large-Scale Bayesian Optimisation Using Uncertainty-Calibrated Product-of-Experts Gaussian Process Models with Trust Regions

cs.LG ·2026-11-01 ·Yean Hoon Ong
similarity 0.75
Abstract

The use of collections of independent local Gaussian processes (GP-ind) combined with trust regions is a widely adopted approach for large-scale and high-dimensional Bayesian optimisation (BO). However, this approach suffers from several limitations, including discontinuities at the boundaries of disjoint regions, an inability to capture long-range spatial correlations, potential information loss, and abrupt increases in computational overhead. To address these limitations, this paper introduces a novel BO algorithm, termed BO-pro-c-tr. The proposed method employs a product-of-experts Gaussian process model with uncertainty calibration (GP-pro-c) as its surrogate model and incorporates a one-to-many association strategy for integrating GP-pro-c with trust regions. Empirical results suggest that, when combined with the GP-LCB acquisition function, BO-pro-c-tr achieves an average reduction of 2.6% in simple regret and 34.9% in computational overhead relative to BO algorithms based on independent local GP models coupled with trust regions.

0.07

Predictively Oriented Gaussian Process Posteriors

cs.LG stat.ML ·2026-10-02 ·Callum Lau, Jeremias Knoblauch, Louis Sharrock
similarity 0.74
Abstract

Gaussian Processes (GPs) are a powerful tool for modelling and quantifying uncertainty in functional relationships. However, they require practitioners to make a number of design decisions, such as the choice of the kernel and the observation model. Suboptimal choices can produce misspecified models that do not capture the underlying data generating process. We introduce Predictively Oriented Gaussian Processes (PrO-GPs), which treat predictive uncertainty as the primary inferential target and provide a robust alternative to standard GPs. Although direct computation of a PrO posterior for nonparametric models is intractable, we derive a reduced formulation and practical sampling scheme for efficient computation. Through synthetic and real data experiments, we show that PrO-GPs produce better calibrated predictive distributions under model misspecification compared to standard GP approaches.

0.05

Calibrating your confidence requires thinking of all the ways you could be wrong

cs.LG stat.ML ·2026-11-06 ·Marco Miani
similarity 0.69
Abstract · PhD Thesis, Technical University of Denmark (DTU), 2026

A primary limitation of modern deep learning is the miscalibration of model confidence; traditional architectures fundamentally lack a mechanism to quantify their own uncertainty. Larger models and larger datasets mitigate this issue, but they do not resolve the underlying structural deficiency. Bayesian methods offer a principled uncertainty-aware learning framework through stochastic parameters. Yet, despite the established benefits, the inherent computational overhead often compromises either scalability or quality. This trade-off currently prevents the widespread adoption of Bayesian methods. This Thesis bridges the gap between high-quality and low-complexity by introducing a novel distribution on parameter space: a degenerate Gaussian whose covariance is a projection matrix aligned with the null space of the Fisher information matrix. We demonstrate the scalability of this Bayesian posterior through two numerically efficient algorithms: one for posterior sampling and another for the direct computation of predictive confidence. Empirically, we show State-Of-The-Art confidence calibration and out-of-distribution detection for large models. We provide theoretical grounding for this approach. We study the degeneracies inherent in Amari's information geometry and highlight its most relevant component: the split of parameter's perturbations into two orthogonal subspaces. From this perspective, our posterior naturally emerges as the simplest choice that captures this split, assigning unitary variance to one subspace and zero variance to the other. Moreover, through the tools of differential geometry, we utilize this split to identify and rectify a critical misalignment issue of the non-linearized Laplace Approximation. Finally, with a focus on PAC-Bayes generalization bounds, we pave the way for a deeper understanding of the learning capabilities of overparametrized models.

0.12

Residual Correlation as a Diagnostic for Joint-Uncertainty Gains from GP Coregionalisation

cs.LG ·2026-09-24 ·Fangqin Zhou, Joaquin Vanschoren
published similarity 0.65
Abstract · Accepted at ACML 2026

In multi-target regression, correlated targets are often coupled through multi-output Gaussian processes with an intrinsic model of coregionalisation (GP-ICM), assuming that sharing statistical strength improves overall performance. In practice, the benefits are inconsistent. Across the settings studied, we find that the main benefit of coregionalisation is joint uncertainty quantification rather than point prediction. Raw target correlation does not predict when coupling helps; in the separable GP-ICM settings studied here, residual correlation, the cross-target dependence left unexplained by independent per-target predictors, is the strongest predictor of joint-uncertainty gains. We introduce a lightweight diagnostic, $D_{\rm logdet}=-\frac{1}{2}\log\det R_{\rm res}$, which represents the idealised joint negative log-likelihood (NLL) gain from modelling a full rather than diagonal residual covariance and is computable from independent GPs alone. Across a controlled synthetic study, 16 multi-target benchmarks, and frozen transformer and convolutional neural network representations for keypoint regression, point prediction remains largely unchanged ($ΔR^2\approx 0$). In contrast, $D_{\rm logdet}$ strongly predicts observed ICM NLL improvements ($ρ_s=-0.83$, $p<0.001$), outperforming heuristics such as the feature-to-sample ratio. We also propose Residual-ICM, which preserves independent marginal variances while adding residual-correlation structure to the joint covariance. Residual-ICM achieves the best average joint NLL among the compared methods, while the diagnostic indicates when covariance coupling is likely to be useful. The diagnostic is specific to global Gaussian residual dependence, the structure captured by separable coregionalisation.

0.23

Input Bias Correction in Gaussian Process Regression

cs.RO cs.LG ·2026-10-22 ·Muzaffar Qureshi, Tochukwu E. Ogri, Trivikram Satharasi, Kyle Volle, Rushikesh Kamalapurkar
top venue similarity 0.65
Abstract · Accepted for publication in IEEE Robotics and Automation Letters (RA-L) (Oct, 2026)

In traditional Gaussian process (GP) regression, the inputs are typically assumed to be known, and uncertainty enters only through the measurements. When scalar fields are mapped using GP regression with measurements obtained from a mobile sensor, the inputs correspond to the estimated pose of the mobile sensor. The goal in this letter is to understand how a recorded GP, trained on erroneous inputs (e.g., pose estimates from dead-reckoning), can be corrected in a numerically efficient way when better estimates of the inputs become available. In the developed bias-aware GP (Ba-GP) framework, the recorded GP posterior-mean gradient is used to propagate estimated input bias and noise into mean and covariance corrections. The resulting additive covariance structure is then exploited to develop numerically efficient first-order correction algorithms. A simulation study and an indoor light-intensity mapping experiment show that Ba-GP improves predictive accuracy and uncertainty calibration relative to noisy-input GP baselines, with an order-of-magnitude reduction in computational cost relative to full GP retraining.

0.33

Scalable Gaussian Process Regression via Deterministic Trigonometric Features: Uniform Bounds for Safe Model Predictive Control

eess.SY ·2026-08-17 ·Julius Jagdt, Johanna Menn, Sebastian Trimpe, Melanie N. Zeilinger, Anna Scampicchio
similarity 0.64
Abstract

Learning-based Model Predictive Control (MPC) using Gaussian processes (GPs) is an effective approach for safe control in the presence of model mismatch. High-probability safety guarantees typically require uncertainty bounds that hold uniformly over the entire state--input domain, but existing bounds are available only for full GP regression. Since exact GP inference scales poorly with the number of data points, its deployment is impractical in large-data regimes. We close this gap by developing a scalable GP framework that admits the derivation of uniform uncertainty bounds. We formalize a deterministic trigonometric feature Gaussian process (DTF-GP), a finite-dimensional kernel approximation based on discretized trigonometric features that reduces GP regression to Bayesian linear regression in feature space. We derive a high-probability uniform uncertainty bound for the proposed DTF-GP and provide its closed-form solution for the squared-exponential kernel case. Finally, we integrate the DTF-GP into a learning-based MPC scheme and demonstrate that it provides high-probability safety guarantees and exploration performance comparable to a full GP while improving computational efficiency in large-data regimes.

0.28

Online Adaptive Kernel Mixing for Gaussian Process Decision Making

cs.LG ·2026-09-17 ·Kavin Aravindan, Mani Tej Sriram, Gautam Dasarathy, Tejas Bodas
published similarity 0.64
Abstract · 35 pages, 9 figures. Accepted as a full paper at IFIP Performance 2026

Gaussian Processes (GPs) are widely used as surrogates for black-box functions in sequential decision-making problems such as Bayesian optimization (BO), level set estimation (LSE), and Bayesian active learning (BAL). GP performance critically depends on kernels, and standard kernels can lead to suboptimal decisions under misspecification. To address this, we introduce HACK GPs (Hedge Adaptive Cumulative Kernels), a method that views kernel selection as an online learning with expert advice problem. HACK treats each candidate kernel as a GP "expert" and updates a distribution over experts online using AdaHedge, based on a loss received as a proxy for their ability to fit the function and align with the task objective. We provide two variants of HACK: (i) Mixture of Gaussians (MoG) and (ii) categorical sampling. We establish general guarantees showing that, under a loss-gap condition, the weight concentrates on the best kernel and the resulting acquisition function is close to that of the best expert. Empirically, we observe robust performance across BO, LSE, and BAL compared to standard kernels such as Squared Exponential and Matern-5/2, as well as simple ensemble baselines.

0.33