CS Paper Radar

Personalized Federated Hierarchical Gaussian Processes for Privacy-Preserving Modeling of Heterogeneous Distributed Systems

Xianjian Xie, Hao Yan

cs.LG ·submitted 2026-09-16 19:06 UTC · v1

Abstract

We present Personalized Federated Hierarchical Gaussian Processes (pFedHGP) for probabilistic regression and classification when data are distributed across heterogeneous clients. Each client's latent function decomposes into (i) a shared global component, (ii) a client-specific deviation that shares the global kernel structure, and (iii) a flexible local residual. Sparse inducing-variable approximations and federated variational inference keep raw data local while the server synchronizes only low-dimensional statistics for the shared component. Full predictive distributions support uncertainty-aware decisions. In application studies, pFedHGP attains perfect fault classification in press tonnage monitoring using 13.77% of labeled cycles and recovers geographic zones in federated air-quality modeling without centralizing station-level time series. An Instantaneous Linear Mixing Model viewpoint links the hierarchy to multi-output Gaussian processes for correlated sensors.

Closest papers

Nearest neighbours by meaning across the whole index.

Private and Distributionally Robust Federated Multiclass Classification

cs.LG math.OC ·2026-10-09 ·Andrea P. Hernandez, Michael Ibrahim, Nagi Gebraeel
benchmark similarity 0.62
Abstract

Fault diagnosis in asset-intensive industries requires multiclass models trained on noisy, sensitive data collected across geographically distributed client sites. Federated learning enables collaborative training without centralizing data, but most approaches do not provide formal privacy guarantees for individual records stored at client sites and can degrade under label noise. To address these limitations, we develop a differentially private, distributionally robust framework for federated multiclass classification that provides sample-level privacy for records stored at each client. We build on a federated Wasserstein distributionally robust multiclass objective with the Crammer-Singer loss, solved using an Alternating Direction Method of Multipliers (ADMM)-based algorithm. To protect the released client updates, our method injects calibrated Gaussian noise into the proximal linearized client updates of both the local model and the local distributionally robust variable before aggregation at the central server, and we quantify the cumulative privacy loss using the moments accountant. We provide per-round sample-level $(ε,δ)$-differential privacy guarantees and an overall composed privacy guarantee across all training rounds, together with convergence results and an explicit privacy-utility tradeoff. Experiments on simulated industrial fault data and UCI benchmark datasets show that our method outperforms differentially private federated empirical risk minimization and stochastic gradient descent baselines in most tested settings, with the largest gains under strong privacy-induced noise.

0.11

Predictively Oriented Gaussian Process Posteriors

cs.LG stat.ML ·2026-10-02 ·Callum Lau, Jeremias Knoblauch, Louis Sharrock
similarity 0.61
Abstract

Gaussian Processes (GPs) are a powerful tool for modelling and quantifying uncertainty in functional relationships. However, they require practitioners to make a number of design decisions, such as the choice of the kernel and the observation model. Suboptimal choices can produce misspecified models that do not capture the underlying data generating process. We introduce Predictively Oriented Gaussian Processes (PrO-GPs), which treat predictive uncertainty as the primary inferential target and provide a robust alternative to standard GPs. Although direct computation of a PrO posterior for nonparametric models is intractable, we derive a reduced formulation and practical sampling scheme for efficient computation. Through synthetic and real data experiments, we show that PrO-GPs produce better calibrated predictive distributions under model misspecification compared to standard GP approaches.

0.05

Resource-Efficient Distributed Recursive Gaussian Processes

cs.LG eess.SP ·2026-09-22 ·Josephine King, Ali Emre Balci, Raj Thilak Rajan
similarity 0.61
Abstract

Gaussian processes (GPs) provide a flexible framework for learning unknown functions from noisy measurements while quantifying predictive uncertainty, making them well suited for estimation in multi-agent systems. However, when measurements are collected by multiple agents, maintaining a unified GP model without centralized processing requires efficient distributed algorithms that can operate using local measurements and communication with neighboring agents. In this work, we develop two distributed recursive GP (RGP) algorithms for multi-output GP regression: ADMM-RGP and PDMM-RGP. We analyze the stability and convergence of both algorithms and develop parameter selection strategies to accelerate convergence, thus reducing the communication burden. The proposed methods are validated on a real-world multi-output wind dataset, and their convergence behavior is examined across communication graphs with varying connectivity. Numerical experiments demonstrate that ADMM-RGP and PDMM-RGP can significantly reduce communication relative to the state of the art, while maintaining comparable estimation accuracy and network-wide consensus.

0.13

Variational Outlier-Robust Gaussian Process Regression with Generative Modeling

cs.LG ·2026-08-17 ·Arslan Majal, Aamir Hussain Chughtai
similarity 0.60
Abstract · 5 pages, 1 figure

Outliers can substantially distort Gaussian process regression (GPR) due to its conventional Gaussian observation likelihood, leading to inaccurate model learning and prediction. To address this limitation, this article introduces a generative GPR model that captures observation-specific contamination and adaptively mitigates the influence of outliers. Subsequently, a variational generalized expectation-maximization procedure is used to learn the latent variables and GPR model parameters. Experiments on synthetic and real datasets under different contamination settings demonstrate that the proposed method remains competitive with-and in several cases outperforms-robust GPR baselines in prediction accuracy. Moreover, the proposed method shares the cubic computational scaling of the compared GPR methods.

0.23

COIN-GP: Cooperative Online Learning in Networked Distributed Systems with Partial Measurements via Gaussian Process Regression

cs.LG cs.MAeess.SY ·2026-09-17 ·Zewen Yang, Xiaobing Dai, Zhenxiao Yin, Hang Zhao, Zhijun Li +1 more
theory similarity 0.59
Abstract

In this paper, we tackle the problem of jointly estimating the system states and partially unknown dynamics within distributed sensor-equipped networks, particularly in scenarios where only partial state observations are available. To address this issue, we propose an observer-based dynamic cooperative learning framework incorporating online distributed Gaussian Process (GP) regression, which enables accurate estimation despite incomplete in measurements and deficient GP models. In addition, a novel data collection strategy is introduced, with theoretical conditions ensuring feasible data acquisition. Moreover, we also derive an error upper bound encompassing state estimation and model estimation, leveraging the deterministic error bounds of GPs. Empirical simulations demonstrate the superiority of our approach compared to existing distributed GP-based methods.

0.33

A Factor Graph Approach to Scalable Multi-Output Gaussian Process Regression

cs.LG ·2026-08-12 ·Wouter W. L. Nuijten, Esther G. van Pelt, Albert Podusenko, İsmail Şenöz, Wouter M. Kouw
similarity 0.58
Abstract · 20 pages, 8 figures, 1 Table. International Conference on Probabilistic Graphical Models 2026

Multi-output Gaussian process regression scales cubically in the number of observations times outputs, and dense kernel-matrix methods need bespoke handling whenever different outputs are observed at different inputs. We express multi-output Gaussian process regression as a Forney-style factor graph in which a nearest-neighbor chain orders a fixed candidate set of $C$ inputs into a one-dimensional sequence. Along this chain, latent Matérn processes evolve through linear-Gaussian transition factors, while the linear model of coregionalization mixes $L$ latent processes into $D$ outputs through a deterministic mixing factor and per-output scalar observation factors. Posterior computation reduces to exact Gaussian message passing on the chain at cost $\mathcal{O}(C(DL^2 + L^3))$ after chain construction, and missing observations omit their local factor without any covariance-matrix restructuring. The formulation therefore scales in the number of data samples and in the rate of missing observations, while remaining best suited to candidate sets in low input dimension. We compare the factor-graph formulation against an exact kernel-matrix baseline, a sparse-variational inducing-point baseline, and a nearest-neighbor baseline on a synthetic input-dimension sweep and on electricity time series forecasting. At low input dimension the factor-graph posterior tracks the exact kernel-matrix posterior closely, and the gap grows gradually as input dimension increases while staying competitive with both approximate baselines. On the electricity time series our factor-graph formulation matches all three baselines in forecast accuracy while scaling linearly in the number of data points, where the exact kernel-matrix method becomes infeasible and the inducing-point baseline remains substantially slower.

0.10

Decentralized Scalar Field Mapping using Gaussian Process

eess.SY ·2026-10-03 ·Hossein Papi, Muzaffar Qureshi, Kyle Volle, Omkar Sudhir Patil, Rushikesh Kamalapurkar
similarity 0.58
Abstract

Decentralized Gaussian process (GP) methods provide a scalable framework for multi-agent scalar-field estimation by allowing each agent to maintain a local GP over its assigned subdomain. Because neighboring agents condition their GP models on different measurements, their GP posteriors generally disagree over overlapping regions. This paper introduces the inducing points decentralized Gaussian process (IPD-GP) framework, which exploits these discrepancies to improve posterior consistency over shared regions. Each sender constructs receiver-specific data packets by selecting inducing locations targeted at the pairwise overlap and transmitting the corresponding posterior means and variances. Each receiver then selectively assimilates informative packets as persistent fictitious measurements in its local GP. The developed architecture preserves decentralized computation and communication without requiring full dataset exchange or centralized inference. Performance is evaluated using normalized root-mean-square error, negative log predictive density, and their overlap-restricted counterparts, with comparisons against non-communicating local GPs, decentralized aggregation methods, and a centralized GP. Simulation results show that IPD-GP achieves competitive predictive accuracy while reducing inter-agent communication.

0.18

Mapping urban heat from citizen-collected data: a scalable Gaussian process approach to correcting for preferential sampling

cs.LG stat.AP ·2026-09-21 ·Ellie K. Kim, Zachary D. Calhoun, Michael H. Bergin, David E. Carlson
similarity 0.57
Abstract

The urban heat island (UHI) effect amplifies heat stress in cities, increasing heat-related illness, mortality, sleep disruption, and energy demand. Because UHI intensity varies at neighborhood scales, intra-city temperature mapping is essential for assessing public health impacts, yet existing monitoring networks are sparse. Crowdsourced weather stations provide dense, hourly observations but suffer from noise and systematic biases, including preferential sampling driven by where residents install sensors. We introduce two scalable Gaussian process frameworks which adjust for preferential sampling and enable statewide, high-resolution temperature prediction. The first integrates sampling correction directly within a sparse variational Gaussian process (SVGPR+PS). The second uses a two-stage estimator involving a log-Gaussian Cox process (LGCP+GP) that estimates the sampling intensity field and incorporates it into temperature prediction. Both methods rely on inducing-point approximations and variational inference to scale beyond traditional Gaussian processes. Applied to approximately 10,000 citizen stations across North Carolina, the models generate hourly temperature fields within minutes. Validation against independent urban heat campaign data shows that the preferential sampling correction improves predictive performance, increasing average afternoon correlations across four cities from 0.26 to 0.43. This scalable framework enables neighborhood-level UHI assessment and supports studies of heat exposure disparities statewide.

0.25