CS Paper Radar

A Bayesian Model Updating Framework for Systems Under Hybrid Uncertainties via Probability Integral Transform and Maximum Mean Discrepancy

Shijie Zhong, Jiangfeng Fu

cs.CE ·submitted 2026-09-16 12:35 UTC · v1
benchmark

Abstract

Model updating under hybrid uncertainty is challenging because aleatory input variability makes the simulator output a probability distribution rather than a scalar, rendering the likelihood analytically intractable. Existing Approximate Bayesian Computation (ABC) methods typically employ nested Monte Carlo sampling, where aleatory samples are redrawn for each epistemic parameter evaluation, introducing sampling noise into the discrepancy and consequently affecting posterior inference and model evidence. This paper eliminates this resampling noise by construction. The probability integral transform (PIT) converts the stochastic simulator into a deterministic map of distribution-free latent variables and epistemic parameters. By freezing a set of stratified quantile particles, the resulting discrepancy becomes a deterministic, sampling-noise-free function of the unknown parameters. Transitional Markov Chain Monte Carlo (TMCMC) is then employed for posterior inference and model evidence estimation. The framework is validated on a two-dimensional benchmark, a high-dimensional transient oscillator, and Subproblem A of the NASA Langley Multidisciplinary Uncertainty Quantification Challenge. The complete Bayesian analysis is achieved in approximately half a minute on a standard desktop workstation.

Closest papers

Nearest neighbours by meaning across the whole index.

A Sequential Monte Carlo Method for Data Assimilation in Subsurface Flow Systems with Hierarchical Uncertainty

cs.LG physics.geo-ph ·2026-11-08 ·Wenchao Teng, Guido Di Federico, Louis J. Durlofsky
similarity 0.65
Abstract

Subsurface flow systems often involve hierarchical geological uncertainty, meaning both higher-level metaparameters and cell-by-cell properties are uncertain. This leads to challenges for data assimilation because both sets of parameters must be estimated. In this work we develop an adaptive-tempered sequential Monte Carlo (SMC) procedure for joint Bayesian estimation of these quantities. The method uses noncentered geological parameterizations, with geomodel realizations represented by metaparameters and Gaussian latent variables that are independent under the prior. Adaptive likelihood tempering, importance weighting, resampling, and blockwise Markov chain Monte Carlo (MCMC) mutation are applied at each iteration. Flow surrogates are used for all function evaluations. The hierarchical SMC procedure is further extended to treat successive data assimilation. In this case, existing posterior particles are updated in a manner that accounts for correlations in surrogate model errors across observation batches. We consider two distinct 3D systems with hierarchical uncertainty. The first is a faulted carbon storage system containing three stacked aquifers and two extensive faults. Principal component analysis is used to represent target-aquifer heterogeneity, while geological metaparameters characterize uncertain aquifer and fault properties. The second is a channelized oil-water system containing three facies with uncertain properties. A latent diffusion model represents variability in channel geometry and facies realizations. For the four synthetic true models considered, SMC posterior metaparameter distributions and flow predictions agree closely with reference MCMC results. SMC, however, requires fewer function evaluations and provides speedups of 2.5--7.0 for single-stage data assimilation. Even greater speedups are achieved with multiple GPUs.

0.22

Broadly Applicable Approximate MCMC for Switching Stochastic Differential Equations Using Uniformization and Time-Conditioned Factorized Neural Likelihood Estimation

cs.LG stat.ML ·2026-10-07 ·Shion Hosoda, Michiaki Hamada
similarity 0.62
Abstract

Switching stochastic differential equations (SSDEs) describe continuous-time dynamics whose parameters switch according to a latent regime process that follows a continuous-time Markov chain (CTMC). By allowing dynamics to change between regimes, SSDEs represent heterogeneous system behavior and have been applied across diverse fields. However, Bayesian inference for SSDEs remains difficult, and existing SSDE inference methods have limited applicability, with restrictions such as noise-free observations, univariate states, linear drift, or state-independent diffusion. In this study, we propose an approximate Markov chain Monte Carlo sampler for SSDEs using uniformization and factorized neural likelihood estimation (FNLE), a simulation-based inference method. Uniformization provides an exact representation of the CTMC but requires SDE transition densities over arbitrary time intervals. We approximate these densities by training a time-conditioned FNLE model. The resulting sampler is broadly applicable to SSDEs without requiring analytically tractable transition densities. In synthetic-data experiments, our method recovered regime paths and parameters for three SSDE models for which previous methods have limited applicability. We also applied our method to a real dataset and detected a regime transition.

0.24

FLARE MCMC: Fidelity-based Layer-Adaptive REcursive proposals for MCMC

cs.AI ·2026-08-13 ·Harini Venkatesan, Christian Shelton, Ming-Feng Ho, Simeon Bird, Mengxuan Wu
published similarity 0.61
Abstract · This is the author's accepted manuscript of an article published in SIAM Journal on Uncertainty Quantification. The final version is availab…

Markov chain Monte Carlo (MCMC) requires only the ability to evaluate the likelihood, making it a common technique for inference in complex models. However, it can have a slow mixing rate, requiring the generation of many samples to obtain good estimates and an overall high computational cost. FLARE MCMC is a multi-fidelity layered MCMC method that exploits lower-fidelity approximations of the true likelihood calculation to improve mixing and leads to overall faster performance. Such lower-fidelity likelihoods are commonly available in scientific and engineering applications where the model involves a simulation whose resolution or accuracy can be tuned. Our technique uses recursive, layered chains with simple layer tuning; it does not require the likelihood to take any form or have any particular internal mathematical structure. We demonstrate experimentally that FLARE MCMC achieves larger effective sample sizes for the same computational time across different scientific domains including hydrology and cosmology.

0.40

Bayesian Inference of Expensive Simulators: Survey and Comparison of Design Choices

cs.LG ·2026-10-27 ·Šimon Soldát, Václav Šmídl
publishedsurvey similarity 0.61
Abstract · 25 pages, 8 figures. Accepted for publication in Statistics and Computing. This is the submitted manuscript; the revised version, including …

We present a survey of approaches to Bayesian inference with simulator forward models, focusing on settings with prohibitively expensive simulators. In this domain, Bayesian optimization-based methods have emerged as the most promising approach. We unify existing methods into a general framework, enabling consistent reasoning about shared algorithmic structure across the fragmented literature. Within this framework, we identify the primary design choices and perform a systematic empirical comparison of their effects on inference efficiency in terms of required simulation budget. A key result of our study is that inference efficiency is often dominated by the choice of surrogate-model response variable, as it directly governs the complexity of the modeled function. Specifically, we demonstrate that the prevalent scalar log-likelihood is a poor response variable, as it discards crucial information. Based on this insight, we introduce the concept of custom \emph{proxy variables}: transformations of simulator outputs engineered to retain more information and provide smoother response surfaces, substantially improving inference efficiency over standard response variables. We provide actionable guidance and a numerically robust Julia package to facilitate practical adoption of Bayesian optimization for simulator inverse problems.

0.41

Behavioral Mapping of Hierarchical Dynamical Models Accelerates Bayesian Parameter Inference

eess.SY stat.MEq-bio.QM ·2026-10-22 ·Jingsong Bai, Ayush Pandey
similarity 0.59
Abstract

Bayesian parameter inference for complex nonlinear dynamical models is often limited by the cost of repeated model simulation. We present a behavioral mapping framework that uses the inferred posterior of a higher-level model to accelerate inference of a lower-level model without requiring state or parameter correspondence between them. Building on approximate simulation relations for hierarchically related models, we define the notion of behavioral compatibility and quantify the behavioral support of detailed model parameters using the reduced model posterior. We learn a level set of behavioral support using a Gaussian process and use it to filter detailed model parameter proposals before expensive simulations during Markov Chain Monte Carlo (MCMC) inference. We derive conditions under which the behavioral support level set retains all detailed model parameters that fit the experimental data within a prescribed tolerance. We apply this method to a hierarchy of gene-expression models fitted to cell-free experimental data. After initial setup costs, our approach speeds up MCMC by 8.02% for the phenomenological model and 15.21% for the mechanistic model.

0.15

Product Monte Carlo: Expectation Under a Product of Densities

cs.LG stat.MEstat.CO ·2026-10-08 ·Pietro Carlotti, Sifan Liu, Yuling Yao
similarity 0.59
Abstract

Aggregating independently computed sub-sample inferences is a well-known challenge in large-scale Bayesian computation, federated learning, and opinion pooling. The core problem is to approximate a distribution whose density is proportional to a product of individual densities. We propose Product Monte Carlo (PMC), an algorithm for computing arbitrary expectations under a product of densities using only Monte Carlo draws from individual densities. We turn the intractable product expectation into a regression problem by expressing it through a family of computable conditional expectations. Besides the point estimate, our fully Bayesian regression formulation provides coherent uncertainty quantification and principled diagnostics. We establish theoretical guarantees on consistency and characterize convergence rates for the PMC estimator. Across simulated and real-data experiments varying the number, dimensionality, and heterogeneity of the component distributions, PMC achieves higher estimation accuracy, better-calibrated uncertainty coverage, and improved scalability at a lower computational cost than existing methods. In addition, PMC requires no density evaluation, making it applicable to simulation-based inference, as we demonstrate in an application with protein images.

0.31

Model-Agnostic Online Certificate-Driven Calibration for Time Series Forecasting Under Distribution Shift

cs.LG ·2026-09-25 ·Chenfeng Huang, Zixuan Ma, George Michailidis
top venuehot directionhighly cited similarity 0.58
Abstract · Oral presentation at the 42nd Conference on Uncertainty in Artificial Intelligence (UAI 2026). Published in Proceedings of Machine Learning …

Time series out-of-distribution generalization requires forecasters to remain reliable when deployment dynamics differ from training conditions due to covariate shift, concept shift, and temporal dependence. Probably Approximately Correct Bayesian domain adaptation provides computable certificates by decomposing target risk into a source risk term, a source-to-target mismatch term, and a complexity term, but standard analyses rely on independent sampling and distributional stability, assumptions that are violated in time series by serial dependence and nonstationary shift. We propose a model-agnostic online martingale Probably Approximately Correct Bayesian framework that yields finite-sample certificates under temporal dependence and distribution shift. The certificate replaces independent-sample concentration with martingale concentration that adapts to loss scale and predictable variation. We use the certificate as a surrogate regularizer for online calibration by training a gated residual Bayesian head on top of a fixed forecasting backbone, producing a corrective update that reverts to the backbone prediction when the gate is closed. Online calibration combines a source risk anchor, a posterior-shift penalty, and a time-adaptive mismatch term computed from target windows observed before forecasting. It follows a predict-then-update protocol in which outcomes become available only after forecasting and are used to update subsequent predictions. Experiments across convolutional, attention-based, and large language model-based forecasters show improved stability and accuracy under covariate and concept shift.

0.76

Exact Bayesian layers via characteristic functions

cs.LG stat.MLstat.CO ·2026-11-19 ·William H. W. Thompson, Guillaume St-Onge, Laurent Hébert-Dufresne, Jean-Gabriel Young
theory similarity 0.58
Abstract

Bayesian neural networks provide principled uncertainty estimates, but training them with Monte Carlo sampling can be expensive. Sampling-free variational inference algorithms make training Bayesian neural networks feasible, but existing approaches assume a Gaussian output distribution, which can be inaccurate in applied problems. Here, we introduce CF-Prop, an uncertainty propagation method that uses characteristic functions to compute the exact output distribution of a Bayesian ReLU layer, without any sampling, even in non-Gaussian settings. Because CF-Prop propagates characteristic functions, the evidence lower bound---the objective of variational training---can be computed in closed form for a broad class of likelihoods, including the Gaussian, Poisson, and Gamma. This allows to implement an efficient resulting variational inference algorithm, CF-VI. On CIFAR-100, CF-VI achieves the lowest calibration error of any tested method, improving on the next best method by about 35%. On a heavy-tailed insurance claim dataset, CF-VI accurately captures tail uncertainty that is severely underestimated by both other sampling-free methods and Monte Carlo methods.

0.14