CS Paper Radar

Amortized Data Borrowing with Exchangeability-Aware Neural Posterior Estimation

Chin-Hung Huang, JooChul Lee, Huan He

cs.LG stat.ML ·submitted 2026-09-30 03:51 UTC · v1
top venuecode/data

Abstract

Augmenting small concurrent studies with external or historical cohorts is attractive in drug development, where enrollment is slow, follow-up is expensive, and closely related trial or real-world data are often already available. Bayesian dynamic borrowing (BDB) provides a principled framework for adaptively controlling the influence of external data, but classical implementations often depend on hand-specified priors and MCMC-based inference, which can be computationally expensive and not generalizable. In this work, we study amortized neural posterior estimation (NPE) as a flexible alternative. A single network is pretrained on simulated current/external dataset pairs spanning covariate shift, outcome drift, and joint non-exchangeability, and then returns an approximate posterior for a scalar current-study target in a single forward pass. Through simulation studies, we find that NPE is most useful under outcome drift and joint mismatch: in the harder outcome-drift regimes, it gives up to about five-fold lower absolute bias than the best classical baseline and keeps Type I error close to nominal. After pretraining, posterior summaries are obtained in about 8 ms per dataset, roughly $10^3\times$ faster than MCMC-based borrowing baselines in our timing experiment. We further analyze Alzheimer's Disease Neuroimaging Initiative (ADNI) data and show that, when mild cognitive impairment outcomes differ across cohorts, the NPE formulation recovers the later-cohort risk level in this example without claiming greater precision. Code is available at https://github.com/ChinHungScott/NPE-for-Bayesian-Dynamic-Borrowing-MLHC-.

Comment
27 pages. Accepted at the 11th Machine Learning for Healthcare Conference (MLHC 2026)
Journal ref
Proceedings of the 11th Machine Learning for Healthcare Conference, PMLR 340:584-610, 2026

Closest papers

Nearest neighbours by meaning across the whole index.

Semi-Amortized Bayesian Inference with Reusable Local Posteriors

cs.LG stat.ML ·2026-10-13 ·Niek Stevenson, Paul-Christian Bürkner, Andrew Heathcote, Stefan T. Radev
similarity 0.70
Abstract

Amortized Bayesian inference uses simulations to perform Bayesian estimation when likelihoods are expensive or analytically unavailable. However, learning the full joint posterior is challenging in high-dimensional models with complex dependencies. We introduce a semi-amortized method that divides parameters into smaller blocks, and uses conditional independence to identify the information needed to infer each block. We train neural estimators for blocks whose posterior updates are computationally expensive or frequently repeated. Each estimator learns from simulations of the relevant model components to approximate the block's full conditional posterior, given the relevant observations and the values of the variables on which it depends. A Markov chain then alternates direct draws from these estimators with conventional updates of the remaining variables. Each neural update uses the current conditioning values and requires neither likelihood evaluation nor an acceptance-rejection step. Where the same conditional structure recurs, one estimator can be reused across blocks and datasets, with an output dimension determined by the block rather than the full model. We illustrate the approach in high-dimensional and computationally expensive hierarchical and spatiotemporal models.

0.21

Simulation-Efficient Double-Flow Neural Posterior Estimation for Agent-Based Models

cs.MA ·2026-10-26 ·Junyao Zhang, Jinglai Li, Junqi Tang
similarity 0.64
Abstract

Neural posterior estimation (NPE) enables Bayesian inference of the posterior parameter in agent-based models (ABMs) with intractable likelihoods, but simulating training data can be computationally expensive. We propose Double-Flow Neural Posterior Estimation (DF-NPE), which combines learned time-series summaries with two conditional normalizing flows for synthetic-data generation and posterior estimation. The framework supplements direct simulations with synthetic training pairs and supports both pooled and ordered training strategies. Experiments in two settings of the Brock-Hommes asset-pricing model show that DF-NPE can reduce direct simulations by 90 percent while achieving lower mean posterior discrepancies and runtime than standard NPE. Performance varies with the allocation of directly simulated and synthetic data and the training strategy. These results establish learned summary augmentation as a practical approach to reducing simulation costs for Bayesian posterior inference of parameters in ABMs.

0.08

When and Why LLM Causal Priors Help: Closed-Loop Prior Selection for Amortized Causal Inference

cs.AI ·2026-09-07 ·Haohao Zhou
similarity 0.64
Abstract

Causal effect estimation asks how an outcome would change under an intervention, and medicine, economics, and public policy all treat it as a foundational task. Prior-data fitted networks (PFNs) amortize the task: a model trained on large numbers of programmatically generated synthetic causal tasks reads a new problem's observational data into context and returns an interventional-effect estimate in a single forward pass. The capability of such models is largely determined by the synthetic training prior, which is currently designed by hand, a bottleneck acknowledged by both Do-PFN and CausalPFN. Large language models (LLMs) can now ``draw'' plausible causal graphs for a given domain, suggesting that LLM-distilled graphs could serve as prior material. Whether injecting such graphs helps at all, where any gain comes from, and when injection helps. Practice has so far relied on manual trial and error. We propose a \emph{closed-loop prior selection framework} that casts prior injection as a budget-constrained optimization over a candidate prior pool. Candidates undergo cheap post-training and are scored by a composite metric dominated by real-domain generalization; the winner then receives full training and paired statistical validation. On a 7.34M-parameter Do-PFN, the framework's winner attains a formally significant $2.75\times$ gain on the primary evaluation domain, and its error falls below that of the uninjected official base. Generalization on an adjacent monitoring domain improves significantly, and no monitored capability degrades. Mechanism experiments show that the gain depends on the semantic content of the distilled graph rather than its structural diversity alone does not produce it (directional evidence). With this framework and this regularity in hand, the use of LLM causal priors stops being manual trial and error and becomes an empirically verifiable selection problem.

0.07

Spectra: Exact Component Transport for Test-Time Prior Adaptation in Simulation-Based Inference

cs.LG stat.ML ·2026-10-06 ·Xin Zhao, Nico Scherf, Robert Trampel, Kerrin J. Pine, Nikolaus Weiskopf
similarity 0.63
Abstract · 41 pages, 7 figures

Simulation-based inference (SBI) has become a powerful approach to Bayesian inference in complex scientific models whose likelihoods are difficult or impossible to evaluate. Amortized SBI learns reusable inference models from simulated data, enabling rapid posterior inference for new observations, and modern generative models have made these models increasingly expressive. However, this reuse is limited to the prior distribution chosen during training, whereas scientific analyses often need revised priors as knowledge accumulates or alternative assumptions are tested. We introduce Spectra, a test-time adaptation method for diffusion-based SBI. Spectra uses an exact score-transport identity to obtain the adapted score from a frozen diffusion model in closed form for structured prior changes, without additional simulation or training. Across six SBI benchmarks, Spectra achieves accurate adaptation under strong prior shifts at low online sampling cost. This enables pretrained SBI models to incorporate updated prior information at test time.

0.19

Prior-Amortized In-Context Bayesian Inference for Generalized Linear Mixed-Effects Models

cs.LG stat.ML ·2026-09-21 ·Alex Kipnis, Marcel Binz, Eric Schulz
datasetcode/data similarity 0.63
Abstract

Hierarchical data is ubiquitous in the empirical sciences and most commonly analyzed with generalized linear mixed-effects models (GLMMs), which have interpretable parameters. Bayesian inference for GLMMs yields calibrated uncertainty estimates but requires Markov Chain Monte Carlo (MCMC); the gold-standard No-U-Turn Sampler (NUTS) is slow and must restart for every new dataset, model and prior specification. We introduce MetaBeta, a pretrained amortized model for in-context Bayesian inference over GLMMs. The full prior and model specification are supplied as context at test time, and the model returns an asymptotically exact posterior at the speed of a forward pass. Two set transformers and two conditional neural spline flows mirror the GLMM's hierarchical structure and are trained jointly on millions of realistic simulated datasets with continuous, binary and count outcomes. The flow posterior is refined by Independence Metropolis-Hastings against the unnormalized posterior, yielding tuning-free asymptotically correct inference. Alternatively, MetaBeta warm-starts NUTS, yielding nearly identical inference 3.5-13x faster than its cold-started counterpart, while removing over 98% of its divergent transitions. On controlled benchmarks with ground-truth parameters, MetaBeta matches NUTS in parameter recovery, posterior calibration and out-of-sample prediction; on real datasets with unknown parameters, priors and model structure, its posteriors closely agree with those of NUTS; and it remains faithful where amortized models typically struggle: under misspecified likelihoods, misspecified priors, out-of-distribution predictors, collinear designs and data-poor regimes. On real data, a full prior sensitivity analysis including model comparison takes a few seconds with MetaBeta against 14-18 core-hours for NUTS, tracing its posteriors throughout. Our model is open-source and open-weights.

0.34

How Well Do Prior-Data Fitted Networks Amortize Bayesian Inference?

cs.LG stat.ML ·2026-10-09 ·Manuel Hülskamp, Julius Kobialka, Emanuel Sommer, Thomas Nagler, David Rügamer
hot direction similarity 0.63
Abstract

Prior-data fitted networks (PFNs) are trained to approximate the Bayesian posterior predictive distribution (PPD) of a synthetic prior, yet results about the approximation quality are surprisingly thin. Taking an empirical risk minimization perspective, we decompose the deployment error of PFNs into prior, context, and amortization error, and measure the latter against a reference PPD under a realistic prior. Results show that PFNs generally recover the targeted PPD very well. Nevertheless, we find that parts of the residual amortization error persist, especially in \textit{rare} settings under the prior pretraining distribution, even when scaling the model size, which manifests in disagreement between independently pretrained PFNs and leaves uncertainty about the predictor at inference time. As a remedy, we study multiple types of Bayesian model averages and find that these can consistently reduce this uncertainty and improve predictions up to full-size TabICLv2 models.

0.35

Guidance for Prior Change via Density Ratio Estimation

cs.LG stat.MLstat.ME ·2026-08-22 ·Yichen Zang, Jiun-Yi Lin, Song Liu
code/data similarity 0.63
Abstract

Simulation-Based Inference (SBI) serves as a vital framework for parameter inference in scientific fields where simulators involve intractable likelihoods, yet while amortized generative models offer rapid posterior estimation, they are often restricted by the specific priors used during training, thereby limiting their flexibility as prior knowledge evolves. To address this prior dependency, PriorGuide was introduced as an inference-time guidance method, but due to its intractable formulation, it relies on Gaussian approximations of the reverse transition kernel and Gaussian mixture model fitting for the prior ratio, both of which introduce systematic bias. Motivated by these limitations, we propose an unbiased test-time guidance framework that leverages Density Ratio Estimation (DRE) to learn a score guidance term, effectively decoupling the inference process from the prior training. Moreover, our framework remains agnostic to the specific density ratio estimators, making it a general and flexible framework for handling prior changes. Experimental results across multiple tasks demonstrate that our method matches or outperforms PriorGuide on C2ST and MMD in most tasks while maintaining robustness even under limited overlap between the training and target priors. Furthermore, we apply our method to Bayesian updating for parameter inference from planetary light-curve data, where it also demonstrates strong effectiveness and robustness. Code is available at https://github.com/a-chenchen/dre-based-prior-guidance .

0.53

LinearPFN: Amortized Variable Selection for Linear Models with Interactions

cs.LG stat.ML ·2026-10-05 ·Louis Schiekiera, Max Zimmer, Christophe Roux, Manuel Arnold, Sebastian Pokutta +1 more
datasetcode/data similarity 0.61
Abstract · 26 pages, 7 figures. Code: https://github.com/schiekiera/LinearPFN

Spike-and-slab regression is a standard Bayesian formulation of variable selection: it returns a posterior distribution over which candidate effects are active rather than a single selected subset, so that every candidate effect carries an inclusion probability. Its cost grows exponentially with the number of candidate effects, so the posterior can be enumerated exactly only when the number of predictors is small. Beyond that reach, the posterior has to be approximated, typically by Markov chain Monte Carlo over the model space, which requires a fresh run for every dataset and, within a fixed budget of steps, may fail to converge. We present LinearPFN, a prior-data fitted transformer network that amortizes spike-and-slab inference for linear models with main effects and pairwise interactions. The network is pretrained once on synthetic datasets, drawn from an explicitly specified prior, and a single forward pass over a new dataset returns posterior inclusion probabilities, posterior-mean coefficients and posterior predictive distributions with no per-dataset fitting. The prior is conjugate by design, so that the posterior for each fixed set of active effects has a closed form, and wherever the exact posterior is still computable by enumeration we verify the network's outputs against it. On real predictor matrices from published social-science datasets, with outcomes drawn from the prior so that the true active set is known, LinearPFN attains a higher per-dataset selection AUC and a higher F1 under the median probability model rule than five classical baselines. The lead holds when the coefficients, the interactions or the noise depart from the prior. Code: https://github.com/schiekiera/LinearPFN. Trained model: https://huggingface.co/schiekiera/LinearPFN.

0.66