CS Paper Radar

Grounding Predictive Reliability through Masked Observability

Shang Gao, Bernhard Sick, Franz Götz-Hahn

cs.LG ·submitted 2026-10-29 09:57 UTC · v1
benchmark

Abstract

An uncertainty estimate is meaningful only if its semantics can be tied to observable predictive behavior. Normal-Inverse-Gamma (NIG) evidential regression provides a structured decomposition of predictive variability into data-scale and predictive-mean components, but marginal likelihood does not identify their allocation. We address this semantic ambiguity in masked representation learning. MORSE (Masked Observability for Reliable Structured Encoding) predicts the same hidden target under controlled changes in visible context and uses normalized cross-context variation to ground predictive-mean uncertainty, while a Gaussian objective learns the conditional mean and data scale. The normalized variation plays the role of the NIG factor $1/ν$, which the marginal leaves unidentified; this yields a policy-relative observability variance that measures dependence on visible evidence, distinct from variation induced by resampling the training set. On a Gaussian-process benchmark with a contextual reference built from analytic Bayes conditional means, MORSE has the lowest log-variance error among compared evidential, ensemble, and dropout estimators in all nine tested settings, while the estimator closest to training-set variation perturbs the model rather than its context in every setting, so the two references separate the estimators empirically. MORSE amortizes observability estimation into one forward pass per observation state. On a heteroscedastic pulse benchmark, its predictor's cross-context variance approaches a large-sample reference as training data increase. MORSE also improves low-label transfer on PTB-XL and in-distribution trajectory prediction on IMPTC.

Closest papers

Nearest neighbours by meaning across the whole index.

ELVAE: Evidential Learning-Based Variational Autoencoder for Uncertainty-Aware Generation

cs.LG cs.AI ·2026-08-11 ·Ge Wang
similarity 0.64
Abstract · 31 pages, 5 figures, 7 tables. Substantially revised: expanded quotient/canonical analysis, added Fashion-MNIST and energy-matched/isotropic…

ELVAE places an input-dependent normal--inverse-gamma (NIG) hierarchy at each VAE latent coordinate, separating location uncertainty $u_{\mathrm{epi}}=β/[ν(α-1)]$ from conditional variability $u_{\mathrm{var}}=β/(α-1)$. The marginalized latent law, however, identifies only the three quotient coordinates $(γ,α,c)$ with $c=β(1+1/ν)$; reconstruction is blind to one $(ν,β)$ fiber direction. A companion theoretical analysis shows that the complete NIG prior and forward KL select a unique prior-relative canonical representative on each fiber, so canonical inverse allocation is not a fourth independent information channel. Empirically, trained inverse evidence $1/ν$ remains the strongest sensitivity-ranking score. At $τ_{\mathrm{epi}}=1$, the three-seed mean high/low-$u_{\mathrm{epi}}$ semantic-transition ratios are 1.98 on MNIST and 1.66 on Fashion-MNIST, falling to 1.33 and 1.16 under scale-matched controls. In the 20-draw MNIST component study with equal per-anchor perturbation energy, $1/ν$ gives high/low ratios 1.69 and 1.65 under the $u_{\mathrm{epi}}$ and $u_{\mathrm{var}}$ fields and 1.69 (95\% interval 1.45--1.97) under a geometry-free isotropic field, whereas $u_{\mathrm{var}}$ reverses the isotropic ordering to 0.76. Under the experimental prior, canonical $1/ν_{\mathrm{can}}$ is a strictly increasing transform of $T=c/[α(γ^2+2)]$ and is bounded above by $3+\sqrt{10}$. Thus ELVAE exposes a controllable sensitivity mechanism whose trained four-output realization is operationally informative, while the exact reconstruction-visible information remains three-dimensional and baseline image quality is a separate question.

0.46

Decoupling Is Not Identification: Supervised Evidential Learning in Next-Token Prediction

cs.AI ·2026-08-14 ·Ge Wang
similarity 0.62
Abstract · 5 pages, 3 figures, and 3 tables

A next-token probability says what a model predicts, not how much training support lies behind it. A Dirichlet head can represent this distinction by separating mean $m$ from concentration $S$, but decoupling does not identify what $S$ means. Here we propose an Evidential Next-Token Prediction (ENTOP) framework to audit this gap on character-level Moby-Dick, using exact 8-gram count as a reproducible lexical-support label and withholding count regression from 20% of context types. Standard implicit evidential training carries essentially no count signal beyond confidence on held-out-label types (partial Spearman $ρ= 0.001 \pm 0.014$), whereas explicit supervision generalizes ($ρ= 0.201 \pm 0.010$; matched-pair win $= 0.822 \pm 0.021$). CE predictive entropy is at chance for unseen 8-grams (AUROC $= 0.490 \pm 0.004$), while supervised vacuity reaches $0.772 \pm 0.003$, comparable with an indexed CE-representation baseline ($0.769$) but below the tautological corpus oracle ($1.000$). Neither longest-suffix nor representation-distance strata explain where amortization succeeds. Increasing count weight under the digamma objective improves support fit only by sacrificing prediction. A constant predictor wins natural log-RMSE, and vacuity does not improve error deferral. These results motivate a minimum evidence protocol---confidence control, matched pairs, held-out labels, a constant baseline, and a decision test---and show that concentration can pass identification while failing calibration and utility.

0.07

Observability-Aligned Evidential Masked Pretraining for Reliable VRU Motion Prediction

cs.LG ·2026-10-29 ·Shang Gao, Franz Götz-Hahn, Konrad Doll, Bernhard Sick
benchmark similarity 0.62
Abstract

Vulnerable road user (VRU) prediction must operate under partial observability and distribution shift, so reliable uncertainty matters alongside accuracy. Masked pretraining is attractive in this setting because it naturally exposes the model to missing content during self-supervision, but existing time-series masked autoencoding methods are typically deterministic and do not constrain how confidence should change with observability. We introduce eCoMAE, an evidential masked pretraining method for multivariate time series that explicitly aligns predictive evidence with what is observed. eCoMAE combines block-wise time-step masking with a Normal-Inverse-Gamma decoder and three observability-aware mechanisms: uncertainty-weighted mean consistency on positions visible in both a masked view and a reference pass, evidence suppression on masked positions, and representation consistency across stochastic masks. On IMPTC benchmark for VRU intention prediction under leave-one-environment-out weather shift, eCoMAE improves the strongest baseline from 0.7971 to 0.8068 in OOD accuracy and from 0.0718 to 0.0648 in OOD AURC. Additional evaluation on pedestrian trajectory forecasting across IMPTC, Waymo, nuScenes, and inD further yields the best fine-tuning MSE on all four datasets. These results show that observability-aligned evidential pretraining improves reliability under shift and remains effective beyond the primary benchmark.

0.55

PEEL-DDPM: Physics-Enabled Evidential Learning for the Denoising Diffusion Probabilistic Model

cs.CV ·2026-09-23 ·Ge Wang
hot direction similarity 0.58
Abstract · 13 pages, 5 figures, 2 tables

Normal-inverse-gamma (NIG) regression is not identifiable from its marginal Student-t likelihood: three combinations of four NIG parameters are determined, leaving one degree of freedom. We introduce PEEL-DDPM, a physics-enabled evidential learning framework for denoising diffusion probabilistic models. A measurement-conditioned DDPM is first trained with epsilon-MSE and then frozen. Its complete reverse trajectory produces a reconstruction, whose residual from the known training object is modeled by a final-image evidential network. The network learns only the identifiable Student-t coordinates, while repeated scanner-noise realizations and repeated diffusion trajectories provide a nested Monte Carlo estimate of final-image aleatoric variance, separated into scanner-induced and sampler-induced components. This measured variance resolves the remaining NIG ambiguity and yields a decomposition of predictive uncertainty into measurement, diffusion, and epistemic terms. The method uses sequential training without a cross-loss weighting coefficient. In a feasibility study on eight held-out objects, empirical central-interval coverages were 49.3%, 80.3%, 90.1%, and 95.6% for nominal 50%, 80%, 90%, and 95% intervals. The mean squared residual was 0.967 times the mean predicted variance, and a single-image aleatoric head achieved pooled Spearman correlation 0.785 against an independent nested reference. Across five dose levels, scanner-induced variance showed a log-log dose slope of -1.15, whereas sampler-induced variance remained nearly dose independent with slope -0.01. These results support PEEL-DDPM as a practical route to identifiable and physically interpretable uncertainty quantification in diffusion-based image reconstruction.

0.27

Predicting Multi-View Rashomon Representation: Can We Learn Where Models Disagree?

cs.LG ·2026-09-30 ·Mingyue Ma, Zongbo Han, Changqing Zhang, Guangyu Wang
similarity 0.58
Abstract · Under review

Foundation models are increasingly adopted across a wide range of applications, often serving as core blocks within AI systems. Yet different foundation models may encode the same input from multiple different views, leading to substantial representation disagreement, which we term Rashomon Representation. Such disagreement often signals inputs that a given model encodes in a way inconsistent with other models, offering a valuable yet underexplored signal for input reliability estimation. While prior work has largely focused on measuring disagreement across multiple models with a representation set, we instead focus on predicting disagreement from a single representation. We hypothesize that this disagreement follows some consistent, input-dependent patterns rather than occurring at random. To test this, we quantify disagreement by comparing each sample's nearest neighbors across different models' representation spaces, then train a lightweight predictor that estimates disagreement from a single model's representation. At inference time, given a new input, the predictor uses that input's representation to tell whether it aligns with or diverges from those of other models. Extensive experiments across diverse foundation models and datasets show that representational disagreement is indeed input-dependent, predictable, and generalizable, enabling efficient reliability estimation of foundation models.

0.22

When Is Accuracy Evidence? A Unified Theory of Generalisation, Validation, and Information Fusion

cs.LG stat.MLphysics.data-an ·2026-10-02 ·JM Gorriz
theory similarity 0.58
Abstract · 52 pages, 30 figures

K-fold cross-validation (CV) is widely used as evidence of out-of-sample performance, although folds are neither independent experiments nor equally informative under heterogeneous data. Cross Upper-Bound Validation (CUBV) replaces point-wise CV accuracy by conservative upper bounds on true risk. Here we generalise CUBV through a single exponential framework in which the moment-generating function of the generalisation gap is controlled by a cumulant envelope gamma(lambda). This yields a family of risk bounds covering Hoeffding-, Bernstein-, dependency-aware, PAC-Bayesian, and heterogeneous source-fusion settings. For K-fold CV, dependence between fold-wise gaps is modelled through a joint sub-Gaussian proxy matrix. Under equicorrelation, this gives an effective number of folds, Keff = K/[1+(K-1)rho], showing that increasing K does not necessarily increase statistical evidence when folds are strongly dependent. The framework is also extended to posterior distributions over predictors and weighted multi-source fusion, where weights are selected by minimising an upper bound on future risk rather than empirical error alone. Experiments with trained linear classifiers on heterogeneous multimodal Gaussian mixtures compare K-fold CV with full-sample resubstitution plus risk correction. Bounds are evaluated by coverage and tightness. In low-dimensional small-sample settings, K-fold partitioning can increase uncertainty because individual folds under-represent minority modes, while corrected resubstitution can remain valid and tighter; this effect disappears as sample size increases. Overall, gamma-CUBV separates observed performance, uncertainty, dependence, model complexity, and confidence into explicit terms, providing a unified route from CV scores to risk statements and a principled validation criterion for heterogeneous small-sample applications such as neuroimaging.

0.26

Information-Based Calibration of Uncertainty Quantification in Product-of-Experts Gaussian Process Models

cs.LG ·2026-08-29 ·Yean Hoon Ong, Paolo Barucca, Wei Pan, Jun Wang
top venue similarity 0.57
Abstract · Published in the Journal of Artificial Intelligence Research, Volume 86 (2026)

Gaussian process (GP) regression with a single global GP (GP-glo) incurs cubic computational cost, limiting scalability to large datasets. Product-of-experts GP models (GP-pro), which combine local GP models to capture global correlations, alleviate this computational burden. However, training local experts on disjoint data subsets can lead to overestimated posterior variances. We propose GP-pro-c, a product-of-experts GP model that calibrates these variances using an information-based method. The method exploits the monotonicity and submodularity of information gain in GPs to define a calibration ratio that reduces the posterior variance of individual local GP models. We evaluate GP-pro-c using negative log-likelihood (NLL), root mean squared error (RMSE), and expected normalised calibration error (ENCE). Experiments on four synthetic functions and six regression datasets show that GP-pro-c achieves average reductions of 2.3% in NLL and 12.0% in ENCE compared with the uncalibrated GP-pro model. The proposed method mitigates posterior variance overestimation while maintaining predictive accuracy and reducing computational complexity. GP-pro-c provides a promising approach for uncertainty estimation in scalable GP models and may serve as a useful surrogate model for Bayesian optimisation with high-dimensional and large-scale data.

0.63

Revealing Epistemic Uncertainty in MLLMs via Causal-Invariant Masking

cs.CV cs.AI ·2026-10-02 ·Haoyang Luo, Linwei Tao, Jie Gui, Xinghao Chen, Chang Xu +2 more
top venue similarity 0.57
Abstract · Accepted by NeurIPS 2026

Multimodal Large Language Models (MLLMs) suffer from hallucinations, creating a critical need for Uncertainty Quantification (UQ) to ensure reliable deployment. However, existing approaches struggle to detect uncertainty caused by superficial associations, especially when the query-relevant signal is weak. We mainly attribute this issue to their bias toward aleatoric uncertainty arising from data ambiguity, overlooking epistemic uncertainty stemming from model limitations. To further decompose uncertainty types for a comprehensive UQ, we propose Causal-Invariant Masking (CIM), which measures the semantic shift between the original predictions and those conditioned on a causally-focused view. Based on this framework, we introduce Semantic Divergence as our core metric for UQ and provide theoretical evidence that it converges to the variance of model's sensitivity to non-causal correlations, establishing its ability to capture MLLM's limitation. To accelerate UQ in MLLMs, we further propose Expected Embedding Drift (EED), a fast geometric proxy metric that estimates semantic shift directly within the hyperspherical embedding space. Experiments show that our method achieves state-of-the-art performance on various benchmarks, while the proposed EED accelerates by nearly 50% with comparable performance.

0.44