CS Paper Radar

Theoretical Study on the Evidential Learning-based Variational Autoencoder

Ge Wang

cs.LG stat.MLcs.NEmath.ST ·submitted 2026-08-19 21:51 UTC · v1

Abstract

A normal--inverse-gamma (NIG) latent hierarchy has four parameters, but its induced latent law does not identify all four. For $σ^2\sim\mathrm{InvGamma}(α,β)$, $μ\midσ^2\sim\mathcal{N}(γ,σ^2/ν)$, and $z\midμ,σ^2\sim\mathcal{N}(μ,σ^2)$, the marginal law of $z$ depends on $(ν,β)$ only through $c=β(1+1/ν)$. Hence the reconstruction-visible parameter space is the three-dimensional quotient $(γ,α,c)$, with a one-dimensional fiber degree of freedom. For a fixed hierarchical variational objective, exact partial minimization of the forward KL divergence to a complete NIG prior selects a unique prior-relative representative on each fiber, yielding an exact three-coordinate reduction with the same optimum as the four-coordinate objective. Writing $ρ_0=2β_0/ν_0$ and $T=c/\{α[(γ-γ_0)^2+ρ_0]\}$, we show that inverse canonical allocation $1/ν_{\rm can}$ is an explicit strictly increasing function of $T$. For $α>1$, the ratio $u_{\rm epi}/u_{\rm var}=1/ν_{\rm can}$ is therefore determined by the quotient state and prior; for rank-only use under a common calibration, $T$ contains the same coordinatewise ordinal information. The residual prior gauge is characterized rather than eliminated: $(γ_0,ρ_0)$ govern ordinal dependence, while $(ν_0,α_0)$ determine numerical calibration and the analytic ceiling of $1/ν_{\rm can}$.

Comment
18 pages, 0 figures. Theoretical study of the NIG latent bottleneck in ELVAE; related to arXiv:2608.10398

Closest papers

Nearest neighbours by meaning across the whole index.

ELVAE: Evidential Learning-Based Variational Autoencoder for Uncertainty-Aware Generation

cs.LG cs.AI ·2026-08-11 ·Ge Wang
similarity 0.72
Abstract · 31 pages, 5 figures, 7 tables. Substantially revised: expanded quotient/canonical analysis, added Fashion-MNIST and energy-matched/isotropic…

ELVAE places an input-dependent normal--inverse-gamma (NIG) hierarchy at each VAE latent coordinate, separating location uncertainty $u_{\mathrm{epi}}=β/[ν(α-1)]$ from conditional variability $u_{\mathrm{var}}=β/(α-1)$. The marginalized latent law, however, identifies only the three quotient coordinates $(γ,α,c)$ with $c=β(1+1/ν)$; reconstruction is blind to one $(ν,β)$ fiber direction. A companion theoretical analysis shows that the complete NIG prior and forward KL select a unique prior-relative canonical representative on each fiber, so canonical inverse allocation is not a fourth independent information channel. Empirically, trained inverse evidence $1/ν$ remains the strongest sensitivity-ranking score. At $τ_{\mathrm{epi}}=1$, the three-seed mean high/low-$u_{\mathrm{epi}}$ semantic-transition ratios are 1.98 on MNIST and 1.66 on Fashion-MNIST, falling to 1.33 and 1.16 under scale-matched controls. In the 20-draw MNIST component study with equal per-anchor perturbation energy, $1/ν$ gives high/low ratios 1.69 and 1.65 under the $u_{\mathrm{epi}}$ and $u_{\mathrm{var}}$ fields and 1.69 (95\% interval 1.45--1.97) under a geometry-free isotropic field, whereas $u_{\mathrm{var}}$ reverses the isotropic ordering to 0.76. Under the experimental prior, canonical $1/ν_{\mathrm{can}}$ is a strictly increasing transform of $T=c/[α(γ^2+2)]$ and is bounded above by $3+\sqrt{10}$. Thus ELVAE exposes a controllable sensitivity mechanism whose trained four-output realization is operationally informative, while the exact reconstruction-visible information remains three-dimensional and baseline image quality is a separate question.

0.46

Variational objectives for amortized Bayesian inference in inverse problems: The role of posterior conditioning

cs.CE stat.MLcs.ITcs.LG+1 ·2026-09-21 ·Abhishek Srivastava, Arijit Hazra, Rajesh Dubbaku
benchmarkhot direction similarity 0.58
Abstract · 50 pages

Variational autoencoders (VAEs) offer an efficient approach to amortized Bayesian inference for inverse problems, but posterior accuracy can depend strongly on the choice of variational regularization, particularly when the inverse problem contains weakly identified parameter directions. This study investigates three objectives: a reverse Kullback--Leibler formulation (VAE-KL), an asymmetric Jensen--Shannon formulation (VAE-JS), and a Jensen--Shannon--Wasserstein formulation (VAE-JSWA), which replaces the reverse Kullback--Leibler regularizer with the squared 2-Wasserstein distance while retaining forward-Kullback--Leibler posterior supervision. A full-covariance Gaussian encoder and a pre-trained physics-based surrogate are used for amortized posterior inference. A local linear--Gaussian analysis in the generalized Fisher basis is developed to characterize the variance-dependent gradients of the three objectives. The formulations are first evaluated using linear--Gaussian benchmarks with known posterior solutions and subsequently tested on nonlinear physics-based inverse problems, including an inverse problem governed by a linear ODE and two PDE-constrained problems. VAE-KL performs slightly better than the other formulations in the well-conditioned benchmark, where all three approaches yield comparable posterior approximations, whereas VAE-JSWA provides substantially lower posterior errors in the strongly ill-conditioned benchmark. The nonlinear physics-based problems exhibit a similar conditioning-dependent trend, with JS-based formulations providing greater benefit as posterior ill-conditioning increases. These results indicate that posterior conditioning is an important factor in selecting variational objectives and motivate geometry-adaptive variational inference for Bayesian inverse problems.

0.36

Generalized Graph Variational Autoencoders: Bounded Divergences Control Posterior Collapse

cs.LG cs.AI ·2026-08-25 ·Kleyton da Costa, Bernardo Modenesi, Ivan F. M. Menezes, Helio Lopes
similarity 0.56
Abstract

The variational graph autoencoder (VGAE) regularizes its posterior toward the prior with the Kullback-Leibler divergence, a choice inherited from the variational autoencoder rather than argued for. We introduce the generalized graph variational autoencoder (GGVA), which replaces that term with any member of the Rényi-Tsallis family of order $q$ while leaving every other part of the model untouched. Both members admit closed forms for diagonal Gaussians and both recover the KL exactly as $q \to 1$, so the VGAE is the $q=1$ arm of our own model rather than a separate baseline, and any measured difference is attributable to a single scalar. Our analysis identifies boundedness, not the order, as the operative property: for $q<1$ the Tsallis divergence is bounded above by $1/(1-q)$, independently of the latent width, whereas the KL and the Rényi divergence of the same order are unbounded. On ten graphs spanning three synthetic families, a social network, three citation networks, a connectome, a power grid and a road network, $q$ moves the retained posterior information by up to $49\times$ relative to the VGAE, while the Rényi arm at the same order stays within $1.02$-$1.30\times$ of it on all six larger real graphs (isolating the bound as the cause). The retained information is usable: probing the frozen embedding for node class, a label absent from the objective, gives GGVA up to $+0.14$ macro-F1 over the VGAE on CiteSeer, with the Rényi control again tracking the VGAE. We also report what the design was built to expose: none of this reaches held-out link-prediction accuracy on any of the six larger real graphs, and boundedness delays posterior collapse rather than preventing it.

0.24

GenFirst: Generation Before Reconstruction for Stable End-to-End Latent Generative Modeling

cs.CV ·2026-08-29 ·Guangting Zheng, Yiyuan Zhang, Tao Yang, Yunpeng Chen, Rui Zhu +2 more
similarity 0.54
Abstract

Latent generative models typically follow a two-stage pipeline, training a variational autoencoder for reconstruction and then a generative model on the frozen latent space. Since reconstruction-optimized latents are not necessarily generation-friendly, jointly training both models is an appealing alternative. However, direct end-to-end training remains challenging, as it is prone to latent collapse and faces a generation-reconstruction conflict. We revisit this problem by analyzing how different objectives shape the latent space and identify two key insights. First, the entropy term in the Kullback-Leibler divergence objective is essential for preventing collapse: reconstruction and prior fitting tend to shrink the posterior, while entropy preserves non-degenerate latent uncertainty. Second, reconstruction and generation exhibit asymmetric learning dynamics: reconstruction is fast and strongly supervised, whereas generation is slower and harder to optimize. Based on these insights, we achieve the first direct end-to-end training without latent collapse and propose GenFirst, a simple generation-before-reconstruction strategy. The generative objective first shapes the latent space under weak reconstruction pressure, after which reconstruction is progressively strengthened to recover visual details. We validate GenFirst with continuous autoregressive priors with exact likelihoods and SiT priors with implicit likelihoods. With our end-to-end objective and GenFirst, SiT achieves a gFID of 0.97 with CFG and 1.45 without CFG on ImageNet-256, while MMDiT reaches a GenEval score of 0.90 on text-to-image generation. Beyond image generation, we extend the framework to shared visual latents for generation and representation learning, and to continuous unified text-image generation. These results demonstrate the generality of stable end-to-end latent learning across generative priors and modalities.

0.53

When Do Gaussian Priors Fail? A Latent Geometry Investigation of RL Fine-Tuning

cs.LG cs.AI ·2026-10-20 ·Wang Ru
similarity 0.52
Abstract · around 49 pages, around 25 figures

Latent diffusion models almost universally place a Gaussian prior on the VAE latent space. For base generation this default is unproblematic; under reinforcement-learning fine-tuning, however, it carries a structural limitation. The standard VAE's KL term does not merely ignore mode structure - it actively taxes separation and tightness. When data are multimodal, the ELBO optimum compresses distinct modes into an overlapping latent. RL fine-tuning amplifies this overlap: the score network is under-resolved on the saddle region between modes, dynamic-to-variational amplification rises sharply, and the collapse boundary beta_c is pushed far above its variational scale. On the real alanine-dipeptide Boltzmann distribution, a Gaussian-prior VAE collapses at beta_c = 0.224 while a mode-tracked Gaussian-mixture prior collapses at 0.00316, a 71x gap from the same pipeline. We trace this gap to a single geometric mediator, d_min/(2 sigma_0), and verify the causal chain through controlled interventions: moving geometry at fixed prior, and switching prior at matched geometry (full mediation; prior-label coefficient 0.995x, 95% CI [0.79, 1.25]). A three-source decomposition localises the amplification to the learned policy (zero ODE-solver contribution, bounded anchor mismatch); the coverage plateau is an unconditional benefit of mode-separating priors that no amount of RL capacity recovers. The remedy is conditional: a non-Gaussian prior is necessary precisely when the latent geometry would fall into overlap and the task is coverage-sensitive. A machine-checked theory skeleton supports the KL pricing structure - a divergent compactness tax under a Gaussian prior versus a finite assignment fee under a mixture - and the position-versus-plateau distinction delimits what is finite-resource from what requires changing the prior.

0.33

Finite-Sample Binary Hypothesis Testing via Rényi Divergences: Strong Converse and Local Privacy

cs.IT math.ST ·2026-09-23 ·Roberto Bruno, Adrien Vandenbroucque, Amedeo Roberto Esposito
published similarity 0.51
Abstract · Extended version of a paper presented at the 2026 IEEE International Symposium on Information Theory (ISIT). Submitted to the IEEE Transacti…

We study asymmetric simple binary hypothesis testing between $H_0:P_0^{n}$ and $H_1:P_1^{n}$, based on $n$ independent and identically distributed observations. Leveraging a variational representation of Rényi divergence of order $α$, we derive our main result: a finite-sample converse with $α>1$. The bound uses both directions of the divergence $D_α(P_1\|P_0)$ and $D_α(P_0\|P_1)$, tensorises under product measures, and contains familiar data-processing converses as boundary cases. For comparison, we apply the same variational approach to general $f$-divergences and specialise it to total variation, $E_γ$, Hellinger, and Kullback Leibler divergences, thereby recovering familiar converses within a unified framework. Together with an achievability bound involving Rényi divergence with $α\in (0,1)$, the main converse recovers the phase transition of the optimal Type II error under the exponentially decaying Type I error constraint $\varepsilon_n=e^{-nr}$. Under regularity conditions, the optimal Type II error vanishes exponentially when $r<D(P_1\|P_0)$ and converges exponentially fast to one when $r>D(P_1\|P_0)$. We also derive sample-complexity bounds and extend both the converse and achievability analyses to locally differentially private observations, quantifying the cost of privacy and recovering the non-private achievability bound as the privacy constraint vanishes.

0.34

The Bayesian Learning Rule Beyond KL Geometry

cs.LG ·2026-10-22 ·Sophia Sklaviadis, Wu Lin, Andre Martins, Mario Figueiredo, Thomas Moellenhoff +1 more
similarity 0.51
Abstract

The Bayesian learning rule (BLR) exploits the relation between Bayesian updating and the Kullback-Leibler divergence (KLD) to derive efficient additive updates for exponential family variational posteriors. The KL geometry is powerful but restrictive: combined with posterior families with parameter-dependent support it can lead to undefined objectives. We ask which non-KL geometries preserve the additive BLR computation. Previous works that use $f$-divergences suggest that this is only possible with the KLD. Here, we show that this is not true. We show that additive information-geometric optimization algorithms can be derived for the class of mixed-space Bregman or Fenchel-Young discrepancies. These discrepancies induce generalized maximum-entropy families, dual coordinates, and generalized-BLR canonical gradient updates, recovering the classical BLR. The generalized BLR leads to new algorithms that \textit{cannot} be derived from the KL-based BLR. For the Tsallis negentropy with quadratic statistics, the generalized BLR yields bounded-support and heavy-tailed $q$-Gaussian posteriors and an easy Newton-like implementation. Experiments up to CIFAR-100 illustrate controllable support and tail behavior and competitive performance.

0.24

Minimax Rates for Conditional Generative Learning under Wasserstein Loss

cs.LG math.STstat.ML ·2026-10-26 ·Sungwon Park, Kyoung-Sook Moon, Hongjoong Kim
theory similarity 0.51
Abstract

We study minimax rates for conditional generative learning under integrated Wasserstein loss, with Hölder continuous conditional laws and an unrestricted covariate distribution. We evaluate unrestricted generators through their conditional output distributions. For expected Wasserstein loss, the minimax rate agrees with the rate under uniform covariates and follows from balancing localization bias against unconditional distribution estimation error. Higher moments of the loss can instead be dominated by unobserved covariate regions, which force an additional risk contribution of order $n^{-1}$, where $n$ is the sample size. An adaptive localization argument gives matching upper bounds. The results cover unrestricted response laws and possibly vanishing Besov densities for $W_1$, and extend to higher Wasserstein orders under uniform response density bounds and integrability conditions. A multiscale lower bound closes the logarithmic gap in unconditional $W_1$ minimax rates in dimension two for both unrestricted distributions and Besov density classes. We also establish minimax rates for conditional Brenier maps under density, integrability, and uniform curvature conditions.

0.12