CS Paper Radar

Generalized Graph Variational Autoencoders: Bounded Divergences Control Posterior Collapse

Kleyton da Costa, Bernardo Modenesi, Ivan F. M. Menezes, Helio Lopes

cs.LG cs.AI ·submitted 2026-08-25 22:59 UTC · v1

Abstract

The variational graph autoencoder (VGAE) regularizes its posterior toward the prior with the Kullback-Leibler divergence, a choice inherited from the variational autoencoder rather than argued for. We introduce the generalized graph variational autoencoder (GGVA), which replaces that term with any member of the Rényi-Tsallis family of order $q$ while leaving every other part of the model untouched. Both members admit closed forms for diagonal Gaussians and both recover the KL exactly as $q \to 1$, so the VGAE is the $q=1$ arm of our own model rather than a separate baseline, and any measured difference is attributable to a single scalar. Our analysis identifies boundedness, not the order, as the operative property: for $q<1$ the Tsallis divergence is bounded above by $1/(1-q)$, independently of the latent width, whereas the KL and the Rényi divergence of the same order are unbounded. On ten graphs spanning three synthetic families, a social network, three citation networks, a connectome, a power grid and a road network, $q$ moves the retained posterior information by up to $49\times$ relative to the VGAE, while the Rényi arm at the same order stays within $1.02$-$1.30\times$ of it on all six larger real graphs (isolating the bound as the cause). The retained information is usable: probing the frozen embedding for node class, a label absent from the objective, gives GGVA up to $+0.14$ macro-F1 over the VGAE on CiteSeer, with the Rényi control again tracking the VGAE. We also report what the design was built to expose: none of this reaches held-out link-prediction accuracy on any of the six larger real graphs, and boundedness delays posterior collapse rather than preventing it.

Closest papers

Nearest neighbours by meaning across the whole index.

When Do Gaussian Priors Fail? A Latent Geometry Investigation of RL Fine-Tuning

cs.LG cs.AI ·2026-10-20 ·Wang Ru
similarity 0.61
Abstract · around 49 pages, around 25 figures

Latent diffusion models almost universally place a Gaussian prior on the VAE latent space. For base generation this default is unproblematic; under reinforcement-learning fine-tuning, however, it carries a structural limitation. The standard VAE's KL term does not merely ignore mode structure - it actively taxes separation and tightness. When data are multimodal, the ELBO optimum compresses distinct modes into an overlapping latent. RL fine-tuning amplifies this overlap: the score network is under-resolved on the saddle region between modes, dynamic-to-variational amplification rises sharply, and the collapse boundary beta_c is pushed far above its variational scale. On the real alanine-dipeptide Boltzmann distribution, a Gaussian-prior VAE collapses at beta_c = 0.224 while a mode-tracked Gaussian-mixture prior collapses at 0.00316, a 71x gap from the same pipeline. We trace this gap to a single geometric mediator, d_min/(2 sigma_0), and verify the causal chain through controlled interventions: moving geometry at fixed prior, and switching prior at matched geometry (full mediation; prior-label coefficient 0.995x, 95% CI [0.79, 1.25]). A three-source decomposition localises the amplification to the learned policy (zero ODE-solver contribution, bounded anchor mismatch); the coverage plateau is an unconditional benefit of mode-separating priors that no amount of RL capacity recovers. The remedy is conditional: a non-Gaussian prior is necessary precisely when the latent geometry would fall into overlap and the task is coverage-sensitive. A machine-checked theory skeleton supports the KL pricing structure - a divergent compactness tax under a Gaussian prior versus a finite assignment fee under a mixture - and the position-versus-plateau distinction delimits what is finite-resource from what requires changing the prior.

0.33

xVGAE: A Hierarchical Variational Graph Autoencoder for Exchangeable Graphs

cs.LG stat.ML ·2026-10-22 ·Daniele Micheletti, Federica Zoe Ricci, Erik B. Sudderth
theory similarity 0.59
Abstract

Graphs play a crucial role in many applications, ranging from drug discovery to social network modeling and astronomy. Generative models for graphs have substantially advanced in recent years, but we identify relatively simple scenarios where state-of-the-art models struggle to converge within a standard compute budget, or exhibit prohibitively slow generation times. We propose a generative architecture that achieves quality competitive with state-of-the-art models and is orders of magnitude faster at generation. Our approach is inspired by the Aldous-Hoover theorem, a classic representation theorem that characterizes any probability distribution of graphs that is invariant to permutations of node indices or exchangeable. This theorem establishes that three ingredients suffice: a graph-level latent variable, a set of exchangeable node-specific latent variables, and a so-called graphon function that maps graph- and node-level variables to the probability of an edge between any node pair. Given a training set of graphs, our exchangeable Variational Graph AutoEncoder (xVGAE) learns an approximation to their underlying graphons, as well as graph- and node-level latent variables. We show that our xVGAE encodes latent representations that correlate with graph- and node-level properties, and can (quickly, in one shot) generate new graphs that closely match key training set statistics.

0.20

WL-SeqVAE: Sequence-Based Graph Generation Using Variational Autoencoders and Weisfeiler-Leman Colors

cs.LG ·2026-10-23 ·Iakovos Evdaimon, Giannis Nikolentzos, Achilleas Tsimichodimos, Christos Xypolopoulos, Michalis Vazirgiannis
code/datahot direction similarity 0.57
Abstract

In this paper, we introduce WL-SeqVAE, a sequence-based variational autoencoder for generating graphs. The key observation behind WL-SeqVAE is that node colors induced by the Weisfeiler-Leman (WL) algorithm provide compact discrete summaries of local graph structure and can therefore serve as structural tokens for generation. Rather than learning a distribution over adjacency matrices directly, our method maps each graph to a sequence of WL colors and employs a variational autoencoder that encodes this sequence into a Gaussian latent distribution, from which an autoregressive decoder reconstructs the graph. During generation, compatibility constraints derived from the WL colors guide node and edge predictions, combining global information from the latent representation with local structural consistency. This formulation turns graph generation into a sequence generation problem and avoids directly parameterizing a quadratic adjacency matrix. We evaluate WL-SeqVAE on molecular and general graph generation tasks. Across the evaluated datasets, WL-SeqVAE is competitive with established baselines on several local structural and molecular quality metrics. Code is available at https://github.com/iakovosevdaimon/WL-SeqVAE

0.32

Perfect Reconstruction Cannot Show Whether Two Codebooks on a VAE's Latent Space Agree

cs.LG cs.IT ·2026-09-29 ·Yusuke Hayashi
theory similarity 0.56
Abstract · 28 pages, 4 figures, 4 tables. Supplementary material (per-run results, precommit documents and verification scripts) is provided as ancilla…

Many systems read the latent space of a variational autoencoder (VAE) as discrete codes, and rely on each latent being read back as the code it was written as. Shannon (1948) took such a shared codebook as given; inside a VAE nothing enforces it. We prove that no item of a standard training report can reveal whether the written and read codes agree on encoded data. We construct two VAEs that reconstruct every input perfectly, infer exactly, and match on every per-example ELBO and every code histogram, yet agreement is one in the first and zero in the second. We then show that the ELBO's own terms bound how far a measured agreement can move: the variational gap bounds the shift to exact inference, and the rate bounds the shift to generation from the prior. For Gaussian posteriors, a mean-field encoder pays an exact price in the gap that depends on its choice of axes. On probabilistic PCA the gap certifies a trained encoder in all 75 settings we test, at agreement between 0.76 and 0.97. On 240 trained VAEs, a floor set by the rate and the dataset size rules out rate-side certification in advance for 66 models, and none of them certifies. The results yield a simple protocol, Certify-or-Measure, for any pipeline that reads a VAE as codes.

0.27

Residual spectral instabilities in representation learning

cs.LG cond-mat.stat-mech ·2026-10-08 ·Zhen Li
theory similarity 0.56
Abstract · 19 pages, 3 figures

Learned representations can lose latent degrees of freedom successively, suggesting a cascade of transitions whose underlying stability principle remains unclear. Here we formulate dimension-wise posterior collapse in variational autoencoder (VAE) as a fluctuation theory around partially collapsed states. Interpreting the negative evidence lower bound as an effective free energy, its quadratic expansion defines a Gaussian theory whose Hessian acts as a mass matrix for latent fluctuations. We show that the collapsed directions form an invariant fluctuation sector and derive its exact mass spectrum in terms of a conditional residual operator. A local reactivation direction lowers the free energy when the decoder variance falls below the residual spectral upper edge, with equality marking marginality. The criterion recovers principal component thresholds in the linear Gaussian VAE limit. Viewed in reverse along continuously connected branches, the reactivation boundary provides a local criterion for successive collapse. Numerical continuation experiments show successive loss of latent dimensions near these spectral marginalities. These results support a spectral cascade interpretation governed by residual information left unexplained by the surviving representation.

0.30

Theoretical Study on the Evidential Learning-based Variational Autoencoder

cs.LG stat.MLcs.NEmath.ST ·2026-08-19 ·Ge Wang
similarity 0.56
Abstract · 18 pages, 0 figures. Theoretical study of the NIG latent bottleneck in ELVAE; related to arXiv:2608.10398

A normal--inverse-gamma (NIG) latent hierarchy has four parameters, but its induced latent law does not identify all four. For $σ^2\sim\mathrm{InvGamma}(α,β)$, $μ\midσ^2\sim\mathcal{N}(γ,σ^2/ν)$, and $z\midμ,σ^2\sim\mathcal{N}(μ,σ^2)$, the marginal law of $z$ depends on $(ν,β)$ only through $c=β(1+1/ν)$. Hence the reconstruction-visible parameter space is the three-dimensional quotient $(γ,α,c)$, with a one-dimensional fiber degree of freedom. For a fixed hierarchical variational objective, exact partial minimization of the forward KL divergence to a complete NIG prior selects a unique prior-relative representative on each fiber, yielding an exact three-coordinate reduction with the same optimum as the four-coordinate objective. Writing $ρ_0=2β_0/ν_0$ and $T=c/\{α[(γ-γ_0)^2+ρ_0]\}$, we show that inverse canonical allocation $1/ν_{\rm can}$ is an explicit strictly increasing function of $T$. For $α>1$, the ratio $u_{\rm epi}/u_{\rm var}=1/ν_{\rm can}$ is therefore determined by the quotient state and prior; for rank-only use under a common calibration, $T$ contains the same coordinatewise ordinal information. The residual prior gauge is characterized rather than eliminated: $(γ_0,ρ_0)$ govern ordinal dependence, while $(ν_0,α_0)$ determine numerical calibration and the analytic ceiling of $1/ν_{\rm can}$.

0.32

ELVAE: Evidential Learning-Based Variational Autoencoder for Uncertainty-Aware Generation

cs.LG cs.AI ·2026-08-11 ·Ge Wang
similarity 0.56
Abstract · 31 pages, 5 figures, 7 tables. Substantially revised: expanded quotient/canonical analysis, added Fashion-MNIST and energy-matched/isotropic…

ELVAE places an input-dependent normal--inverse-gamma (NIG) hierarchy at each VAE latent coordinate, separating location uncertainty $u_{\mathrm{epi}}=β/[ν(α-1)]$ from conditional variability $u_{\mathrm{var}}=β/(α-1)$. The marginalized latent law, however, identifies only the three quotient coordinates $(γ,α,c)$ with $c=β(1+1/ν)$; reconstruction is blind to one $(ν,β)$ fiber direction. A companion theoretical analysis shows that the complete NIG prior and forward KL select a unique prior-relative canonical representative on each fiber, so canonical inverse allocation is not a fourth independent information channel. Empirically, trained inverse evidence $1/ν$ remains the strongest sensitivity-ranking score. At $τ_{\mathrm{epi}}=1$, the three-seed mean high/low-$u_{\mathrm{epi}}$ semantic-transition ratios are 1.98 on MNIST and 1.66 on Fashion-MNIST, falling to 1.33 and 1.16 under scale-matched controls. In the 20-draw MNIST component study with equal per-anchor perturbation energy, $1/ν$ gives high/low ratios 1.69 and 1.65 under the $u_{\mathrm{epi}}$ and $u_{\mathrm{var}}$ fields and 1.69 (95\% interval 1.45--1.97) under a geometry-free isotropic field, whereas $u_{\mathrm{var}}$ reverses the isotropic ordering to 0.76. Under the experimental prior, canonical $1/ν_{\mathrm{can}}$ is a strictly increasing transform of $T=c/[α(γ^2+2)]$ and is bounded above by $3+\sqrt{10}$. Thus ELVAE exposes a controllable sensitivity mechanism whose trained four-output realization is operationally informative, while the exact reconstruction-visible information remains three-dimensional and baseline image quality is a separate question.

0.46

Universality and Convergence of Generative Flows

cs.LG math.PR ·2026-10-04 ·Leo Brunswic
theory similarity 0.56
Abstract · pending corporate approval

Generative flows sample from an unnormalized target by training a flow to be balanced, and the training loss is the signal a practitioner watches. We ask what that signal is worth: whether a small loss certifies an accurate sampler, whether the loss can be driven to zero, and how fast gradient descent does so. The loss decides the first. Losses that compare the two sides of the balance by their difference bound, in total variation, the error of the sampler the flow implies, with explicit constants that do not involve the policy; flow-matching losses that compare them through a ratio admit no such bound, already on a single cycle, whenever their generator is continuous at balance. On graphs, the backward policy decides the other two. Once it is frozen, balance becomes invariance under the backward chain, so that existence is free on finite graphs, and one constant --- the norm of that chain's Green operator, which plays the role of an inverse spectral gap --- fixes the order of the curvature of the loss around the balanced flow, from above and below, and sets a floor under the rate at which training converges near it. The mechanism is that gradient descent diffuses the flow along the backward policy. For the squared-logarithm generator of detailed and trajectory balance, training the balance loss on states converges globally on every finite path-connected graph, from every positive initialization. The constant can be infinite while backward trajectories are short on average, and exact flow matching can then fail. The bounds and rates are tested by exact computation on enumerable state spaces, and every theorem carries a certification status computed from a Lean~4 development.

0.24