CS Paper Radar

Radial Tail Repair: Post-Training Correction for Diffusion and Flow Models

Hamza Cherkaoui, Hélène Halconruy, Antonio Ocello

cs.LG stat.ML ·submitted 2026-10-12 14:24 UTC · v1

Abstract

Diffusion and flow-matching models may struggle to accurately represent distributional tails, especially for heavy-tailed data. Existing approaches often use heavy-tailed noise or modify the generative dynamics, requiring dedicated retraining and more difficult estimation. We take a simpler approach: rather than relearning the full distribution, we correct only the model's behavior in the extreme regime. Our method builds on the radial-angular decomposition of multivariate heavy tails, where radius and direction become asymptotically independent. This allows us to preserve the directional structure learned by the pretrained model and correct only its radial tail. We estimate the tail onset from reference and generated samples, then rescale generated extremes so that their radial survival matches the target distribution. The resulting method is a lightweight post-training correction applied to a frozen diffusion or flow-matching model, requiring neither retraining nor changes to its sampling dynamics. Experiments on challenging synthetic heavy-tailed distributions and real weather and wildfire datasets show that this simple correction substantially improves extreme-tail coverage using only a small number of fitting samples.

Comment
30 pages

Closest papers

Nearest neighbours by meaning across the whole index.

Fine-Tuning Generative Models for Extreme Events via CVaR-Penalized Wasserstein Gradient Flows

cs.LG stat.ML ·2026-08-12 ·Thejani Gamage, Hyemin Gu, Zhizhen Zhang, Ziyu Chen, Markos A. Katsoulakis +1 more
similarity 0.63
Abstract

In many high-stakes domains, extreme events carry substantial consequences, yet learning the heavy-tailed distributions that govern them from finite samples remains challenging: the quantities of interest are driven by a few extreme observations, so the tail is under-sampled relative to its importance. Even generative models tailored for heavy tails capture the tail region inadequately in practice. We propose the Conditional Value-at-Risk (CVaR)-penalized Generative Particle Algorithm (CVaR-GPA), a tail-agnostic algorithm for fine-tuning generative models toward heavy-tailed targets, built as a time discretization of the Wasserstein gradient flow of the Lipschitz-regularized KL divergence penalized by a CVaR discrepancy term. Such a flow can be initialized from the output samples of the pre-trained model, which are then transported along the gradient descent of the loss functional, without requiring access to the pre-trained model's internal architecture. The Lipschitz-regularized KL divergence requires minimal assumptions on the target, while the CVaR penalty focuses the flow on the tail. The CVaR penalty depends on the target only through a scalar tail statistic, inducing a velocity field that remains active in the under-sampled tail region at a dimension-free estimation cost. The resulting velocity field, and hence the depth of the transport map, implicitly adapts to the target, without target-specific modifications. Across four targets, including two real-world, high-dimensional datasets (daily streamflow in the Ohio River basin ($d=64$) and the Fama-French portfolios ($d=25$) with tail indices ranging from $1.05$ to $3.34$), fine-tuning with CVaR-GPA reduces global and tail errors by geometric-mean factors of $14.0 \times$ and $9.8 \times$, respectively, across seven pre-trained models spanning GANs, diffusion models, and other generative flows, with a single set of hyperparameters.

0.34

Ranked Right, Sized Wrong: Ensemble Spread Fails Silently at Streamflow Extremes

cs.LG stat.MLphysics.geo-ph ·2026-10-14 ·Justin Gallant, Tyler Wilson
published similarity 0.60
Abstract · Accepted at the Tackling Climate Change with Machine Learning workshop at NeurIPS 2026. 13 pages, 2 figures, 6 tables. Code: https://doi.org…

Deep learning models, particularly Long Short-Term Memory networks, are increasingly used for operational streamflow forecasting, with ensemble spread providing a signal of predictive uncertainty. As climate change increases the frequency of hydroclimatic extremes, these models will increasingly operate under flow conditions that are rare in their historical records. We therefore evaluate whether ensemble spread remains informative under increasing flows, examining both its magnitude relative to error and its ability to rank errors. Prediction error is approximately $2\times$ the corrected spread at ordinary flows and up to $4\times$ at the highest flows, with the discrepancy increasing with flow (log-log slope $1.30$, 95% CI: $[1.20,1.41]$). Under-dispersion is strongest in observed extreme flows beyond the training record, reaching an $18\times$ discrepancy. This is pervasive and not explainable by conditional mean bias. Area Under the Sparsification Error, a standard distribution-free metric of uncertainty ranking, describing how well a model knows when it is uncertain, indicates near-oracle ranking for ensemble spread ($0.071$), yet ordering by predicted discharge alone nearly matches it. Informative uncertainty ranking can coexist with severe under-dispersion. In deployment, error ranking cannot be verified, yet the spread is available, and it is relied on as the uncertainty signal under increasingly extreme conditions, while systematically understating risk, most severely for the extreme events that matter most.

0.44

FUND: Density Flow for Sampling Unnormalised Distributions

cs.LG hep-lat ·2026-09-26 ·Vikas Kanaujia, Vipul Arora
published similarity 0.59
Abstract · 24 pages, 6 figures

Efficient sampling from Boltzmann distributions is central to modelling complex physical systems. Markov Chain Monte Carlo (MCMC) methods suffer from critical slowing down, high autocorrelation, and poor mode-mixing, limiting their scalability. Recent advances, like Boltzmann Generators, offer a promising alternative but remain constrained by costly MCMC-based training, inefficient sampling, and poor ergodicity. We introduce an algorithm for learning Boltzmann distributions that does not require any true samples for training. Our approach draws inspiration from flow matching but departs fundamentally from sample-trajectory matching to distribution-trajectory matching. The algorithm iteratively reshapes the target distribution, using model generated samples to guide learning and ensure comprehensive mode coverage. We validate our method on standard benchmarks, including a 2D Gaussian mixture, Many-Well distributions, and high-dimensional scalar $φ^4$ theory. The proposed approach not only improves sampling performance and accuracy over traditional MCMC and flow-based baselines but also establishes a new method for sample-free learning of complex physical distributions.

0.65

Cumulative Marginal Matching for One-Step Diffusion

cs.LG ·2026-10-25 ·Paribesh Regmi, Rui Li
similarity 0.59
Abstract

The sampling procedures of diffusion and flow-matching models involve hundreds of denoising steps incurring substantial inference time, which has motivated a growing body of work on training schemes for few-step and one-step generation. Existing approaches primarily seek to align the generation trajectory of few-step and base models, yet their training objectives only involve discrepancies at a single point in the trajectory, encouraging only local \emph{point-level} alignment. While recent methods such as \emph{Shortcut-CSL} introduce cumulative objective that enforce alignment over the entire trajectory, this is suboptimal as they do not directly enforce alignment between the distributions induced by the two models throughout the generation process. We propose \emph{Cumulative Marginal Matching} (CMM), a principled objective for few-step generation: by viewing flow-matching as a sequence of intermediate marginal distributions connecting the prior and data distributions, CMM directly optimizes the cumulative discrepancy between the base and few-step marginal distributions over the entire sequence. Specifically, we formulate this discrepancy using the Wasserstein-2 distance. By exploiting the pushforward representation of flow-matching, we transform an intractable cumulative Wasserstein objective into a tractable surrogate that can be optimized using samples from the intermediate marginal, which are readily generated through the conditional interpolation process of flow-matching. Furthermore, we show that the existing \emph{Shortcut-CSL} objective arises as a first-order approximation of that of the CMM, establishing a theoretical connection between the two objectives. Experiments on standard benchmarks demonstrate improvement in generation quality over \emph{Shortcut-CSL} and other state-of-the-art one- and few-step baselines under identical training budgets.

0.06

Mini-batch Sampling Strategies for Long-Tailed Image Classification: An Empirical Study on CIFAR-100-LT

cs.CV stat.MLcs.LG ·2026-09-14 ·Siyu Yuan
code/data similarity 0.58
Abstract · 38 pages, 12 figures, 17 tables. Code and per-run logs: https://github.com/SiyuYuan0117/longtail-sampling

Real-world datasets often exhibit long-tailed class distributions, where a few head classes contain a large number of training samples while a large number of tail classes have only a few. The composition of each mini-batch, determined by the sampling strategy, governs which classes contribute to the stochastic gradient estimate, and therefore affects convergence behaviour and generalisation across the whole class spectrum. We provide a systematic theoretical and empirical comparison of four mini-batch sampling strategies for long-tailed image classification: uniform instance sampling, class-balanced sampling, square-root sampling, and progressively balanced sampling. We place all four in a unified bias-variance framework describing their effect on gradient estimation, which exposes the tension between unbiased optimisation of the empirical loss and fair representation of rare classes. We then evaluate them under controlled conditions using ResNet-32 on CIFAR-100-LT at three imbalance ratios (rho = 10, 50, 100), with every strategy sharing the same long-tailed subsets and initialisation within a seed. Progressive sampling improves tail-class accuracy by 25% relative to the uniform baseline at rho = 100 (13.5% versus 10.8%), consistently across all three seeds, while its overall accuracy is not distinguishable from that of uniform sampling given the seed-to-seed variation (40.0% versus 39.7%); the tail-class gain, not the overall gain, is the robust effect. At rho = 100, class-balanced sampling degrades accuracy on every class group, including the tail classes it is designed to help, which we attribute to overfitting caused by extreme oversampling of scarce data; at rho = 50 this failure is confined to head and medium classes. These results indicate that when rebalancing is applied during training matters as much as how much rebalancing is applied.

0.44

Annealed Lévy Diffusion via Gaussian Surrogate Reverse Dynamics

cs.LG stat.ML ·2026-10-14 ·Yuling Jiao, Lican Kang, Jerry Zhijian Yang, Cheng Yuan, Ruizhe Zhang
similarity 0.58
Abstract

Gaussian diffusion models offer a practical framework for generative modeling. Denoising Lévy Probabilistic Models (DLPM) introduce heavy-tailed perturbations with a fixed tail index throughout denoising. We introduce Annealed Lévy Diffusion (ALD). ALD varies the stability index $α(t)$ to combine heavy-tailed exploration with Gaussian refinement. Our marginal-first formulation represents stable noise as a scalar Gaussian scale mixture. This formulation enables standard Gaussian noise-prediction training with one additional scale input. We study two sampling processes. ALD-renoise estimates the clean sample and injects fresh marginal noise. ALD-posterior preserves dependence between successive states through an accumulated conditional variance and a Gaussian bridge update. Both samplers reuse the same trained predictor with different scalar inputs. We evaluate ALD on four 2D benchmarks and an imbalanced nine-mode mixture. We also examine training efficiency and the effects of prediction perturbations on clipped Holtsmark data. Additional experiments evaluate MNIST and CelebA64 generation and CIFAR-10LT augmentation. ALD preserves local geometry on the evaluated light-tailed and manifold-structured targets.

0.23

Wet-Biased Data Pipelines Mislead the Training and Evaluation of Generative Precipitation Nowcasting

cs.LG physics.ao-ph ·2026-10-19 ·Christoph Metzl, Stefan Kapsreiter, Kianusch Vahid Yousefnia, Virginia Poli, Richard Müller +2 more
similarity 0.57
Abstract · 63 pages, 7 figures and 3 tables in the main text, 11 supplementary figures and 9 supplementary tables. This work has been submitted to npj …

Generative models for precipitation nowcasting are trained and evaluated predominantly on wet-biased datasets, which over-represent precipitation events to emphasize rare extremes. Selecting test cases based on target observations is known to favor forecasts that match the selected distribution rather than the intended deployment distribution. We show analytically how this target-dependent sampling also shifts the conditional distribution learned by generative models. When training and test sampling match, even an ideal model learns precisely the distribution favored by evaluation, allowing the evaluation bias to conceal the training bias. Using a fixed satellite-to-radar diffusion model, we demonstrate that target-dependent test sampling can reverse the calibration assessment. The model appears under-predictive on a wet-biased test set but over-predictive on a deployment-representative test set. This result is replicated outside the model's training domain after a seven-year gap. Utilizing classifier-free guidance as an example, we demonstrate how the contradictory calibration assessment can misdirect downstream model development.

0.15

Climate downscaling with likelihood functional flow matching

cs.LG ·2026-10-24 ·Zhi-Song Liu, Helmi Toropainen, Ammar Kheder, Michael Boy
similarity 0.57
Abstract · 32 pages, 14 figures

Decisions about heat, frost and flooding are made locally, yet reanalyses and climate models resolve tens of kilometres. Generative downscalers add kilometre-scale detail with uncertainty, but are tied to a grid, costly to sample and lack a tractable likelihood. Here we introduce CD-FFM, a functional flow-matching model that treats the high-resolution field as a continuous random field that one model can query at any location and resolution. Its velocity acts pointwise, so spatial variability comes from a noise prior fitted to the observed spectrum and every observation has an exact likelihood. Downscaling ERA5 to 1 km daily temperature over Fennoscandia, CD-FFM lowers the continuous ranked probability score by 27-31% against regression and 12-18% against a forty-times larger neural operator in each of three test years, and outperforms a CorrDiff-style diffusion model while sampling about fifty times faster. The same model transfers to precipitation and makes large kilometre-scale ensembles affordable.

0.17