CS Paper Radar

Improving Function Space Flow Matching with Kernel Optimal Transport

Fred Xu, Thomas Markovich, Barbora Barancikova, Yizhou Sun

cs.LG ·submitted 2026-09-29 17:20 UTC · v1
publishedbenchmarktheory

Abstract

Generative models for function-valued data, such as time series and solutions of partial differential equations, must learn distributions over infinite-dimensional spaces. Functional Flow Matching (FFM) extends Flow Matching to this setting, learning a velocity field whose flow transports a Gaussian prior to the data distribution, but it inherits the independent endpoint pairing of standard Flow Matching: in each batch, prior and data samples are matched arbitrarily, so the conditional bridge must traverse both the shared global structure of the dataset and instance-specific residuals. In function space this is harder to fix than in finite dimensions, since optimal transport (OT) on function spaces is delicate to formulate and a flat Euclidean surrogate ignores the geometry that distinguishes function-valued data. We propose kernel Functional Flow Matching (kFFM), which replaces the independent pairing by entropic OT under a kernel-induced cost, the coupling underlying the Hilbert Sinkhorn Divergence (HSD), leaving the FFM neural-operator architecture unchanged. We prove that the kernel cost and the HSD objective are uniformly bounded and well-posed on Banach ambient spaces, derive an error decomposition against quadratic-cost OT on compact metric spaces that isolates an irreducible kernel-cost mismatch term, and prove a discretization-invariance bound whose rate is governed by Sobolev regularity. Empirically, kFFM improves distributional matching over FFM, diffusion, adversarial, and finite-dimensional OT baselines on time-series and PDE benchmarks, with significant paired-seed gains over FFM and improvements that persist under non-kernel and physics-based diagnostics, including a turbulent Navier-Stokes benchmark. Bounded kernel costs already outperform raw $L^2$ Sinkhorn, and function-space-aware kernels (signature, Sobolev RBF) give further gains on rough or path-valued data.

Comment
Paper is already accepted at Neurips

Closest papers

Nearest neighbours by meaning across the whole index.

Generative Modeling with Continuous Flows: Sample Complexity of Flow Matching

cs.LG cs.AI ·2026-11-01 ·Mudit Gaur, Prashant Trivedi, Shuchin Aeron, Amrit Singh Bedi, George K. Atia +1 more
theoryhighly cited similarity 0.70
Abstract

Flow matching has emerged as a promising alternative to diffusion-based generative modeling, offering a continuous-time ordinary differential equation formulation with fast sampling and a simple regression-based training objective. Despite its empirical success, some theoretical analyses idealize the learning procedure by assuming that the velocity field is obtained by exact empirical risk minimization. This assumption does not reflect the practical finite-dataset setting, where the empirical flow-matching loss is only approximately minimized by a finite number of stochastic-gradient updates. We develop an empirical-risk-minimizer-free analysis for flow matching that explicitly accounts for both finite-sample statistical error and finite-time empirical optimization error. Our analysis considers a finite dataset of \(N\) flow-matching samples and a learned parameter \(θ_T\) obtained after \(T\) stochastic gradient updates on the empirical objective. Under regularity assumptions on the interpolation path, sub-exponential source and target distributions, approximation by a bounded-parameter smooth neural velocity class, empirical-loss optimization, and uniform flow stability, we prove a guarantee that holds with high probability over the finite training dataset and in expectation over the stochastic-gradient randomness. The resulting bound separates approximation error, statistical error from the finite dataset, tail error from unbounded source and target distributions, and empirical optimization error from approximate minimization of the finite-sample objective.

0.32

Invertible Transport Between Sample-Only Distributions via Projected Characteristic Functions

cs.LG cs.CV ·2026-11-19 ·Juan Gutiérrez, Victor Gutiérrez-García, José Luis Blanco-Murillo
benchmark similarity 0.68
Abstract

Many useful latent distributions can be sampled but do not have a tractable density. Such distributions may be represented by empirical collections of real-world observations or generated by scientific simulators and black-box generative models. These sample-only priors cannot be used directly in likelihood-based normalizing flows, which require evaluating the prior density. We propose Projected Characteristic-Function matching (PCF), a density-free framework for learning an invertible map between a data distribution and a prior specified only through samples. Our method aligns distributions using projected empirical characteristic functions, eliminating the need for density estimation, adversarial training, or paired observations. To address the non-uniqueness of the transport map, we introduce an optional transport regularizer that selects lower-displacement transformations while preserving endpoint alignment. We benchmark PCF on controlled transports, sample-efficiency studies, and high-dimensional alignment tasks. PCF matches the accuracy of kernel MMD on the controlled benchmark and on raw-pixel MNIST--USPS at a cost linear in the batch size, and it is more sample-efficient, achieving lower endpoint error than MMD in 13 of 15 paired runs across library sizes. By avoiding pairwise kernel evaluation, it reduces discrepancy memory by 29.8% and trains at batch sizes where kernel MMD exceeds GPU memory. These results show that PCF provides accurate and efficient invertible transport, particularly in sample- and memory-limited settings.

0.22

VTV-FM: Flow Matching through Variational Terminal-Velocity Closure

cs.CV ·2026-09-30 ·Haoyang Jiang, Yuheng Li, Di Yang, Yanhai Xiong, Haipeng Chen +1 more
top venuecode/data similarity 0.67
Abstract · Accepted at NeurIPS 2026. Code: https://github.com/HaoyangJiang-WM/VTV-FM

Flow matching (FM) learns generative transport by fitting continuous-time motion from a simple source distribution to the data distribution. Most existing methods use first-order bridges: once a source and a target sample are paired, the path is a straight motion with constant velocity. FM with optimal transport (OT) improves the pairing, but the bridge itself remains linear, limiting its ability to model curved motion, acceleration, and changing directions. A natural remedy is to use second-order phase-space dynamics; however, learning the bridge requires target-side terminal-velocity information that static datasets do not provide. We propose Variational Terminal-Velocity Flow Matching (VTV-FM), a second-order FM framework that derives the missing velocity by minimizing acceleration energy, yielding a closed-form closure for static data. The same minimum-acceleration variational construction also defines the OT pairing cost and the acceleration targets used for training. Experiments on low-dimensional datasets, PDE-governed physical fields, and CIFAR-10 show that VTV-FM improves transport geometry and generation quality over first-order and high-order FM baselines.

0.63

Probabilistic Geodesic Flow Matching on Location-Scale Families

cs.LG stat.MLmath.PR ·2026-09-28 ·Zeyuan Yu, Zhi Chang, Shiwei Lan
position similarity 0.67
Abstract · 25 pages, 10 figures

Flow matching (FM) has recently emerged as a promising framework for generative modeling due to its conceptual simplicity and strong empirical performance. In FM, samples are transported along a vector field parameterized by a neural network, inducing a probability path that evolves from a simple noise distribution to the target data distribution, governed by an ordinary differential equation (ODE). However, existing FM approaches predominantly rely on probability paths derived from optimal transport (OT) between Gaussian distributions, which may be suboptimal for capturing complex data with inhomogeneous structures such as heavy tail or sharp contrast. In this work, we generalize FM to the broader class of location-scale families for handling data inhomogeneity and introduce a novel class of probability paths defined as geodesics on the manifold of probability distributions. We name this approach probabilistic geodesic flow matching to distinguish it from prior geodesic (Riemannian) FM methods defined in input space. We argue that Euclidean OT-based paths are not necessarily optimal in probability space and may limit modeling flexibility. Through synthetic benchmarks and scientific datasets at different scales, we demonstrate that the proposed method more effectively captures complex distributions, leading to improved or comparable performance compared with SOTA geometry-motivated generative models.

0.22

Optimal Control of RKHS Diffeomorphic Flows for Classification and Generative Modeling

cs.LG math.NAmath.OC ·2026-11-17 ·Christopher Chukwuemeka, Andreas Mang
benchmark similarity 0.66
Abstract · 55 pages, 21 figures, 16 tables, 6 algorithms

We develop a unified optimal-control framework that solves supervised classification and likelihood-based density estimation with a single diffeomorphic flow whose velocity lives in a reproducing kernel Hilbert space (RKHS); the two tasks share all structure except the objective. Variable projection eliminates the terminal model (a classifier, or a Gaussian-mixture (GMM) reference), leaving a reduced control problem whose gradient comes from an analytic discrete adjoint. The RKHS parameterization makes the flow's divergence exact and deterministic at the cost of one kernel matrix-vector product, avoiding the stochastic trace estimators and potential-flow restrictions of existing continuous normalizing flows. Because the objective and gradient are deterministic, matrix-free reduced Gauss-Newton (GN) and related second-order solvers are practical for both tasks, which we benchmark against a range of first- and second-order optimizers; the learned map is a diffeomorphism with explicitly controlled regularity. In experiments from dimension 2 to 256, the reduced GN solvers use the fewest iterations of any optimizer tested and, together with joint L-BFGS, the least work; a multiscale kernel improves the density fit, and a trainable GMM reference reduces the deformation needed for multimodal transport at comparable likelihood while lowering the likelihood on real data. The full-model solve is quadratic in the particle count and sub-linear in dimension; a control-point reduction makes the particle-count scaling linear at near-lossless accuracy.

0.33

When do explicit penalties improve flow matching?

cs.LG stat.MLmath.ST ·2026-10-11 ·Lea Kunkel, Alberto González-Sanz, Sophie Langer
similarity 0.65
Abstract

Flow matching can learn structure directly from data, raising the question of when additional constraints improve generation. We study penalties that encode such constraints, with physics-informed flow matching as a motivating example. Satisfying a constraint more closely need not make the generated distribution more accurate. We therefore compare penalised and unpenalised generative modelling through their expected squared $2$-Wasserstein error. Our analysis covers both vector-valued and functional data in a Hilbert-space setting. We first derive geometric criteria that account for constraint violations and the distribution of samples after projection onto the constraint set. We then solve the statistical comparison explicitly for specific velocity-field classes and constraints via flow matching. In this model, penalisation improves accuracy exactly when the reduction in estimation variance outweighs the increase in squared bias. The resulting formula shows how this trade-off depends on sample size, the model class, the noise schedule, and the penalty weight.

0.09

Explicit Regularization for Flow Matching

cs.LG math.NA ·2026-11-14 ·Lars Ruthotto, Eran Treister
similarity 0.65
Abstract · 40 pages (main text pp. 1-24, references pp. 24-26, appendices A-C pp. 27-40), 4 figures, 11 tables

Learning the underlying probability distribution from samples is a core problem in statistics and applied mathematics. Generative models based on diffusion and flow matching learn a velocity field whose flow transports a known distribution to the data distribution. They produce novel, realistic samples, although the empirical learning problem is ill posed near the data distribution. Its minimizer transports the samples onto the training data, and small perturbations of the data change the velocity field strongly near the end of the flow. The observed generalization is often attributed to implicit regularization by the network parameterization, early stopping of the training, and the time integration during sampling. Explicit regularization has advanced the theory and computation of many inverse problems and has recently been proposed to prevent memorization in generative models. With a regularizer on the magnitude of the velocity, however, the field approaches the memorizing velocity unless the regularization parameter grows to infinity fast enough, and it then stays bounded by a constant that is small only for large parameters. We show that penalizing the derivatives of order $k$ of the velocity renders the $D$-dimensional empirical problem well posed uniformly in time if $2k > D$ and the training points are in general position. Our experiments in one and two dimensions discretize the velocity with splines and use direct solvers for the regularized least-squares problems to exclude optimization error. Refining the discretization in space and time changes the error of the Hessian-regularized flows only marginally, whereas without the regularizer the error tends to that of memorization. These flows generalize, with errors comparable to or lower than those of well-tuned kernel density estimates.

0.19

Barron Optimal Transport I: Generative Modeling

cs.LG math.AP ·2026-10-07 ·Evan Dogariu, Joan Bruna
similarity 0.65
Abstract

Motivated by recent applications in generative modeling and sampling, we introduce a framework for optimal measure transport where cost captures the notion of neural network complexity. In transport-based generative models, samples from a reference distribution (e.g. Gaussian) are mapped to samples of a target distribution along ordinary or stochastic differential equations. These are implemented as deep residual networks when discretized in time, where each hidden layer approximates the associated instantaneous velocity. Thus, given a pair of target and reference measures, a natural question is to search for the most efficient neural representation that implements this transport. Our starting point is the kinetic formulation of OT, due to Benamou and Brenier. We replace the average kinetic $L^2$ energy by the \emph{Barron} energy \cite{bach2017breaking, ma2022barron}, a natural norm which measures the complexity of representing a given vector field with a neural hidden layer, and which captures the adaptive properties of feature learning. This defines a metric on the space of probability measures, complementing existing Wasserstein and Stein geometries. In this work we examine the properties of this metric in the context of generative modeling. As a first application, we quantify the suboptimality of diffusion generative modeling in the Barron geometry by establishing super-polynomial score approximation lower bounds for data generated by neural network pushforwards of the Gaussian. We then investigate the benefit of adaptivity as a way to study alternative generative models. In a companion paper \cite{companionpaper} we leverage the Barron transport geometry for sampling applications, extending the scope of Stein variational gradient methods via feature adaptation.

0.11