CS Paper Radar

ParetoTransport: Generative Optimization by Mass Transport Toward The Pareto Front

Stephanie Holly, Sepp Hochreiter, Werner Zellinger

cs.LG ·submitted 2026-09-07 16:19 UTC · v1

Abstract

Offline multi-objective optimization requires not only moving the objective vectors of candidate designs toward the Pareto front, but also distributing them effectively along it. Generative methods have recently emerged as a natural approach because they learn a distribution over feasible designs while allowing generation to be steered toward promising designs. Existing methods, however, largely retain classical sample-wise guidance strategies, leaving the distribution-level modeling capability of generative methods underused. We propose ParetoTransport, a training-free guidance method for pre-trained flow-matching models that explicitly specifies and refines a population-level distribution in objective space. ParetoTransport guides a flow-matching sampler to iteratively transport the empirical offline distribution toward the Pareto front, with Wasserstein matching to intermediate proxy distributions. This directly controls distributional displacement and mass allocation along the front. We establish a convergence result and demonstrate state-of-the-art performance on standard offline MOO benchmarks, extending recent evaluations beyond hypervolume to generational distance, inverted generational distance, and Wasserstein distance.

Closest papers

Nearest neighbours by meaning across the whole index.

Learning Where to Steer: Noise-Space Geometry for Efficient Offline Multi-Objective Optimization with Generative Models

cs.LG ·2026-09-30 ·Yuan Lu, Esha Singh, Yi-An Ma, Yusu Wang
theory similarity 0.70
Abstract · 65 pages, including appendices

Offline multi-objective optimization (MOO) seeks solutions with better objective trade-offs using only a fixed dataset, without querying the objectives. Diffusion models trained on such data have emerged as a promising approach, but their samples are not inherently better than the data and must be steered toward the Pareto front. Existing methods guide or condition every sampling step. We instead act on the initial noise and leave the sampling process unchanged. Across Off-MOO-Bench, we observe that the objectives, as functions of the noise, are sensitive to only a few directions. We estimate these directions once per task via a Recursive Feature Machine using function values alone, and a small cache serves every trade-off, so each candidate costs one noise displacement and one ODE solve. We prove that this displacement increases the learned scalarized objective in expectation, and that sweeping trade-offs recovers the flow's attainable front up to proxy and steering errors. With additional guidance, for which we introduce novel data-adaptive and Pareto-aware operators, our method attains the best average hypervolume rank among generative methods on 47 tasks, at comparable or lower sampling cost. Steering alone outranks the best prior generative method at a fraction of its sampling cost.

0.51

Fenchel Tilting: Weighted Correction for Efficient Finetuning of Generative Models

cs.LG cs.AI ·2026-09-30 ·Maksim Bobrin, Maksim Zhdanov, Dmitry Dylov
similarity 0.67
Abstract

Adapting a pretrained generative model to an arbitrary preference expressed as a utility function underlies reward alignment, guided design, and constraint satisfaction, enabling diverse applications. Existing fine-tuning methods trade off generality against computational cost: they either restrict the family class of supported preferences to keep optimization simple or preserve generality at the expense of efficiency. We introduce Fenchel Tilt Flow Control (FTFC), which decouples utility optimization from generative-model fitting. FTFC first optimizes for a target distribution by jointly fitting an effective reward and density-ratio weights on pretrained samples. Method combines the utility's variational structure with Fenchel duality, supporting general $f$-divergence penalties that determine how rewards are transformed into an distribution-correction weights. These weights are then frozen and used to modify a diffusion or flow model in a single stage of importance-weighted denoising or flow matching, without differentiating through sampling trajectories. We establish exact duality for concave utilities under suitable conditions and show that weighted fitting reproduces the optimal target distribution for a given utility. Across image and molecule generation benchmarks, FTFC improves over baselines on diverse preference functions, while also being up to $20\times$ more efficient. roposed method enables adaptation beyond expected-reward maximization without complex optimization, while preserving robustness for more general class of the utility functions compared to baselines.

0.23

When do explicit penalties improve flow matching?

cs.LG stat.MLmath.ST ·2026-10-11 ·Lea Kunkel, Alberto González-Sanz, Sophie Langer
similarity 0.66
Abstract

Flow matching can learn structure directly from data, raising the question of when additional constraints improve generation. We study penalties that encode such constraints, with physics-informed flow matching as a motivating example. Satisfying a constraint more closely need not make the generated distribution more accurate. We therefore compare penalised and unpenalised generative modelling through their expected squared $2$-Wasserstein error. Our analysis covers both vector-valued and functional data in a Hilbert-space setting. We first derive geometric criteria that account for constraint violations and the distribution of samples after projection onto the constraint set. We then solve the statistical comparison explicitly for specific velocity-field classes and constraints via flow matching. In this model, penalisation improves accuracy exactly when the reduction in estimation variance outweighs the increase in squared bias. The resulting formula shows how this trade-off depends on sample size, the model class, the noise schedule, and the penalty weight.

0.09

MintFlow: Minimal Trajectory Intervention for Constrained Flow Matching

cs.AI ·2026-09-30 ·Yesom Park, Kelvin Kan, Qifan Chen, Thomas Flynn, Hayden Schaeffer +1 more
hot direction similarity 0.66
Abstract

Flow matching models excel at generative modeling, and many downstream applications require their samples to satisfy prescribed constraints, such as observed measurements and physical laws. However, existing constrained samplers often face a trade-off: \textit{enforcing constraints can substantially displace samples from the pretrained data distribution}. To address this trade-off, we introduce \textbf{MintFlow}, a training-free constrained sampling framework that formulates constraint enforcement as a minimal intervention on the pretrained flow trajectory. MintFlow seeks the minimal perturbation of an intermediate flow state such that its subsequent evolution under the pretrained flow field satisfies the target constraint. By minimally perturbing the flow state while keeping the pretrained flow field unchanged, MintFlow enforces the constraint while minimizing unnecessary deviation from the pretrained distribution. An adjoint formulation yields a closed-form expression for this perturbation, eliminating expensive iterative optimization. Furthermore, MintFlow adaptively selects the intervention time to balance the required perturbation magnitude with its amplification by the remaining flow. Across a range of tasks in generative vision and physical system modeling, MintFlow achieves competitive constraint satisfaction while preserving the pretrained generative distribution substantially better than state-of-the-art constrained methods.

0.28

Acquisition Function-Guided Generation for Bayesian Optimization over Discrete Spaces

cs.LG ·2026-10-16 ·Tommaso Ancilli, Thomas Wedenig, Henry Moss, Robert Peharz
code/data similarity 0.64
Abstract

Bayesian optimization (BO) over discrete spaces is challenging due to missing gradients and the sparsity of valid solutions. Latent-space BO sidesteps both issues by fitting a surrogate in the latent space of a generative model, which however substantially complicates the direct incorporation of domain knowledge that is naturally expressed in the original discrete space. In this work, we instead combine a pretrained generative model with a surrogate defined directly in data space, where inductive biases such as domain-specific kernels can be readily incorporated. Rather than maximizing the acquisition function over the discrete design space, our method samples candidates from the generative model under a distribution tilted by the acquisition function. To recover useful gradients for sampling, we embed this tilted distribution into a diffusion bridge that yields a smooth, posterior-averaged acquisition function. This leads to \emph{acquisition function guided generation} (AFGG), which we instantiate with a categorical diffusion model and an autoregressive normalizing flow. Across a broad suite of molecular benchmarks, the diffusion model variant performs on par with leading BO methods, while the flow model variant outperforms state-of-the-art methods in most cases and remains highly competitive otherwise. Our implementation is available at https://anonymous.4open.science/r/afgg.

0.24

Knowledge Transfer for Conditional Flow Matching: Frozen Estimation and Adaptation

cs.LG stat.ME ·2026-11-11 ·Yian Huang, Jian Huang
theory similarity 0.64
Abstract

Conditional generative modeling with limited target data can benefit from representations learned in a related source population. We develop a transfer learning framework for conditional flow matching that accommodates distinct source and target response distributions and dimensions. Two procedures exploit a representation learned from source data: TFM-Freeze estimates a target velocity head with the representation fixed, whereas TFM-Adapt jointly learns the head and a representation adaptation under a functional anchoring penalty. For explicit neural network classes, we establish bounds on conditional squared Wasserstein risk that account for source estimation, target fitting, representation mismatch, and numerical integration. When the source representation admits a smooth correction, adaptation replaces the frozen mismatch contribution by the cost of learning the correction and a controllable regularization bias. An information lower bound identifies regimes in which adaptation attains a strictly faster conditional squared Wasserstein risk rate than any generator restricted to the frozen context information. We also develop independent validation procedure for selecting among fitted conditional generators. We show that the selection procedure satisfies an oracle inequality, and the switching rule guarantees, with high probability, no increase in conditional energy risk relative to a target-only generator. Simulations and real data applications illustrate the gains from our proposed methods and the effectiveness of target validation in guarding against negative transfer.

0.09

One-Step Generative Modeling via Unbalanced Optimal Transport

cs.IT cs.LG ·2026-09-26 ·Yirong Shen, Mengfei Xia, Junpeng Jing, Lu Gan, Cong Ling
similarity 0.63
Abstract

Drifting models enable one-step generation by amortizing distribution transport into training, but this efficiency places greater demands on the transport field estimated at each update. In large-scale training, the field is computed from finite mini-batches of generated and real samples, which provide only imperfect approximations to the underlying distributions. Balanced optimal transport enforces exact mass matching within every mini-batch, making the estimated field sensitive to the particular composition of the real-data batch. We find that generated and real samples should be treated asymmetrically: letting the mass assigned to real samples adapt while keeping every generated sample fully transported improves generation across six feature-space metrics in controlled ablations, and is more robust to the relaxation strength than relaxing both marginals simultaneously, which falls below balanced transport under stronger relaxation. Motivated by this observation, we propose Unbalanced Optimal Transport Gradient Flow (UOT-GF), which keeps the generated-sample marginal fixed and relaxes only the real-data marginal. Under identical settings at DiT-B/2 on ImageNet-256, UOT-GF improves Fréchet Inception Distance (FID) from 1.53 to 1.46 over the balanced W-Flow baseline; scaling the same recipe yields 1.34 and 1.22 FID at L/2 and XL/2, the best FID among the one-step models we compare. We further derive the induced UOT transport force, establish a kinetic Vlasov--Fokker--Planck formulation whose overdamped zero-temperature limit recovers the drifting dynamics, and characterize non-target stationary states together with sufficient conditions for convergence.

0.25

Few-Step Generation via Data-Space Iteration

cs.LG cs.CV ·2026-10-08 ·Shanchuan Lin, Yansong Peng, Fu-Yun Wang, Haoqi Fan
similarity 0.63
Abstract

Flow matching has emerged as a scalable paradigm for training high-quality generative models, but sampling from the learned probability flow requires many network evaluations. Distillation can reduce this cost to one or a few evaluations; however, one-step generation often sacrifices quality, making few-step generation the practical operating regime. Existing few-step methods perform their iterative computation along the probability flow and therefore require a fixed, manually chosen timestep discretization. This discretization is often chosen heuristically and is expensive to tune; it may also be restrictive when refinement difficulty differs across samples or spatial locations. We introduce data-space iteration, a few-step generation framework that removes flow discretization altogether. Starting from noise, a shared generator directly refines its prediction in data space, with every iteration trained to produce the best sample permitted by its capacity. Our formulation integrates with distribution matching distillation (DMD) with minimal changes, enabling a controlled comparison between iteration methods under matched training settings. On class-conditional ImageNet 256x256, data-space iteration outperforms standard discretization baselines and matches or improves upon variants selected through schedule search, without requiring schedule-specific training. These results show that data-space iteration provides a simple and effective alternative to discretized flow-space iteration for fast generation.

0.15