CS Paper Radar

MeanFlowAdvantage: Stable Reward Fine-Tuning for Few-Step Average-Velocity Generators

Haocheng Tang, Tianchi Xie, Xingqiao Lin

cs.AI cs.CEcs.CV ·submitted 2026-09-29 14:25 UTC · v1

Abstract

MeanFlow enables efficient few-step generation by predicting interval-average velocities, but this representation creates a mismatch for reward fine-tuning: existing advantage-based objectives are typically defined on instantaneous velocities or equivalent $x_0$-space predictions, whereas inference directly uses the learned average-velocity map. We introduce MeanFlowAdvantage, a signed advantage-weighted least-squares objective for average-velocity generators. Our key construction uses a shared, detached MeanFlow derivative correction to express the reward objective in prediction space while making rollout and reference regularization exact penalties on the average-velocity network deployed at inference. The resulting formulation preserves MeanFlow's native few-step sampler and provides a direct mechanism for transferring reward improvements to the deployed flow map. On SD3.5-Medium, MeanFlowAdvantage improves all eight reported metrics over the matched four-step MeanFlowNFT baseline and, with only four NFEs, matches or exceeds the 40-step DiffusionNFT baseline on six of eight metrics. The same objective also transfers to DNA promoter design, where it supports both teacher-free on-policy RL for a generator defined on a manifold and teacher-guided reward-graded distillation, with the latter yielding the lowest one-step Sei profile MSE among the compared configurations.

Closest papers

Nearest neighbours by meaning across the whole index.

Continuous Adversarial MeanFlow for Few-Step Generative Adaptation from Any Parameterization

cs.LG cs.CV ·2026-08-20 ·Yara Bahram, Zahra Dehghani, Mélodie Desbos, Eric Granger, Pablo Piantanida +1 more
similarity 0.66
Abstract · Paper under review, Version 2.0

Training fast generators on new domains with limited data remains challenging for two reasons. First, adapting a pretrained diffusion or flow model leaves its costly multi-step sampling intact, and existing acceleration methods are tied to the source parameterization (x, epsilon, v, or u), so heterogeneous pretrained models share no common acceleration target. Second, while adversarial post-training sharpens generation quality, existing criteria each cover only one end of the step range. Discrete criteria compare jump endpoints and lose their signal as the jump shrinks, while continuous criteria score the instantaneous velocity and misjudge the large jumps of few-step sampling. Neither fits MeanFlow (MF) models, which predict average velocities over intervals of any length. We propose MeanFlow-Transfer (MF-T), which adapts a source domain model of any parameterization and compresses it into a few-step MF generator through a common velocity map. We then introduce Continuous Adversarial MeanFlow (CAMF), which scores the change of a learned potential along each predicted interval jump of MF. CAMF needs no gradient penalty or Jacobian-vector product (JVP). We also prove it stays well-conditioned for every interval length and recovers the instantaneous criterion as the interval vanishes. Adapting four ImageNet-based source models (DiT, SiT, JiT, iMF) to five target domains, MF-T + CAMF approaches or exceeds the fine-tuned many-step teacher in FID and FDD at up to 125x fewer Neural Function Evaluations (NFEs). Further, CAMF improves MF-T's few-step FID by about 30% on average, outperforming prior adversarial post-training on iMF at every NFE.

0.12

EbbFlow: PINN-Inspired Progressive Curriculum for MeanFlow Training

cs.LG ·2026-11-08 ·Mengfan Yuan, Xunheng Kuang, Jianbin Chen, Lei He
similarity 0.63
Abstract · 20 pages, 6 figures

MeanFlow learns interval-average velocity for few-step generation, but its fixed temporal sampler mixes diagonal and interior supervision throughout training. We interpret temporal coordinates as collocation points: diagonal samples provide a stochastic boundary target, whereas finite intervals supply a differential-consistency target. The diagonal fraction controls how often the consistency component appears, while interval length enters its gradient through an explicit factor. Motivated by this distinction, EbbFlow uses Temporal Supervision Allocation (TSA) to schedule the diagonal fraction and interval geometry. It starts with diagonal supervision, then admits interior samples and expands their maximum interval; a left-anchored interval law preserves coverage across time. The per-sample loss, target, backbone, and inference procedure remain unchanged. In matched class-conditional ImageNet $256\times 256$ runs without classifier-free guidance (DiT-B/2, one training seed per method, 400k updates), EbbFlow achieves 50k-sample FID of 39.65 at one network evaluation and 36.02 at two-evaluation ODE sampling, versus 44.82 and 39.86 for MeanFlow. Gradient probes show a mid-training difference in component geometry. Ablations support allocation scheduling and left-anchored intervals, although a fixed full-horizon variant performs better at one evaluation. These results suggest that temporal sampling is a useful training control whose benefits depend on the inference protocol.

0.23

GuidanceNFT: NFT-Based Reward Fine-Tuning of Guided Diffusion Models

cs.LG ·2026-10-11 ·Jiayin Liu, Zeyu Michael Li, Bingshuo Qian, Xiang Cheng
similarity 0.62
Abstract

DiffusionNFT (NFT) fine-tunes diffusion models toward task-specific rewards, but optimizing these rewards does not guarantee fidelity to the full prompt. Although classifier-free guidance (CFG) strengthens prompt alignment, NFT is formulated as a CFG-free method, and applying CFG to the fine-tuned velocity field can reduce the optimized reward. This motivates optimizing rewards over the guided output distribution while seeking to retain its prompt alignment. However, applying standard NFT directly to guided samples can cause reward degradation or training collapse. We trace this failure to a mismatch between the reference velocity used in the NFT loss and the score of the guided sample distribution, which biases the improvement direction. We propose GuidanceNFT, which estimates the matching reference velocity through online flow matching on guided samples and uses it to correct both regression terms. With an exact reference, the corrected objective recovers NFT's intended update. In our experiments, the learned correction sustains reward improvement under both explicit and built-in guidance. On SD3.5-M, GuidanceNFT achieves higher optimized rewards than NFT with and without CFG across GenEval, text-rendering, and multi-reward objectives, together with better overall prompt alignment as measured by off-domain evaluation metrics. Experiments on models with built-in guidance, including Model Guidance and ELF, further demonstrate improved ImageNet generation, a better perplexity-entropy tradeoff on OpenWebText, and higher accuracy on GSM8K.

0.13

Velocity Scaling in Flow Matching

cs.CV ·2026-10-07 ·Youssef Saied, François Fleuret
similarity 0.62
Abstract · 41 pages, including appendix

Scaling a learned flow-matching velocity field $v_θ$ by a gain $γ(t)$ was recently shown to greatly improve generation quality. Prior work argued that velocity fields trained with mean-squared error (MSE) systematically underestimate velocity magnitude and that scaling corrects this error. We show that MSE training does not create a velocity-magnitude deficit. We find instead that velocity scaling reduces population time lag: sampled states at model time $t$ resemble training states from an earlier time. Velocity scaling and moving model time back are two ways to address this population time lag. Across architectures and model sizes, measuring population time lag and using it to select a gain greatly improves generation quality, reducing FID from 28.0 to 12.2 (estimated by linear interpolation between FID measurements at neighboring gains) on ImageNet-256 at NFE 25 without guidance.

0.23

Classifier-Free Guidance in Flow Matching: Non-Autonomous Potentials, Overshoot, and Posterior-Mean Control

cs.CV ·2026-09-21 ·Jishen Peng, Zheng Ma
similarity 0.60
Abstract

Classifier-free guidance (CFG) improves conditional generation in Flow Matching, but strong guidance can distort the generated distribution and reduce diversity. We provide a geometric account of this behavior by viewing Flow Matching as a time-varying gradient flow and characterizing how CFG reshapes its underlying potential. This view explains how stronger alignment can be accompanied by mean displacement and trajectory concentration, and motivates controlling guidance through the model-implied terminal posterior mean. We therefore propose Posterior-Mean-Capped CFG (PMC-CFG), a training-free, per-sample method that adaptively retains the strongest feasible guidance without additional network evaluations. Experiments on synthetic and large-scale image-generation benchmarks show that PMC-CFG limits guidance-induced distortion and concentration while improving the alignment--diversity trade-off, with particularly strong benefits when nominal guidance is large.

0.52

From Suboptimal to Optimal: One-Step Latent Flow Policies for Offline Reinforcement Learning

cs.LG cs.AI ·2026-10-12 ·Shenghua Lu, Zixuan Yang, Rongxu Zhao, Bohao Qu, Jifeng Hu +1 more
benchmark similarity 0.60
Abstract

Offline reinforcement learning aims to learn effective policies from fixed datasets without environment interaction. Although generative models such as diffusion and flow matching enhance policy expressiveness, they typically start from uninformative Gaussian noise and generate actions through multi-step iteration, leading to high computational costs and failing to effectively leverage suboptimal trajectories in datasets. We propose MeanFlow2Better (MF2B), which reformulates offline RL as generative decision chunk optimization from current performance status to better performance status in a high-level latent decision space. MF2B maps trajectory chunks to high-level latent decision representations through an encoder, placing generation starting points within a structured decision space sculpted by behavioral data. The model learns a shared conditional mean velocity field in this space, constructing two flow mappings: from Gaussian-Induced Decision Sources toward positive behavioral decision regions, and from suboptimal decisions toward high-value decision regions. Both flow mappings complete decision refinement through one-step ODE integration over the unit time interval $Δt=1$, requiring neither multi-step iterative sampling nor backpropagation through time. Furthermore, we propose hierarchical chunk-level advantage to guide the flow direction toward high-return regions, where the high-level decisions cover multiple low-level actions, leading to sequence length compression from $T$ to $K \approx T/H$ and significantly lower computational overhead. On the D4RL Gym-MuJoCo benchmark, MF2B achieves an average score of 90.91, matching or surpassing strong baselines.

0.15

Elite-Weighted Supervised Fine-tuning for Goal-Directed Molecular Optimization

cs.LG ·2026-08-31 ·Shiyun Wa, Yifei Wang, Anna G. Green, Simone Sciabola, Ye Wang
benchmark similarity 0.59
Abstract

Goal-directed optimization is essential for steering molecular generators to propose candidates with desired properties. However, it is often implemented with policy-gradient reinforcement learning, which requires a generation-trajectory log-probability whose form depends on the model architecture and generation procedure. This makes an optimizer difficult to reuse across architectures and conditional generative designs. Supervised fine-tuning needs none of that machinery, but its update is driven by a fixed dataset, so the reward never enters the update. We introduce Elite-Weighted Supervised Fine-tuning (EW-SFT), which uses reward to guide elite selection of high-scoring molecules, and updates the model by its own pretraining loss on that set. Ablations show that reward information is passed primarily through elite selection, rather than through continuous weighting within the selected set. Because the update consumes only scored molecules and the model's native loss, the same rule applies across autoregressive, masked-diffusion, and discrete-flow generators, and across de novo, motif-extension, and linker-design tasks. Under a fixed budget of 3D shape alignment oracle calls on two kinase reference compounds, EW-SFT consistently outperforms the corresponding native optimizers. It further improves goal-directed optimization under a 2D similarity oracle on four held-out references and achieves comparable performance on a sample-efficiency benchmark without a trajectory-level RL formulation. These results demonstrate that EW-SFT is a unified and effective optimizer across molecular generators, design constraints, references, and oracles.

0.21

Expanding the Quality-Coverage Frontier with Competitive Flow Matching

cs.CV ·2026-11-04 ·Wenxuan Peng, Richard Zhang, Haitian Zheng, Jaskirat Singh, Tsai-Shien Chen +4 more
code/data similarity 0.59
Abstract · 40 pages, 24 figures. Project page: https://lilydaytoy.github.io/CFM/

Flow matching (FM) generators often face a quality-coverage tradeoff. Inference-time techniques such as classifier-free guidance raise quality, but at the cost of coverage. We hypothesize this limitation stems from a basic property of the objective: an intermediate noisy image could correspond to many clean images, yet the model can only predict a single velocity, at best averaging over the multiple possibilities. To mitigate this, we introduce competition among predictions during training. We achieve this by augmenting the predictor with a randomly drawn low-dimensional conditioning code $z$. For each training example, we sample $K$ values of $z$ to produce $K$ candidate velocities that compete to explain the target, and apply the loss only to the best-matching candidate. This winner-take-all objective allows different modes to shape different predictions, rather than being averaged into a single compromise. Furthermore, this term naturally enables a new form of guidance, auxiliary conditioning scaling, which complements traditional guidance methods. Together, these two guidance mechanisms show that our method substantially improves the quality-coverage frontier. We validate on ImageNet-256, using two different testbeds, SiT and RAEv2. Given less than 3% of the original SiT pretraining compute for post-training, at the 67% Recall point, our method reduces $\mathrm{FD}_{r,6}$ from 19.22 to 5.91. On RAEv2, at the 70% Recall point, it reduces $\mathrm{FD}_{r,6}$ from 16.83 to 3.18.

0.25