CS Paper Radar

Rethinking Classifier-Free Guidance for TI2I Flow Matching

Andreas Floros, Elena Koritska, Teofilo de Campos, Christoph Vogel, Chu-Tak Li, Anuj Sharma, Alexandros Neophytou

cs.CV cs.LG ·submitted 2026-10-15 09:13 UTC · v1
position

Abstract

We study Text-and-Image-to-Image (TI2I) flow matching methods for controllable image editing. In a broader generative modeling context, Classifier-Free Guidance (CFG) is established as a key component responsible for high quality outputs. However, we argue that its role in TI2I modeling is poorly understood. Specifically, we make the surprising observation that the CFG mechanism is implicitly harmful for editing in the noisy phase of flow modeling, hindering instruction following and controllability. We propose a simple correction in the early flow steps, enabling a richer alignment-preservation tradeoff at no extra cost. The effectiveness of our approach is demonstrated on both pixel-space and latent TI2I flows.

Closest papers

Nearest neighbours by meaning across the whole index.

DecFlowEdit: Self-Localized Flow-based Image Editing via Guidance Decoupling

cs.CV ·2026-09-28 ·Zheyuan Zhan, Can Wang, Jiawei Chen, Chun Chen, Siwei Lyu +2 more
similarity 0.76
Abstract

Flow-based image editing (FlowEdit) enables inversion-free semantic changes through the difference between source and target velocities. In this paper, we observe that FlowEdit's default classifier-free guidance (CFG) configuration, with asymmetric source and target scales, causes substantial background leakage. Matching these guidance scales, for example by removing CFG, improves edit-relevant localization but severely degrades editability. To get the best of both worlds, we propose DecFlowEdit, which decouples the optimal guidance scales for localization and for editing in flow-based generative models. In particular, DecFlowEdit first extracts an edit-relevant prior by temporally aggregating velocity differences evaluated without CFG, and then uses this prior to reweight the original updates under default CFG. Our method remains training-free and inversion-free, requiring neither external spatial masks nor attention manipulation. Experiments on PIE-Bench across FLUX, SD3, and SD3.5 show that DecFlowEdit improves background preservation, reducing structure distance by approximately 61 to 73 percent and background LPIPS by 68 to 80 percent relative to FlowEdit at comparable editing fidelity.

0.22

Constrained Look-ahead Guidance for Interference-Aware Flow Editing

cs.LG cs.CV ·2026-11-07 ·Doudou Zhang, Qi Chen
similarity 0.73
Abstract

Diffusion and flow models enable powerful generative modeling, but controlled editing remains challenging due to the need to balance source fidelity and edit compliance along the generative trajectory. We propose a principled, inversion-free framework for flow-based image editing from a trajectory centric perspective. Rather than relying on inversion or heuristic guidance, we introduce a one-step look-ahead mechanism that evaluates the effect of candidate updates on both edit and fidelity rewards estimated via denoising, and adjusts the current update based on this evaluation. At each step, our method applies reward-driven corrections guided by the look-ahead evaluation under an explicit fidelity constraint, resulting in stable and coherent trajectory evolution. This look-ahead design mitigates gradient interference between edit and fidelity objectives by anticipating their interaction in future states, leading to improved trade-offs. Experiments demonstrate that the proposed approach achieves stronger semantic edits while better preserving source content compared to existing strong baselines.

0.05

Instance-Specific Classifier-Free Guidance for Text-to-Image Diffusion/Flow-Based Editing

cs.CV eess.IV ·2026-09-29 ·Wajeeh Atrash, Tom Tirer
benchmark similarity 0.71
Abstract

In recent years, a variety of prompt-based image editing methods have been developed based on text-to-image (T2I) diffusion/flow models. Image editing introduces an inherent trade-off between adhering to the original source image and realizing the change specified by the target prompt, which are typically measured by the LPIPS and CLIP-T metrics, respectively. Intuitively, this trade-off can be controlled by the classifier-free guidance (CFG) scale. Most existing editing methods, though, use a fixed CFG scale, ignoring that different edits require varying levels of modification. In this paper, we first explore the potential benefits of using an instance-specific CFG by exhaustively re-running prominent editing methods with a wide range of CFG scale values on benchmark datasets. We show that instance-specific setting can significantly improve LPIPS without compromising CLIP-T. Yet, tuning based on repeatedly executing an editing pipeline is inefficient. To prevent slow inference, we train a light-weight neural network that predicts the metrics of the underlying editing method for a given edit instance and CFG scale. At inference time, these predictions enable fast selection of an instance-specific CFG scale. Experiments across multiple editing methods and benchmark settings show the performance advantage of our approach over the baseline methods.

0.22

FlowTool: Controlling Tool Parameter in Image Retouching via Flow Matching

cs.CV ·2026-09-28 ·Thanh-Long V. Le, Steven Walton, Seunghyun Yoon, Branislav Kveton, Trung Bui +2 more
similarity 0.69
Abstract

Tool-based image editing (image retouching) is commonly formulated with autoregressive multimodal large language models (MLLMs) that sequentially generate reasoning, tool selections, and parameter values. In this work, we present a novel approach to tool-based image editing by framing the task as a flow matching problem. We introduce FlowTool, a framework that directly models the distribution of high-quality tool parameters conditioned on the input image and user instruction using conditional rectified flow. FlowTool combines a vision-language model backbone for multimodal understanding with a Diffusion Transformer parameter generator that transforms Gaussian noise into an editing plan. We train FlowTool with a two-stage supervised flow-matching curriculum, followed by reward-based post-training. Across MMArt-Bench, FlowTool-Eval, ArtEdit-Bench, and MIT-Adobe5K, FlowTool achieves significantly stronger reference-based performance than specialized MLLM editing agents and proprietary MLLMs, while remaining competitive with proprietary models under reference-free evaluation. Moreover, FlowTool significantly improves inference efficiency, reducing latency by at least $50\times$ while requiring nearly $2\times$ less memory than the compared baselines. These results demonstrate that tool-based image editing can be effectively modeled as conditional generation over structured continuous editing parameters, without autoregressive reasoning.

0.24

VLM as an Editing Pilot: VLM-Steered Velocity Control for Flow-Based Image Editing

cs.CV ·2026-11-06 ·Kuan-Shen Chiu, Jia-Wei Liao, Yu-Chieh Chao, Li-Xuan Peng, Wei-Yun Ma +1 more
top venuecode/data similarity 0.69
Abstract

Text-guided image editing requires realizing the requested semantic change while preserving content unrelated to the instruction. FlowEdit provides an efficient inversion-free solution, but its source-anchored coupling can become overly restrictive, causing zero-edit, where the output remains close to the source and the requested change fails to emerge. We find that source coupling directly governs the preservation--editing trade-off, while a fixed coupling schedule cannot adapt to timestep-specific editing needs. Motivated by this observation, we propose V2C-Edit, a training-free VLM-steered velocity control framework. V2C-Edit constructs complementary coupling-controlled velocities and dynamically selects among them using Instruction-Level Similarity (ILS), which evaluates whether the predicted transformation follows the editing instruction. Adaptive spatial masking further preserves regions unrelated to the edit. Experiments on MagicBrush and PIE-Bench with FLUX and Stable Diffusion 3 show that V2C-Edit consistently improves instruction alignment over FlowEdit while maintaining competitive structural preservation, and that ILS better reflects editing effectiveness than target-caption CLIP similarity. Our code is available at https://github.com/Kuanshen77/V2C-Edit.

0.43

Classifier-Free Guidance in Flow Matching: Non-Autonomous Potentials, Overshoot, and Posterior-Mean Control

cs.CV ·2026-09-21 ·Jishen Peng, Zheng Ma
similarity 0.68
Abstract

Classifier-free guidance (CFG) improves conditional generation in Flow Matching, but strong guidance can distort the generated distribution and reduce diversity. We provide a geometric account of this behavior by viewing Flow Matching as a time-varying gradient flow and characterizing how CFG reshapes its underlying potential. This view explains how stronger alignment can be accompanied by mean displacement and trajectory concentration, and motivates controlling guidance through the model-implied terminal posterior mean. We therefore propose Posterior-Mean-Capped CFG (PMC-CFG), a training-free, per-sample method that adaptively retains the strongest feasible guidance without additional network evaluations. Experiments on synthetic and large-scale image-generation benchmarks show that PMC-CFG limits guidance-induced distortion and concentration while improving the alignment--diversity trade-off, with particularly strong benefits when nominal guidance is large.

0.52

FlowTranslator: Training-Free In-Image Machine Translation with Flow-Based Editing

cs.CV cs.AI ·2026-11-03 ·Zhixian Li, Xingsong Ye, Yongkun Du, Yunyan Xiao, Yanglin Huang +4 more
position similarity 0.68
Abstract

In-image machine translation (IIMT) replaces text in an image with its translation while preserving the surrounding visual context. Existing task-specific models achieve convincing performance but require massive paired training images before and after translation. As advanced text-to-image (T2I) models have reached a practically usable level of multilingual text generation, we argue that this capability can be transferred to IIMT and present FlowTranslator. It is a training-free IIMT framework that first employs an OCR model and an LLM to generate the translation prompt, which is injected into the T2I-based diffusion model. To optimize the accurate and natural generation of translated text, we introduce two trajectory controls: Region Coupling Release (RCR) partially decorrelates target-branch noise inside the text region early in the trajectory, while Coupling Expansion (CE) amplifies the emerging edit displacement mid-trajectory. Under the EasyOCR read-back protocol, FlowTranslator achieves average PRIM BLEU and COMET scores of 20.6 and 57.2, respectively, substantially outperforming the strong training-based baseline UniTranslator. On the more challenging and realistic whole-image ScenePair photographs translated into Chinese, it reaches 40.1\% exact-match accuracy and surpasses existing editing systems, some of which only address the text-rendering stage rather than the complete IIMT task. These results establish flow-based editing as a strong, fully training-free baseline for IIMT.

0.30

RAGE: Residual-Adaptive Gating for Flow-Based Image Editing

cs.CV cs.LG ·2026-11-16 ·Ruofeng Mei, Xiangyu Xu
similarity 0.68
Abstract

Recent inversion-free, flow-based image editing methods evolve the source image toward the target via a differential velocity field. We observe that this velocity field does not explicitly account for which regions should be preserved, leading to unintended drift in non-editing regions. To address this, we formulate source fidelity control as a stochastic optimal control problem in which a selective fidelity term on the editing residual is enforced along the editing trajectory. Solving this problem and approximating its solution yields the Residual Gating Principle: each latent component is pulled back toward the source with a gain that decays exponentially with the local residual magnitude, so that small deviations are suppressed while large, semantically meaningful edits pass through unimpeded. The proposed method, RAGE, requires only one lightweight gated interpolation per sampling step, involves no mask estimation, attention maps, retraining, or architecture-specific modifications, and is applied on top of FlowEdit-based editors. On PIE-Bench, RAGE achieves the best overall rank among 17 methods and substantially improves background preservation on both Stable Diffusion 3 and FLUX backbones while maintaining competitive editing quality.

0.14