CS Paper Radar

Model the Edit, Not the Image: Visual Autoregressive Editing from a Source-Centric Perspective

Hongyi Fang, Chuwen Xie, Benjia Zhou, Yu-Xuan Qiu, Chenggong Hu, Zhibin Wang, Chao Chen, Jianbin Qin, Rui Mao

cs.CV ·submitted 2026-08-10 03:05 UTC · v2

Abstract

Next-scale visual autoregressive models (VARs) have emerged as a powerful generative paradigm, producing high-quality images through efficient coarse-to-fine prediction. However, their potential for text-guided image editing remains largely underexplored. Existing training-free VAR editing approaches often formulate editing as target-conditioned regeneration guided or constrained by the source image, and may rely on inversion, test-time optimization, attention control, or user-provided masks. This generation-centric formulation does not fully exploit the multiscale source representations provided by VARs and may introduce additional computation or intervention. We instead take a source-centric perspective on VAR editing, in which the encoded source image tokens serve as the primary visual state and the editing process focuses on condition-induced changes. Based on this perspective, we propose \textbf{EditMod}, which compares source- and target-conditioned predictions under a shared autoregressive context, treats their difference as a scale-wise editing direction, and applies it as a residual update to source tokens at selected scales. Experiments show that EditMod achieves leading source-image fidelity while maintaining strong text alignment, and completes end-to-end editing of a 1K image in only 1.57 seconds on a single A100 GPU without per-image preparation.

Closest papers

Nearest neighbours by meaning across the whole index.

Edit-VAR: Taming Visual Autoregressive Model for Precise Video Editing

cs.CV ·2026-09-18 ·Chongbo Zhao, Jiangming Wang, Xilai Wang, Xinyu Wang, Jingyi Tang +3 more
code/data similarity 0.80
Abstract · Project page: https://chongbozhao3-coder.github.io/Edit-VAR. Code: https://github.com/chongbozhao3-coder/Edit-VAR

Text-guided video editing modifies target content while preserving the appearance and temporal coherence of unedited regions. Training-based approaches provide strong control but demand substantial data and computation. Training-free methods fall into inversion-free and inversion-based paradigms. Inversion-free approaches avoid trajectory recovery, but their source-preserving guidance can limit editing strength and leave semantic changes incomplete. Inversion-based approaches recover a latent trajectory before regeneration, where approximation errors can accumulate and cause source-content drift and temporal inconsistency. We introduce Edit-VAR, the first training-free and inversion-free framework for text-guided video editing with a pretrained visual autoregressive video model. Edit-VAR directly encodes the source video into multi-scale discrete tokens and performs probability-guided conditional token replacement for source preservation. Attention-guided token-wise and scale-aware modulation selectively relaxes source constraints over edit-relevant positions and generation stages. Scale-Decoupled Generation, implemented as late-scale constraint release, regenerates motion-consistent details and reduces texture fragmentation. Residual-guided token pruning further exploits redundancy at the final two high-resolution scales to reduce inference cost. Extensive experiments and a blind user study demonstrate that Edit-VAR outperforms existing training-free video editing methods overall in editing fidelity, source preservation, temporal coherence, and inference efficiency.

0.19

Instance-Specific Classifier-Free Guidance for Text-to-Image Diffusion/Flow-Based Editing

cs.CV eess.IV ·2026-09-29 ·Wajeeh Atrash, Tom Tirer
benchmark similarity 0.75
Abstract

In recent years, a variety of prompt-based image editing methods have been developed based on text-to-image (T2I) diffusion/flow models. Image editing introduces an inherent trade-off between adhering to the original source image and realizing the change specified by the target prompt, which are typically measured by the LPIPS and CLIP-T metrics, respectively. Intuitively, this trade-off can be controlled by the classifier-free guidance (CFG) scale. Most existing editing methods, though, use a fixed CFG scale, ignoring that different edits require varying levels of modification. In this paper, we first explore the potential benefits of using an instance-specific CFG by exhaustively re-running prominent editing methods with a wide range of CFG scale values on benchmark datasets. We show that instance-specific setting can significantly improve LPIPS without compromising CLIP-T. Yet, tuning based on repeatedly executing an editing pipeline is inefficient. To prevent slow inference, we train a light-weight neural network that predicts the metrics of the underlying editing method for a given edit instance and CFG scale. At inference time, these predictions enable fast selection of an instance-specific CFG scale. Experiments across multiple editing methods and benchmark settings show the performance advantage of our approach over the baseline methods.

0.22

KINESYS: Test-Time Adaptation for Reference-Based Effect Editing in Images

cs.CV ·2026-10-23 ·Noah Frahm, Luchao Qi, Haoze Li, Dan B Goldman, Sambit Bhattacharya +1 more
code/data similarity 0.74
Abstract · Project page: https://noahfrahm.github.io/KINESYS-Project-Page

Visual effects are often easier to demonstrate via reference than to describe in words. We study effect-reference editing, where a reference image shows a desired visual effect and language identifies which properties should guide the edit. Existing single-pass editing techniques often fail to make fine-grained edits, resulting in unintended changes to source content or deviations from the reference effect. We introduce KINESYS, a test-time adaptation framework for effect-reference editing that combines reference-aware and source-aware feedback to transfer the desired effect while preserving the composition and content of the original shot. To ensure KINESYS can produce a diverse set of edits that preserve the source, we develop a robust optimization strategy in which the generative model is fine-tuned across multiple noise samples and optimization runs. Evaluation with VLM judges, source preservation metrics (LPIPS, TPIPS), and a user study shows that KINESYS outperforms state-of-the-art image editors in source preservation; on effect transfer, it remains competitive with Nano Banana 2 and outperforms open-weight editors.

0.28

RAGE: Residual-Adaptive Gating for Flow-Based Image Editing

cs.CV cs.LG ·2026-11-16 ·Ruofeng Mei, Xiangyu Xu
similarity 0.74
Abstract

Recent inversion-free, flow-based image editing methods evolve the source image toward the target via a differential velocity field. We observe that this velocity field does not explicitly account for which regions should be preserved, leading to unintended drift in non-editing regions. To address this, we formulate source fidelity control as a stochastic optimal control problem in which a selective fidelity term on the editing residual is enforced along the editing trajectory. Solving this problem and approximating its solution yields the Residual Gating Principle: each latent component is pulled back toward the source with a gain that decays exponentially with the local residual magnitude, so that small deviations are suppressed while large, semantically meaningful edits pass through unimpeded. The proposed method, RAGE, requires only one lightweight gated interpolation per sampling step, involves no mask estimation, attention maps, retraining, or architecture-specific modifications, and is applied on top of FlowEdit-based editors. On PIE-Bench, RAGE achieves the best overall rank among 17 methods and substantially improves background preservation on both Stable Diffusion 3 and FLUX backbones while maintaining competitive editing quality.

0.14

FASTER: Fast Adjoint Stochastic Transport for Endpoint Refinement in Reward-Guided Image Editing

cs.CV ·2026-10-03 ·Yimiao Zhou, Zejia Zhong, Jingya Wang, Ye Shi
similarity 0.74
Abstract

Reward-guided image editing at test time seeks to improve a specified reward while preserving source content and visual plausibility. Many existing approaches optimize candidates through pretrained generation processes, making repeated adjustment depend on costly large-model execution and, in some cases, backbone backpropagation. We develop a theoretical framework that jointly accounts for reward, source preservation, and pretrained-prior preferences, allowing the desired output distribution to be specified separately from the dynamics used to realize it. Based on this framework, we introduce FASTER, which trains a small network for each source and objective to perform inexpensive editing, while pretrained and reward models provide feedback on candidate outputs. By reusing each candidate and its feedback across multiple small-network updates, FASTER reduces repeated sampling and supervision queries without placing the pretrained generative backbone inside the inner optimization loop. On SD3, FASTER leads all four target metrics and several validation metrics among the evaluated methods. Compared with the evaluated baseline that optimizes controls along pretrained generation trajectories, FASTER achieves editing-time speedups of up to \({6.91\times}\) on Stable Diffusion 3 and \({24.14\times}\) on Stable Diffusion 1.5.

0.24

QK-Edit: Revisiting Attention-based Injection in MM-DiT for Image and Video Editing

cs.CV cs.AI ·2026-10-13 ·Tiancheng Shen, Zilong Huang, Xiangtai Li, Zhijie Lin, Jiyang Liu +4 more
top venuehighly cited similarity 0.73
Abstract · Accepted at ICCV 2025

Multimodal Diffusion Transformers (MM-DiTs) have recently emerged as a powerful framework for unified text-vision synthesis, surpassing traditional U-Net architectures in generative tasks. One key innovation lies in its Multimodal Self-Attention (MM-SA) interaction, where image and text tokens are concatenated and processed via self-attention. However, this mechanism poses significant challenges for editing, rendering conventional U-Net-based attention manipulation methods ineffective. To address this limitation, we propose QK-Edit, a training-free framework that exploits the unique attention dynamics of MM-DiTs for precise text-guided image and video editing. By introducing a novel query-key manipulation strategy, our method isolates and adjusts critical attention components to achieve an optimal balance between prompt fidelity and structural consistency. This enables seamless edits across various tasks, including object addition, object removal, object replacement, changing background, changing material, changing color, and style transformation. Notably, it can be easily implemented with feature replacement in inference. QK-Edit demonstrates superior editing performance on state-of-the-art models, such as FLUX and HunyuanVideo, effectively bridging the gap between generative power and editable flexibility in MM-DiTs, and paving the way for scalable multimodal content creation.

0.56

Refinement Is Inherently Editable: Training-Free Prompt-to-Prompt Image Editing with Generative Refinement Network

cs.CV ·2026-09-17 ·Yulong Chen, Ziqian Zhang, Haoyu Zhang, Ao He, Yaxing Wang +2 more
code/data similarity 0.73
Abstract

Text-guided image editing must introduce the requested changes while preserving unrelated source content. In training-free editing, diffusion editors often use spatial controls whose inaccuracies can leave edits incomplete or alter unrelated regions. Causal autoregressive editors face a further constraint: their fixed decoding order limits revision of earlier decisions. As the first to explore training-free image editing with Generative Refinement Networks (GRN), we observe that its refinement process is inherently suitable for editing and offers a promising way to address these limitations. Motivated by this observation, we introduce RefineEdit, a training-free prompt-to-prompt image editing framework built on the GRN. Our key idea is to couple edit localization with content generation through the global refinement of binary image codes, allowing editing evidence to be revised as the image evolves. More specifically, RefineEdit combines bit routing with two stabilization mechanisms: adaptive spatial freezing and finite bit locking. Bit routing starts from an intermediate source state and uses signed probability differences between the two branches to identify editable positions and bits. It directs selected bits toward editing refinement while anchoring the rest to the evolving source trajectory. Adaptive spatial freezing limits unnecessary expansion of the editing region, while finite bit locking maintains recent bit activations to support continued editing. The overall framework requires no additional training, external masks, or attention control. Across nine editing categories of PIE-Bench, RefineEdit achieves the best background-preservation scores in PSNR, LPIPS, MSE, and SSIM, together with the highest whole-image and edited-region CLIP scores among the evaluated methods. Code is available at https://github.com/mura1n/RefineEdit.

0.17

Diagnosing and Mitigating Evidence Misgrounding in Multimodal Knowledge Editing

cs.CV cs.AI ·2026-10-21 ·Zhiwu Wang, Yichen Wu, Jinqian Yang, Yizhou Yang, Renzhen Wang +2 more
top venuecode/data similarity 0.73
Abstract · Accepted to Findings of EMNLP 2026

Multimodal knowledge editing aims to update visually grounded facts in Multimodal Large Language Models (MLLMs) while preserving unrelated text and visual behaviors. However, standard answer-level metrics can hide evidence misgrounding: an edit may produce the target answer on the original image yet remain sensitive to incidental context rather than the intended visual evidence. To examine this issue in a controlled setting, we construct WaterBird-Edit, an analysis set that decouples bird habitat labels from background context and measures whether edited predictions remain stable when the target evidence is preserved but the background changes. Motivated by this diagnosis, we propose VICE (Visual In-Context Editing), a retrieval-based framework that stores edits in external memory and applies them through visual in-context demonstrations designed to expose task-relevant evidence. Experiments on three standard benchmarks show that VICE achieves strong lifelong editing performance, while results on WaterBird-Edit show that it substantially reduces sensitivity to the controlled background shifts. Code and dataset are available in https://github.com/Wang-Z-W/VICE-main.

0.44