CS Paper Radar

Dino Forcing Flow Models: Do not denoise what you can predict

Arijit Ghosh, Lucas Degeorge, Paul Couairon, Alexei A Efros, Vicky Kalogeiton, David Picard

cs.CV ·submitted 2026-10-08 11:41 UTC · v1
code/datahot direction

Abstract

Co-denoising pretrained representations such as DINO can substantially improve the training speed and quality of flow matching models, but it introduces a second denoising trajectory and requires carefully designed schedules. We propose a simpler alternative: predict the pretrained representation directly, then condition the model on its own prediction. This removes the need for a second ODE and any representation-specific denoising schedules, while retaining the benefits of representation guidance. Our approach converges substantially faster and achieves better generation quality as measured by FID score. On ImageNet, it outperforms the state of the art in latent space at 2x fewer epochs than prior methods; in pixel space, it improves FID over comparable prior methods by more than 20%. These results support a simple principle: do not denoise what you can predict. Our code is openly available at https://github.com/arijit-hub/dino_forcing.

Closest papers

Nearest neighbours by meaning across the whole index.

PixelDiT2: Representation-Grounded Pixel Diffusion Transformers

cs.CV ·2026-09-21 ·Yongsheng Yu, Wei Xiong, Yichen Sheng, Shiqiu Liu, Jiebo Luo
top venuepositioncode/data similarity 0.65
Abstract · Accepted to NeurIPS 2026 Code: https://github.com/NVlabs/PixelDiT

Recent advances in pixel-space diffusion models have narrowed the image quality gap with latent-space diffusion, but still converge more slowly and lag behind in final image quality. We argue that a key reason is the lack of an explicit representation prior: unlike latent diffusion, which usually denoises in a compact and structured latent space, pixel diffusion needs to learn denoising-friendly representations and pixel generation simultaneously from raw RGB space. To address this problem, we propose PixelDiT2, an end-to-end pixel-space diffusion model designed to decouple representation learning from pixel generation without introducing an autoencoder or latent reconstruction bottleneck. We propose representation grounding that uses a frozen pretrained vision foundation model to provide explicit per-patch representation guidance throughout denoising, allowing the pixel diffusion transformer to focus more on pixel generation. On ImageNet-256x256, PixelDiT2 achieves an FID of 1.46 after 600 epochs; at 512x512 resolution, PixelDiT2 achieves an FID of 1.48 after 680 epochs. Project page: https://pixeldit.github.io/pixeldit2/

0.70

On the Diffusibility of High-Dimensional Latents

cs.CV cs.LG ·2026-09-23 ·Chao Feng, Zhiyang Xu, Bowei Chen, Yuanjun Xiong, Xiyao Wang +5 more
top venuecode/data similarity 0.63
Abstract · Accepted to ECCV 2026. Project page: https://cfeng16.github.io/on_the_diffusibility/

Representation Autoencoders (RAEs) enable diffusion models to operate in the feature spaces of pretrained visual encoders. However, many off-the-shelf encoders are not optimized for faithful reconstruction, discarding fine-grained visual details. As expected, finetuning these encoders for image reconstruction recovers such details. However, perhaps counterintuitively, this procedure reduces the effective dimensionality of the resulting representation, and the altered geometry has downstream effects on generation. Specifically, we show that using the standard velocity prediction in flow matching in this high-dimensional space requires the model to fit orthogonal noise directions outside the low-dimensional signal manifold, making optimization inefficient. This motivates using the clean data parameterization ($\boldsymbol{x}_{0}$-prediction) instead, which focuses learning on the underlying signal manifold. Across experiments with multiple strong-reconstruction encoders, we show that $\boldsymbol{x}_{0}$-prediction consistently improves text-to-image generation performance.

0.58

Magnitude-Direction Decoupling for Fast Video Generation with Flow Matching Models

cs.CV ·2026-08-18 ·Haonan Xu, Feiyang Chen, Songkui Chen, Hongpeng Pan, Zhefeng Wang +3 more
similarity 0.63
Abstract

Flow matching models for video generation achieve impressive performance but suffer from high computational overhead due to iterative denoising. In fact, the original model is not necessary for all denoising steps, allowing some steps to use lightweight alternatives for faster sampling. However, directly using caching or lightweight models can deviate from the original denoising trajectory, resulting in suboptimal performance. Through empirical analysis, we find that lightweight models can robustly capture the magnitude components of the original model's output, while caching provides reliable directional guidance. Building on this insight, we propose the Magnitude-Direction Decoupling (MDD) method, which adaptively employs a direction-calibrated lightweight model as a substitute for the original model to accelerate inference and effectively correct deviations in the denoising trajectory. Moreover, MDD further reduces inference costs by reusing magnitude information under classifier-free guidance (CFG). As a result, MDD offers a more reliable and lightweight solution to accelerate sampling. Experiments show that MDD outperforms existing acceleration methods, delivering promising speedups (e.g., up to 2.95x on Wan2.1) while preserving high visual fidelity and content richness.

0.16

One Step Is Optimal: Unconditional Rectified Flows as Noise2Noise Denoisers for Low-Dose CT

cs.CV ·2026-09-15 ·Timothy Sereda, Debesh Jha
benchmark similarity 0.63
Abstract · 15 pages, 3 figures

We study rectified flows for denoising without clean training targets. For an unconditional rectified flow trained on Noise2Noise pairs, the optimal velocity field yields the minimum mean squared error (MMSE) denoiser in one step. In a jointly Gaussian model, we show that integrating the flow before applying its denoising readout increases error with integration time. The flow transports between noisy distributions, so its endpoint is not a clean-image estimate. Experiments on low-dose CT and Gaussian-corrupted natural images follow this pattern: one-step flow performs similarly to direct regression on the same pairs, whereas additional Euler steps reduce PSNR. A direct regressor using spatially separated, similar patches gives our best label-free CT result, a $1.99$\,dB gain over the noisy input. We report these results in CTDenoiser, a benchmark covering five architectures, supervised and self-supervised training, and two per-image methods. Supervised CNNs gain about $4$\,dB, while label-free performance depends strongly on how training pairs account for correlated CT noise. A channelized Hotelling observer also shows that fidelity gains need not improve lesion detectability: detectability falls at $160$\,HU despite higher PSNR and SSIM, while changes at $40$--$80$\,HU are small. Code, model definitions, and data-loading utilities will be released publicly; the repository link is withheld for anonymous review.

0.31

Expanding the Quality-Coverage Frontier with Competitive Flow Matching

cs.CV ·2026-11-04 ·Wenxuan Peng, Richard Zhang, Haitian Zheng, Jaskirat Singh, Tsai-Shien Chen +4 more
code/data similarity 0.63
Abstract · 40 pages, 24 figures. Project page: https://lilydaytoy.github.io/CFM/

Flow matching (FM) generators often face a quality-coverage tradeoff. Inference-time techniques such as classifier-free guidance raise quality, but at the cost of coverage. We hypothesize this limitation stems from a basic property of the objective: an intermediate noisy image could correspond to many clean images, yet the model can only predict a single velocity, at best averaging over the multiple possibilities. To mitigate this, we introduce competition among predictions during training. We achieve this by augmenting the predictor with a randomly drawn low-dimensional conditioning code $z$. For each training example, we sample $K$ values of $z$ to produce $K$ candidate velocities that compete to explain the target, and apply the loss only to the best-matching candidate. This winner-take-all objective allows different modes to shape different predictions, rather than being averaged into a single compromise. Furthermore, this term naturally enables a new form of guidance, auxiliary conditioning scaling, which complements traditional guidance methods. Together, these two guidance mechanisms show that our method substantially improves the quality-coverage frontier. We validate on ImageNet-256, using two different testbeds, SiT and RAEv2. Given less than 3% of the original SiT pretraining compute for post-training, at the 67% Recall point, our method reduces $\mathrm{FD}_{r,6}$ from 19.22 to 5.91. On RAEv2, at the 70% Recall point, it reduces $\mathrm{FD}_{r,6}$ from 16.83 to 3.18.

0.25

Just Image Transformers with Late-Stage Pixel Refinement

cs.CV cs.AI ·2026-11-09 ·Denis Lukovnikov, Andreas Müller, Asja Fischer
similarity 0.63
Abstract

Recent work on latent-free diffusion transformers has demonstrated the feasibility of training image generators without a latent VAE. This avoids compression artifacts from the VAE and simplifies the training pipeline. In particular, JiT showed that even a standard transformer can be trained to generate images when the model is trained to predict the final image ($x$-prediction). Even though JiT unlocks training efficiency comparable to latent models, it still suffers from reduced image quality likely arising from modeling the image as a collection of large patches. In this work, we revisit JiT and show that adding simple convolutional pixel-pathway layers in a post-training phase significantly improves generation quality. Moreover, we show that the added pixel-pathway does not need to be active in the early denoising stages, and can be activated as late as in the last $20\%$ of the denoising process. Our experiments on ImageNet@256 show improvements compared to the original JiT results, achieving an FID of $1.78$ (from $2.34$ with corrected sampling), without using additional losses.

0.08

PEEK: One-Step-Look-Ahead Exposure Bias Correction for Diffusion Sampling

cs.CV cs.AI ·2026-10-20 ·Yunwei Bai, Ying Kiat Tan, Tsuhan Chen
published similarity 0.62
Abstract · Accepted to Neurips 2026

Diffusion image generators owe much of their practical success to accurate denoising networks and strong numerical samplers. However, during sampling, the denoiser predicts the next step from flawed inputs that differ substantially from real data. This train-test gap, or exposure bias, caps image generation quality (i.e., blurry images, missing details, or distorted shapes). To mitigate the problem, we propose PEEK, a training-free deterministic sampler plugin. At each eligible denoising step, PEEK first performs a provisional one-step update, evaluates the denoiser once at the look-ahead state, and interpolates the current clean prediction with this one-step-later prediction to form the next noisy state. By taking the additional step, we obtain a generally more image like prediction from the higher-signal noisy state, mitigating exposure bias. To optimize the interpolation positions, PEEK searches for an interpolation schedule with a greedy metric-driven search made efficient by the PEEK prefix and baseline noise-schedule-rebasing caches. Across pixel-space and latent-space diffusion models with unconditional, label-guided, and text-guided settings on CIFAR-10, LSUNBedroom,I mageNet, and MS-COCO,the look-ahead correction consistently improves over baselines matched by number of function evaluations (NFE), with Fréchet Inception Distance (FID) reductions up to 49.3%. The one-time per-model calibration, the only extra cost, takes at most 2.2 minutes in our experiments. Project website is available at https://dachengruoque.github.io/PEEK-Video/.

0.33

Beyond the Estimate: Wiener-Informed Decoding for Easier Training of Pixel-Space Flow Models

cs.CV cs.AI ·2026-10-26 ·Ziqiao Wang, Yang You
similarity 0.62
Abstract · 23 pages

Learning velocity fields directly in pixel space is difficult and slow. We show that coarse second-order image statistics can ease this task: frozen Wiener estimates improve velocity learning, with gains extending beyond the linear predictions they explicitly provide. Moreover, we observe similar but complementary gains from frozen bases and pixel decoding. These observations motivate Wiener-informed Decoding (WiD), which uses statistical knowledge both for direct prediction and to organize decoder computation without modifying native blocks. WiD combines a trainable Wiener prediction path with coefficient interfaces that inject features into the native decoder and return its predicted coefficient corrections, with these components initialized with data statistics. All introduced parameters, including the spectrum and coordinate transforms, are trained jointly under the original pixel-space velocity objective. Across model scales and decoder architectures, WiD accelerates training and improves generation quality with modest parameter overhead. It also yields smoother gradient trajectories and remains effective with representation alignment. WiD improves class-conditional and text-to-image generation across different resolutions. On ImageNet 256 x 256, PixelDiT-XL with WiD reaches FID 1.53 after 500 epochs, achieving comparable quality to PixelDiT with 37.5% fewer training epochs. Statistical denoisers contribute beyond the estimate: their computational structure can also ease pixel-space training.

0.25