CS Paper Radar

Background Gradients Shape Memorization in Flow Matching

Xuanhua Yin, Boyu Wei, Shuyi Zhang, Shunqi Mao, Chuanzhi Xu, Weidong Cai

cs.CV ·submitted 2026-09-27 04:50 UTC · v1
hot direction

Abstract

Repetition is closely associated with memorization in generative models, but how other training images affect the retention and copying of targets remains unclear. We study this question in class-conditioned flow matching, where images outside the target set form the background. At fixed target repetition and same-class background row count, replacing repeated same-class images with distinct images reduces the target extraction rate from 80.7% to 18.0%. To explain this effect, we develop a paired-trajectory framework that isolates target-induced parameter displacement and the background gradient response to it. This response has an exact path-integrated curvature representation, connecting background loss geometry to target learning. Reciprocal response transfer between repeated and distinct backgrounds changes target retention and copying in both directions, establishing the response's causal role. After target removal, the response correction parallel to the target-induced displacement preserves approximately 90% of the copying effects of full response transfer. Directly scaling the displacement also changes copying without further training. The post-removal copying effects of reciprocal transfer are reproduced across datasets and architectures. Together, these results identify the background gradient response as a mechanism through which same-class training data shape the retention of target learning and the reproduction of target images.

Comment
38 pages, 8 figures, 34 tables

Closest papers

Nearest neighbours by meaning across the whole index.

Localizing Transfer Between Memorization Tasks

cs.LG ·2026-09-30 ·Yimiao Yu, Florentin Guth
published similarity 0.64
Abstract · 17 pages, 10 figures. Accepted as a poster at the NeurIPS 2026 Workshop on Neural Network Artifacts as a New Data Modality

A central puzzle in transfer learning is why pre-training on one task can accelerate training or improve performance on another task, and what mechanisms underlie this transfer. In this work, we examine the transfer between memorization tasks of random input-output mappings. We find two surprising transfer patterns: equivalent transfer, where each additional pre-training epoch saves approximately one downstream fine-tuning epoch; and non-equivalent transfer, where pre-training on a mismatched task can be even more efficient than directly training on the downstream task itself. Through ablation experiments, we decompose and localize the transfer into two separate effects: a "trivial" magnitude-driven transfer in the last layer, and a "non-trivial" structure-driven transfer, partially attributable to the covariance of the other layers. These results advance our understanding of the underlying mechanisms of transfer learning and have the potential to lead to principled pre-training strategies.

0.44

When Anchors Shift the Target: Advantage-Gated Adaptation for Diffusion Reinforcement Learning

cs.CV cs.LG ·2026-11-17 ·Pu Cao, Min Zhou, Ying Shu, Tiezheng Ge, Qing Song +1 more
similarity 0.63
Abstract · 12 pages, 4 figures, including supplementary material

Forward-process reinforcement learning adapts diffusion and flow-matching models by re-noising rewarded generations into regression targets. Because negative advantages induce concave updates with unbounded repulsive gradients, existing methods stabilize training using cached predictions from a moving policy copy, yet these anchors also alter what positive feedback teaches the model to fit. We formalize this conflict as anchor coupling: after separating frozen-reference regularizers, affine moving-anchor rules cannot eliminate negative curvature while preserving every positive regression target. We propose Advantage-Gated Adaptation (AGA) to decouple these roles. Positive samples retain standard target fitting, whose self-limiting update vanishes at the observed target independently of the moving anchor. Negative samples instead use the first-order tangent of the concave objective at the moving-policy prediction, preserving the local repulsion direction with a prediction-independent gradient. Within the quadratic-anchor family, these requirements uniquely determine the parameter-free gate $ω(a)=\max(-a,0)$. Because the moving copy provides local linearization rather than retention, AGA assigns long-horizon behavior preservation to a frozen pretrained reference regularized at its operating classifier-free guidance scale, interpreted through an implicit-reward lens. Across compositional generation, text rendering, and preference alignment, our experiments show that displacing positive targets degrades visual quality and heightens sensitivity to policy-copy schedules, whereas AGA consistently improves the reward--quality trade-off.

0.28

Learning a Flow to Self-Supervised Representations

cs.CV cs.LGstat.ML ·2026-09-24 ·Yuling Jiao, Wensen Ma, Houduo Qi, Defeng Sun
similarity 0.63
Abstract · 38 pages, including references and appendices

We introduce Flow-Based Distribution Matching (FBDM), which learns self-supervised representations through non-adversarial velocity regression. The encoder and auxiliary field are trained jointly by regressing analytic velocities along spherical paths from learned representations to reference targets. Both augmented views share a reference target, with a capacity limit on each reference center and an explicit view-alignment loss. A structured reference supports more components than the auxiliary flow dimension while retaining geometric separation. The auxiliary field is discarded after pretraining; only the backbone is retained downstream. On three ResNet-18 benchmarks with matched auxiliary parameter counts, FBDM provides 2.17 to 2.36 times higher per-epoch training throughput than critic-based Distribution Matching (DM), with about 0.21 GiB additional peak allocated GPU memory. Experiments show that FBDM is competitive with existing SSL methods from CIFAR to ImageNet and performs comparably to DM on their shared benchmarks. Under stated conditions, our population analysis bounds nearest-centroid classification error on the initial normalized pretraining representation by the FBDM loss.

0.36

Bridging the Data Gap: Source-Informed Task-Space Transport for Few-Shot Diffusion

cs.LG cs.AI ·2026-10-22 ·Jinjian Jiang, Jing Peng, Rehan Deen, Yu Cao, Ziquan Liu +2 more
similarity 0.63
Abstract

When an object category is well represented in one domain but scarcely observed in another, adapting a diffusion model to the target can be unreliable. Yet the same source-target relationship may have been observed for other categories. We ask whether this recurring relation can be learned and reused for a new semantic task. We introduce Meta-Diff, which embeds source and target distributions in a shared low-dimensional task space, learns an affine transport map across paired training tasks, and generates from the transported source coordinate using a shared diffusion backbone. At deployment, it requires source examples and the identity of a previously learned relation, but neither target images nor target-specific optimisation. Gaussian-mixture experiments examine how transfer depends on task information retained by the encoder, relation predictability and the size of the source--target shift. On held-out CIFAR-100 validation tasks, correct-source transport achieves 13.9% lower mean sliced Wasserstein distance than wrong-source transport. In the one-target final-test comparison, it achieves 14.7% lower mean distance than target-only adaptation and 32.7% lower than full fine-tuning under validation-selected settings. These gains have clear limits: directly applying the known corruption to clean-source generations performs better on CIFAR, and requested-class identity remains weak. Office-Home shows domain alignment without reliable object identity; clinical stress tests show no resolved source-specific advantage. The results characterise when reusable task-space relations help and where their generative limitations remain.

0.19

DR.WILSS: Diffusion-Based Replay for Weakly Supervised Continual Semantic Segmentation

cs.CV ·2026-09-16 ·Leon Arthur Marx, Francesco Barbato, Matteo Caligiuri, Pietro Zanuttigh
published similarity 0.62
Abstract · Accepted at MMSP 2026, 6 pages, 4 figures

Weakly supervised class-incremental semantic segmentation (WILSS) aims to train a segmentation model over multiple steps, each introducing new concepts to be learned with only image-level supervision. We introduce DR$.$WILSS, an innovative approach to address catastrophic forgetting in continual learning using diffusion-based generative replay. Our framework leverages language clues to guide the diffusion process, employing self-inpainting and regularization techniques to efficiently produce replay data, aiding the learning process. By generating high-quality replay data, the information from previously learned classes can be preserved during continual updates, a critical challenge in incremental learning scenarios. To further align the statistics of replay data with those of training samples, we apply LoRAs to the generative model. Experimental results demonstrate state-of-the-art performance across multiple benchmarks and generative architectures, while avoiding storage of training data and the use of additional resource-demanding tools during training. The proposed technique enables an optimal tradeoff between training complexity and inference-time accuracy, making DR$.$WILSS a promising solution for real-world applications.

0.17

Drift-adaptive Representation Preservation for Domain-Incremental Learning

cs.CV cs.AI ·2026-11-04 ·Hyunmin Kong, Gyutae Oh, Jitae Shin
publishedbenchmark similarity 0.62
Abstract · Accepted to ACCV 2026

Pretrained vision models provide strong general-purpose representations, but in Domain-Incremental Learning (DIL), where the input distribution changes sequentially, adapting to new domains often degrades performance on previously learned ones. We observe that forgetting is not uniform across domains and classes and is not explained by training order alone. This motivates us to track each past domain-class concept individually. We use representation drift, defined as the deviation of a past concept's current representation from a fixed anchor, as a measurable surrogate for forgetting risk. The measured drift then serves two complementary roles. A consolidation loss assigns stronger preservation to high-drift pairs, while prototype construction down-weights highly drifted domains to reduce their influence on the class reference. Preservation is thus allocated by measured drift rather than assumed importance. To obtain this signal during training without storing past raw data, we repurpose a Joint-Embedding Predictive Architecture (JEPA). For each domain-class pair, we store only a compact context summary in the backbone feature space and rerun it through the learned context-to-target relation to measure its drift from the fixed anchor. Across DomainNet, Office-Home, CORe50, and CDDB-Hard, our method sustains a strong accuracy-forgetting balance on every benchmark, using substantially less training memory than accuracy-competitive methods and the smallest storage among compared methods.

0.21

Learning to Absorb Corrections: Critic-Weighted Residual Flow for Vision-Based Insertion

cs.RO ·2026-10-21 ·Yiqing Yuan, Zhi Li, Jinrong Yang, Mingshan Sun, Mei Yu Zhi +3 more
similarity 0.62
Abstract · 9 pages, 7 figures

Vision-based insertion requires correcting alignment errors while retaining useful behavior learned from demonstrations. We present ACT--Flow, an online learning framework that couples an adaptive Action Chunking Transformer (ACT) with a conditional residual flow policy. Following anchor-guided warmup, ACT absorbs successful executed actions through behavioral cloning, while residual targets are recomputed relative to the updated prior. A chunk-level critic learns from successful and failed interactions and evaluates final composed actions. Its estimates weight generated residual candidates for flow matching, complemented by supervision from successful real executions. Candidate scoring is confined to training; deployment uses RGB observations and samples one residual per action chunk. We evaluate the framework on power-plug, USB, and key insertion, together with cable routing and needle threading. Results show a five-task mean success rate of 66% at 20,000 post-initialization interaction steps, exceeding the strongest baseline by 18 percentage points at this checkpoint. Evaluations at matched checkpoints with the residual disabled show ACT's mean success increasing from 16% to 76%. Component ablations further support the complementary roles of behavior absorption, value weighting, and successful-action supervision.

0.20

One-to-Many: Diverse and Bidirectional Image-to-Image Translation via Rectified Flow

cs.CV ·2026-11-19 ·Sho Shimotsumagari, Seiichi Uchida, Shumpei Takezaki
similarity 0.62
Abstract · 12 pages, 13 figures, 4 tables. Includes supplementary material

Many paired image-to-image translation tasks admit multiple valid target appearances, while training provides only one target per source. We propose One-to-Many Flow (O2MFlow), a latent-conditioned Rectified Flow (RF) model for one-to-many generation and backward translation (target$\rightarrow$source) within a single transport framework. Conditioning the RF dynamics on an appearance latent turns transport into a family of latent-indexed bijections: a fixed latent gives a mapping that is inverted by time-reversed integration, and sampled latents produce diverse outputs for the same input. We formalize reusable appearance variations as Shared Latent Transport and show that Kullback--Leibler (KL) regularization directly controls the source--latent distribution gap between training and generation. Across three benchmarks, O2MFlow offers dataset-dependent forward diversity--fidelity trade-offs and the lowest backward LPIPS-Dist among evaluated methods, with the lowest backward FID on two benchmarks.

0.23