CS Paper Radar

Distill Locally, Schedule Globally: Flow Maps for Few-Step Text-to-Speech

Yentl Collin, Evan Dufraisse, Amr Mohamed, Amine Khelif Khelif, Dani Bouch, Guokan Shang

cs.SD cs.LG ·submitted 2026-09-28 21:57 UTC · v1

Abstract

Flow-matching text-to-speech (TTS) models achieve high synthesis quality but require many neural function evaluations (NFEs) to integrate their generative trajectories. Recent few-step flow-map distillation approaches for TTS construct targets from numerically integrated teacher trajectories, creating a trade-off between target accuracy and training cost. We propose Local Flow-Map Distillation (LFMD), which adapts Eulerian Map Distillation to conditional TTS and avoids teacher trajectory integration during target construction. For inference, we derive a sampling schedule (TD-DP) from teacher dynamics and consistency of the learned maps, with a single cost graph supporting multiple NFE budgets without external audio-metric evaluation. Because scheduling offers no flexibility at one NFE, we refine this regime with alignment-aware temporal self-distillation using soft-DTW. Across Seed-TTS and LibriSpeech-PC, LFMD improves low-NFE synthesis over a matched integral-distillation baseline. On Seed-TTS, the refined student reaches 1.80% WER with 1-NFE, compared with 1.76% for its 32-NFE teacher.

Comment
5 pages, Submitted to ICASSP 2027

Closest papers

Nearest neighbours by meaning across the whole index.

Simplifying and Improving On-Policy Distillation for Few-Step Diffusion

cs.CV cs.LGeess.IV ·2026-10-13 ·Jiawei Zhang, Ziyu Wan, Nanye Ma, Long Zhao, Dong Chen +3 more
code/datacommunity pick similarity 0.72
Abstract · Project Page: https://zjwfufu.github.io/SMD-Page/

Few-step distillation reduces the sampling cost of diffusion and flow models, but existing approaches often require auxiliary score models or specialized architectures. We show that simple on-policy velocity regression already provides a strong foundation for 4-NFE generation. To represent teacher velocity variation within each coarse transition, we introduce Pivot, a training-only residual proxy that separates local corrections from the student's interval-mean prediction. We further repel the student from weaker teacher predictions, providing a lightweight mechanism for avoiding undesirable modes. The resulting framework, Simplified Matching Distillation (SMD), remains entirely regression-based, requires neither online score estimation nor adversarial training, and leaves the deployed generator unchanged. We evaluate SMD on image generation and editing, 3D generation, video synthesis, and joint audio--video generation, scaling to 33 billion parameters. At 4-NFE, SMD outperforms strong few-step baselines and surpasses multi-step teachers on several benchmarks. It also supports cross-model, multi-teacher, and causal-video distillation.

0.49

Geometry-Aware Time Reparameterization for Flow-Map Distillation

cs.LG cs.AI ·2026-10-01 ·Félix Dedek, Makoto Yamada
hot direction similarity 0.71
Abstract

Flow-map distillation enables one- and few-step generation by learning finite-time transitions of a pretrained generative ODE. We investigate whether changing the teacher's time parameterization can make these transitions easier to learn. Motivated by the hypothesis that trajectory segments with large normal acceleration are harder to distill, we propose a geometry-aware time reparameterization that allocates more student time to these regions while preserving the teacher's geometric paths and terminal distribution. We derive a shared clock that equalizes a population normal-acceleration statistic under suitable assumptions, and construct a practical approximation from robust, regularized estimates across teacher trajectories. We incorporate this clock into Lagrangian flow-map distillation, using the transformed time coordinate to condition the student. The clock is estimated once before distillation and requires neither teacher retraining nor additional student parameters or inference-time network evaluations. Experiments on synthetic data, CIFAR-10, and CelebA-64 show improved sample quality over identity-time distillation at matched inference budgets, including improvements in one- and two-step image generation. The gains in one-step generation, where no intermediate sampling times can be adjusted, highlight the benefits of time reparameterization during distillation.

0.26

MixiSpeech: Progressive State Distillation for One-Step Vietnamese TTS

cs.SD cs.MMeess.AS ·2026-10-18 ·Tran Quoc Bao Le, Binh Mai, Hung Dinh, Lam Nguyen, Cong Tran
publishedcode/data similarity 0.71
Abstract · Accepted at SOICT2026

Conditional-flow text-to-speech (TTS) models can achieve high synthesis quality with relatively few sampling steps, but collapsing the remaining trajectory to a single update can still cause substantial quality degradation. We propose MixiSpeech, a progressive state distillation framework that converts a multi-step conditional-flow acoustic model into a one-step generator while preserving its architecture and speaker conditioning. Rather than training the one-step model directly, MixiSpeech progressively reduces the global sampling budget through a 10-4-2-1 curriculum, with each student trained to match local teacher-integration endpoints using a masked state-matching objective. We instantiate the framework on a 710-speaker Vietnamese Matcha-TTS model trained from scratch on PhoAudiobook. On 100 shared test text-speaker pairs, direct one-step inference substantially degrades synthesis quality, obtaining 1.6846 UTMOS22-Strong and 6.21% WER compared with 2.2923 and 4.65% for the 10-step teacher. In contrast, the distilled one-step model achieves 2.2678 UTMOS22-Strong, 4.63% WER, and 2.83% CER, recovering most of the quality lost at the one-step operating point while requiring only a single acoustic vector-field evaluation. End-to-end inference time decreases from 0.0662 s for the 10-step teacher to 0.0312 s for MixiSpeech, corresponding to a 2.12 measured speedup. These results show that progressive state distillation can make one-step conditional-flow TTS practical without relying on a large multi-step sampling budget. Project page: https://mixispeech.solanai.us/

0.29

Trajectory-Regularized Distribution Distillation for One-Step Generation

cs.CV cs.AI ·2026-11-08 ·An Zhao, Yixiang Zhou, Zejian Li, Yi Guo, Pei Chen +1 more
similarity 0.71
Abstract

Distilling flow matching models for one-step generation generally follows two paradigms. Trajectory alignment supervises the student with the teacher's probability flow ODE, but its performance degrades under very small sampling budgets. Distribution matching directly optimizes the generated distribution, but estimating its reverse Kullback-Leibler gradient typically requires a separate network to track the evolving student score. We propose trajectory-regularized distribution distillation (TRD), which combines distribution-level and trajectory-level supervision through a momentum-mixture policy predicted in a single student forward pass. This policy serves simultaneously as a one-step generator, an estimator of the student's marginal velocity that removes the need for a second trainable network in distribution matching, and a closed-form time-dependent velocity field supervised along the teacher ODE trajectory. Experiments on SANA-1.6B and Stable Diffusion 3.5 show that TRD outperforms representative consistency and distribution matching baselines in generation quality and training efficiency for both one-step and few-step generation, with a trade-off in sample diversity.

0.05

Route-DMD: Distribution Matching Distillation with Task-Routed Mechanism for Speech Foundation Models

cs.SD ·2026-11-05 ·Yanru Huo, Zhikang Niu, Wenming Tu, Junxi Liu, Tian Tan +5 more
similarity 0.70
Abstract

Speech foundation models built on flow matching handle generation, editing, and processing in one system, but every request costs 20-50 iterative steps, and classifier-free guidance (CFG) doubles each of them. Distribution matching distillation (DMD) trains a student that approaches the teacher's quality in a few steps, yet existing speech distillation work applies it to generation alone. We show that a distribution-matching objective does not serve every task: in separation, the noised states of erroneous student outputs deviate far from the teacher's training distribution, where its prediction is unreliable. Route-DMD therefore makes the objective task-dependent, routing the affected tasks to a regression loss and keeping decoupled distribution matching elsewhere. The student is initialized by consistency distillation, and teacher rollouts use adaptive projected guidance (APG) instead of CFG. Without CFG, the 4-step student matches the 32-step teacher on generation, editing, and processing tasks at a 4.5$\times$ end-to-end speedup.

0.16

DriftTTS: One-stage Single Step Text-to-Speech via Generative Drifting

cs.SD ·2026-10-16 ·Jaehyun Ahn, Dohyun Kim, Heeseung Kim
code/data similarity 0.69
Abstract · Project page: https://drifttts.github.io/

Text-to-speech models built on score matching and flow matching produce natural speech, but only by calling the decoder many times, which makes synthesis slow. Existing ways to cut the number of calls either distill from a multi-step teacher or add a stage on top of a pretrained model, leaving single-step TTS without a pretrained TTS model unexplored. We question whether pretraining a TTS model is necessary, and propose DriftTTS, which adopts generative drifting, an objective that moves iterative sampling from inference into training. Drifting assumes samples of fixed shape, whereas speech has variable length; we address this issue with segment drifting. On LJSpeech, DriftTTS matches the multi-step quality of Matcha-TTS and RapFlow-TTS in a single decoder call, while being faster. To our knowledge, this is the first application of generative drifting to text-to-speech.

0.16

VA-TDM: A Unified View for Improving Few-Step Diffusion Models by Velocity Adversarial Distillation

cs.CV ·2026-11-04 ·Yihong Luo, Tianyu Pang, Tianyang Hu
code/data similarity 0.69
Abstract · Project Page: https://luo-yihong.github.io/VA-TDM/ Code: https://github.com/Luo-Yihong/VA-TDM

Few-step distillation offers efficient generation with diffusion and flow models, but preserving quality with only a few sampling steps remains challenging. Distribution-matching methods via score distillation provide a strong foundation, yet they rely on accurately tracking the student's changing distribution. At intermediate generation steps in few-step sampling, student-score estimation can be difficult, and commonly used estimators may describe a different distribution from the one used to update the student. Flow matching allows a more flexible approach to supervision through velocity fields constructed from the student's sampling process. Building on this perspective, we introduce Velocity Adversarial Distillation (VAD), which uses a learned critic to compare student and teacher fields and turn their disagreement into feedback for learning. VAD unifies score distillation, Fisher-based matching, and moment matching, also explaining existing heuristic MMD updates and delivering new training objectives. We further develop VA-TDM, which follows trajectory distribution matching: providing supervision for partial denoising along the student trajectory. Beyond this, VA-TDM combines transition-aware online velocity learning and dual-form VAD supervision across intermediate student transitions. Extensive results show that VA-TDM is powerful for learning few-step generators, achieving state-of-the-art four-step performance in both text-to-image and text-to-video generation across various backbones and scaling effectively to 33B MiniMax-H3. In particular, our VA-TDM-H3 notably outperforms the few-step Fast-H3 and is closer to the original many-step H3 according to human preference.

0.28

Consistent Distribution Matching for Data-Free Diffusion Distillation

cs.LG cs.AIcs.CV ·2026-10-06 ·Yuxiang Fu, Qi Yan, Zike Wu, Yongxing Zhang, Purang Abolmaesumi +2 more
theorycode/data similarity 0.68
Abstract

Flow and diffusion models suffer from slow inference due to computationally expensive numerical integration. Distillation provides a promising way for a student model to learn from a teacher's dynamics, enabling one-step or few-step generation. However, existing methods often depend on curated distillation datasets, costly teacher rollouts, or auxiliary proxy networks, which complicate model training and scaling. In this work, we propose Consistent Distribution Matching, a simulation-free and data-free distillation method for accelerating diffusion and flow models while preserving strong generative capacity. Our key insight is to unify sample generation and score estimation with one student network. Thus, our framework uses only two models, a frozen teacher and a trainable student, and optimizes one objective. We prove that minimizing our objective indicates Wasserstein convergence of the student flow-map pushforwards to the teacher marginals. On ImageNet 256$\times$256, our method attains an FID of 2.04 with a single function evaluation (1-NFE) and a 4-NFE FID of 1.37 within 40 epochs of training, surpassing the state-of-the-art distillation baselines without data. Our code code and model are available at https://consistentdmd.github.io/.

0.45