CS Paper Radar

DiffVQE2: An Efficient Low-delay Diffusion Model for Acoustic Echo and Noise Control

Haljan Lugo, Ernst Seidel, Pejman Mowlaee, Ziyue Zhao, Tim Fingscheidt

eess.AS cs.SD ·submitted 2026-09-19 19:26 UTC · v1
published

Abstract

Hands-free communication devices and speakerphones are inherently affected by acoustic echo and background noise. To mitigate these impairments, end-to-end discriminatively trained neural networks have emerged as the best-performing approach in research and deployment. While recent advancements in generative methods have provided remarkable results for various speech enhancement tasks, diffusion-based acoustic echo control (AEC) research is still restricted to non-causal, utterance-level processing, thereby not widely applicable in practice. With this work, we are the first to propose low-delay (i.e., causal) diffusion-based joint AEC and noise control models DiffVQE2 / DiffVQE2-S, excelling the so-far state of the art DeepVQE / DeepVQE-S models in multiple objective metrics, and, most importantly, in subjective MOS, respectively. In addition, our models are less complex. Furthermore, we show that a limited lookahead applied to the efficient DiffVQE2-S model allows for an even higher performance. These results have been obtained on the ICASSP 2023 AEC Challenge blind test set. We claim the first streaming-capable, diffusion-based acoustic echo and noise control that excels state-of-the-art discriminative approaches.

Comment
accepted at IWAENC 2026

Closest papers

Nearest neighbours by meaning across the whole index.

Knowledge Distillation for Efficient Acoustic Echo Control

eess.AS ·2026-08-26 ·Ernst Seidel, Pejman Mowlaee, Tim Fingscheidt
published similarity 0.64
Abstract · 5 pages, accepted to EUSIPCO 2026

In recent years, many efforts have been made to supersede classical acoustic echo control (AEC) algorithms with more powerful machine-learned approaches. While surpassing the performance of well-established adaptive filters is very much possible, a remaining challenge is computational complexity. Popular architectures, such as convolutional recurrent networks (CRNs), are by multiple orders of magnitude computationally more expensive than classical signal processing solutions. Scaling down such models is usually straight-forward, but it comes at the cost of a notably reduced performance. We show - to the author's knowledge for the first time in AEC - how these performance drops can be successfully alleviated to a large degree by employing an effective knowledge distillation (KD) process, enabling more potent efficient AEC. Our proposed CGGN16 student AEC models show significantly less near-end speech distortion at only 2% of its teacher's computational complexity, surpass the overall performance of a six times more complex model trained on ground-truth labels, and outperform other AEC-focused architectures from recent literature.

0.22

SlimDiffuSE: Towards Efficient Diffusion-Based Speech Enhancement using Slimmable Networks

eess.AS ·2026-08-21 ·Nagashree K. S. Rao, Shrishti Saha Shetu, Mohamed Elminshawi, Emanuël A. P. Habets, Andreas Brendel
benchmark similarity 0.64
Abstract

Diffusion-based models are emerging in the speech enhancement domain and are achieving state-of-the-art performance across various benchmark datasets. A major downside of diffusion models is that data generation requires many evaluations of a typically large neural network, which results in high overall complexity. In this work, we propose a slimmable diffusion model that employs adaptive network widths throughout the data generation process to reduce computational cost. By using a greedy search algorithm to optimize the network width schedule, our method achieves performance comparable to baseline diffusion models with significantly reduced computational complexity. Notably, our approach reduces the computational complexity by up to $87.5\%$ without a significant drop in objective metrics, such as perceptual evaluation of speech quality (PESQ) and SI-SDR.

0.18

SE-MSB: End-to-End Unpaired Speech Enhancement using Mamba Schrödinger Bridges

eess.AS cs.AIcs.SD ·2026-09-22 ·Andreas Bagge, Andreas Nymand, Michael Riis Andersen, Bjørn Sand Jensen
similarity 0.64
Abstract · 15 pages, 4 figures, 4 tables

Speech enhancement (SE) models typically rely on supervised learning with paired data examples where clean speech is synthetically degraded. This paradigm limits performance in real-world scenarios where the target environment's specific acoustic characteristics are unknown. We propose a fully unpaired SE framework that uses principled Diffusion Schrödinger Bridges (DSB) to learn a stochastic transport process between a clean and a degraded speech distribution. Algorithms for learning transport maps are computationally heavy since they require simulating differential equations during training, usually at each training step. Therefore, we propose using a high-efficiency Mamba Diffusion Model designed for end-to-end waveform processing. We compare against state-of-the-art methods for speech enhancement, both paired and unpaired, as well as a classical signal processing algorithm. Experimental results show that we are on par or better than the baselines while being orders of magnitude faster during inference. Furthermore, we show that the flexibility of the DSB formulation allows our model to generalize across SE tasks, offering a robust and efficient solution for real-world speech restoration.

0.25

StreamSE: Neural Audio Codec-based Streaming Speech Enhancement for On-device Inference

eess.AS ·2026-10-27 ·Hyunjin Choi, Zinu Kim, Junwon Moon, Jeongyun Chae, Seungbeom Kim +1 more
published similarity 0.62
Abstract · NeurIPS 2026 Workshop on On-Device Intelligence

Recent generative speech enhancement (SE) models achieve high perceptual quality, but their non-causal processing and expensive computation make them difficult to use for streaming, on-device inference. We propose StreamSE, a 20M-parameter streaming SE model that operates in the latent space of a frozen pretrained neural audio codec. At each frame, StreamSE directly regresses a clean continuous latent from the noisy continuous latent while conditioning on its own predictions from previous frames. Enhancement therefore requires a single forward pass per frame and no iterative sampling or look-ahead. We further truncate the codec's residual vector quantizer (RVQ) to its coarse levels, halving the number of tokens per frame while improving robustness to reverberation. StreamSE is the only streaming generative model in our comparison, yet it matches or exceeds the perceptual quality of the offline generative baselines. Furthermore, the complete StreamSE pipeline runs in real time on a single smartphone CPU core. Our results show that continuous latent regression provides a practical route to generative SE under fully causal, on-device streaming constraints.

0.14

Does per-frame early exit pay? A compute-matched study of dynamic depth for on-device speech enhancement

eess.AS cs.LGcs.SD ·2026-09-24 ·Clément Laroche, Riccardo Miccini
similarity 0.62
Abstract

Deep learning-based speech enhancement is increasingly deployed on-device in hearing aids, headsets, and earbuds. Most of these devices, however, can only accelerate static int8 graphs, so a depth-varying network must be implemented as several graphs, orchestrated by a policy. In this paper, we supervise every intermediate depth of one causal model, then we fine-tune its output heads to guarantee that deeper outputs are never worse than shallower ones. Using this training protocol, we can derive a family of static models that are more Pareto-efficient than their equivalently-sized counterparts trained from scratch on the same budget. Specifically, we achieve up to 0.11 higher PESQ for equivalent compute, and match the best PESQ at 30% less compute. We then quantize the models to int8 and measure the latency-quality frontier on an STM32N6 microcontroller. On VoiceBank-DEMAND, the dynamic enhancer lies on the same frontier as the static models, rather than trading quality for dynamic execution. Running the policy on the companion Cortex-M55 takes only 26 $μ$s per frame, while splitting the enhancer into separate NPU graphs adds 2.2% latency overhead. The cost of dynamic execution is therefore small.

0.24

Teacher-Free Self-Distilled Consistency Trajectory Learning for Fast Speech Enhancement

eess.AS ·2026-09-09 ·Shuubham Ojha, Carol Espy-Wilson
similarity 0.61
Abstract

Consistency trajectory models offer a route to fast, high- quality speech enhancement, collapsing the many reverse steps of diffusion-based enhancers into a handful. When instantiated on a Schrödinger bridge (SB), which pins the generative process to fixed clean and noisy endpoints, exist- ing consistency-trajectory enhancers (SBCTMs) still require a pretrained teacher to supply trajectory supervision, which raises training cost and ties the final quality to that of the teacher. We propose a teacher-free, self-distilled consistency- trajectory framework that removes the external teacher result- ing in a 5X reduction in per epoch training time. Our model is trained with a three-stage curriculum of clean speech pre- diction, a self-distilled shortcut objective, and perceptual fine-tuning with a multi-resolution short-time Fourier trans- form (MR-STFT) loss. Using the same NCSN++ backbone as SBCTM, our model attains a wide-band PESQ of 3.01, ES- TOI 0.87, and SI-SDR 19.07 dB on VoiceBank+DEMAND compared to 3.57, 0.87 and 12.8 dB for the teacher based model. Further, we find that a geometric schedule at low reverse step count maximizes perceptual quality, while a higher-step uniform schedule favors signal fidelity.

0.25

Personalized Causal Low-Bitrate Coded Speech Restoration with Speaker Embeddings

eess.AS eess.SP ·2026-11-17 ·Eray Özgünay, Christian Rollwage, Kishan Gupta, Andreas Brendel, Xavier Domont +3 more
similarity 0.61
Abstract · 5 pages, 3 figures, 1 table

Legacy low-bitrate speech codecs such as AMR-NB remain widely used in telephony, where they restrict speech bandwidth and degrade listening quality. Coded speech restoration (CSR) aims to restore the speech bandwidth while improving the overall quality of coded speech, but typically without adapting to different speakers. In this paper, we systematically investigate personalization for restoration of speech coded at low bitrates, using a causal, low-complexity Universal Bandwidth Extension Generative Adversarial Network (UBGAN) model as the basis. We propose two personalization methods based on speaker embeddings, which incur little additional computational cost at inference: (i) conditioning a single CSR model on speaker embeddings, and (ii) cluster-based model selection, where multiple personalized models are trained on clusters of acoustically similar speakers. Experiments are performed on AMR-NB coded speech at 4.75 kbps. Experimental results with disjoint seen- and unseen-speaker test sets show that cluster-based model selection does not consistently outperform the non-personalized UBGAN baseline, whereas embedding conditioning significantly improves ViSQOL and WB-PESQ scores for both seen and unseen speakers.

0.19

Rethinking Language Model-Based Generative Speech Enhancement in the Latent Space of a Neural Audio Codec

eess.AS ·2026-08-12 ·Yihui Fu, Zhengyang Li, Tim Fingscheidt
published similarity 0.61
Abstract · Accepted at IWAENC 2026

Language model (LM)-based speech enhancement (SE) has recently emerged rapidly using latent space features of neural audio codecs (NACs). In this paper, first, we present a unified framework covering six popular LM-based generative SE modeling paradigms based on discrete/continuous latent NAC features: discrete or continuous autoregressive (D/CAR) SE, discrete or continuous non-autoregressive (D/CNAR) SE, discrete diffusion (DDiff) SE, and continuous flow matching (CFM) SE. Second, we are the first to compare their performance in a unified experimental setup and synopsis with diverse intrusive and non-intrusive metrics, enabling a fair and comprehensive evaluation. Third, we propose a fine-tuning strategy with auxiliary losses on reconstructed speech to improve both intrusive and non-intrusive metrics. Trained and evaluated on URGENT 2025 Speech Enhancement Challenge data splits, all continuous-domain paradigms excel their discrete-domain counterparts. The overall best approach turns out to be CNAR. We further show that our proposed auxiliary loss fine-tuning strategy helps to improve DNSMOS, NISQA, PESQ, and POLQA consistently in all six paradigms.

0.19