CS Paper Radar

SCNet: Enhancing GAN-based Speech Generation with Subband Condition Network and Magnitude-aware Phase Loss

Nan Xu, Mingxue Yang

eess.AS ·submitted 2026-09-09 10:59 UTC · v1
top venue

Abstract

Recent speech generation has been predominantly driven by GAN-based networks aimed at high-quality waveform synthesis from mel-spectrograms. However, these methods often operate as black-box models, leading to the loss of inherent spectral information. In this work, we propose SCNet, a GAN-based vocoder augmented with a Subband Condition Network to address this issue. Specifically, SCNet leverages a subband signal predicted by a lightweight condition network as prior knowledge. This subband signal is then transformed via STFT to obtain Fourier coefficients, which are integrated into the backbone for the enhanced reconstruction. Additionally, to mitigate the phase wrapping, we introduce a magnitude-aware phase loss that computes instantaneous phase errors weighted by the corresponding magnitude, emphasizing regions with higher energy. Experimental results demonstrate that SCNet achieves superior performance in both objective and subjective evaluations for high-quality speech generation.

Comment
Accepted by Interspeech 2026

Closest papers

Nearest neighbours by meaning across the whole index.

PhaseGAN: High-Fidelity Vocoder via Decoupled Amplitude and GAN-Driven Phase Reconstruction

cs.SD ·2026-09-11 ·Wenzheng Zhang, Xueliang Zhang, Shulin He, Fei Zhao, Xin Liu +5 more
code/data similarity 0.83
Abstract · 15 pages, 1 figure

A vocoder is a pivotal component of modern text-to-speech (TTS) systems. Despite the significant progress of neural network-based vocoders, accurate phase reconstruction remains the main challenge limiting both audio quality and modeling efficiency. We introduce PhaseGAN, a lightweight vocoder that addresses this limitation through a "mel $\rightarrow$ Amplitude $\rightarrow$ Phase" reconstruction pipeline. By reconstructing amplitude and phase spectra via distinct methodologies, the proposed PhaseGAN outperforms state-of-the-art baselines while utilizing fewer model parameters and reduced computational requirements. The compact version generates high-fidelity audio with approximately 500K parameters and 1 GMAC computational load, making it highly suitable for real-time applications on edge devices. In addition, our approach exhibits exceptional musical audio synthesis capabilities despite no training on musical data, illustrating unprecedented cross-domain generalization. See https://github.com/phasegan/phasegan-audio-demo for demos of our work.

0.35

Back to Mel: Learning to Bridge Fidelity and Perception in Generative Speech Enhancement

cs.SD ·2026-11-17 ·Andong Li, Kun Zhou, Xiaodong Li, Chengshi Zheng
code/data similarity 0.71
Abstract · Submitted to IEEE TPAMI, 18 pages, 15 figures, 16 tables

Recent generative techniques have facilitated the rapid proliferation of speech enhancement approaches built upon learnable compact latent representations and large generative backbones. Nevertheless, it remains inherently challenging to balance between acoustic fidelity, perceptual quality, and processing efficiency. In this paper, we revisit the Mel spectrum as a structured acoustic representation and propose BAME, a flow-matching-based framework that exploits the complementary priors for generative speech enhancement task. First, leveraging the explicit hierarchical frequency distribution of Mel spectrum, we elaborately devise a subband-aware Transformer (SaT) for efficient generative modeling. Using only 2.9% of the parameters, the proposed SaT achieves superior enhancement performance compared with a 331 M DiT backbone. Second, motivated by range-null decomposition theory of Mel filtering in the magnitude-spectral domain, we introduce a phase-aware neural vocoder that incorporates degraded-phase prior, together with a quality-guided dual-phase strategy to adaptively mitigate the effects of unreliable phase information. Third, we devise a simple yet effective one-step fidelity-perception alignment strategy that fine-tunes the flow model with complementary constraints from fidelity and perception domains. Extensive experiments are conducted on DNS-Challenge, EARS, GSR, and URGENT benchmarks, and both objective and subjective results demonstrate promising performance on various intrusive and non-intrusive evaluation metrics. These results further advocate the potential of exploiting structured acoustic representations for reliable and efficient generative speech enhancement. Audio samples and codes are available at https://github.com/AndongLi-speech/BAME.

0.25

ESDCodec: Ultra-Low Bitrate Speech Codec with Enhanced Semantic Disentanglement and Subband Decoding

eess.AS ·2026-11-10 ·Nan Xu, Kecan Mao, Xin Gu, Yi Zhao, Zhi Gong
similarity 0.70
Abstract · 5 pages, 2 figures

Despite recent advances in neural speech codecs, achieving highfidelity speech reconstruction at low bitrates remains a formidable challenge. To address this limitation, we propose ESDCodec, a speech codec that integrates a thoroughly enhanced semantic quantizer and a conditioned decoder network. Specifically, we employ a randomly initialized and frozen codebook, followed by a lightweight projector, to encode semantic details entirely within a linear space while enhancing codebook utilization. To further improve perceptual quality, we design a condition network that explicitly injects prior subband knowledge into the upsampling decoder. Taking the de-quantized feature as input, this network predicts subband signals, thereby providing fine-grained guidance for waveform reconstruction. Extensive experimental results show that ESDCodec achieves superior reconstruction performance at an ultra-low bitrate of 0.85kbps. For LLM-based speech generation tasks, ESDCodec also consistently outperforms existing codec models.

0.15

Bridge, Distill, Refine: High-Fidelity One-Step Generation for Multi-Bandwidth Speech Extension

eess.AS ·2026-10-16 ·Zhiqi Ai, Xu Shen, Mou Wang, Yongjin Zhou, Shugong Xu
similarity 0.67
Abstract · 5 pages, 4 tables, 1 fugure

Speech bandwidth extension (BWE) aims to recover missing spectral content from band-limited speech. However, most generative BWE methods are designed for fixed input bandwidths and rely on iterative sampling, limiting their flexibility and efficiency. We propose BDR-BWE, a three-stage generative framework for multi-bandwidth speech extension. First, to enable conditional generation from the observed band-limited spectrum, we design a cutoff-aware conditional flow bridge in the compressed complex STFT domain, focusing generation on the missing high-frequency band while preserving observed spectral information. Second, progressive consistency distillation compresses the multi-step flow transport into a one-step generator. Third, adversarial refinement further improves high-frequency detail and harmonic coherence. BDR-BWE achieves state-of-the-art log-spectral distance (LSD) on VCTK across all evaluated bandwidths with one-step inference, while maintaining competitive perceptual quality and generalizing well to EARS. Audio samples are available at https://aizhiqi-work.github.io/bdr-bwe-demo.

0.23

MPASCNet: Magnitude-Phase-Aware Adaptive Spectral Compression for Compressed Music Restoration

cs.SD ·2026-10-12 ·Xinlong Deng, Yu Xia, Qiuqiang Kong, Yin Cao, Jie Jiang
code/data similarity 0.66
Abstract · Submitted to ICASSP 2027

Compressed music restoration aims to recover high-fidelity audio from lossy codecs such as MP3 by restoring missing spectral content and reducing coding artifacts. Recently, generative adversarial networks and band-split models have made progress in compressed music restoration. However, existing methods either operate without explicitly modeling phase or rely on predefined frequency partitions, which may limit restoration performance and the use of input-dependent frequency information. To address these issues, we propose MPASCNet, a Magnitude-Phase-Aware Adaptive Spectral Compression network. Instead of using fixed bands, MPASCNet combines magnitude and phase features to guide cross-attention, adaptively compressing full-resolution spectral features into compact tokens. These tokens are then processed by a spectro-temporal backbone to model spectral-temporal dependencies. Finally, separate decoder branches reconstruct the full-resolution magnitude and wrapped phase spectra. Experiments on MP3 restoration show that MPASCNet outperforms the Apollo and AEROMamba baselines across multiple restoration metrics while using only 6.49 M parameters. The code is available at https://github.com/xinghour/MPASCNet.

0.26

P2Flow: Phoneme-aware Progressive Flow Matching for Extreme Speech Super-Resolution

eess.AS cs.LGcs.SD ·2026-09-21 ·Ningyuan Yang, Yize Li, Pu Zhao, Diego A. Cuji, Kanad Sarkar +3 more
similarity 0.63
Abstract · Under review

Generative models have recently demonstrated considerable promise in speech super-resolution (SSR). Nevertheless, the majority of existing work has concentrated on standard or versatile SSR configurations, leaving the extreme setting with severely limited spectral inputs largely unexplored. In this regime, current approaches exhibit marked performance degradation, underscoring the need for dedicated solutions. To bridge this gap, we introduce P2Flow, a phoneme-aware progressive flow matching (FM) framework designed for extreme SSR with three main strategies. First, our model leverages phonetic information to reconstruct missing spectral components. Furthermore, it employs a progressive architectural design that hierarchically restores distinct frequency regions. Finally, we incorporate post-training of the vocoder to enhance overall waveform fidelity. Extensive experiments are conducted on the TIMIT and VCTK datasets under both 1 kHz to 16 kHz and 2 kHz to 16 kHz settings, demonstrating that P2Flow yields state-of-the-art results across multiple evaluation metrics.

0.31

Vocal Intensity Conversion with Continuous SPL Control via Adversarial Training

eess.AS cs.LGcs.SD ·2026-10-09 ·Quentin Le Tellier, Albert Rilliard, Olivier Perrotin, Marc Evrard
code/data similarity 0.63
Abstract · 5 pages, 2 figures, 1 table. Submitted to IEEE ICASSP 2027. Code: https://github.com/vers-project/vocal-intensity-conversion. This work has …

Vocal intensity conversion renders a recorded utterance at a different vocal intensity, rather than simply scaling its amplitude, while preserving its content and speaker identity. Existing systems control intensity through a reference utterance or a few discrete categories. Instead, we control the conversion with any target sound pressure level (SPL) on a continuous dB scale. A lightweight converter, trained as a conditional cycle-consistent GAN without parallel data, transforms the features of a frozen speech encoder, from which a pretrained vocoder resynthesizes the speech. We compare three representations, log-mel spectrograms, WavLM, and SpeechTokenizer, and introduce an objective protocol that measures the achieved intensity change against the target one on a calibrated corpus. With WavLM, the converter achieves a measured 83% of the target change in intensity and also shifts the fundamental frequency. A perceptual evaluation confirms that perceived vocal intensity increases with the target level on amplitude-normalized stimuli.

0.30

StreamWSR: Streamable and Lightweight Waveform-Domain Neural Speech Super-Resolution

eess.AS ·2026-09-03 ·Yuan Tian, Yang Ai, Hui-Peng Du, Zhen-Hua Ling
similarity 0.62
Abstract

This paper proposes StreamWSR, a Streamable neural Waveform-domain model for speech Super-Resolution (SR). By adopting a fully causal architecture with compact frame-level waveform representation, the proposed StreamWSR supports zero-look-ahead streaming inference while avoiding vocoder-based reconstruction and explicit phase prediction. Specifically, StreamWSR downsamples the input waveform into a compact frame-level representation using strided causal convolutions. Then, a lightweight causal long-short-term modeling backbone is employed to capture both local waveform structures and long-range historical dependencies under causal constraints. Finally, the modeled output is converted back to the waveform domain through a causal transposed-convolution and combined with the input waveform via a residual connection to generate the final high-resolution speech. Experimental results on 16 kHz speech SR show that StreamWSR achieves competitive or superior speech quality and intelligibility compared with representative waveform- and spectrum-based baselines, while maintaining a zero-look-ahead streaming advantage with only 9M parameters and 2G FLOPs.

0.31