CS Paper Radar

The NPU-ASLP System for Music Restoration in the CCF-AATC 2026 Challenge

Zeyu Zhu, Yike Zhu, Xingchen Li, Chaoren Yu, Jingyi Wang, Yurun Li, Jierui Guo, Chunjiang He, Lei Xie

eess.AS ·submitted 2026-10-11 09:03 UTC · v1

Abstract

This paper presents the NPU-ASLP music restoration system developed for Track 1 of the China Computer Federation (CCF) Advanced Audio Technology Competition (AATC) 2026 Challenge. The task aims to recover clean music from mobile-device-captured recordings affected by coupled degradations from playback devices, acoustic propagation, environmental noise and recording terminals. System development is further constrained by limited and imperfectly aligned real data pairs and a strict model-complexity budget. In our system, we adopt TF-MoE as the backbone for its dual-axis modeling of cross-band and temporal dependencies, whose Top-1 routing activates a single expert per token to bound computation under the strict budget. We extend it with two task-specific adaptations: a scale-aware encoder and a complex mapping decoder. The former encodes normalized complex spectra jointly with per-band log-RMS to preserve the band-energy cues discarded by band-wise normalization. The latter directly estimates the clean complex spectrum rather than masking the input, thereby repairing coupled degradations. To address the limited, imperfectly aligned real recordings, we further build a challenge-calibrated playback--propagation--recording simulator that generates 1,572.12 h of sample-aligned data across 202 acoustic conditions. The final system contains 2.951M parameters and requires 82.659 GMACs for a 10-s input. On the official blind test, it achieved -49.067 dB SI-SNR, 1.273 LSD, 2.540 FAD, and 3.495 ViSQOL. It ranked third in LSD, FAD, and ViSQOL and received the third prize in the challenge.

Closest papers

Nearest neighbours by meaning across the whole index.

MPASCNet: Magnitude-Phase-Aware Adaptive Spectral Compression for Compressed Music Restoration

cs.SD ·2026-10-12 ·Xinlong Deng, Yu Xia, Qiuqiang Kong, Yin Cao, Jie Jiang
code/data similarity 0.66
Abstract · Submitted to ICASSP 2027

Compressed music restoration aims to recover high-fidelity audio from lossy codecs such as MP3 by restoring missing spectral content and reducing coding artifacts. Recently, generative adversarial networks and band-split models have made progress in compressed music restoration. However, existing methods either operate without explicitly modeling phase or rely on predefined frequency partitions, which may limit restoration performance and the use of input-dependent frequency information. To address these issues, we propose MPASCNet, a Magnitude-Phase-Aware Adaptive Spectral Compression network. Instead of using fixed bands, MPASCNet combines magnitude and phase features to guide cross-attention, adaptively compressing full-resolution spectral features into compact tokens. These tokens are then processed by a spectro-temporal backbone to model spectral-temporal dependencies. Finally, separate decoder branches reconstruct the full-resolution magnitude and wrapped phase spectra. Experiments on MP3 restoration show that MPASCNet outperforms the Apollo and AEROMamba baselines across multiple restoration metrics while using only 6.49 M parameters. The code is available at https://github.com/xinghour/MPASCNet.

0.26

How Much Audio Is Left In An Embedding? An Inversion Audit Of Audio Encoders

cs.SD cs.AI ·2026-10-08 ·Marios Glytsos, Brian McFee
similarity 0.66
Abstract · ICASSP 2027 submission; 5 pages, 2 figures, 2 tables

Pretrained audio encoders are reused for downstream tasks that are often unknown when the encoder is trained, so their usefulness depends partly on which signal properties survive the pretext objective. We study this retained information through paired source reconstruction. Using a shared Stable Audio Open latent diffusion decoder, we reconstruct five-second, 44.1-kHz stereo music from frozen representations produced by supervised classifiers (VGGish, ConvNeXt), an audio-text contrastive model (CLAP), and a waveform-reconstruction model (EnCodec). These objectives impose different pressures to preserve source detail, while their exposed interfaces vary substantially in temporal and spectral resolution. Evaluating on the Million Song Dataset (MSD), we find clear differences in reconstructability across encoder families, while within encoder comparisons show improved recovery when finer temporal or spectral structure is exposed. Even compressed task oriented embeddings support reconstructions that preserve measurable source specificity and high level musical content.

0.22

Fourier is Frontier: Frequency-Aware Autoencoding for High-Fidelity Music Reconstruction

cs.SD cs.MM ·2026-08-20 ·Kangdi Wang, Yusheng Dai, Jin Xu
similarity 0.63
Abstract

Continuous-latent audio autoencoders form the backbone of latent music generators, yet decoders at high compression rates commonly exhibit three failure modes: high-frequency loss, phase incoherence, and stereo-image collapse. These share a structural root: waveform autoencoders lack an explicit frequency axis, leaving no handle for targeted per-band correction. Among five matched-budget representations, the complex STFT achieves the lowest full-band and high-frequency spectral distances, providing direct access to magnitude and phase at every bin. Building on this, we present ear-VAE2, a complex-spectral autoencoder with cross-channel interaction. Spec-SnakeBeta learns a periodic activation per frequency bin with frequency-dependent initialization, outperforming other activation variants while using fewer parameters than the fully independent variant. Duplex-Aware Refiner applies band-specific corrections to magnitude and phase following duplex theory of sound localization. On the 546-track Song Describer Dataset, ear-VAE2 achieves the best point estimates on five of seven reconstruction metrics. The Duplex-Aware Refiner reduces Mel Distance by 19.4% and uses ~45% fewer residual-output dimensions than the Unconstrained Refiner, while also lowering spectral distances, spatial-cue errors, and receiving higher ratings from professional engineers. The downstream generator using ear-VAE2 latents achieves better point estimates on all 12 automatic metrics.Demo page is available at https://eps-acoustic-revolution-lab.github.io/EAR_VAE2/.

0.30

UniAE-MoE: A Unified Audio Encoder via Mixture of Experts

cs.SD cs.AI ·2026-09-30 ·Shengbo Cai, Zhisheng Zhang, Zichao Nie, Jing Peng, Jingran Xie +1 more
benchmark similarity 0.63
Abstract

Large Audio Language Models (LALMs) rely on effective audio encoders for multi-task performance. We introduce UniAE-MoE, a unified audio encoder designed to model cross-domain audio representations and achieve outstanding downstream understanding performance via a Mixture-of-Experts (MoE) architecture. Specifically, we explore mainstream audio encoders and integrate those from Qwen2-Audio and Audio-Flamingo 3, which demonstrate superior downstream capabilities. To facilitate effective model fusion, we improve our encoder using SwiGLU with shared experts to decouple encoder networks, and we further introduce a two-stage instruction-tuning strategy to better adapt the model to diverse downstream tasks. Moreover, we propose the task-specific data scaling (TSDS) technique to enhance UniAE-MoE's understanding capabilities. On the XARES-LLM benchmark, UniAE-MoE attains a score of 0.802, achieving state-of-the-art performance. It also delivers top-tier performance in the official Interspeech 2026 Audio Encoder Capability Challenge, further demonstrating robust generalization across diverse audio tasks. Together, these results validate the effectiveness of UniAE-MoE for unified audio understanding across speech, music, and general audio domains.

0.13

End-to-End Historical Music Restoration in Latent Space

eess.AS cs.LGcs.SDeess.SP ·2026-09-30 ·Steven Cho, Junghyun Koo, Raphael Lafargue, Tushar Dhyani, Eloi Moliner +1 more
benchmarkcode/data similarity 0.62
Abstract · 5 pages, 2 figures, 3 tables; submitted to ICASSP 2027. Code and audio demos available at the project repository

Historical music restoration (HMR) has almost exclusively focused on constrained problems such as Super-Resolution or the restoration of solo pieces, under-exploring the general task of restoring orchestral historical music, which has multiple instruments. This under-exploration is largely because the HMR domain, early-20th-century recordings, has no pre-degradation ground-truth pairs, making the restoration task unsupervised and more challenging. This paper presents a supervised end-to-end orchestral HMR benchmark by exploring both the synthetic degradation functions and the end-to-end generative deep-learning restoration methods. We simulate the historical recording degradation chain more faithfully than prior work, which makes orchestral restoration into a tractable supervised problem. A latent flow-matching model trained on the resulting synthetic pairs outperforms existing HMR baselines on intrusive, non-intrusive, and subjective evaluations. We also curate and release a 9.3-hour license-free, unpaired, historical classical-music test set, along with code and audio demos.

0.37

TUTTI: Toward generalizable audio-to-score transcription via fully synthesized data

cs.SD cs.AI ·2026-09-01 ·Jianhuai Hu, Yashan Wang, Shangda Wu, Zhancheng Guo, Shijie Liang +5 more
code/data similarity 0.61
Abstract

Generalizable Audio-to-Score (A2S) transcription is fundamentally constrained by the severe scarcity of high-quality, real-world paired data. Relying solely on existing human-annotated datasets often restricts the generalization of A2S models, limiting their efficacy primarily to single-instrumentation domains. To break this dependency on scarce real-world data, we introduce TUTTI (Transformer for Unified audio-To-score Transcription trained on Synthetic multi-Instrumentation Data), a pre-training paradigm driven by a purely synthetic, large-scale dataset. Rather than using human-composed scores, we leverage a symbolic music generation model to generate a massive, highly scalable multi-instrumentation corpus and create audio-score pairs with expressive acoustic characteristics. Capitalizing on the generated data, we employ a standard Transformer encoder-decoder architecture. We empirically demonstrate that pre-training a unified attention-based model on generated, multi-instrumentation data yields a consistently stronger foundational representation than single-instrumentation training. When fine-tuned with downstream real-world datasets, TUTTI outperforms previous approaches, establishing new overall state-of-the-art results across various A2S baselines. Notably, TUTTI shows remarkable cross-instrument transferability, effectively adapting to unseen instruments with highly competitive performance. The source code and the TuttiCorpus dataset will be made publicly available at https://github.com/a-musiclover/TUTTI.

0.55

AT-ADD: All-Type Audio Deepfake Detection Challenge Summary

cs.SD ·2026-08-14 ·Yuankun Xie, Haonan Cheng, Jiayi Zhou, Xiaoxuan Guo, Tao Wang +9 more
top venuebenchmark similarity 0.61
Abstract · Accepted to ACM MM 2026

This paper summarizes the ACM Multimedia 2026 AT-ADD Grand Challenge on all-type audio deepfake detection. AT-ADD contains two tracks: robust speech deepfake detection under realistic acoustic and channel variations, and type-agnostic detection over speech, environmental sound, singing voice, and music. We describe the challenge tasks, dataset and evaluation-set design, official leaderboard results, and common design patterns observed in participating systems. The best Track 1 system achieved 90.71% Macro-F1 on the final evaluation set, while the best Track 2 system achieved 96.10% Macro-F1. The final submissions show that strong systems commonly combine large-scale self-supervised audio representations, data augmentation, multi-crop inference, and structured fusion or routing. The results also reveal remaining challenges in generalization to unseen generators, robustness to realistic speech-domain distortions, and balanced performance across heterogeneous audio types.

0.64

Sound Matching with a Differentiable Karplus-Strong Algorithm

eess.AS ·2026-10-20 ·Pablo Tablas de Paula, David Marttila, Rodrigo Díaz, Irán Román, Emmanouil Benetos +1 more
published similarity 0.60
Abstract · 8 pages, 4 figures. Audio examples and code: https://ptablasdpaula.github.io/DAFx26-Karplus/

We present a self-supervised, event-based sound matching model using a differentiable extended Karplus-Strong algorithm. To avoid relying on external onset and fundamental frequency detectors, we explore training methodologies combining parameter losses on synthetic data with audio losses. We demonstrate that time-domain fractional delay interpolation provides gradient accuracy comparable to frequency-sampling while avoiding time-aliasing in highly resonant time-varying scenarios. Through systematic gradient analysis, we reveal that standard spectral losses provide no meaningful directional gradients for onset times, heavily degrading joint training. Training exclusively with parameter losses on synthetic data effectively learns fundamental frequency, timbral parameters, and onset times, but struggles to generalise to monophonic studio recordings of plucked guitar. External detectors combined with audio losses generalise best, isolating the model to timbre optimisation. While our Karplus-Strong decoder recovers interpretable parameters and naturally captures the transient characteristics of plucked guitar, Harmonics plus Noise baselines yield higher reconstruction fidelity by most metrics.

0.38