CS Paper Radar

PADP: Perceptual Audio Data Perturbation for Probing Perception Awareness in Audio Quality Models

Guanxin Jiang, Andreas Brendel, Pablo M. Delgado, Jürgen Herre

eess.AS ·submitted 2026-10-01 10:08 UTC · v1

Abstract

This paper presents a collection of audio transforms that introduce perceptually irrelevant distortions and demonstrates their use as perceptual stress tests for audio quality models. We refer to these methods as Perceptual Audio Data Perturbation (PADP). PADP exploits the insensitivity of the human auditory system to certain fine-grained signal variations to substantially alter the waveform while preserving perceived audio quality and content. The audibility of PADP and the selection of its parameters are evaluated through controlled listening tests, ensuring that the transformations achieve transparent or near-transparent quality for both non-critical and critical items. We further probe the robustness of state-of-the-art (SOTA) perception-motivated objective audio quality models and foundation models. The results reveal a misalignment between model responses and human auditory perception, highlighting the limited perceptual awareness of these models for certain proposed transforms.

Comment
5 pages, 3 figures

Closest papers

Nearest neighbours by meaning across the whole index.

ODAQ-Collection: Expanding the Open Dataset of Audio Quality with a Focus on Spatial Distortions

eess.AS ·2026-10-27 ·Chih-Wei Wu, Matteo Torcoli, Mhd Modar Halimeh, Pablo M. Delgado, Sascha Dick +3 more
code/data similarity 0.67
Abstract · 5 pages, 3 figures, 2 tables

The Open Dataset of Audio Quality (ODAQ) is an expanding resource that addresses the scarcity of openly available data in audio quality assessment research. While previous contributions focused mainly on timbral distortions, with spatial processing limited to basic mid-side and left-right stereo conditions, we present a new contribution featuring high-quality music and speech recordings with diverse stereo images, controlled spatial distortions, and subjective listening test results from trained listeners in a laboratory setting. With 148 pairs of stereo audio samples processed by three different method classes, two applying controlled binaural cue distortions and one applying dialogue enhancement, this addition expands the collection to 688 pairs of processed and unprocessed audio signals spanning 17 processing method classes, along with 18,984 raw subjective ratings from listeners. All associated data and code are released as open resources to facilitate future expansions and research on objective audio quality metrics.

0.41

Auditory Illusion Benchmark for Large Audio Language Models

cs.SD cs.AI ·2026-09-02 ·Hayoon Kim, Eunice Hong, Kyogu Lee
publishedbenchmarkcode/data similarity 0.66
Abstract · Accepted to the 2026 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2026)

Perceptual illusions have long served as crucial probes into human cognition, revealing biases and limitations of perception. In the auditory domain, such illusions provide a unique lens for testing whether Large Audio Language Models (LALMs) replicate human perceptual tendencies. Despite their importance, most benchmarks focus on visual illusions or general audio tasks, leaving auditory illusions underexplored. To this end, we present AIB, the first auditory illusion benchmark for LALMs, covering ten representative illusions across music, sound, and speech, each annotated for the presence of knowledge-based priors. Our methodology pairs model evaluation with controlled human listening studies, enabling direct comparison of responses. Results show systematic differences: while most LALMs remain signal-faithful on low-level acoustic illusions, several exhibit more human-like responses when linguistic or musical priors are involved, although no model matches the human perceptual profile. These findings highlight the current limitations of LALMs as cognitive models. By establishing auditory illusions as a rigorous testbed, our work offers a new perspective for probing neural black-box models and advancing understanding of auditory cognition. AIB is publicly available at https://github.com/gillosae/aib.

0.47

Perceptual Quality Loss or Loss of Perceptual Quality?

eess.AS cs.LG ·2026-09-28 ·Danilo de Oliveira, Tal Peer, Maurício do V. M. da Costa, Timo Gerkmann
similarity 0.64
Abstract · Submitted to ICASSP 27

Contemporary deep speech enhancement (SE) models are often trained with specific auxiliary terms in the loss function as a way to improve their performance in terms of perceptual metrics. Nevertheless, a higher score on a perceptual metric does not necessarily correlate with an improved listening experience. Through objective and subjective experiments, we assess the performance of SE models trained with two different types of auxiliary PESQ loss terms. The numerical evaluation on a suite of standard metrics suggests that, while models optimized for PESQ naturally obtain higher PESQ scores in the test set, for most other metrics the scores do not significantly change. In some cases, the PESQ loss even results in worse PESQ scores on mismatched data. A formal listening experiment reveals that the models without a PESQ loss were generally preferred over models that include it, across all settings. Finally, we analyze the relative importance of PESQ in the composite metrics CSIG, CBAK and COVL, and find that PESQ dominates all of them. Our study highlights the perils of over-reliance on PESQ and stresses the importance of a complete evaluation procedure for SE.

0.14

PRISM-Bench: An Audio-Centric Diagnostic Benchmark for Text-to-Audio-Video Generation

cs.MM cs.AIcs.SD ·2026-09-04 ·Yuchen Sun, Qian Yang, Jun Wang, Detai Xin, Guoqiao Yu +2 more
top venuebenchmarkcode/data similarity 0.63
Abstract · 19 pages, 10 figures, 4 tables. Accepted at ACM Multimedia 2026 (MM '26). This arXiv version includes supplementary appendices not included …

Text-to-audio-video (T2AV) generation has advanced rapidly, but its evaluation still underestimates the audio modality. Existing benchmarks either treat audio as an auxiliary component of video quality or assess it in isolation from audiovisual grounding, making it difficult to diagnose where current systems truly succeed or fail in audio generation. We present PRISM-Bench, the first audio-centric diagnostic benchmark for T2AV generation. Built from a rigorously curated dataset of 900 human-verified samples, PRISM-Bench factorizes audio evaluation along two orthogonal axes: audio type (Speech, Music, and Sound) and sound-source visibility (On-screen vs. Off-screen). It evaluates generated content across four perceptual dimensions (Audio-Visual Coherence, Audio Quality, Audio Expressiveness, and Prompt Following) with 35 fine-grained criteria. To ensure reliable assessment, we adopt an enhanced MLLM-as-a-Judge protocol based on blind, side-by-side comparison against ground-truth references, demonstrating strong alignment (over 70% mean agreement) with human raters. Our evaluation of recent T2AV systems highlights a significant performance gap between frontier and open-source models. Furthermore, we demonstrate that current generation paradigms overfit to perceptual fidelity while struggling with complex grounding and control tasks, particularly in generating music and synchronized On-screen audio.

0.46

Ensemble-Based Perceptual Audio Quality Assessment with Confidence Intervals

eess.AS cs.SD ·2026-10-05 ·Pablo M. Delgado, Andreas Brendel, Konstantin Schmidt, Jürgen Herre
similarity 0.62
Abstract · 5 Pages. 3 Figures. Submitted to 2027 IEEE International Conference on Acoustics, Speech, and Signal Processing

Objective audio quality metrics typically provide point estimates, whereas listening tests yield score distributions from which mean opinion scores (MOS), confidence intervals (CIs), and significance decisions are derived. We propose a lightweight intrusive metric that combines a PEAQ-style perceptual front-end (ITU-R BS.1387) with a bagging ensemble of regressors. Calibration with subjective data aligns ensemble outputs with listener scores. The resulting item-dependent score distributions enable uncertainty assessment, panel-size-matched CIs, and identification of less conclusive predictions. Calibration improves agreement with subjective distributions and CI coverage across all evaluated datasets while preserving MOS accuracy. Using only 11 fixed PEAQ features, low-capacity regressors, and public training data, the method performs comparably to more data-intensive end-to-end approaches. Its output can support uncertainty-aware assessment and target listening tests towards uncertain conditions. The distributions also enable approximate pairwise comparisons, but not yet reliable significance inference.

0.21

Evaluating perceptual transparency of lossy music audio compression with Opus codec

cs.SD ·2026-10-14 ·Benjamin Martin, Romain Hennequin
code/data similarity 0.62
Abstract · This work has been submitted to the IEEE ICASSP 2027 for possible publication. Copyright may be transferred without notice, after which this…

Although the open-source Opus audio codec is widely used for digital audio streaming, its perceptual transparency threshold for high-bitrate music compression remains insufficiently evaluated. To establish this threshold, this study evaluates Opus across a two-phase ITU-R BS.1116-3 double-blind framework, analyzing Difference Grades using a Linear Mixed-Effects Model (LMM) and Two One-Sided Tests (TOST) for statistical equivalence. In a first phase with 21 qualified listeners and 23 music excerpts, 128 kbps achieves statistical in-distinguishability ($p = 0.456$) but narrowly fails strict equivalence to the uncompressed reference ($p = 0.064$). In Phase 2, Best Linear Unbiased Predictors (BLUPs) isolate the 10 most critical listeners and 8 most challenging samples for a stress-test. Under these worst-case conditions, 128 kbps is conclusively detected as degraded ($p = 0.030$), whereas 160 kbps successfully reaches perceptual transparency ($p = 0.747$) and strict statistical equivalence ($p = 0.024$). The study concludes that 160 kbps demonstrates perceptual transparency to uncompressed CD quality in critical listening environments. For reproducibility, we publicly release results for both phases, anonymized subject background info and sample characteristics.

0.31

Is Semantics Enough for Speech Mean Opinion Score Prediction?

cs.SD ·2026-09-03 ·Tianyu Lan, Yufei Shi, Yang Ai, Honghao Sun, Huipeng Du +1 more
publishedtheory similarity 0.61
Abstract · 5 pages, 2 figures, accepted by ISCSLP 2026

Mean Opinion Score (MOS) is the gold standard for evaluating synthesized speech naturalness. However, current automatic MOS predictors are dominated by self-supervised learning (SSL) models that prioritize high-level semantics, potentially compromising their ability to capture critical acoustic details. In this paper, we systematically investigate representations from three paradigms: SSLs, acoustic-only neural audio codecs (NACs), and unified NACs that integrate semantics into reconstruction-based architectures. Extensive benchmarking on the standard BVCC and multiple out-of-domain (OOD) datasets demonstrates that features synergizing semantic understanding with fine-grained acoustic modeling achieve a higher performance upper bound in speech quality assessment. Ultimately, our findings highlight that semantics alone are not enough; a dual focus on semantic content and acoustic fidelity is essential for robust MOS prediction.

0.31

SPHERE: A Multi-Attribute No-Reference metric for Binaural Speech Quality And Spatial Fidelity

cs.SD eess.AS ·2026-10-30 ·Bala Manikya Prasad Puram, Chandan K A Reddy, Artem Dementyev
similarity 0.60
Abstract

In this paper, we introduce SPHERE (Spatial and Perceptual Human Evaluation and Rating Estimator), a multidimensional no-reference perceptual evaluation framework for binaural speech. SPHERE defines nine distinct perceptual and spatial attributes that characterize the immersive listening experience. We detail the crowdsourcing protocol and evaluation template used to collect ground-truth human ratings. To demonstrate the tractability of automating this framework, we evaluate six baseline configurations across single-branch and Siamese architectures utilizing Mel-spectrograms, complex STFTs, and pretrained spatial embeddings (PhaseCoder). Experimental results demonstrate that these baseline estimators achieve strong correlation with human subjective ratings, proving the feasibility of automated multidimensional spatial audio quality assessment.

0.23