CS Paper Radar

Metadata-Conditioned Audio Transformers for Adaptive Respiratory Sound Classification

George Kontogiannis, Pantelis Tzamalis, Sotiris Nikoletseas

cs.SD cs.AIeess.AS ·submitted 2026-11-02 15:53 UTC · v1
publishedbenchmark

Abstract

Respiratory sound classification from auscultation recordings is confounded by heterogeneous acquisition conditions, including recording device, auscultation site, and patient characteristics, that introduce systematic variability unrelated to the underlying pathology. Although such metadata is often available, existing approaches either ignore it or rely on naïve fusion strategies that do not model how distinct metadata sources influence different components of the acoustic representation. We propose a family of metadata-conditioning mechanisms for the Audio Spectrogram Transformer (AST), progressing from gated residual fusion to Feature-wise Linear Modulation (FiLM) adapted to the Transformer architecture. We further introduce two novel variants: Token-Aware FiLM (TAFiLM), which applies distinct modulation to classification and patch tokens, and Soft-Factorized FiLM (SoftFiLM), which learns soft masks to align metadata factors with disentangled feature subspaces. Experiments on the ICBHI benchmark demonstrate consistent gains across all conditioning strategies. SoftFiLM achieves 64.11% on the official 4-class task (+3.94% over the baseline) and 72.40% on the 2-class task, establishing a new state-of-the-art on the 2-class task and competitive performance on the 4-class benchmark.

Journal ref
Proc. 34th European Signal Processing Conference (EUSIPCO), 2026

Closest papers

Nearest neighbours by meaning across the whole index.

Muting Tokens: Exploring Token Sparsification for Efficient Audio Transformers

cs.SD cs.LGeess.AS ·2026-10-22 ·Shreya Sajid, Fabrizio Julian Piva, Hiram Rayo Torres Rodriguez, Elif Ceren Gok Yildirim, Murat Onur Yildirim +1 more
similarity 0.71
Abstract · This work was conducted in 2024; the related work reflects the literature at that time

Transformer-based models have shown strong performance across diverse tasks, but their reliance on self-attention mechanisms often leads to high computational costs. Token sparsification, which reduces the number of tokens processed in transformer layers, offers a promising solution to maintain performance while improving efficiency. However, although widely studied in the vision and language domains, token sparsification remains underexplored in audio, with existing work often limited to a single method, model size, or dataset. This work presents a comprehensive evaluation of token sparsification techniques for audio transformers. We assess a range of token pruning and merging strategies across multiple datasets and model sizes, under both post-training and fine-tuning settings. Furthermore, we introduce a novel sparsification approach that combines attention scores with spectral cues derived from the frequency content of token embeddings. Our experiments show that frequency-aware approaches, particularly those based on spectral flatness and low-frequency energy, consistently outperform attention-only methods. We also find that sparsification based on spectrogram patch intensity is especially effective for longer audio inputs. These results highlight the value of incorporating frequency-domain and input-specific information into sparsification strategies to enhance performance in audio tasks.

0.07

Clinically Informed Acoustic-Aware Attribute Modeling for Respiratory Sound Classification

cs.SD ·2026-10-16 ·Xinhong Li, Li Xiao, Ziyang Chen, Yuhong Yang, Weiping Tu
similarity 0.69
Abstract · Accecpted by BIBM2026 regular paper

Automated respiratory sound classification (RSC) aims to identify normal and adventitious respiratory sounds, such as crackles and wheezes. Most existing methods formulate RSC as a flat four-class classification problem over norma, crackle, wheeze and both. However, unlike conventional classification tasks with semantically independent and mutually exclusive categories, the commonly used four-class RSC labels have a compositional structure and can be represented by two clinically meaningful sound attributes: crackle presence and wheeze presence. Existing flat classifiers do not explicitly exploit this structure, which may increase the learning burden by treating the four labels as independent targets. Meanwhile, crackles and wheezes exhibit distinct temporal characteristics, with sparse transient crackle events and sustained continuous wheeze patterns, making a single temporal aggregation strategy less suitable for modeling different adventitious sound evidence. To address these limitations, we propose an Acoustic-Aware Asymmetric Framework (AAAF) for RSC. AAAF reformulates four-class recognition as two sound-attribute detection tasks and recovers the final category through logical composition during inference. Built upon a pretrained BEATs encoder, AAAF employs branch-specific multi-layer fusion and asymmetric temporal aggregation to capture localized crackle evidence and sustained wheeze patterns. Auxiliary normal--abnormal supervision and a batch-adaptive dynamic loss are further introduced to improve global discrimination and robustness under imbalanced labels. Experiments on ICBHI and SPRSound demonstrate that AAAF improves overall RSC performance and generalizes across datasets, achieving a state-of-the-art four-class Score of 65.30% on ICBHI.

0.24

LAST: Looped Audio Spectrogram Transformer

cs.SD cs.LGeess.AS ·2026-10-01 ·Haider Al-Tahan, Sean O'Brien, Anastasia Razdaibiedina, N. Apurva Ratan Murty
similarity 0.68
Abstract · 6 pages, 4 figures, 1 table

Increasing depth of transformer models improves recognition, but it comes at a substantial cost. Each additional layer requires more parameters, which makes the process computationally inefficient. We ask whether additional processing can focus on integrating features already computed. Looped Audio Spectrogram Transformer (LAST) first processes all tokens, then reuses the same blocks to refine only the class token over fixed audio features, thereby making later passes inexpensive. On AudioSet, ten-pass LAST achieves 0.345 mean average precision, exceeding a twelve-layer sequential transformer by 2.1% relative with 49.4% fewer parameters, 42% fewer multiply-accumulate operations, and 9.8% higher measured throughput. Across separately trained models, increasing the pass count from two to ten improves accuracy while adding only 1.2% computation. Further evaluations show improved robustness to temporal masking and various other auditory augmentations, with better generalization on classification tasks with music, environmental, and event sounds.

0.07

Listening for Airway Stenosis: A Foundation Model-Based Method for Rapid and Accessible Detection

cs.SD ·2026-09-14 ·Jean Groeninger, Zihao Zhao, Juliana de Castilhos, Sven Nebelung, Daniel Truhn
code/data similarity 0.66
Abstract · Jean Groeninger and Zihao Zhao contributed equally. Code is available at https://github.com/JeanGrng/listening-for-airway-stenosis

Airway stenosis can cause severe respiratory complications, yet its detection often relies on specialized examinations and medical imaging. This study explores the potential of acoustic AI for rapid and accessible airway stenosis detection using readily acquired patient voice recordings. We systematically investigate whether acoustic foundation models (AFMs) can extract acoustic representations associated with airway stenosis-related speech patterns. Experiments are conducted on a cohort of 748 participants from the Bridge2AI-Voice dataset, 134 with airway stenosis and 614 without. The best-performing model achieves an AUROC of 0.952 and an accuracy of 0.924 (means over five-fold cross-validation), highlighting the potential of AFMs to transfer beyond general-purpose speech applications to clinical diagnostic tasks. Further analysis reveals that the model primarily relies on connected-speech recordings rather than isolated acoustic tasks, such as sustained phonation and breathing. Overall, these results suggest that voice-based acoustic AI could complement existing diagnostic workflows by enabling rapid, low-burden, and widely accessible screening for airway stenosis.

0.43

Feature Scaling Beats Architecture Choice in Offline Respiratory Screening for Low-Resource Clinics

cs.LG cs.AIcs.SD ·2026-10-12 ·Abubakar Chilala, Cleophas Kadima, Innocent Anyaele, Victor Olufemi
similarity 0.66
Abstract

Deep models are the usual choice for respiratory sound classification, but in clinics without GPUs or connectivity that choice carries a cost. We compare classical and deep classifiers on a five-class respiratory-sound corpus of 1,211 clips (asthma, COPD, pneumonia, Bronchial, healthy) from 201 patient groups, with a 3.86:1 class imbalance, using pre-registered patient-grouped cross-validation (25 runs per configuration). An SVM over 30 summary features (MFCC, ZCR and RMS statistics) reaches macro recall 0.78 and 0.71 recall on the rarest class. A 1.13M-parameter convolutional recurrent network over log-Mel spectrograms reaches 0.75 to 0.77, and no difference between model families is resolved. The largest single effect we measure is a preprocessing effect rather than an architectural one. Standardizing the SVM's features, a routine step for RBF kernels, moves its macro recall from 0.51 to 0.78, a 27-point swing wider than any gap between model families here. Augmentation did not help: SMOTE, waveform augmentation and SpecAugment have no measurable effect. Latency does not separate the models (51 to 74 ms per clip on a CPU-only laptop), but the classical models avoid a 449 MB PyTorch runtime. Clip-level splits, which let a patient's recordings fall on both sides, inflate macro recall by 7.5 to 9.8 points. Recording source is strongly tied to label (Cramer's V = 0.65 between native sampling rate and class), and in an exploratory analysis clips recorded at a class's minority rate are mostly assigned to classes typical of that rate; we state this confound as the paper's main limitation.

0.13

Zero-Shot Respiratory Sound Classification through LLM-Augmented Audio-Text Alignment

cs.CL cs.AIcs.SD ·2026-08-30 ·Mustafa Talha İlerisoy, Hung Manh Pham, Mathias Funk, Mykola Pechenizkiy, Aaqib Saeed
top venue similarity 0.66
Abstract · Accepted to INTERSPEECH 2026

Self-supervised respiratory encoders lack semantic grounding in clinical domain needed for zero-shot inference, limiting their utility without task-specific labeled data. We propose a framework that aligns these encoders with medical terminology in a shared latent space turning them into a zero-shot-capable foundation model. To address paired data scarcity, we use a medical LLM to synthesize structured reports from metadata, creating dense semantic anchors for contrastive learning. Our training combines a sigmoid-based contrastive loss with encoder's native SSL objective and similarity-aware negative sampling to sharpen pathological boundaries. Across 9 tasks on 6 datasets, our method achieves a 61.3% mean zero-shot AUC, surpassing CLAP (51.4%) and Qwen2-Audio (54.9%) while reaching the highest linear probing AUC (71.6%) with only 43% of data used by full-scale baselines, showing that structured semantic alignment outperforms large-scale, general-purpose models in clinical diagnostics.

0.54

RespCCT and FreqCCT: Compact, Explainable and Cost-Aware Convolutional Transformers for Multi-Class Respiratory Disease Classification

cs.SD cs.LG ·2026-11-18 ·Auntor Chakma, Mst. Farhana Akter, Abu Henaf Rashid Ahmad Mufti, Asmana Yasmin
similarity 0.65
Abstract · 22 pages. Manuscript to be submitted to Biomedical Signal Processing and Control

Chest auscultation depends on the listener, and software that classifies lung sounds could give a second reading where trained listeners are scarce. The most accurate published classifiers fine-tune large pretrained networks, and their accuracy is rarely reported with running cost or paired statistics. This study makes three contributions. (i) It proposes RespCCT and FreqCCT, two compact convolutional transformers of 173,187 and 124,999 parameters that learn from random initialization on mel, constant-Q and cepstral images of 5 s frames from three public databases, ICBHI, KAUH and RD@TR, with 1,665 audio files in seven classes. (ii) It compares both models with five ImageNet-pretrained networks and larger distilled variants on one test set, using paired bootstrap intervals. (iii) It measures the cost of every pipeline stage on one CPU thread, tests whether attribution maps depend on the trained weights, and probes whether the learned features reveal the recording source. On the test fold, RespCCT reaches an accuracy of 98.13% and a macro F1 of 0.970, and FreqCCT reaches 97.58% and 0.951. Over five folds, RespCCT with three seeds reaches 97.65% and 0.959, and FreqCCT with one seed reaches 96.95% and 0.941. The two strongest pretrained networks reach a macro F1 of 0.984, whereas RespCCT needs 2.0 to 64.6 times less weight storage. Computing the three input images takes longer than running either model, and a linear probe identifies the source database with 0.958 accuracy. A team can weigh accuracy against storage and time before device testing.

0.17

RespFocus: Learning Temporal Evidence for Weakly Supervised Respiratory Sound Classification

cs.LG cs.AI ·2026-10-11 ·Yi Tian, Xiaohuan Chen, Yufei tang, Xinyu Lin, Hongliang Li +3 more
similarity 0.65
Abstract · 9 pages, 5 figures,2 tables

Respiratory auscultation provides a low-cost means of assessing pulmonary function, yet transient crackles and sustained wheezes can be difficult to recognize in noisy clinical recordings. Conventional deep learning classifiers often operate as black boxes, offering limited insight into the temporal acoustic evidence supporting their predictions. We present RespFocus, a weakly supervised framework that uses multiple instance learning (MIL) to learn inspectable temporal evidence from cycle-level labels and integrate it into classification. We also introduce PedResp2026, a multicenter pediatric respiratory sound cohort collected in routine clinical environments with adjudicated cycle-level annotations. In classification experiments, RespFocus achieves four-class Scores of 69.55\% on public ICBHI 2017 and 72.91\% on PedResp2026. Evidence diagnostics and matched temporal interventions support the functional contribution of the learned evidence to model predictions. Together, these findings highlight RespFocus's potential as a tool for automated respiratory sound diagnosis.

0.30