CS Paper Radar

Label-Free Parkinson's Disease Screening from Face and Voice through Mechanistic Interpretability

Jiaheng Su, Yu Sun

cs.LG ·submitted 2026-08-10 00:43 UTC · v1
benchmark

Abstract

Parkinson's disease (PD) is the second most common neurodegenerative disorder. Typical machine learning screening methods require PD labels, but the available data is limited by privacy concerns and the need for expert annotation. We propose a label-free face-plus-voice PD screen built entirely on frozen pretrained encoders--a face-expression Vision Transformer and HuBERT--in which no PD label touches any fit; the reference is training controls only. The voice modality uses a synthetic-dysarthria contrastive activation addition (CAA) direction built from time-stretch and breathy degradation of healthy speech; the face modality uses a k-nearest-neighbor anomaly score to the control embedding cluster. We introduce the alignment principle, a post-hoc analysis showing that a synthetic-degradation CAA detector works when the cosine similarity between the synthetic and real disease directions exceeds zero. Measured on the YouTubePD benchmark, this cosine is +0.37 for voice (CAA works, AUROC 0.765) and -0.48 for face (CAA fails; anomaly succeeds, AUROC 0.751). Equal-weight late fusion reaches AUROC 0.802 (95% CI [0.70,0.89]) with NPV 0.95, supporting a rule-out triage interpretation. An overfitting audit shows the voice detector transfers cleanly, while the face-side--and thus fused--AUROC is potentially optimistic pending external validation.

Closest papers

Nearest neighbours by meaning across the whole index.

Cross-Lingual Parkinson's Disease Severity Assessment Using Pre-trained Speech Embeddings: A Multi-Class Evaluation

eess.AS cs.CLcs.SDeess.SP ·2026-09-16 ·Simon Pals, Cristian Tejedor-Garcia
publishedcode/data similarity 0.67
Abstract · Accepted and published at IEEE SLT 2026 - IEEE Spoken Language Technology 2026. OneVoice-MSD 2026: Multilingual Speech Technologies for Moto…

Parkinson's disease (PD) often manifests through speech impairments, facilitating accessible, non-invasive, and cost-effective severity assessment for early diagnosis and progression tracking. Despite advances in speech foundation models (SFMs), their cross-lingual generalization for PD severity multi-class classification remains underexplored due to limited labeled data, a lack of explainable methods and variability across languages and datasets. In this work, we evaluate pre-trained embeddings from four state-of-the-art open-source SFMs across three datasets in zero-shot and k-shot cross-lingual settings for multi-class PD severity assessment. Our results show that pre-trained speech embeddings enable meaningful cross-lingual transfer, although performance is sensitive to dataset properties, preprocessing, and adaptation strategy. Misclassifications under these conditions related to inter-speaker variability and atypical speech patterns highlight the need for more robust feature extraction and modeling for PD severity assessment while emphasizing the importance of explainability for reliable clinical insights.

0.40

Language Orthogonalization of Self-Supervised Speech Representations for Cross-lingual Parkinson's Detection

eess.AS ·2026-09-08 ·Minu Kim, Eunjung Yeo, Kwanghee Choi, June-Woo Kim
similarity 0.67
Abstract · IEEE SLT 2026 submission

Self-supervised speech models (S3Ms) provide powerful representations for Parkinson's disease (PD) detection, making cross-lingual transfer attractive for languages lacking labeled patient speech. However, these representations also encode language identity, which can confound this transfer: without target-language PD speech, classifiers may separate languages rather than pathology, yielding high specificity but low sensitivity on target patients. We propose \emph{language orthogonalization}, a closed-form ridge residualization of S3M features against external VoxLingua107 language embeddings, fitted using only healthy-control (HC) speech. By removing language-predictable components while retaining pathology-related variation, it produces a less language-dependent geometry in which HC representations concentrate while PD representations disperse. Across five S3M backbones, three speech tasks, and three target languages, our method consistently improves cross-lingual PD-detection performance while correcting the high-specificity/low-sensitivity failure.

0.44

Exploiting Speech LLM Representations for Multilingual and Cross-Lingual Parkinson's Disease Detection

cs.SD eess.AS ·2026-09-13 ·Sarthak Giri, Zi Haur Pang, Tatsuya Kawahara
hot direction similarity 0.65
Abstract · Accepted at IEEE Spoken Language Technology Workshop (SLT) 2026

Speech Large Language Models (Speech LLMs) have shown strong performance across diverse tasks, yet their utility for pathological speech analysis remains underexplored. In this work, we investigate the effectiveness of internal representations from encoder and decoder components of Speech LLMs for Parkinson's Disease (PD) detection across multilingual and cross-lingual settings. Our findings reveal that encoder representations consistently outperform their decoder counterparts in most models and settings and that pathological cues may be progressively attenuated as audio representations are projected into the language model space. We further show that generative outputs are less reliable for clinical tasks compared to internal representations. To leverage information spread across multiple layers, we propose a Squeeze-and-Excitation (SE)-based dynamic layer aggregation framework, which surpasses best-layer selection in multiple experiments, suggesting that PD-relevant acoustic cues are distributed across transformer layers rather than concentrated in one.

0.27

From Visual Attribution to Clinical Reasoning: Explainable Parkinson's Disease Screening from Hand-Drawn Patterns

cs.CV cs.AI ·2026-09-13 ·Aritra Dey, Utsav Kumar Nareti, Chandranath Adak, Soumi Chattopadhyay, Krishna Gopal Sasmal +1 more
similarity 0.64
Abstract

Parkinson's disease (PD) manifests early neuromotor impairments that become observable in controlled hand-drawn patterns such as spirals and meanders, where tremor-induced oscillations, stroke irregularity, and curvature instability reflect underlying motor degradation. In this work, we present an explainable framework for PD screening from offline hand-drawn patterns that integrates discriminative visual modeling with clinically grounded reasoning. The predictive model captures distributed structural distortions and fine-grained texture variations. It is evaluated under subject-disjoint protocols to ensure reliable generalization. To move beyond black-box classification, we introduce a multi-stage explainability pipeline that combines visual attribution with structured symptom abstraction. Salient regions are identified using attention- and gradient-based localization, followed by extraction of clinically meaningful motor descriptors quantifying contour roughness, curvature irregularity, stroke variability, and tremor-frequency energy. These descriptors are subsequently translated into coherent clinical rationales through a language-based reasoning module, linking model evidence to established PD symptomatology. By bridging visual attribution and clinical interpretation, the proposed framework advances interpretable document intelligence for neurological screening using hand-drawn patterns. Experimental results on publicly available Parkinson's disease handwriting datasets demonstrate competitive predictive performance and clinically consistent explanations.

0.33

AlignUS: MRI-Guided Ultrasound Representation Learning for ALS Classification from Tongue Images

cs.CV ·2026-09-14 ·Kadija Abdel Ghader, Emani Babe, Lorenzo Pettinari, Meya Haroune, Sidaty El Hadramy
similarity 0.61
Abstract

Amyotrophic lateral sclerosis (ALS) is a progressive neurodegenerative disease in which early assessment remains challenging, particularly in low-resource settings where MRI is often unavailable. High-resolution ultrasound (HRUS) of the tongue offers a portable and low-cost alternative for evaluating bulbar involvement, but learning reliable diagnostic models is limited by small datasets and the difficulty of extracting robust representations from ultrasound alone. We propose AlignUS, a cross-modal knowledge distillation framework that transfers anatomical knowledge from MRI to a HRUS-based classifier while requiring only HRUS at inference time. The model combines classification loss, supervised contrastive learning, and feature-level distillation to align HRUS representations with MRI embeddings. AlignUS achieves a patient-level balanced accuracy of 0.958, macro-F1 of 0.963, and ROC-AUC of 0.990, aggregated across four patient-level cross-validation folds, with consistent improvements over HRUS baselines and cross-modal alternatives. These results demonstrate that MRI-derived supervision can substantially improve ultrasound-based ALS assessment while preserving low-cost, inference-time independence from MRI.

0.28

Siam-ALS: An Explainable Siamese Deep Learning Framework for Early ALS Detection Using EEG and Eye-Tracking Signals

cs.LG cs.AI ·2026-10-11 ·Abdur Rahman, A. A. M. Muzahid, Istiak Ahmed, Dipto Biswas, Hua Han +2 more
similarity 0.60
Abstract

Amyotrophic lateral sclerosis (ALS) diagnosis remains challenging due to heterogeneous symptoms and reliance on clinical assessment, underscoring the need for objective, data-driven approaches that leverage patient-generated physiological signals. We propose Siam-ALS, an explainable Siamese deep learning framework that jointly models synchronized electroencephalography (EEG) and eye-tracking (ET) signals. EEG captures ALS-related changes in brain activity, while ET provides information about oculomotor control, allowing the model to learn complementary patterns associated with ALS. The framework employs two modality-specific encoders: a convolutional EEG encoder that captures spatiotemporal neural patterns and an MLP-based ET encoder that models oculomotor dynamics. These modality-specific features are processed through weight-sharing Siamese branches and fused into a compact 64-dimensional joint embedding. A lightweight classifier is used to predict ALS, while pairwise metric learning helps the model learn more effective patterns from the small, diverse ALS datasets. Performance was evaluated on the public EEGET-ALS dataset using strict subject-level cross-validation, where Siam-ALS achieved an accuracy of 0.988 and an AUROC of 0.998, demonstrating strong performance compared with classical machine learning, DL, and non-Siamese fusion approaches. Cross-dataset evaluation on the independent BigP3BCI dataset achieved an AUROC of approximately 0.98, supporting the generalization capability of Siam-ALS across different BCI paradigms. To enhance interpretability, six XAI methods were evaluated in the learned embedding space, with SHAP achieving the highest, though statistically comparable, fidelity in identifying important EEG and ET patterns that contribute to ALS predictions. Overall, Siam-ALS provides an accurate, generalizable, and explainable framework for multimodal ALS detection.

0.17

Advancing Sentence-Level Dysarthric Audio-Visual Speech Recognition via Test-Time Adaptation

cs.SD ·2026-10-16 ·Shiyao Wang, Hui Wang, Jiaming Zhou, Shiwan Zhao, Yong Qin
similarity 0.60
Abstract · Accepted for Poster Presentation at NCMMSC 2026

Dysarthric audio-visual speech recognition (DAVSR) enhances performance by integrating complementary visual cues. However, existing research is largely confined to word-level tasks with limited vocabularies, which severely restricts real-world communication for dysarthric speakers. Furthermore, performance remains heavily dependent on supervised fine-tuning with speaker-specific data; significant variations in age, etiology, and severity cause traditional methods to struggle with unseen speakers. To address these domain shifts and the scarcity of continuous multimodal resources, we first introduce a "source speech search-segmentation" pipeline to construct a structured sentence-level corpus from raw CDSD videos. We then propose the use of unsupervised Test-Time Adaptation (TTA) for sentence-level DAVSR using Sequential-level Generalized Entropy Minimization (SGEM). Our evaluations across the state-of-the-art AV-SUPERB and ViSpeR frameworks demonstrate that TTA consistently alleviates Character Error Rate (CER) degradation for unseen pathological speakers. Paired t-tests verify that these improvements are statistically significant (p < 0.05), highlighting TTA as a promising paradigm for advancing speaker-adaptive assistive speech interfaces.

0.22

Self-Supervised Speech Representations for Cross-Speaker Dysarthria Detection During Awake Craniotomy

cs.LG cs.SDeess.SP ·2026-10-08 ·Kanthila Chinmayi, Abdallah Nassib, Misy Harrison, Panheleux Celine, Saliou Vanessa +2 more
similarity 0.60
Abstract

Detecting intra-operative speech impairment during awake craniotomy is essential for preserving language function. However, automated detection remains challenging because operating-room recordings contain substantial acoustic interference, clinically relevant speech events are rare, and available cohorts are small and heterogeneous across speakers. This study presents a systematic component-wise evaluation of a pipeline for distinguishing dysarthric from no-trouble speech in the DATABRASE corpus of awake-craniotomy recordings. The pipeline incorporates speaker diarization to isolate patient speech, a multi-view representation combining handcrafted acoustic descriptors with multilayer wav2vec 2.0 embeddings, speaker-conditional normalization and transferability-based feature selection to improve cross-speaker robustness, and a cascaded classifier comprising a gradient-boosted first stage and a neural second stage. Evaluation was conducted under strict speaker-independent conditions using leave-one-speaker-out cross-validation. The results show that cross-speaker performance is influenced more strongly by the speech representation than by classifier choice. The AUCs of three classifiers differed by no more than 4.7%, whereas replacing conventional acoustic descriptors with the multilayer self-supervised representation produced AUC improvements of 18.2%-26.1%. Diarization-conditioned feature extraction and the proposed classifier cascade provided additional consistent gains. These findings indicate that reliable patient-specific speech isolation and strong pretrained representations are more important than increased classifier complexity in low-resource intra-operative settings. They also quantify the potential performance gains that may be achieved through patient-specific preoperative calibration.

0.09