CS Paper Radar

BioSync: Transformer-Based Cross-Modal Fusion for a Multimodal Physiological Digital Biomarker

Seyed Mahmoud Sajjadi Mohammadabadi

cs.AI cs.IR ·submitted 2026-09-03 21:45 UTC · v1
benchmark

Abstract

Cardiac, neural, behavioral, and speech measurements from wearable and mobile devices provide partial, noise-sensitive views of physiological state. BioSync combines these measurements into the \textbf{BioSync Index (BSI)}, a continuous composite digital biomarker defined under the BEST framework. The model applies multi-head self-attention to modality tokens and adds a linear branch whose hypothesis class includes standard feature concatenation. This architecture is motivated by latent-variable measurement theory and by the possibility that joint observations contain information unavailable from individual modalities. We evaluated BioSync on two literature-informed synthetic cohorts: a four-modality cognitive-decline cohort using HRV, EEG, actigraphy, and speech, and a metabolic-autonomic cohort structured around the public AI-READI wearable schema. In the cognitive cohort, BioSync and concatenation obtained AUCs of 0.928 and 0.926, respectively. In the metabolic cohort, BioSync obtained accuracy/F1 of 0.764/0.766, compared with 0.756/0.758 for concatenation. The BSI correlated with latent severity in both cohorts ($r=0.91$ and $r=0.68$). A pure-attention ablation obtained cognitive-cohort AUC 0.911, locating the increase to 0.928 in the combined wide-and-deep architecture. With matched modality-dropout training, BioSync led concatenation at five of six cognitive-cohort corruption rates and at the highest metabolic-cohort rate. Its cognitive-cohort AUC was also higher than five published digital-biomarker reference values, although differences in datasets and tasks preclude a controlled benchmark claim. Comparison with single-modality, early-fusion, and late-fusion designs across six prespecified criteria identifies the model's computational properties; validation on real cohorts remains necessary.

Closest papers

Nearest neighbours by meaning across the whole index.

WearableQA: A Benchmark for Health Reasoning over Real-World Wearable Data

cs.CL ·2026-09-04 ·Ji Soo Lee, Xilun Chen, Pierce Chuang, Ashish Shenoy, Jason Wei +3 more
benchmarkcode/data similarity 0.67
Abstract

Recent advances in wearable sensing enable continuous monitoring of physiological and behavioral signals, yet existing benchmarks rarely evaluate whether AI systems can reason over a real user's longitudinal wearable record. We introduce WearableQA, a benchmark comprising 4,084 10-option multiple-choice questions constructed from the wearable time series, blood biomarkers, and demographics of 200 real users, each with up to 500 days of daily measurements. WearableQA preserves authentic wearable distributions that include device noise and inter-individual variability. To evaluate distinct reasoning capabilities, we introduce 16 question types organized along two complementary axes: data versus health reasoning, which distinguishes computation over longitudinal measurements from physiological interpretation; and single- versus cross-signal reasoning, which separates reasoning about individual signals from the integration of multiple signals. To construct reliable questions at scale, we adopt a dual-grounding framework that combines literature-grounded physiological findings with statistically validated population-grounded physiological patterns. This enables the capture of meaningful relationships observed in real-world wearable data. Evaluation of 14 proprietary and open-source LLMs demonstrates that WearableQA effectively differentiates model capabilities, with performance ranging from 19.6% to 72.9% against a 10% chance baseline. Moreover, WearableQA remains far from solved: most models achieve accuracies below 60%. Overall, WearableQA provides a realistic and diagnostic benchmark for evaluating LLM reasoning over real-world wearable data.

0.18

Towards clinical adoption of voice and speech as measures of health: the need for harmonization

cs.ET cs.SD ·2026-09-24 ·Nicholas Cummins, Vikram Ramanarayanan, Daniel Low, Fabio Catania, Si-Ioi Ng +7 more
similarity 0.66
Abstract

Speech and voice are multidimensional signals that capture both communicative intent and underlying physiological processes, providing a unique, non-invasive window into health. Analyzing these signals has the potential to yield digital biomarkers that (i) provide scalable, objective measurement tools for research and clinical care and (ii) reflect the presence or progression of diverse conditions, including neurological, psychiatric, respiratory, and cardiovascular disorders. Realizing this promise, however, requires the field to overcome pervasive reproducibility and generalizability issues due to heterogeneous data collection, processing, and analysis practices. A major source of this heterogeneity is how underlying acoustic measures themselves are defined and computed. In this paper, we outline key considerations across the speech biomarker discovery lifecycle, from data collection through machine learning modeling to clinical interpretation, needed to achieve reliable, reproducible, and clinically translatable results. Chief among these is the need for harmonization efforts to start from common, precisely specified measure definitions. As a first step, we therefore provide definitions, physiological correlates, and computational implementations for a minimal, clinically interpretable set of core speech measures spanning respiration, phonation, articulation, and fluency. We close by discussing ongoing standardization efforts and the open challenges that remain in advancing the adoption of speech- and voice-based digital biomarkers.

0.08

PPG-Bench: A Unified Benchmark for PPG Foundation Models

cs.LG ·2026-10-09 ·Arvasu Kulkarni, Mahir Jain, Aditya Ray Mishra, Sandeep Singh, Siddharth Panwar
benchmarkcode/data similarity 0.63
Abstract

Photoplethysmography (PPG) is increasingly recorded by wearable and clinical devices at enormous scale, and a growing number of PPG foundation models (FMs) have been developed to compute reusable representations for various downstream tasks. These models are difficult to compare: different evaluations use different datasets, task definitions, preprocessing, downstream heads, hyperparameter tuning, and baselines. We present PPG-Bench, a unifying open-source benchmark evaluating five public PPG FMs across nine open-access datasets and 26 tasks spanning cardiovascular state estimation, activity, sleep, screening, and identification. We evaluate models under four downstream regimes of increasing adaptability (linear probing, random forests, frozen MLP probes, and end-to-end finetuning), with model-specific hyperparameter optimization and subject-disjoint cross-validation. We also evaluate against three controls: random/constant baseline, a deterministic pulse-rate-variability (PRV) feature set, and a generic time-series FM (MOMENT). Across all 68 task-regime combinations, 35.3% show no meaningful FM advantage over the random/constant baseline, and in 83.8% no PPG-specific FM outperforms both PRV and MOMENT. We also find that added downstream head capacity generally improves performance, but can cause significant degradation if the model is performing at chance/constant level. Current PPG FMs therefore cannot yet claim to provide generalised and transferrable representations of PPG signals, and future evaluation in this field should include checks against simple, non-learned controls. Code for the benchmark and instructions to evaluate more models can be found here - https://github.com/mannas-ai/ppg-bench.

0.31

TAMI: Temporally Aligned, Missingness-Aware, and Interpretable Multimodal Fusion for Mental Health Assessment in Older Adults with Mild Cognitive Impairment

cs.CV cs.AI ·2026-08-31 ·Merna Bibars, Bolaji Omofojoye, Allan I. Levey, Rachel Hershenberg, Gari D. Clifford +1 more
similarity 0.62
Abstract · 19 pages, submitted to IEEE Transactions on Affective Computing

Depression and anxiety in older adults with Mild Cognitive Impairment (MCI) are frequently underdiagnosed due to limited access to care. Multimodal analysis of remote clinical interviews is a scalable screening approach, but existing methods have three limitations. First, they do not correct temporal misalignment across multimodal features extracted at different resolutions, inducing spurious cross-modal associations. Second, remote recordings exhibit uneven modality dropout, but missing values are often zero-filled, making them indistinguishable from valid near-zero measurements. Finally, they do not jointly attribute predictions to modalities, questions, and interview moments, limiting fine-grained clinical interpretation. We propose a Temporally-Aligned, Missingness-Aware, Interpretable (TAMI) multimodal fusion framework. TAMI aligns speech, language, facial, and physiological features within question-answer segments on a shared timeline, encodes modality-level missingness over time, and conditions fusion on question context. In interviews with 49 older adults with MCI, TAMI achieved area under the receiver operating characteristic curve (AUROC) scores of 0.68 (depression) and 0.69 (anxiety). Fine-grained temporal alignment of multimodal features produced the largest performance gain ($Δ{\geq}0.1$). Multi-level interpretability analysis revealed that depression classification relied on eyegaze and open-ended questions, while anxiety classification depended on eyegaze and head pose, with attribution uniformly distributed across questions. Using only responses to the open-ended questions (5.1min), the depression model achieved an AUROC score of 0.67, which was not significantly different from using the full interview (19min) ($p>0.05$). Our findings support designing interview protocols centered on open-ended questions for depression screening in older adults with MCI.

0.30

SOTER: A Generative Time-Series Foundation Model for Wearable Human Physiological Signals

cs.LG cs.AI ·2026-09-15 ·Fangke Chen, Sirry Chen, Wei Chen, Zhongyu Wei
similarity 0.61
Abstract

Time-series foundation models have demonstrated strong cross-domain transfer, yet their common architectural assumptions remain poorly aligned with wearable physiological signals, which are multichannel, irregularly sampled, noisy, and governed by coupled continuous-time dynamics spanning distinct spectral scales. We present SOTER, a generative foundation model for wearable physiological time series that unifies cross-channel coupling, spectrum-guided expert specialization, and continuous-time latent evolution within a single pre-training framework. SOTER combines a spatial feature-aware backbone that models inter-signal dependencies, a power spectral density (PSD)-guided mixture-of-experts layer that routes representations to experts associated with fixed spectral bands through an inspectable, non-learned rule, and a neural controlled differential equation decoder that supports prediction and imputation at arbitrary timestamps. We pre-train SOTER on 226 billion time points from five public physiological datasets and evaluate the same pre-trained model across out-of-distribution zero-shot forecasting, frozen-encoder linear-probe classification, and continuous-time imputation on wearable benchmarks. SOTER achieves the best RMSE on 4 of 6 datasets and the best MAE on 5 of 6 in zero-shot forecasting, the highest average Macro-AUROC in classification, and the lowest imputation error on all six datasets at 75% missingness. It further remains robust to additive acquisition noise, matching or surpassing baselines evaluated on clean inputs even under the strongest corruption. These results indicate that domain-specialized foundation models for wearable physiology benefit from jointly modeling channel structure, spectral scale, and continuous-time dynamics.

0.46

MC-TRCM: Observation-Aware Recursive Fusion for Incomplete Mobile and Wearable Mental-Health Feature Views

cs.LG ·2026-10-08 ·Wentao Wang, Lifeng Han, Zining Ren, Hengyu Zhong, Guangyu Zou
publishedsurveycode/data similarity 0.61
Abstract · Accepted to ICONIP 2026. Code is available at https://github.com/Botwwt/MC-TRCM

Public mobile and wearable mental-health datasets often provide summarized feature tables rather than synchronized raw sensor streams. In these releases, each anchor corresponds to a survey or label time and may combine phone or wearable summaries, prior symptom scores, demographics, clinical variables, and source-availability indicators. We propose the Modality-Conditioned Temporal Recursive Context Model (MC-TRCM), which preserves each feature source as a separate token and incorporates missingness as part of the input context. Observed sources are encoded with values and missingness summaries, absent sources use learned absence tokens, dataset and task embeddings condition fusion, and a recursive prediction head refines each output over validation-selected steps. We evaluated MC-TRCM on six predefined endpoints from DepreST-CAT and Prediction of Severity Change-Depression (PSYCHE-D) using participant-level splits and validation-only model selection. MC-TRCM achieved the lowest mean absolute error on DepreST-CAT Patient Health Questionnaire-9 (PHQ-9) and Generalized Anxiety Disorder-7 (GAD-7) severity, improving over the best tabular reference by 0.181 and 0.217 scale points. Classification endpoints showed task-dependent behavior: MC-TRCM matched the best rounded GAD-7 category balanced accuracy, was numerically highest by 0.002 balanced-accuracy points on PSYCHE-D multiclass prediction, and remained close to the strongest references on PHQ-9 category and PSYCHE-D binary prediction. Ablations support Feature-wise Linear Modulation, absence tokens, missingness projections, and recursive refinement, while calibration and feature-source controls characterize endpoint behavior. Our code is available at https://github.com/Botwwt/MC-TRCM.

0.48

NeoTriFuse: Reliability-Aware Multimodal Fusion under Missingness Heterogeneity for Neonatal Mortality Risk Prediction

cs.LG ·2026-08-26 ·Jiyuan Tian, Qincheng Shen, Ye Lin, Yu Gao, Haohui Lu
similarity 0.61
Abstract · ICONIP 2026

Neonatal mortality risk prediction from bedside monitoring data remains challenging due to extreme class imbalance, heterogeneous clinical risk factors, multi-scale temporal dynamics, and substantial missingness. We propose NeoTriFuse, a reliability-aware multimodal fusion framework for missingness-heterogeneous neonatal monitoring data. Unlike conventional multimodal approaches that treat missingness primarily as a preprocessing issue, NeoTriFuse models missingness as an explicit reliability signal that dynamically modulates modality contributions during fusion. The framework integrates static perinatal variables, local-global temporal encoders, and patient-level statistical summaries through reliability-guided gating mechanisms, while jointly optimizing mortality prediction and an auxiliary length-of-stay objective. NeoTriFuse achieves competitive performance, with an F1 score of 0.6736 +/- 0.0216 and an AUROC of 0.9454 +/- 0.0056. Ablation studies indicate that the local-global temporal architecture and patient-level summary branch contribute most substantially to predictive performance, while reliability-aware gating provides additional improvements on threshold-dependent metrics under heterogeneous observation completeness. Sensitivity analyses further suggest stable performance across nearby hyperparameter settings. Overall, the findings support reliability-aware multimodal fusion as a practical approach for neonatal mortality prediction under realistic clinical missingness conditions.

0.17

Evaluating Generalization and Long-Term Stability of Smartwatch-Based Cuffless Blood Pressure Monitoring

eess.SP ·2026-10-29 ·Hongda Huang, Xiao Peng, Xiaoyu Li, Shuailong Tang, Guangpu Zhu +4 more
published similarity 0.61
Abstract · Accepted by the 2026 IEEE EMBS International Conference on Biomedical and Health Informatics (BHI 2026)

Smartwatch-based cuffless blood pressure (BP) monitoring is a promising low-burden solution for long-term cardiovascular management, but practical deployment remains constrained by model complexity, uncertain cross-dataset generalization, and insufficient evidence on long-term tracking stability. This study presents an efficient Signal-Context Gated Fusion Network (SCGBPNet) for cuffless BP estimation from photoplethysmography (PPG) signals. SCGBPNet integrates signal-level representations through self-attention and further incorporates subject-specific contextual information via a compact gated fusion module. On Aurora-BP, SCGBPNet achieved SBP/DBP mean absolute errors (MAEs) of 7.08/4.90 mmHg, compared with 6.44/4.89 mmHg under within-dataset validation on the private smartwatch dataset, indicating a modest within-dataset discrepancy. Cross-dataset transfer from Aurora-BP to the smartwatch dataset increased the MAEs to 8.98/6.47 mmHg, demonstrating substantial cross-dataset degradation. For long-term stability, the proposed method showed low week-to-week performance variation and a low daily decay rate, indicating good stability over a follow-up period of more than one month. Nevertheless, estimation accuracy deteriorated when BP deviated substantially from an individual's baseline. These findings indicate that favorable aggregate MAE alone should not be interpreted as evidence of deployment-ready performance. More systematic and stratified evaluation across dataset settings, BP ranges, and baseline-deviation levels is needed to assess the practical reliability of smartwatch-based cuffless BP monitoring.

0.25