CS Paper Radar

A Multimodal Autonomic Sensing Framework for Objective Assessment of Patient Responses to Dental Pulp Stimulation

Youngsun Kong, Yubin Choi, Dongjin Song, Dong-Guk Shin, I-Ping Chen, Ki Chon

cs.LG eess.SP ·submitted 2026-09-28 13:18 UTC · v1
unusual

Abstract

Patient responses to dental pulp testing, ranging from no sensation to intense pain, provide important information for assessing pulp status in endodontic diagnosis. However, pain is a subjective sensory and emotional experience that varies considerably across individuals and can be difficult to communicate. We investigated whether complementary autonomic signals could support objective assessment of responses during dental examination. Forty-nine patients underwent cold pulp testing, yielding no-response, mild-response, and intense-response conditions. The framework integrated ECG-derived skin nerve activity (SKNA) and R-R intervals (RRI), together with electrodermal activity (EDA), using temporal convolutional network encoders with attention-based mid-level fusion. Individual baseline signals and subject-level covariates, including anxiety scores and biological sex, were also incorporated. The framework achieved 80.2% balanced accuracy, 75.2% sensitivity, and 85.2% specificity for binary classification of no response versus mild or intense response. For three-class classification, it achieved 60.0% balanced accuracy and a 58.8% macro-averaged F1 score. Ablation and attention-weight analyses indicated that EDA contributed most strongly to model performance, followed by RRI, while SKNA improved balanced accuracy by approximately five percentage points. Age was significantly associated with model performance. These findings support the feasibility of multimodal autonomic sensing for objective, non-invasive assessment of responses to dental pulp stimulation.

Comment
14 pages, 9 figures

Closest papers

Nearest neighbours by meaning across the whole index.

Multimodal BVP and Respiration for Automated Pain Recognition with Hankel-FCN

cs.CV ·2026-11-01 ·Valentina Becchetti, Muhammad Umar Khan, Stefanos Gkikas, Alessandro Giuseppi, Raul Fernandez Rojas
similarity 0.59
Abstract

Automated pain recognition is a fundamental challenge for the advancement of clinical care and empathetic human-computer interaction, due to the subjective and dynamic nature of pain. In this study, we propose a deep learning framework for objective pain assessment based on multimodal physiological signals from the AI4Pain dataset. The approach focuses on Blood Volume Pulse (BVP) and Respiration (RESP), capturing complementary aspects of autonomic responses to nociceptive stimulation. To explicitly model their temporal dynamics, the signals are transformed into structured trajectory representations using Hankel matrix embeddings, enabling the learning of dynamical patterns beyond conventional time-series representations. These embeddings are processed by modality-specific Fully Convolutional Networks, and their outputs are integrated through a late fusion strategy for three-class classification (No Pain, Low Pain, High Pain). Experimental results demonstrate that the proposed method achieves an accuracy of 67.09% and an F1-score of 66.41%, providing competitive performance compared to existing approaches, while relying on no handcrafted feature engineering. Multimodal fusion significantly outperformed either modality alone, and the Hankel embedding showed a trend towards further improvement. Further validation is required to establish generalisability to broader populations and clinical settings.

0.24

MUPA${}^2$S: Multimodal Unified Perception with Asymmetric Attention for Stress Assessment

cs.CV cs.AI ·2026-11-01 ·Stefanos Gkikas, Thomas Kassiotis, Yu Fang, Christian Arzate Cruz, Miao Cheng +6 more
similarity 0.55
Abstract

Stress is a leading contributor to long-term physical and mental health decline. The automatic assessment of stress is a promising approach for continuous monitoring and mitigating human biases. However, the development of automated systems remains challenging due to the heterogeneity of physiological and behavioral signals. This paper presents MUPA${}^2$S, a multimodal framework for automatic stress assessment that combines facial video and ECG signals within a unified perception architecture. Both modalities are mapped into a shared token representation without modality-specific components, and processed through asymmetric attention comprising segment-local cross-attention and global self-attention across segment states. Experiments on a newly collected 58-subject stress dataset, evaluated under a stratified subject-level protocol, show that facial video achieves 69.44% test accuracy, while ECG-only performance remains close to chance under the proposed representation. Unified Video-ECG fusion achieves a test accuracy of 69.58% at approximately half the number of parameters of feature-level fusion, with negligible additional computational cost. Feature-level fusion does not exceed the video-only baseline, while unified input-level integration provides a parameter-efficient mechanism for jointly processing the two modalities.

0.14

From Stress to Affect: Multimodal Deep Learning for Physiological Emotion Recognition Across Wearable Sensor Modalities

cs.LG ·2026-09-17 ·Desta Haileselassie Hagos, Saurav Keshari Aryal, Legand L. Burge
similarity 0.53
Abstract · Under review

Physiological emotion recognition using wearable sensors has important applications in mental health monitoring, affective computing, and human-computer interaction. However, existing studies typically evaluate a single model, sensing configuration, or dataset, limiting our understanding of how these factors influence recognition performance. We present a comparative study of temporal deep learning architectures for physiological emotion recognition using two multimodal wearable datasets: WESAD and EmoWear. Bidirectional long short-term memory (LSTM), temporal convolutional network (TCN), and Transformer models are evaluated under wrist-only, chest-only, and multimodal sensing configurations using participant-independent leave-one-subject-out cross-validation (LOSO-CV). We also investigate soft-voting ensembles, sensor ablation, sampling frequency, and gradient-based saliency. The Transformer achieved the highest multimodal accuracy on WESAD (99.02% +/- 0.51%), whereas the LSTM achieved the best multimodal accuracy on EmoWear for both arousal (91.80% +/- 1.06%) and valence (89.96% +/- 0.36%). These results show that relative architecture performance depends on dataset characteristics rather than one architecture being uniformly superior. Multimodal sensing consistently outperformed wrist-only and chest-only configurations across both datasets. Sampling-frequency analysis showed that 4 Hz provides a practical operating point, with performance comparable to higher frequencies at substantially lower training cost. These findings provide guidance for selecting architectures, sensing modalities, and sampling frequencies for wearable physiological emotion recognition.

0.09

Electrodermal Activity (EDA) for Stress Detection: A Comprehensive Survey and Benchmark

eess.SP ·2026-09-18 ·Gang Liu, Zixiong Ning
surveybenchmark similarity 0.52
Abstract · 19 pages, 5 figures, 7 tables

Electrodermal activity (EDA) is widely used for pervasive stress monitoring because of its sensitivity to sympathetic arousal and its computational efficiency. However, research remains fragmented: studies concentrate on a small subset of available datasets, preprocessing and evaluation protocols are inconsistent, and widely adopted methodological conventions remain unvalidated. Existing reviews summarize these practices without evaluating them under a common protocol. To bridge these gaps, this paper presents a comprehensive survey and benchmark for EDA-based stress detection. We review the technical pipeline from preprocessing to modeling and compile 26 public datasets and 22 representative unimodal studies to inform future dataset selection and methodological design. Under a subject-independent protocol spanning five diverse datasets, we benchmark decomposition, filtering, and channel-input choices alongside a broad range of machine learning (ML) and deep learning (DL) models in both in-domain and cross-domain transfer settings. Our results provide the first multi-dataset validation of cvxEDA, the default decomposition method, show that feature-based ML outperforms end-to-end (E2E) DL, and identify ecological validity as a key factor in cross-domain transfer. We conclude with directions toward robust, data-efficient, and trustworthy deployment.

0.22

Multiscale Temporal-Spectral-Spatial EEG Learning With Modality-Specific and Modality-Invariant Representations for Emotion Recognition

cs.CV ·2026-10-20 ·Jian Zhang, Manrong Liu, Yilei Huang, Wenhua Liu, Xiaohui Yang +1 more
similarity 0.49
Abstract

Multimodal physiological signals provide complementary cues for emotion recognition. They also raise two related challenges. Emotion-related EEG patterns vary across time, frequency bands, electrodes, and brain regions. Different physiological modalities also contain both modality-specific and shared affective information. This paper presents GMSI-Net, an EEG-centered multimodal emotion recognition framework. It has three key components. The temporal-spatial-spectral-region (TSSR) encoder models multiscale temporal-channel patterns, differential entropy features, and local-to-global brain-region relations. The modality-specific representation (MSR) branch uses bidirectional cross-modal attention and learnable gates. It controls information transfer while retaining modality-specific cues. The modality-invariant representation (MIR) branch learns a shared affective representation. An adaptively weighted contrastive loss aligns this representation across modalities. Independent multiscale Conv-Transformer encoders process the peripheral physiological signals. GMSI-Net is evaluated on DEAP and MAHNOB-HCI using within-subject random ten-fold cross-validation. The model reaches 96.21 (1.47) and 96.41 (1.60) for valence and arousal on DEAP. It reaches 94.10 (2.86) and 94.48 (3.77) on MAHNOB-HCI. These results indicate that combining EEG features with complementary physiological information can improve emotion recognition.

0.19

Context-Aware Emotionally Adaptive Voice Assistants: A Multimodal Framework for Empathetic Human-Agent Interaction

cs.HC ·2026-09-14 ·Tapon Kumer Ray, Rajkumar Yesuraj
similarity 0.49
Abstract · 7 pages, 1 figures, 5 tables

Voice-assistant interruptions tend to be intrusive because existing systems fail to consider the affective state, cognitive load and situational context of the user when deciding when and how to interrupt.Voice-assistant interruptions tend to be intrusive, since existing systems do not consider the affective state, cognitive load or situational context of the user when determining when and how to interrupt. In this paper, EmpathicVA, a closed-loop framework integrating physiological sensing, vocal-affect analysis, contextual modeling and reinforcementlearning interruption policy, is introduced. A hierarchical fusion model involves integrating HRA, EDA, respiration, acousticprosodic features, linguistic embeddings, and contextual cues and computing the probabilities of five affective states. A Double Deep Q-Network selects immediate response, brief or extended delay, empathetic response, or silent mode based on these probabilities, context and interaction history. The multimodal model obtained an accuracy of 92.3% and an F1-score of 0.922 at the macro level on a held-out test set, outperforming the highest accuracy unimodal model by 6.0 percentage points. Comparing the six-week within-subject field study with 48 participants with a baseline and context-only assistants, there was a corresponding increase in satisfaction, trust, and appropriateness of timing, as well as a large reduction in interruption-related stress episodes. The results suggest that affect-aware timing and restraint are both important in voice interaction in addition to the response wording.

0.27

A Multi-Scale Temporal Framework with Dynamic Fusion for EEG-Based Emotion Recognition

cs.AI cs.LGcs.SD ·2026-08-10 ·Stefanos Gkikas, Yang Guo, Guangliang Li, Raul Fernandez Rojas, Giorgos Giannakakis +1 more
top venue similarity 0.47
Abstract · The paper has been accepted at: IEEE | 2026 9th International Conference on Pattern Recognition and Artificial Intelligence (PRAI 2026)

Mixed emotions represent a clinically relevant but still underexplored target for automatic emotion recognition. EEG provides millisecond-level access to neural activity, yet most EEG pipelines analyze the signal through a single temporal window, thereby fixing the temporal structure available to the model. This study introduces a multi-scale temporal framework for EEG-based emotion recognition. The EEG waveform is decomposed into windows of one or several durations, processed by a shared attention-based encoder, and integrated through a dynamic fusion module that assigns sample-specific weights across temporal scales. The framework is evaluated under a subject-independent protocol in binary and three-class settings, with the three-class task including the mixed affective category. The best results are 65.22% for the two-class task and 45.43% for the three-class task. Both are obtained with three-scale dynamic-fusion configurations and remain substantially above the full-signal baseline. The best-performing temporal scales differ between the two tasks. Dynamic fusion outperforms concatenation in the highest-scoring two-class configuration and slightly exceeds it in the highest-scoring three-class configuration, although these multi-scale settings require substantially more computation than the full-signal baseline.

0.68

On Edge in the Dental Chair: Designing VR Support for Moments of Dental Anxiety

cs.HC ·2026-09-14 ·Zhu Guo, Junjie Zhao, Haofan He, Jiaming Zhang, Mingshi Deng +11 more
similarity 0.47
Abstract · 44 pages, 9 figures, including appendices and references

Dental anxiety can change as a procedure unfolds, yet dental virtual reality (VR) commonly provides continuous distraction or relaxation. We investigate how support can be coordinated with specific simulated dental events. Stakeholder interviews (N=36), participatory design with three returning dentists, and patient walkthroughs of a no-intervention prototype (N=12) informed five Anxiety Events and an intervention-module framework. Drawing on cognitive vulnerability and emotion regulation, we implemented a standardized event-contingent VR system with predefined event-module assignments and shared agency and safety controls. A randomized study (N=24) compared the intervention package with no-intervention VR. The adjusted intervention-minus-control difference averaged -12.83 VAS-A points across events (95% CI [-24.42, -1.70]). Physiological, behavioural, and qualitative measures contextualized participants' experiences. The findings inform timely, comprehensible support and reassuring social presence in simulated dental VR; they concern the complete package rather than individual modules or clinical effectiveness.

0.28