CS Paper Radar

Understanding emotional and behavioural responses through physiological signal analysis recorded during theatrical performance

Aditi Site, Annariina Lohiranta, Tarmo Lipping

eess.SP cs.HC ·submitted 2026-09-08 06:45 UTC · v1

Abstract

Among various cultural interventions, theatre offers a powerful medium for emotional expression, reflection, and social connection. Live performances can positively influence mental well-being. Therefore, studying audience responses during theatrical performances can provide valuable insights into the relationship between theatre, emotional experiences, and well-being. Physiological responses have emerged as an important modality for identifying patterns associated with different emotional states. However, in naturalistic environments such as live performances, analysing these signals is challenging due to the absence of explicit labels. In this study, we focus on understanding emotional and behavioural responses using physiological data collected during a live theatrical performance. Data were recorded from participants using multiple sensing modalities. In addition, a comprehensive analytical framework was employed, which included demographic analysis, visual exploration of physiological signals, participant-level observations, scene-wise physiological analysis, and intensity-based analysis using z-score normalization to summarize the findings. The findings revealed that dramatic, surprising, and intense scenes produced the strongest activation, often with concurrent increases in HR and SCL, whereas empathy-driven scenes showed lower and more stable responses. Humorous scenes elicited moderate-to-high activation, particularly in the presence of sudden reactions or loud sounds. Analyses also identified shared physiological response patterns across participants, with activation concentrated around key emotional events. These results indicate that combined HR and SCL measures can effectively capture variations in audience engagement during live theatrical performances.

Closest papers

Nearest neighbours by meaning across the whole index.

EmoTrace-Bench: Diagnosing Emotional Expression in Text-to-Video Generation through Controlled Cues and Intensity

cs.CV ·2026-10-11 ·Yeheng Wang, Liudan Shi, Yi Zheng, Jingke Li, Shiyuan Liu +3 more
benchmarkcode/data similarity 0.58
Abstract · 30 pages. Code and data: https://github.com/Tonyheng6/EmoTrace-Bench

Text-to-video (T2V) models are increasingly used to generate videos with targeted emotional expressions, yet existing evaluation methods mainly rely on holistic video-text alignment and provide limited insight into whether failures arise from target-emotion expression, intensity control, or specific affective cues. We introduce EmoTrace-Bench, a fine-grained diagnostic benchmark that controls emotion intensity and selected affective cues in prompts and evaluates generated videos at multiple levels. EmoTrace-Bench contains 448 controlled prompts covering eight emotions, two intensity levels, and diverse cue conditions. Using 12 T2V models, we generate 5,376 videos and evaluate target-emotion presence, perceived intensity, and ten audiovisual dimensions of emotional expression with an automated evaluator calibrated and validated against human annotations on a subset of the data. Results reveal substantial differences across models and prompt designs. Prompts that explicitly describe a character's emotional reaction are followed more reliably than prompts that describe only events or scenes, whereas adding more affective cues or using more explicit cinematic audiovisual language does not consistently improve emotional generation. These results illustrate how aggregate alignment scores may overlook localized differences in target-emotion expression, intensity control, and cue realization. EmoTrace-Bench provides a traceable and interpretable framework for diagnosing emotional video generation without modifying existing generation pipelines, and we release the prompts and the automated evaluation rules.

0.35

Reconstruction and Reflection of Positive Experiences through Resurfacing Laughter-indexed Everyday Moments

cs.HC ·2026-09-11 ·Jun Fang, Jiajin Li, Yuntao Wang, Kexin Miao, Susu Wang +3 more
similarity 0.58
Abstract

Positive everyday moments often escape deliberate recording, while continuous self-tracking can generate extensive records that are difficult to revisit. We explore laughter as a naturally occurring, sparse index for constructing contextualized personal records to support later reconstruction and reflection. A formative study with 12 participants characterized laughter as an affective but semantically incomplete index and informed \textit{LaughAnchor}, a mobile and wearable self-tracking system. During participant-initiated recording, the system assembles detected laughter and aligned context into candidate moments for later reconstruction and reflection, with layered context disclosure, user-controlled curation, and near-term and long-term resurfacing. In a three-week field deployment with 12 participants, passive indexing preserved moments they considered unlikely to record deliberately but valued retrospectively. During resurfacing, participants attributed affective re-experiencing to laughter and used additional context both to reconstruct episodes and to explore already-recalled experiences. Across moments and reviews, resurfacing supported rediscovery and broader awareness of relationships, routines, and emotional states. These findings inform self-tracking designs that use sparse affective indices to organize contextual records for reconstruction and reflection, while keeping interpretation and retention under user control.

0.25

PAIR: Perceptual Affective Inference and Regulation in a Real-Time Multimodal Conversational Agent

cs.HC ·2026-10-05 ·Kexin Quan, Zijian Ding, Jiaye Yong, Qinshi Zhang, Dong Wang +1 more
similarity 0.58
Abstract

Sustained emotional support requires generative agents to connect momentary emotion inference and regulation with continuity across encounters. We present PAIR (Perceptual Affective Inference and Regulation), a real-time multimodal agent that reconstructs how an event is appraised into an emotional state. Appraisal scaffolds produce a valence-arousal-dominance estimate and select regulation guidance, delivered through conversation with coordinated speech, color, and avatar cues. Rolling memory carries context across sessions, and the scaffold re-runs after guidance. In a 14-day deployment with 19 participants, 1,093 sessions paired initial and post-guidance estimates with unanchored self-reports. Initial valence reached MAE 1.20 on the 9-point SAM scale (r=.68), dominance reached MAE 1.30, and arousal showed weak agreement even after coarsening. Self-reported emotional change varied with initial state, with the largest valence increases in sessions that began at negative valence. Perceived understanding was associated with greater valence increase and showed little correspondence with numerical prediction error. Over two weeks, helpfulness increased while input shortened; interviews traced personalization and companionship to relevant recall, context updates, and familiar dialogue. These findings connect inference accuracy to conversational and temporal patterns of support through per-event, first-person evaluation.

0.19

ReactionBench: A Human-Anchored Benchmark for Modeling Reactions to Video

cs.CV cs.AI ·2026-10-11 ·Yibo Zhao, Ao Qu, Xuan Jiang, Jiahao Xu, Keane Ong +7 more
publishedbenchmarkcode/data similarity 0.57
Abstract · Accepted at the NeurIPS 2026 Workshop on Evaluation of Interactive Agents (IAEval). 49 pages (10 main text), 9 figures, 46 tables. Project p…

Understanding what happens in a video is not sufficient for socially responsive multimodal systems: such systems also aim to relate what a person observes to how that person responds. Existing video benchmarks largely evaluate events, temporal reasoning, or people depicted in the video, while affective and viewer-response datasets often focus on emotion labels, aggregate responses, or prediction from the stimulus alone. This leaves an underexplored question: can current multimodal models ground an observed viewer reaction in the stimulus context associated with it? We introduce ReactionBench, a human-anchored benchmark for reaction-specific video understanding. Built from a candidate pool of 31,834 synchronized stimulus-reaction clips spanning 433 hours and eight domains, ReactionBench evaluates three complementary abilities: ranking the intensity of a viewer's reactions within a video, matching a reaction to its temporally aligned stimulus, and explaining the concrete stimulus cues linked to that reaction. Human annotation provides 14,894 ratings over 3,257 moments, and the frozen evaluation comprises 250 intensity profiles, 112 matching items, and 200 rationale items with 447 blind-written human references. Across seven multimodal models, reaction-specific grounding remains challenging: the best model reaches 50.9% accuracy on four-way stimulus-reaction matching, while GPT-5 shows no detectable loss under reaction substitution. Performance is also sensitive to evaluation protocol. Project page: https://yibozh.github.io/ReactionBench-page/

0.37

Understanding the relationship between retrospective soundscape attributes and continuous pleasantness and eventfulness ratings

eess.AS cs.SD ·2026-11-01 ·Yuanbo Hou, Kening Guo, Francesco Aletta, Woon-Seng Gan, Dick Botteldooren +1 more
unusual similarity 0.57
Abstract

Soundscape studies ask listeners to rate perceived affective quality (PAQ) attributes, such as pleasant and vibrant, after listening to recordings or experiencing acoustic environments. These retrospective ratings summarise an experience during which impressions may change. International Organization for Standardization (ISO) procedures convert these PAQ ratings into Pleasantness and Eventfulness scores. Pleasantness and Eventfulness (P/E) can also be rated directly and continuously during listening, capturing changes in reported perception over time. Retrospective PAQ ratings and continuous direct P/E describe the same soundscape experience from different perspectives. We examine how PAQs relate to direct P/E and how ISO-derived P/E corresponds to direct judgements, using 1,782 paired 30-s trials involving audiovisual recordings from 18 streets. The results show that PAQs retain a predominantly two-dimensional structure. Vibrant and Monotonous lie closer to the Eventfulness-Uneventfulness axis in the direct P/E plane than in the ISO 2025 reference geometry. The full 30-s direct P/E mean has lower calibrated error against ISO-derived P/E than three 10-s means. An empirical mapping using the same 8 PAQs reduces held-out joint root-mean-square error by 4.9% relative to affine-calibrated ISO 2025 scoring. The findings show that retrospective and continuous assessments offer related but distinct soundscape descriptions.

0.27

Cognition-Oriented Emotion Tracing from Causes to Consequences in Real-World Social Scenes

cs.AI ·2026-10-08 ·Hao Li, Jinye Zhang, Bobo Li, Mong-Li Lee, Wynne Hsu +3 more
code/data similarity 0.54
Abstract

Affective computing has progressed from categorical emotion recognition to open-ended affective analysis with large multimodal models. Yet affective science describes emotion as an unfolding process shaped by appraisal, regulation, and social interpretation, which remains underexplored computationally. We propose TRACE, a cognition-oriented framework that formalizes an affective episode through three interrelated stages: Condition, Affect, and Effect, integrating observable cues with cognitive factors such as internal stance and regulation of emotional display. Based on this formulation, TRACE-Bench evaluates multimodal models in real-world social scenes through five tasks spanning grounded affect recognition, regulation decoding, cause reasoning, effect reasoning, and full-chain reconstruction, with 3,746 structured question-answer pairs over 646 videos. A matched human-model comparison reveals a substantial performance gap, while affect-specialized models also generally lag behind general-purpose MLLMs. Model outputs show recurring failures, including treating displayed behavior as genuine feeling and fabricating unsupported events during long-chain generation. We further propose TRACER, a cognition-grounded structured reasoning method that couples each inference with explicit premises from factual observations, cognitive appraisals, and established upstream conclusions, forming a traceable graph of intermediate and target conclusions. TRACER outperforms all evaluated model baselines on each of the five tasks. Project page: https://cogaffc.github.io/TRACE

0.28

PI-MLLM: Psychophysics-Inspired Multimodal Large Language Model for Oddly Satisfying Video Emotion Analysis

cs.CV cs.AI ·2026-10-20 ·Yuheng Zhang, Jie Zhang, Yuecong Min, Yunwei Zhao, Meichen Liu +2 more
dataset similarity 0.54
Abstract · 12 pages, 5 figures, 8 tables; includes supplementary material

Oddly Satisfying Videos (OSVs) can evoke strong viewer responses through repetitive motion, visual tactility, and ASMR-like sounds despite containing few explicit semantic emotion cues. This makes OSVs a challenging setting for Video Emotion Analysis (VEA), since most existing methods focus on semantic video content and under-model the physical sensory dynamics that shape viewer responses. To support Oddly Satisfying Video Emotion Analysis (OSVEA), we construct the Oddly Satisfying Video Dataset (OSVD), comprising 12,000 annotated 5-second clips. Each clip provides viewer-emotion labels under video-only, audio-only, and audio-visual conditions, together with explanations grounded in visual and acoustic sensory evidence. Building on OSVD, we propose PI-MLLM, a psychophysics-inspired multimodal large language model that integrates Modality-Specific Dynamics Alignment (MSDA) and Coalition-Based Audio-Visual Attribution (CAVA). MSDA uses auxiliary prediction objectives to align modality representations with visual motion and acoustic energy-onset dynamics. CAVA evaluates emotion scores across modality coalitions and applies Shapley attribution to identify the dominant modality and determine whether visual and auditory evidence is synergistic or conflicting. Experiments show that PI-MLLM improves both viewer-side emotion prediction and sensory-grounded explanation quality, suggesting the value of physical sensory cues to video emotion analysis.

0.23

Slide to Report: Continuous Reporting Symptom-Specific Cybersickness in VR

cs.HC ·2026-11-05 ·Purnata Saha, Rifatul Islam, Qiliang He, John Quarles
similarity 0.53
Abstract

Cybersickness is often assessed during VR exposure through discrete overall sickness, which cannot capture specific symptoms or their changes over time. Post-exposure questionnaires such as the Simulator Sickness Questionnaire provide symptom-specific measures but cannot be collected during immersion. We investigate whether separating cybersickness into three symptom-specific dimensions -- nausea, oculomotor discomfort, and disorientation -- during immersion provides more informative continuous measurements. We conducted a between-subjects study with 52 participants in a VR-rollercoaster, comparing a three-slider interface with a single-slider interface. Concurrently, participants also completed an auditory Simon task to measure attention. We additionally collected multimodal set of physiological signals, including GSR, ECG, PPG, eye and head tracking. Results show that each slider correlated most strongly with its corresponding SSQ subscale, whereas the single slider correlated with all SSQ subscales. Using three sliders did not significantly affect usability, workload, or attention. We publicly release the multimodal dataset to support future cybersickness research.

0.20