CS Paper Radar

Reconciling Kinesthetic Mismatches: A Somatic Alignment Mindset for Musical Body Transformation

Ziyue Piao, Isabelle Cossette, Marcelo M. Wanderley

cs.HC ·submitted 2026-09-02 01:35 UTC · v1
published

Abstract

Mastering musical performance requires precise multisensory coordination, yet learners encounter a kinesthetic mismatch, which is a discrepancy between the internal perception of an action and the actual physiological state of the body. While multisensory Body Transformation Experiences (BTE) provide tools to bridge this gap, existing designs often focus on external correction rather than internal alignment. To address this, we propose the Somatic Alignment Mindset (SAM), a conceptual lens that integrates Taoist philosophy to shift the focus of HCI design from prescriptive feedback toward holistic embodied unity. By positioning technology as a reflective medium, SAM operationalizes the principles of Adaptation, Assessment, and Awareness to reconcile somatic discrepancies and foster deep, self-aligned musical mastery.

Comment
5 pages, 1 figures. Accepted to the CHI 2026 Workshop on Body Transformation Experiences: A Workshop on How to Elicit, Assess and Support them through Multisensory Technology

Closest papers

Nearest neighbours by meaning across the whole index.

Exploring Breathing-Music Coupling: Using the Breathing Mirror for Somatic Reflection in Piano Performance

cs.HC ·2026-09-02 ·Ziyue Piao, Yohei Wada, Isabelle Cossette, Marcelo M. Wanderley, Akira Maezawa
published similarity 0.58
Abstract · Proceedings of the International Conference on New Interfaces for Musical Expression (NIME)

While breathing is essential to living and for sound production in some instruments, for pianists, it is often a hidden and automatic process, making it difficult to analyze or refine. A critical gap exists between data and awareness: while sensors record precise physical metrics, they fail to capture the performer's somatic experience. Conversely, the high cognitive load of performance makes it nearly impossible for musicians to recall their internal states with temporal precision. To address this, we present a system, Breathing Mirror, and associated methodology designed to externalize the pianist's internal somatic experience through three analytical lenses: a Baseline View (synchronized signals), a First-Person View (subjective recall), and an Interpersonal View (collaborative reflection). Through a four-week longitudinal study with a skilled amateur pianist (35 years of experience), we evaluated the system's effectiveness by recording respiratory data using textile-integrated strain sensor belts. The results show that the Breathing Mirror reveals some patterns of breathing-music coupling and identifies critical blind spots where objective data diverges from subjective perception. Furthermore, we propose four somatic themes regarding the link between breathing and musical elements, offering a foundation for future large-scale validation across a broader range of pianists. This work provides a new way to study body signals, transforming breathing from an internal biological function into an articulate expressive parameter.

0.62

Navigating relational agency in AI feedback musicianship: a diffractive dialogue

cs.SD cs.HC ·2026-11-10 ·Yichen Wang, Charles Patrick Martin, David Gunawan, David Cooper
similarity 0.56
Abstract

Musical AI systems behave in non-linear, dynamic ways that exceed the tool-oriented framings inherited from earlier instrument design. Existing frames of control and ensemble interaction do not articulate the entangled, relational agency that emerges when performers work with AI feedback instruments over time. We examine a duet practice with two AI-enabled feedback instruments, Sù and Agentier, in which feedback operates at the inner level of the AI's recurrent processes and the outer loop of musical interaction between performer, instrument, and co-performer. Our work draws on agential realism and a diffractive dialogue methodology to read our independent reflections through one another across five performances, examining how agency is enacted between performers and AI instruments. We find that control and agency are relational and shift across performances; musical textures enact feedback moments of control with performers; reaching a productive feedback environment is itself a shared exploratory act; and performers maintain their instruments through funnelling and injection, developing the skill of navigating agency along a continuous spectrum. Feedback musicianship with AI instruments demands navigation of this relational agency, displacing the leading-versus-responding binary that initially framed our reflections. This work contributes a materially-grounded vocabulary for AI feedback musicianship and a model of human-AI creative collaboration in which mutual agency is worked for through practice rather than designed in.

0.19

Rescuing Performance from the Demo: Co-Designing Drum Gesture Mappings with a Percussionist

cs.HC cs.SDeess.AS ·2026-09-08 ·Jordie Shier, Teresa Pelinski, Charalampos Saitis, Andrew Robertson, Andrew McPherson
published similarity 0.52
Abstract · Accepted for publication at AIMC 2026

Augmenting instruments with sensors and neural network mappings is a well-explored digital musical instrument design approach. While augmentations can create new expressive opportunities, they also exert aesthetic influence and can constrain musicians' gestural language, which, if left unchecked, can lead to technological capture. To examine this, we conducted a study with a professional percussionist, co-developing a gesture mapping toolkit and recording a ten-track album. Drawing on the concept of productive dissonance, our study aimed to hold the musician's aesthetic in tension with technological constraints. This, along with a practice-based reflective approach, supported the development of a continuous gesture recognition method for percussive mapping and surfaced insights into the design process. We identify knowing-when as a form of tacit knowledge that supported productive dissonance, and raise an open question: absent a musician's broader social context, how do we know whether a technology's influence is genuinely supporting their practice?

0.30

Grounding Mindfulness in Embodied Tangibles: A Scoping Review & Theoretical Framework for HCI Design

cs.HC ·2026-08-20 ·Tharaka Sachintha Ratnayake, Samangi Wadinambiarachchi, Sarah Schömbs, Jonathan Eden, Denny Oetomo +1 more
similarity 0.52
Abstract

Embodied and tangible devices are increasingly used to support mindfulness practices across meditation, yoga, and everyday routines. However, existing HCI research lacks a coherent theoretical foundation for explaining how such systems support distinct mindfulness processes and outcomes. First, we report findings from a scoping review of tangible devices (n=65) for mindfulness in HCI based on the mechanisms of action of mindfulness. The review found that most systems primarily target attentional regulation and body awareness, while emotion regulation and change in perspective on the self remain comparatively underexplored. Also, the evaluation methods used to assess the effectiveness of tangible systems for mindfulness-related outcomes were found to be fragmented and weakly grounded in theory. Building on these findings, a theoretical framework grounded in the Self-Awareness, Self-Regulation, and Self-Transcendence (S-ART) framework is proposed to explain how tangible systems support different forms of mindfulness practice. Within the framework, we present two complementary models: (1) Embodied Sensory Expansion (ESE) for focused-attention meditation (FAM) and (2) Embodied Sensory Anchoring (ESA) for open-monitoring meditation (OMM). Together, we show how this framework can provide a principled basis for explaining designs, generating hypotheses, and evaluating tangible mindfulness technologies in HCI, while highlighting key gaps for future research.

0.27

Somatosensory Activation and Attentional States in Creative Making

cs.HC ·2026-09-09 ·Katherine Rees
similarity 0.51
Abstract · In Proceedings of The First Reflection in Creative Experience (RiCE) Workshop (RiCE W1) arXiv:2607.24558

The methods for capturing the creative process come with associated tensions around memory recall, articulation, and communication during the act of making, as well as how to record these considerations. This paper has a twofold purpose: first, to offer an example of a mixed methodology, drawn from dance anthropology, sensory ethnography, and design, that applies embodied methods as an alternative for documenting creative making. Specifically, this incorporates the researcher-as-participant and the collation of fieldnotes, embodied knowledge/movement recall, with notation forms, and participant interviews. These are existing methods in dance anthropology; however, using them alongside exploratory prototyping and workshop approaches broadened this work into transdisciplinary practice. Second, it discusses the activation of somatosensory systems through wearable technology and the facilitation of heightened sensory awareness for the practitioner, leading to a subsequent ability to focus on creative decisions linked to reflection and metacognition.

0.28

OJOx: Specification-Conditioned Demonstrations for Embodied AI in Construction

cs.RO ·2026-09-13 ·Mohamed Dawod
similarity 0.49
Abstract · 10 pages, 6 figures. Project website: https://ojox.ai

Large-scale egocentric and whole-body human demonstrations are becoming a primary source of data for embodied intelligence. They record what people perceive and do, but rarely the external specification that gave an action its purpose. In construction that omission is consequential: skilled work is directed at project-specific configurations defined in a design model - configurations not yet present in the environment being observed. A mason's transferable competence is not the geometry of one wall but the ability to realise a new geometry from a specification. We introduce the specification-conditioned demonstration: a synchronised record of the physical state a demonstrator perceives, the intended state supplied to them by an external design, and the behaviour connecting the two. We present OJOx, a capture interface that realises this for construction - delivering design geometry to a headset, anchoring it in the physical workspace, rendering it into a demonstrator's stereo passthrough view, and recording that view synchronously with whole-body and hand motion. We report one fully instrumented session - a 33-component wall laid against a specification that changes while the work proceeds - and check the record against the physical scene through an external camera registered independently of the capture. Recorded sessions remain compatible with existing humanoid retargeting infrastructure and replay onto a Unitree G1 in simulation. The result is a data interface for testing whether embodied policies can learn not merely to imitate demonstrated actions, but to act toward specifications absent from their training experience.

0.31

GAP: Generalized Adaptation Prior for Dexterous Piano Playing

cs.RO ·2026-11-09 ·Jinyi Chen, Hongyi Li, Lei Jiang, Xianwei Liu, Yanhong Liang +4 more
similarity 0.49
Abstract · See video at https://www.bilibili.com/video/BV1hbWx6QE9u

Dexterous hands enable versatile manipulation, but policies trained in simulation often degrade on hardware owing to non-idealities, and the high-dimensional action space makes real-world adaptation costly. Piano playing, which requires precise and coordinated multifinger motion, offers a representative testbed. We propose GAP (Generalized Adaptation Prior for Dexterous Piano Playing), a sample-efficient adaptation framework that learns how to adapt, rather than how to act, during simulation pretraining and transfers this adaptation prior to hardware. Between repeated executions, GAP uses feedback to adjust the actions of the next one: a gradient guidance network learns from paired executions how adjusting actions affects playing performance, and its gradients guide these adjustments. In simulation, GAP yields approximately 2.4 times the F1 scores improvement of the existing fast-adaptation baseline for dexterous piano playing under the same interaction budget. The learned prior also generalizes to musical pieces held out from pretraining. On a physical dexterous hand, the same simulation-pretrained prior improves F1 scores by 3.6%-23% across three pieces of varying difficulty after approximately 7 min of adaptation each. These results suggest a practical route to reduce the real-world interaction needed to deploy dexterous hands.

0.16

Generalized Audio-Driven Synthesis of Precise Drummer Motion

cs.CV cs.GRcs.SD ·2026-08-19 ·Álvaro G. Iñesta, Mattia Ryffel, Amit H. Bermano, Robert W. Sumner, Martin Guay
published similarity 0.49
Abstract · Best Paper Award at the 25th ACM SIGGRAPH / Eurographics Symposium on Computer Animation (SCA 2026). For Supplementary Video, see https://st…

Music-driven character animation enables and enhances transformative applications in entertainment and interactive education. However, synthesizing realistic drumming motion from audio remains challenging due to the inherent tension between high-acceleration dynamics and the need for extreme spatial-temporal precision. Existing approaches, often reliant on motion matching or MIDI input, struggle with generalizing to diverse real-world audio. Moreover, the field lacks standardized evaluation metrics capable of distinguishing precise drumming from noisy motion. In this paper, we introduce a generative diffusion framework featuring a dual-objective loss function that decouples skeletal integrity from drumstick precision, thus enabling centimeter-level stick precision without sacrificing natural body dynamics. Additionally, leveraging our own dataset and data augmentation strategy, the model generalizes to non-curated, in-the-wild audio. To rigorously evaluate performance, we propose two novel metrics: an impact-to-target distance to quantify spatial precision and an audio-motion correlation score to assess temporal alignment. Our quantitative analysis and user studies demonstrate that our system generates high-quality motion that is often indistinguishable from ground-truth performances.

0.28