CS Paper Radar

Exploring Breathing-Music Coupling: Using the Breathing Mirror for Somatic Reflection in Piano Performance

Ziyue Piao, Yohei Wada, Isabelle Cossette, Marcelo M. Wanderley, Akira Maezawa

cs.HC ·submitted 2026-09-02 01:16 UTC · v1
published

Abstract

While breathing is essential to living and for sound production in some instruments, for pianists, it is often a hidden and automatic process, making it difficult to analyze or refine. A critical gap exists between data and awareness: while sensors record precise physical metrics, they fail to capture the performer's somatic experience. Conversely, the high cognitive load of performance makes it nearly impossible for musicians to recall their internal states with temporal precision. To address this, we present a system, Breathing Mirror, and associated methodology designed to externalize the pianist's internal somatic experience through three analytical lenses: a Baseline View (synchronized signals), a First-Person View (subjective recall), and an Interpersonal View (collaborative reflection). Through a four-week longitudinal study with a skilled amateur pianist (35 years of experience), we evaluated the system's effectiveness by recording respiratory data using textile-integrated strain sensor belts. The results show that the Breathing Mirror reveals some patterns of breathing-music coupling and identifies critical blind spots where objective data diverges from subjective perception. Furthermore, we propose four somatic themes regarding the link between breathing and musical elements, offering a foundation for future large-scale validation across a broader range of pianists. This work provides a new way to study body signals, transforming breathing from an internal biological function into an articulate expressive parameter.

Comment
Proceedings of the International Conference on New Interfaces for Musical Expression (NIME)
Journal ref
Piao, Z., Wada, Y., Corssette, I., Wanderley, M., & Maezawa, A. (2026). Proceedings of the International Conference on New Interfaces for Musical Expression, 60--69
DOI
10.5281/zenodo.20784052

Closest papers

Nearest neighbours by meaning across the whole index.

The BEATS Dataset: Towards Understanding Music-Physiology Dynamics in Real-World Scenarios

eess.SP cs.AI ·2026-11-11 ·Benjamin Gutierrez Serafin, Tanaya Guha, Fahim Kawsar
dataset similarity 0.59
Abstract · Under review in Nature Scientific Data

We introduce the BEATS dataset, a robust compilation of psycho-physiological data collected in the wild from 30 participants while they listened to curated playlists intended to promote well-being. An acronym for Biomarkers, Experience and Affect from Therapeutic Soundscapes, the BEATS dataset aims to support research on wellbeing, music psychology, affective computing and signal processing. More specifically, this dataset includes: (1) sensor data collected during music listening sessions using a research-grade wearable device in free-movement environments; (2) music listening activity records, stimulus metadata, acoustic features, and self-reports associated with the listening sessions (3) individual characteristics, such as demographics, music preference, musical skills, and personality traits; and (4) self-reported psychological measures before and after their experimental participation spanning for a number of weeks. By situating data collection outside of lab settings, the BEATS dataset offers the opportunity to study real-world music-listening scenarios, allowing for a richer exploration of the physiological, subjective and therapeutic effects of music under real-world conditions.

0.29

ResonaVis: Visualizing Interactive Music Data to Support Reflective Music Composition for Therapeutic Contexts

cs.HC ·2026-08-11 ·Abhishek Karwankar, Elise Ruggiero, Daniel Stevens, Matthew Louis Mauriello
published similarity 0.59
Abstract · 9 pages core, 2 pages acknowledgements + references, 2 pages appendices. Accepted at IEEE VIS 2026, to appear in November 2026

Designing music for therapeutic contexts requires navigating complex relationships between musical structure and listeners' sensory responses, yet composers often lack structured representations of these interactions, relying instead on intuition. We present ResonaVis, an interactive visualization system that helps composers analyze interaction and audio data from prior sessions with children with Autism Spectrum Disorder (ASD), informing future compositions. ResonaVis integrates audio features and interaction logs to capture how children engage with layered musical compositions, representing this engagement through coordinated visualizations of temporal transitions, layer co-occurrence, rhythmic activity, and spectral characteristics. Rather than prescribing strategies or supporting therapy sessions directly, the system surfaces patterns in past session data to support data-informed reflection during composition. We evaluated ResonaVis through a mixed-methods study with eight music students and a follow-up case study with two experienced composers. Results demonstrate good usability (SUS = 72.23), exceeding benchmarks for early prototypes, and show that participants could identify interaction patterns, reason about layer relationships, and make informed compositional decisions with high perceived performance and low frustration. Confidence in interpreting interaction and acoustic data for ASD-focused composition increased significantly across multiple dimensions (p < 0.05), with qualitative findings suggesting a shift toward more adaptive, data-informed composition. This work contributes a visualization design space for therapeutic music interaction data, an integrated system for compositional reflection, and empirical evidence that visualization tools support analytical reasoning and confidence in data-informed creative practice. (Abstract shortened for arXiv.)

0.25

Navigating relational agency in AI feedback musicianship: a diffractive dialogue

cs.SD cs.HC ·2026-11-10 ·Yichen Wang, Charles Patrick Martin, David Gunawan, David Cooper
similarity 0.59
Abstract

Musical AI systems behave in non-linear, dynamic ways that exceed the tool-oriented framings inherited from earlier instrument design. Existing frames of control and ensemble interaction do not articulate the entangled, relational agency that emerges when performers work with AI feedback instruments over time. We examine a duet practice with two AI-enabled feedback instruments, Sù and Agentier, in which feedback operates at the inner level of the AI's recurrent processes and the outer loop of musical interaction between performer, instrument, and co-performer. Our work draws on agential realism and a diffractive dialogue methodology to read our independent reflections through one another across five performances, examining how agency is enacted between performers and AI instruments. We find that control and agency are relational and shift across performances; musical textures enact feedback moments of control with performers; reaching a productive feedback environment is itself a shared exploratory act; and performers maintain their instruments through funnelling and injection, developing the skill of navigating agency along a continuous spectrum. Feedback musicianship with AI instruments demands navigation of this relational agency, displacing the leading-versus-responding binary that initially framed our reflections. This work contributes a materially-grounded vocabulary for AI feedback musicianship and a model of human-AI creative collaboration in which mutual agency is worked for through practice rather than designed in.

0.19

Reconciling Kinesthetic Mismatches: A Somatic Alignment Mindset for Musical Body Transformation

cs.HC ·2026-09-02 ·Ziyue Piao, Isabelle Cossette, Marcelo M. Wanderley
published similarity 0.58
Abstract · 5 pages, 1 figures. Accepted to the CHI 2026 Workshop on Body Transformation Experiences: A Workshop on How to Elicit, Assess and Support th…

Mastering musical performance requires precise multisensory coordination, yet learners encounter a kinesthetic mismatch, which is a discrepancy between the internal perception of an action and the actual physiological state of the body. While multisensory Body Transformation Experiences (BTE) provide tools to bridge this gap, existing designs often focus on external correction rather than internal alignment. To address this, we propose the Somatic Alignment Mindset (SAM), a conceptual lens that integrates Taoist philosophy to shift the focus of HCI design from prescriptive feedback toward holistic embodied unity. By positioning technology as a reflective medium, SAM operationalizes the principles of Adaptation, Assessment, and Awareness to reconcile somatic discrepancies and foster deep, self-aligned musical mastery.

0.67

Rescuing Performance from the Demo: Co-Designing Drum Gesture Mappings with a Percussionist

cs.HC cs.SDeess.AS ·2026-09-08 ·Jordie Shier, Teresa Pelinski, Charalampos Saitis, Andrew Robertson, Andrew McPherson
published similarity 0.57
Abstract · Accepted for publication at AIMC 2026

Augmenting instruments with sensors and neural network mappings is a well-explored digital musical instrument design approach. While augmentations can create new expressive opportunities, they also exert aesthetic influence and can constrain musicians' gestural language, which, if left unchecked, can lead to technological capture. To examine this, we conducted a study with a professional percussionist, co-developing a gesture mapping toolkit and recording a ten-track album. Drawing on the concept of productive dissonance, our study aimed to hold the musician's aesthetic in tension with technological constraints. This, along with a practice-based reflective approach, supported the development of a continuous gesture recognition method for percussive mapping and surfaced insights into the design process. We identify knowing-when as a form of tacit knowledge that supported productive dissonance, and raise an open question: absent a musician's broader social context, how do we know whether a technology's influence is genuinely supporting their practice?

0.30

Generalized Audio-Driven Synthesis of Precise Drummer Motion

cs.CV cs.GRcs.SD ·2026-08-19 ·Álvaro G. Iñesta, Mattia Ryffel, Amit H. Bermano, Robert W. Sumner, Martin Guay
published similarity 0.54
Abstract · Best Paper Award at the 25th ACM SIGGRAPH / Eurographics Symposium on Computer Animation (SCA 2026). For Supplementary Video, see https://st…

Music-driven character animation enables and enhances transformative applications in entertainment and interactive education. However, synthesizing realistic drumming motion from audio remains challenging due to the inherent tension between high-acceleration dynamics and the need for extreme spatial-temporal precision. Existing approaches, often reliant on motion matching or MIDI input, struggle with generalizing to diverse real-world audio. Moreover, the field lacks standardized evaluation metrics capable of distinguishing precise drumming from noisy motion. In this paper, we introduce a generative diffusion framework featuring a dual-objective loss function that decouples skeletal integrity from drumstick precision, thus enabling centimeter-level stick precision without sacrificing natural body dynamics. Additionally, leveraging our own dataset and data augmentation strategy, the model generalizes to non-curated, in-the-wild audio. To rigorously evaluate performance, we propose two novel metrics: an impact-to-target distance to quantify spatial precision and an audio-motion correlation score to assess temporal alignment. Our quantitative analysis and user studies demonstrate that our system generates high-quality motion that is often indistinguishable from ground-truth performances.

0.28

Learning Jazz Pianist Style with Cross-Attention Conditioning

cs.SD cs.LGeess.AS ·2026-10-02 ·Drew Edwards, Akira Maezawa, Simon Dixon
published similarity 0.54
Abstract · 8 pages, 6 figures. Accepted at ISMIR 2026. Audio demos, code and checkpoints: https://almostimplemented.github.io/jazz-pianist-style

Jazz pianists develop distinctive traits that experienced listeners can often identify within seconds, yet the features underlying this recognition resist formal description. We study jazz pianist style through the lens of a pretrained symbolic music transformer, showing that its learned representations already encode pianist identity well enough for highly accurate classification across two benchmarks. We then augment the transformer with cross-attention over learned pianist identity embeddings, enabling it to generate music conditioned on a specific artist's style. Two evaluation protocols confirm that the generator captures meaningful stylistic structure: a sliding-window classifier consistently attributes conditioned continuations to the correct artist, far above unconditioned baselines; and a classifier trained entirely on synthetic generations identifies real pianists across 12 classes with 87% chunk-level and 95% song-level accuracy. Finally, we repurpose the classifier to locate the most characteristic moments within a performance, surfacing the specific musical gestures that distinguish each pianist's voice.

0.39

Resonance and Depth: Composing Timbral Identity and Spatial Form in an Artist-in-the-Loop Co-Creative Multi-Agent Percussive Performance

cs.SD cs.MAcs.MM ·2026-10-21 ·Keon Ju Maverick Lee
published similarity 0.54
Abstract · 4 pages, 1 figure. Proceedings of Timbre 2026, 4th International Conference on Timbre, Montréal, Québec, Canada. Research-creation artistic …

This research-creation artistic report presents Resonance and Depth, a 9-minute immersive electroacoustic/acousmatic performance that treats timbre and space as primary compositional materials. Built from live electronic percussion and a personal corpus recorded, curated, and composed by the author, the work investigates how timbral identity and musical form can emerge through real-time recombination, spatialisation, and performer-system interaction rather than conventional harmonic development. The performance is realised as an artist-in-the-loop co-creative multi-agent system integrating MACAT, MACataRT, a MIDI-based musical agent, a spatialisation layer, and a conductor agent. These components recombine and transform the performer's material through concatenative synthesis, audio mosaicing, symbolic mediation, physically inspired synthesis, and object-based spatialisation. The work approaches spatialisation as an active compositional process in which position, trajectory, depth, diffusion, width, and apparent size participate in musical phrasing and form. Through this practice-based framework, Resonance and Depth examines how performer identity can be retained, transformed, and redistributed across human, algorithmic, timbral, and spatial processes within an immersive co-creative performance ecology.

0.36