CS Paper Radar

Enhancing Communication in Speech Therapy: Exploring the Cognitive Synergy Between Gesture and Speech

Paul-Peter Arslan, Xiao Xiao

cs.HC ·submitted 2026-09-08 09:29 UTC · v1

Abstract

This paper examines the adaptation of a rhythm-based interface, originally designed for manual dexterity rehabilitation, for use in speech therapy. The interface allows users to control synthesized vocal phrases through finger tapping, leveraging the cognitive link between gesture and speech. Through interviews with four therapists and pilot tests with one speech therapist and eight children with speech impairments (autism, Down syndrome, verbal apraxia, dyslexia), we found that the interface improves motivation and therapeutic outcomes by facilitating more interactions between verbally challenged patients and the therapist. Our findings suggest that rhythmic gestures can enhance verbal communication, offering potential for broader therapeutic and educational applications.

Comment
18 pages, 6 figures, 3 tables

Closest papers

Nearest neighbours by meaning across the whole index.

Articulatory Entrainment and Coordination Complexity in Spontaneous Autistic and Non-autistic Dialogue

cs.LG ·2026-10-02 ·Thanushi Withanage, Carol Espy-Wilson, Elizabeth Redcay, Desi Jones, Noah Sasson
top venue similarity 0.57
Abstract · Accepted to be published in Interspeech 2026

Articulatory entrainment, the adaptation of vocal tract coordination to facilitate interaction remains underexplored in spontaneous dialogue, particularly among autistic speakers. Many prior studies have utilized task-based, phoneme-level analyses with invasive measurement techniques. Here, we introduce a speaker-independent framework to quantify articulatory entrainment in spontaneous dyadic conversations among autistic and non-autistic adults using acoustic-to-articulatory inversion and coordination complexity metrics. We examine temporal changes in articulatory coordination across interaction. Non-autistic dyads exhibit increasing coordination complexity and stronger entrainment over time, autistic dyads show moderate effects, and mixed dyads demonstrate the least alignment. Greater articulatory entrainment correlates with higher self-reported conversational success, indicating increase in coordination complexity as a marker of effective social interaction.

0.68

Toward Neuroadaptive Communication Interfaces: A Framework for Personalized Speech and Language Support

cs.HC cs.CL ·2026-10-25 ·Carlos Bruno, Luis Ricardo Santos
similarity 0.56
Abstract

Speech and language technologies have expanded access to communication, but the support they provide is often fixed. The same linguistic or visual aid may help one user and hinder another, or become less useful as task demands change. This project proposes a framework for adapting language presentation to individual abilities and ongoing interaction. It separates access to a message from the estimation of user needs and from the selection of support, including sentence segmentation, pictograms, and optional spoken output. Behavioral feedback provides the primary evidence for adaptation; electroencephalography is considered an additional, uncertainty-aware source of information rather than a direct measure of comprehension. We specify a staged evaluation comparing fixed, behavior-adaptive, and neuroadaptive interfaces. The central question is whether neural information improves comprehension and communication efficiency beyond a strong behavioral baseline.

0.12

"Just Like This": Manner Deixis at the Graphics Interface

cs.HC cs.GR ·2026-10-06 ·Hamza El Alaoui, Jeffrey P. Bigham, Jun Rekimoto
hot direction similarity 0.53
Abstract

People communicate how things should move by combining words with demonstrations: "open it like this." We present an interaction technique that brings this expressive resource to conversational 3D authoring. Building on "Put-That-There," our system combines speech, pointing, and spatiotemporal demonstrations to specify editable behavior. A hand movement supplies evidence for a mechanism's axis, pivot, range, and pace; an animated preview makes the interpretation inspectable. Users refine behavior through further words or demonstrations, and the system can request a demonstration to clarify intent. In a twelve-participant study comparing three input configurations, our system achieved 89% participant-declared completion versus 47% with speech alone, had the highest observed match rates on six categorical accuracy measures and tied on two, and was preferred by nine participants. This work makes demonstration part of an ongoing authoring conversation: behavior can be shown, inspected, and revised.

0.32

Industry case studies of adapting TTS for smooth interactions

cs.HC cs.ROcs.SDeess.AS ·2026-08-26 ·Paige Tuttösi, Eléonore Ferrier-Barbut, Anais Scipioni, Olivia Yung, Ha Eun Shim +3 more
published similarity 0.52
Abstract · Published at RO-MAN 2026

Modern, neural-based ASR and TTS systems for speech communication are now very sophisticated, but they are frequently designed for average users in more idealized scenarios, which can underperform when interacting with users having specialized listening needs, or who speak in non-standard ways. Here we review recent TTS research, and note that industrial, production-level implementations of adaptive TTS systems are scarce. We describe how we implemented changes to TTS systems based on prior research, and then report qualitative results from two case studies, where adapted TTS techniques are applied to real-world HRI settings: First, in an older adult care home, and second, in an airport, where noise levels create comprehension problems. Results suggest promising directions using relatively simple modifications to pitch, duration, and intensity of the robot's speech, and offer a framework for larger, more formal studies of how TTS adaptations can improve HRI.

0.33

ResonaVis: Visualizing Interactive Music Data to Support Reflective Music Composition for Therapeutic Contexts

cs.HC ·2026-08-11 ·Abhishek Karwankar, Elise Ruggiero, Daniel Stevens, Matthew Louis Mauriello
published similarity 0.51
Abstract · 9 pages core, 2 pages acknowledgements + references, 2 pages appendices. Accepted at IEEE VIS 2026, to appear in November 2026

Designing music for therapeutic contexts requires navigating complex relationships between musical structure and listeners' sensory responses, yet composers often lack structured representations of these interactions, relying instead on intuition. We present ResonaVis, an interactive visualization system that helps composers analyze interaction and audio data from prior sessions with children with Autism Spectrum Disorder (ASD), informing future compositions. ResonaVis integrates audio features and interaction logs to capture how children engage with layered musical compositions, representing this engagement through coordinated visualizations of temporal transitions, layer co-occurrence, rhythmic activity, and spectral characteristics. Rather than prescribing strategies or supporting therapy sessions directly, the system surfaces patterns in past session data to support data-informed reflection during composition. We evaluated ResonaVis through a mixed-methods study with eight music students and a follow-up case study with two experienced composers. Results demonstrate good usability (SUS = 72.23), exceeding benchmarks for early prototypes, and show that participants could identify interaction patterns, reason about layer relationships, and make informed compositional decisions with high perceived performance and low frustration. Confidence in interpreting interaction and acoustic data for ASD-focused composition increased significantly across multiple dimensions (p < 0.05), with qualitative findings suggesting a shift toward more adaptive, data-informed composition. This work contributes a visualization design space for therapeutic music interaction data, an integrated system for compositional reflection, and empirical evidence that visualization tools support analytical reasoning and confidence in data-informed creative practice. (Abstract shortened for arXiv.)

0.25

Beyond Intelligibility: Fine-Grained Perceptual Assessment of Dysarthric Speech with Auditory Large Language Models

eess.AS cs.SD ·2026-11-18 ·Haotian Wei, Can Long, Wen Xian Jim, Guangzhi Sun, Zengrui Jin +1 more
similarity 0.50
Abstract · 5 pages; Haotian Wei, Can Long, and Wen Xian Jim contributed equally

Accurate assessment of dysarthric speech is vital for clinical diagnosis and rehabilitation, yet existing automated methods rely heavily on isolated, single-task models or generic speech quality metrics that fail to disentangle heterogeneous neuromotor deficits. This work presents the first systematic investigation into adapting auditory large language models (ALLMs) for multi-dimensional dysarthric speech assessment under a unified, prompt-conditioned framework. Four representative ALLMs are benchmarked across five clinical perceptual dimensions from the Speech Accessibility Project (SAP): naturalness, monopitch, intelligibility, imprecise consonants, and inappropriate silences. To bridge generic perceptual quality and clinical evaluation, a two-stage training paradigm is explored, leveraging out-of-domain pre-training followed by in-domain clinical adaptation. Experimental results demonstrate that leading ALLMs reliably capture fine-grained pathological variations, exhibiting distinct and complementary strengths across prosodic and articulatory attributes. These findings establish ALLMs as a scalable foundation for clinical speech evaluation and provide practical insights into model selection and multi-task transfer potential.

0.14

An Evaluation of AI-Supported Evidence-Based Learning for Public Speaking Skill Development

cs.CL ·2026-09-16 ·Sashini Hettiarachchi, Shahbaz Siddeeq, Mika Saari, Pekka Abrahamsson
publishedunusual similarity 0.50
Abstract · Proceedings of the 54th Annual Conference of the European Society for Engineering Education (SEFI 2026)

Public speaking is an essential skill in academic and professional contexts, but it often causes anxiety. Although several AI-based speech coaching tools exist, they typically provide generic feedback and lack model speeches for learning. This study addresses these gaps by developing a system that incorporates speech context into evaluation criteria, enabling more tailored feedback. It also generates improved model speeches in both text and audio formats to support example-based learning. The system was evaluated in a pilot study with seven undergraduate students over one to three weeks. Participants used the system repeatedly and practiced the same speech at least three times. Results showed improvements in public speaking performance for all participants. Filler word usage decreased, and anxiety levels dropped by 15.3% to 34.4%. Participants found the context-aware feedback and revised speeches useful and confidence-building. These findings suggest that context-aware feedback and model speeches can enhance public speaking skills and reduce anxiety. Future studies could explore video-based analysis of physical behavior during speeches.

0.39

LipCoder: Voice-Enabled Coding Toolkit

cs.HC cs.SD ·2026-08-31 ·Hayoon Kim, Sungho Lee, Juhwi Kim, Bongwon Suh, Kyogu Lee
top venue similarity 0.50
Abstract · Accepted to the Posters and Demos Track at the 28th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2026)

AI-assisted programming environments have accelerated software development, giving rise to new paradigms like vibe coding. However, their benefits remain largely inaccessible to visually impaired programmers, as existing screen readers and assistive tools offer limited support for these emerging workflows. We introduce LipCoder, a voice-centric programming toolkit designed to deliver editor-level functionality through auditory and speech-based interfaces. LipCoder offers features comprising speech feedback and earcon cues for comprehension and validation, as well as natural language input for navigation and modification. In an exploratory evaluation, 5 visually impaired programmers performed a series of coding tasks comparing LipCoder with a baseline of VSCode, Copilot, and VoiceOver. Quantitative trends and qualitative feedback point to directions for auditory-first design that may broaden accessibility in speech-driven coding environments.

0.51