CS Paper Radar

Seeing the Voice, Preserving the Self: A Participatory Design Approach to Deaf-Centric Text-to-Speech

Shela Atemnkeng, Patrick Boudreault, Paige DeVries, Lloyd May, Christian Vogler

cs.HC ·submitted 2026-09-09 14:04 UTC · v1
top venue

Abstract

We describe a participatory design approach toward developing Deaf-centric text-to-speech (TTS) technologies. While TTS is growing rapidly in the mainstream, it has received little attention to date in the deaf and hard of hearing (DHH) technology space. Critical problems have remained unaddressed for DHH users, including the ability to manipulate tone, emotions and delivery via non-auditory means. Verifying that the generated speech matches intent and is appropriate for a given situation without having to listen to it is another challenge. Respecting cultural and identity factors in the generated speech is also important. This work explores the design space with DHH participants through two focus groups, three co-design sessions, and four one-on-one early-stage design evaluation sessions. Participants included people both familiar and unfamiliar with TTS, as well as DHH content creators. We describe key findings, design ideas, results, and implications for future Deaf-centric TTS development. We also identify unmet technology requirements that pose barriers to adoption of Deaf-centric TTS technology.

Comment
Accepted for publication at ACM ASSETS 2026

Closest papers

Nearest neighbours by meaning across the whole index.

Bigger than the EAR BOX: A Theory-Grounded Review of XR Accessibility Research for Deaf and Hard of Hearing Communities

cs.HC ·2026-09-10 ·Shuxu Huffman, Michaela Okosi, Jose Merino, Tifanie Bouchara, Abraham Glasser +2 more
top venue similarity 0.63
Abstract

Extended Reality (XR) technologies have received growing attention in accessibility research involving Deaf and Hard of Hearing (DHH) communities. Yet less attention has been given to the assumptions shaping this work. We present a theory-grounded review of XR research involving DHH users. Drawing on Disability Studies, Deaf Studies, and DeafSpace, we develop a theoretical framework with four analytical dimensions: orientation toward access, distribution of responsibility, conceptualization of DHH communities, and spatial and perceptual assumptions. We apply this framework to 53 XR studies involving DHH users, identified through a search and screening of ACM publications from 2015 to 2025. Our analysis shows that XR accessibility is frequently framed as supporting communication within hearing-default environments, while overlooking the diversity of DHH communities. We identify directions for redistributing accessibility labor and reconfiguring space, and offer our framework as a tool for future XR research involving DHH communities.

0.46

VoCa: Designing Speech-Canvas Interaction for Voice-Based Conversational Agents

cs.HC cs.AI ·2026-10-03 ·Yate Ge, Run Yuan, Yueran Qi, Wenjie He, Jiaqi Mo +5 more
similarity 0.62
Abstract · 22 pages, 11 figures, 5 tables

People write and sketch while speaking to explain, organize, and develop content together. Inspired by these practices, we investigate how voice agents can use a canvas alongside speech in multi-turn conversations with users. We conducted a two-part formative study: an observational study of how pairs coordinated speech and boardwork, followed by a design workshop that informed a design space for speech-canvas interaction with voice agents. Building on these insights, we developed VoCa, a voice agent that coordinates speech with visual object creation, annotation, and attention guidance. A five-day deployment with 18 participants examined usability, experiences of speech-canvas interaction, patterns of use, and desired improvements. Participants' experiences highlighted opportunities for speech-canvas interaction in learning, work, and daily life, alongside challenges in coordinating what agents say and show in ways users can follow and influence. These findings inform how voice agents can use a canvas alongside speech in conversation.

0.36

Lived Experiences of Power and Agency: Achieving Voice Sovereignty in Assistive Speech Technology for Nonbinary Users

cs.CY cs.HC ·2026-11-16 ·Maxwell Hope, Juliana Francis, Joakim Gustafson, Éva Székely
top venue similarity 0.61
Abstract · Proceedings of Interspeech 2026

Speech-generating devices (SGDs) are essential tools for many speech-disabled individuals, yet current systems often reinforce normative assumptions about identity and communication. Prior research on gender expansive SGD users has largely focused on gendered voice options, with less attention to broader identity alignment and the technological conditions that shape users' agency. This study examines how gender expansive SGD users experience power over their voices using thematic analysis of qualitative survey data. Three themes emerged: (1) integration of SGDs into daily life, (2) identity expression, and (3) power and agency. Participants expressed the need for more responsive and reliable technology, inclusive and customizable voices, and greater autonomy in defining how their voices sound and are perceived. Ultimately, their perspectives call for a reimagining of SGD systems as tools for reclaiming the power of one's voice and the ability to define and enact one's will through it.

0.49

Souffleur: A Glanceable Display for Mobile Voice Conversations

cs.HC ·2026-11-05 ·Anton Wolter, William Benjamin, Niklas Elmqvist
similarity 0.59
Abstract

The speech capabilities of mobile devices have grown, but their displays still rely on a text-based chat. When resuming a voice chat after a context switch, the last messages may not convey the key points of the conversation. Autobiographical use and sessions with external participants led to design goals for a mobile-first, speech-driven, glanceable display that helps users continue their reasoning from where they left off. We implement these goals in Souffleur and evaluate it in a between-subjects study with 48 participants. Compared with a conventional chat transcript, we found no reliable difference in how easily participants followed and resumed short voice conversations, nor in effort, attention or recall, although Souffleur reduces the conversation to a few short phrases.

0.09

LipCoder: Voice-Enabled Coding Toolkit

cs.HC cs.SD ·2026-08-31 ·Hayoon Kim, Sungho Lee, Juhwi Kim, Bongwon Suh, Kyogu Lee
top venue similarity 0.57
Abstract · Accepted to the Posters and Demos Track at the 28th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2026)

AI-assisted programming environments have accelerated software development, giving rise to new paradigms like vibe coding. However, their benefits remain largely inaccessible to visually impaired programmers, as existing screen readers and assistive tools offer limited support for these emerging workflows. We introduce LipCoder, a voice-centric programming toolkit designed to deliver editor-level functionality through auditory and speech-based interfaces. LipCoder offers features comprising speech feedback and earcon cues for comprehension and validation, as well as natural language input for navigation and modification. In an exploratory evaluation, 5 visually impaired programmers performed a series of coding tasks comparing LipCoder with a baseline of VSCode, Copilot, and VoiceOver. Quantitative trends and qualitative feedback point to directions for auditory-first design that may broaden accessibility in speech-driven coding environments.

0.51

Immersive Commons: Designing Ephemeral Spatial Dialogue for Co-Located Mixed Reality Environments

cs.HC cs.MM ·2026-10-26 ·Wuwu Ran, Mohammed Faraj Al-Suwaidi
published similarity 0.57
Abstract · 14 pages, 8 figures. Accepted at SIGraDi 2026, the Conference of the Iberoamerican Society of Digital Graphics

Head-mounted mixed reality (MR) often prioritises individual immersion, potentially drawing attention away from others sharing the room. This paper asks how MR can support co-located spatial dialogue through embodied interaction and activity visible to non-headset observers. We introduce Immersive Commons, a design framework, and Atmos Messages, a prototype combining gesture-based, ephemeral spatial messages with voice transcription, preset text, and a public floor-plan overview. Following a Research through Design approach, fifteen participants created 236 messages across two sessions, including 48 voice-authored messages. Questionnaire responses highlighted playfulness and social awareness, while observations suggested opportunities for exchange grounded in the architectural setting. Interaction clustered near the public display, although this was also where participants put on their headsets. These findings suggest the potential of ephemeral spatial messaging to support co-located exchange, with playfulness, shared visibility, and architectural context informing the design of more social and situated MR experiences.

0.22

"What I See is What I Hear": Deepfake Detection Across Diverse Hearing Abilities

cs.CR ·2026-09-23 ·Magdalena Pasternak, Malvika Jadhav, Palavi V. Bhole, Aviva Smith, Elaina Trapatsos +6 more
top venue similarity 0.56
Abstract · Proceedings of the Network and Distributed System Security (NDSS) Symposium 2027

The proliferation of audiovisual deepfakes has lowered the cost of fraud, impersonation, and misinformation, but their success ultimately depends on human perception. Detection requires integrating auditory and visual cues, yet security and privacy research has largely overlooked d/Deaf and hard-of-hearing (DHH) populations. We address this gap with an in-person, mixed-methods study of 80 participants: 31 hearing persons (HPs), 15 hard-of-hearing (HoH) participants, 17 d/Deaf participants, and 17 cochlear implant (CI) users. Each participant judged the authenticity of 30 clips, where manipulations spanned text-to-speech, voice conversion, lip-sync, or face-swap. DHH participants were less accurate than HPs overall (76.4% vs. 88.0%, p<.001), primarily because they more often classified authentic clips as manipulated (FPR: 29.7% vs. 11.2%). Differences depended strongly on the manipulated channel. For audio-only manipulations, HoH participants matched HPs (90.0% vs. 90.3%), followed by CI users (79.4%) and d/Deaf participants (41.2%). When clips contained an audiovisual manipulation, accuracy clustered between 84% and 87%, although performance still varied by manipulation method. Our work systematically characterizes how deepfakes affect DHH populations, highlighting the asymmetric risks audiovisual manipulations may pose to groups with different hearing abilities and the need for accessible, tailored defenses that support all users.

0.47

VocalEyes: Speaker-Aware Augmented Reality Captioning through In-Conversation Registration

cs.HC ·2026-09-26 ·Yuxiao Wang, Xulong Tang, Chen Chen, Rawan alghofaili
similarity 0.56
Abstract

Co-located augmented reality (AR) captions make speech readable, but they can separate an utterance from the person who produced it. In unfamiliar groups, losing that source complicates immediate responses and later review: users must recover not only what was said, but also who said it. Conventional diarization returns anonymous clusters, while speaker recognition typically assumes pre-meeting enrollment. We built VocalEyes, a speaker-aware AR captioning system that creates named voice profiles from natural self-introductions. The interface coordinates speaker-attributed captions, a fixed profile card, and a visual cue that marks the articulating face. In a controlled within-subjects study with 20 participants who reported typical hearing, VocalEyes identified speakers with 88.0% accuracy and increased participant speaker-tracking accuracy from 47.2% with caption-only AR to 87.3% with the complete interface. Participants also reported lower workload with the complete interface. These findings show how in-conversation registration can preserve speaker attribution across live captions and meeting records in scripted small-group meetings.

0.27