CS Paper Radar

Sensing Bone-Conducted Speech with Earbuds

Christoph Weyer, Peter Jax

eess.AS ·submitted 2026-09-02 06:25 UTC · v2
published

Abstract

Clear capture of the wearer's own voice (OV) is essential when using earbuds for mobile communication. However, OV capture remains challenging in noisy environments. Bone-conducted (BC) speech, which can be sensed as vibrations of the earbud housing, can be used to improve OV capture. However, neither bandwidth nor spatial characteristics of OV-induced earbud vibrations have been analyzed in detail, despite both characteristics being relevant, e.g., for sensor choice and placement. This study investigates both characteristics, based on measurements with two earbud models. Spectrally, results indicate that OV-induced earbud vibrations exhibit a low-pass characteristic, with a steep roll-off of -93 dB per decade above 400 Hz. Thus, sensors with comparatively low noise floors are required to sense the vibrations above \SI{1}{\kilo\hertz}. Spatially, results indicate that the earbuds mainly vibrate in and out of the ear canal entrance, with high consistency between subjects and fits. Simulations confirm that this enables capture of the high-power vibrations below 400 Hz by a single-axis sensor with less than 1.5 dB mean attenuation.

Comment
14 pages, 12 figures, journal
Journal ref
J. Audio Eng. Soc., vol. 74, no. 11, pp. 796-810, 2026
DOI
10.17743/jaes.2026.0301

Closest papers

Nearest neighbours by meaning across the whole index.

A Device to Control and Manipulate Occlusion Effects for Own Voice Perception Studies

eess.AS ·2026-09-11 ·Rouben Rehman, Simon Kersten, Aron Schliep, Janina Fels
similarity 0.68
Abstract

The occlusion effect (OE) refers to changes of the eardrum sound pressure through ear canal occlusion. It consists of two phenomena: an insertion loss (IL) attenuating air-conducted sounds, and an occlusion gain (OG) amplifying bone-conduction. Perceptual research on this is hindered by high variability of the OE across individuals, complicating repeatable presentation of precise OE conditions. Consequently, a method to control OE conditions reproducibly during perceptual experiments is needed. For such investigations, the system must enable separate control of IL and OG. We present an approach based on modified commercial earmuffs with integrated microphones. The design integrates dedicated impedance measurements and the derivation of digital filters, which are applied to the microphone signals to emulate arbitrary OE curves. The system is evaluated objectively through appropriate measurements. Results show that the headphones' inherent OG is guaranteed to be below 6dB above 115Hz, falling below 0dB above 155Hz. Emulation is shown to work accurately over the entire frequency range of interest. Limitations arise mainly due to a slight residual inherent OG for deep voices and the processing delay of the system. Future work will focus on the perceptual evaluation of the system and its application in perceptual studies.

0.20

EarLog: Opportunistic Ear Physiology Sensing toward Personalized Hearing Risk Awareness

eess.AS cs.LGcs.SDeess.SP ·2026-09-30 ·Yongzhi Huang, Jiayi Zhao, Kaishun Wu
top venuebenchmark similarity 0.58
Abstract · 22 pages. Accepted at ACM MobiCom 2026. Includes appendix

Many headphone users risk irreversible hearing damage during high-load listening, yet current protection still relies on sound-pressure limits. Clinical tests detect damage after it becomes measurable, leaving the risk-accumulating process unobserved. The acoustic reflex (AR), an involuntary response to high acoustic load, marks when the ear enters a protective state. AR traces are weak, and existing measurements require dedicated probes and controlled conditions that are difficult to maintain during everyday audio. We present EarLog, which turns playback, the dominant interference, into an opportunistic probe. EarLog introduces content-conditioned residual sensing to suppress variations caused by nonstationary content, individual ear acoustics, and device response. It identifies weak AR through loudness-dependent changes and localized spectral irregularities in the residual. A resonance-constrained sparse finite impulse response (FIR) model captures the short-window response. Dual-microphone observations, mild-distortion recovery, and cross-session reference updates keep detection usable under low signal-to-noise ratio (SNR), fit changes, and environmental drift. EarLog runs in real time on an embedded controller using a low-cost dual-microphone add-on. We evaluate it with 81 participants, 16 headphone models, and more than 400 hours of recordings. EarLog achieves 99.0% sensitivity and 98.7% specificity on a clinically referenced benchmark. EarLog records listener-specific AR events during everyday listening, providing data for future longitudinal studies of hearing risk.

0.45

Speech-muscle coupling at jaw and ear electrodes: a volume-conductor model with orientation and electrode-count controls

cs.HC eess.SP ·2026-08-12 ·Carl Vincent Kho
code/data similarity 0.56
Abstract · Revised title, abstract and text. Reported both derived fibre models; clarified protocol history and current-normalization limits. Added a g…

Objective. We compared how strongly speech-muscle sources couple to electrodes on the jaw and around the ear, and tested sensitivity to source orientation, electrode count, and anatomical detail. Approach. Using reciprocity in MIDA (116 labelled compartments, 500 micrometres), we evaluated 22 electrode positions against ten segmented articulators. Muscle was modelled as both source and conducting tissue. Comparisons used equal electrode counts and a hemisphere orientation sweep, with anatomy-derived fibre models for temporalis and lateral pterygoid. Results. Orbicularis oris, buccinator, mentalis, depressor anguli oris, and platysma favoured the jaw at every sampled orientation and tested electrode subset. Temporalis showed a 2.57 dB ear advantage under the uniform sweep, reduced to 1.15 dB with the derived fibre field. Its jaw-minus-ear interval [-1.45, +5.46] dB crossed zero; 50.5% of four-site ear subsets favoured the ear. The derived lateral-pterygoid interval [-5.33, -1.12] dB favoured the ear at the fixed jaw basis. Both derived constructions have unresolved anatomical limitations. Simplifying non-muscle soft-tissue conductivities preserved montage assignments while changing gaps by up to 3.27 dB. Significance. In this head model, jaw placement better captures lip and chin activity. Ear-side preferences depend on source assumptions and electrode selection. Targeted four-site placement improved lateral-pterygoid coupling by 1.07 dB over the median random ear montage. These coupling predictions require experimental testing.

0.24

The Hearpiece Database of Binaural Room Impulse Responses and Driver Responses on KEMAR

eess.AS ·2026-11-04 ·Tong Xiao, Reinhild Roden, Matthias Blau, Simon Doclo
similarity 0.54
Abstract · 13 pages, 7 figures, 5 tables. Database and MATLAB loading script available at https://doi.org/10.5281/zenodo.23667308

This document describes the measurement setup, signal processing, data organization, and MATLAB file format of a database of acoustic responses measured on a Knowles Electronics Manikin for Acoustic Research (KEMAR) using the Hearpiece, a multi-microphone, multi-driver in-ear research device. The data include binaural room impulse responses (BRIRs) and Hearpiece driver responses, together with center-microphone room impulse responses and a loopback impulse response of the playback and recording system. The BRIRs cover two Hearpiece configurations, three room conditions, three independent insertions per configuration, and 24 horizontal source directions. The database supports reproducible research on binaural and hearing-device signal processing, with potential applications in speech enhancement, spatial filtering and source separation, dereverberation, active noise and occlusion control, and acoustic feedback cancellation. The database, MATLAB loading script, and technical documentation are publicly available at https://doi.org/10.5281/zenodo.23667308.

0.35

Finger-to-Ear ECG: Systematic Evaluation of an Earbud-Based Cardiac Monitoring Approach

cs.HC ·2026-09-21 ·Philipp Lepold, Paula Breitling, Jonas Hummel, Tobias Röddiger, Michael Beigl
published similarity 0.52
Abstract

Ear-ECG enables unobtrusive cardiac monitoring, but existing approaches often struggle with low signal amplitudes and limited morphology preservation. We evaluate a finger-to-ear ECG paradigm that combines an in-ear electrode with a rear housing finger-contact electrode in a speaker-equipped earbud. A study with 30 participants investigated four finger-to-ear electrode geometries against chest-reference ECG and additionally assessed robustness under music playback, talking, and walking disturbances. Left-finger configurations achieved R peak detection F1-scores >99% with high agreement of key ECG morphology features. In contrast, right finger configurations as well as walking led to severe signal degradation. The results reveal a trade-off: while cross-body setups provided the highest signal fidelity, users preferred same-side contacts for comfort reasons. This establishes finger-to-ear ECG as a robust, morphology-aware paradigm for opportunistic sensing in earables.

0.38

Machine Hearing: Can Audio LLMs Generalize Beyond Human Auditory Perception?

cs.SD cs.CL ·2026-11-07 ·Liwei Liu, Jun Tao, Xingyuan Li, Xuenan Xu, Chen Zhang +6 more
benchmarkcode/data similarity 0.50
Abstract · 22 pages, 7 figures, 5 tables

Multimodal LLMs have made remarkable progress in perceiving the physical world, while their perceptual scope remains largely human-centric, focusing on what humans can directly see or hear. This progress raises the prospect of perception beyond human limits, such as extending hearing to sounds that are inaudible to humans. More broadly, the physical world is full of waves, vibrations, and oscillations, only a small fraction of which are directly audible to humans. This invites the extension of multimodal LLMs toward a broader form of machine hearing, which remains largely underexplored. To investigate this direction, we introduce the Machine Hearing Benchmark (MHB), comprising 6,128 instances across 13 tasks and seven domains. MHB spans highly heterogeneous input regimes, with sampling rates from approximately $10^{-5}$ to $10^6$ Hz, sequence lengths from $10^1$ to $10^6$ samples, and both single- and multi-channel inputs. Evaluations of current open-source and proprietary Audio LLMs (ALLMs) reveal limited zero-shot transfer to machine hearing. Supervised adaptation on machine-hearing data alone does not close this gap, suggesting that the challenge extends beyond data exposure to a systematic mismatch between existing audio front ends and machine-hearing inputs. To address this mismatch, we introduce MachEar, a lightweight encoder extension that complements the pretrained Audio LLM for machine-hearing inputs. The results show that MachEar equips a pretrained Audio LLM with stronger machine-hearing capabilities while largely preserving its existing audio understanding performance.

0.32

Active noise cancellation extends the benefit of AI speech enhancement in real hearing-aid recordings

eess.AS ·2026-10-12 ·Andrew T. Sabin, Linkai Li
similarity 0.49
Abstract · 8 pages, 5 figures, 1 table

Active noise cancellation (ANC) is common in headphones but rare in hearing aids, and its value for understanding speech in noise is not established. Algorithms that improve signal-to-noise ratio (SNR) act only on the hearing aid's amplified path, while unprocessed sound also reaches the eardrum directly through the vent, limiting the SNR improvement that is realized. Our earlier simulations predicted that attenuating this direct path with ANC would substantially increase the benefit of such processing. We tested that prediction in three hardware conditions: a standard receiver-in-canal (RIC) hearing aid, and an ANC-capable RIC with ANC off and on. Each was fit to NAL-NL2 targets for an N3 hearing loss and recorded on KEMAR with an AI speech-enhancement algorithm (AI-SE) off and on, in seven realistic scenes above 70 dB SPL. Speech and noise spectra were estimated from each recording with a binary time-frequency mask derived from the clean speech. AI-SE produced a similar peak benefit of about 4 dB in the high frequencies in all three conditions, but that benefit stayed at or above 1 dB only above 1500, 600, and 130 Hz for the standard RIC, ANC RIC (ANC off), and ANC RIC (ANC on), respectively, closely tracking the measured attenuation of the direct path. With AI-SE on, ANC RIC (ANC on) improved SII-weighted SNR by 5.1 dB relative to the open ear, compared with 3.3 dB for ANC RIC (ANC off) and 1.6 dB for the standard RIC. With AI-SE off, enabling ANC added only 0.2 dB. ANC is therefore an enabling technology: on its own it added little, but by attenuating the direct path it allowed SNR-enhancing processing to deliver substantially greater benefit at the eardrum, extending that benefit to much lower frequencies.

0.21

Window Size Matters: Benchmarking Voice Activity Detectors in Noise

cs.SD ·2026-10-10 ·Alok Kumar, Mohit Srivastava, Anupam Purwar, Kuldeep Kumar Nigam, Yash Anchaliya +2 more
benchmark similarity 0.49
Abstract · Offline frame-level benchmark of TEN VAD, Silero VAD and an RMS energy baseline on LibriVAD

Voice activity detection (VAD) sits in front of almost every streaming speech recognizer, yet the choice of detector is often made by feel rather than by measurement. We report a frame-level offline benchmark of three detectors on the test-clean split of LibriVAD-small: the neural TEN VAD, the neural Silero VAD and a simple RMS energy baseline. The evaluation covers 2,214 noisy utterances (about 7.1 hours) built from LibriSpeech, nine noise families and six SNR levels from -5 to 20 dB. At a 16 ms hop, TEN VAD reaches a ROC AUC of 0.917, an average precision of 0.968 and an F1 of 0.913. A sweep over 20, 30, 40 and 50 ms windows shows TEN to be almost insensitive to frame length (AUC 0.916 to 0.919), while RMS stays near 0.75 to 0.77. Silero matches TEN only near its native 32 ms input size and drops to 0.63 at 20 ms. We hypothesize that this is likely caused by our fixed-frame protocol rather than by the model itself. Zero padding, sub-chunk averaging and unreset model state are the likely factors, but we have not ablated them. In a per-condition breakdown of TEN, babble noise at low SNR stays close to chance. We also show that F1 at a fixed 0.5 threshold is a weak metric on this data, because a trivial all-speech detector already scores 0.854. All settings needed to reproduce the results are given in the paper.

0.19