CS Paper Radar

A Novel Binaural Cue Preservation Loss for DNN-Based Binaural Speech Enhancement

Jayteerth Amble, Thomas Haubner, Hendrik Schröter, Christoph Hoog Antink, Henning Puder

eess.AS cs.SD ·submitted 2026-08-17 09:12 UTC · v1

Abstract

Binaural speech enhancement for hearing aids aims to reduce noise while preserving the interaural cues needed for spatial localization. Although deep neural network-based methods achieve strong noise reduction, they often distort the rela- tionship between the left and right signals. In this paper, we propose two novel binaural cue preservation losses. First, a binaural reconstruction error loss that directly penalizes masking-induced distortion in the relationship between the left and right spectra, providing a more direct measure of the binaural consistency than conventional separate interaural level differences (ILD) and interaural phase differences (IPD) errors as in prior work. Second, a binaural cue loss that jointly models ILD and IPD to better preserve the binaural structure. Experimental results show that both proposed losses maintain strong noise reduction performance and reduce masking- induced distortion compared to the state-of-the-art baseline cue loss, while the second proposed joint binaural cue loss also outperforms the baseline in ILD preservation.

Comment
Accepted at the International Workshop on Acoustic Signal Enhancement (IWAENC) 2026

Closest papers

Nearest neighbours by meaning across the whole index.

Mask-Based Speech Enhancement for Spatial Audio: A Comparison of Ambisonics, Beamforming, and Microphone Channels

eess.AS ·2026-09-16 ·Sheli Hendel, Boaz Rafaely, Dorothea Kolossa
similarity 0.70
Abstract

Mask-based speech enhancement is widely used for suppressing noise and interference, but its performance in spatial audio algorithms with multichannel output has not been studied extensively. In such settings, speech enhancement must improve speech quality while preserving spatial cues that are essential for localization, spatial awareness, and spatial release from masking. In this work, we systematically compare time frequency masking applied to three signal representations: microphone signals, beamformer outputs, and Ambisonics signals. Performance is evaluated in terms of speech quality, intelligibility, binaural cue preservation, and reverberation preservation. Results reveal a clear trade-off between enhancement and spatial fidelity: beamformer-domain masking achieves the highest speech enhancement scores, while Ambisonics-domain masking better preserves the spatial attributes of the residual interference. All methods preserve the target's localization cues.

0.15

BAmbiDrop: Array-Agnostic Speech Enhancement with Zero-Shot binaural Reproduction using Multi-center Ambisonics Encoding

eess.AS ·2026-11-07 ·Michael Tatarjitzky, Vladimir Tourbabin, Boaz Rafaely
similarity 0.67
Abstract · Submitted for ICASSP 2027

Array-agnostic enhancement networks generalize across microphone geometries but produce a single channel, while binaural enhancement methods preserve interaural cues only for one fixed device geometry. We propose BAmbiDrop, a framework that extends array-agnostic enhancement to binaural reproduction without retraining. Exploiting the fact that the Ambisonics origin is an arbitrary point in space, a frozen single-channel AmbiDrop network estimates a complex mask at multiple centers across the array; these masks are fused by a complex mean into a single shared mask, applied to the microphone signals and rendered by fixed BSM filters. Across 21 free-field and rigid-sphere arrays, the fused mask improves enhancement and interaural cue reproduction simultaneously, whereas masking each microphone independently degrades the latter without a consistent gain in the former.

0.25

Directivity-Conditioned Low-Latency Neural Filtering for Speech Enhancement in Hearing Aids

eess.AS ·2026-09-14 ·Lennart Uphaus, André Merboldt, Markus Hofbauer, Timo Gerkmann
similarity 0.66
Abstract · Accepted at the International Workshop on Acoustic Signal Enhancement (IWAENC) 2026

Latest advances in neural directional filtering show exceptional results in adapting the direction and shape of directivity patterns during the inference phase. However, in the existing methods for adapting directivity patterns during inference, important real-world constraints have been disregarded. Particularly for hearing devices, scenarios are often much more dynamic, microphone positions vary with head diameter and hearing aid placement, head-shadow effects occur, and strict latency constraints apply. In this work, we propose a novel low-latency (10 ms) deep neural network (DNN) taking the above requirements of hearing devices into account. As in recent work, we use feature-wise linear modulation (FiLM) to steer the directivity patterns during testing. To preserve the desired directivity pattern, a loss function is proposed that maintains the spectral cross-channel relationships. Interestingly, we are able to achieve similar results to methods with relaxed latency constraints.

0.23

BiTSE: Binaural Target Speaker Extraction in Noisy Multi-Talker Environments for AR Glass Arrays

eess.AS ·2026-08-10 ·Selani A. Indrapala, Wageesha N. Manamperi
similarity 0.64
Abstract · This is the preprint version of the paper accepted at APSIPA ASC 2026

Isolating a desired speech signal in noisy multi-talker conversational scenarios is a key requirement for augmented reality (AR) wearable microphone array systems. In this work, a binaural target speaker extraction (TSE) framework, termed BiTSE, is proposed. It leverages both spatial and temporal cues, specifically the direction-of-arrival (DoA) of the target speaker and corresponding voice activity information, to guide the extraction process. Built upon a binaural signal denoising architecture, our model integrates three key enhancements: (i) a DoA-aware attention mechanism using cyclic positional embeddings, (ii) a timestamp-based masking strategy that utilizes speaker activity to suppress non-target segments, and (iii) a novel two-stage loss optimization strategy that first trains the model for robust denoising and then fine-tunes it to improve perceptual quality. Evaluations on the SPeech Enhancement for Augmented Reality (SPEAR) challenge dataset demonstrate that the proposed BiTSE consistently improves upon conventional approaches, leading to enhanced signal fidelity and perceptual quality.

0.36

Neural Field-of-View for Binaural Signal Matching with Wearable Microphone Arrays

eess.AS ·2026-09-23 ·Matan Yifrach, Boaz Rafaely
similarity 0.64
Abstract

The growing use of spatial audio in applications such as augmented and virtual reality has driven the development of binaural reproduction methods for wearable arrays with a limited number of microphones. Binaural signal matching (BSM) is one such method, producing high-quality binaural signals under a diffuse-field assumption, but degrading at high direct-to-reverberant ratios (DRR) where the direct sound dominates. Previous extensions incorporate Field-of-View (FoV) weighting, either with fixed apertures or based on explicit source localization, but these approaches are limited by coarse spatial coverage or reliance on localization estimation accuracy. This paper introduces FoV-BSM-Net, a signal-dependent FoV-BSM formulation that avoids explicit source estimation by learning the FoV parameters end-to-end from the microphone signals using a Convolutional Recurrent Neural Network. The method is evaluated in simulated rooms across varying reverberation conditions, and compared against BSM and a fixed FoV-BSM baseline. Results show that FoV-BSM-Net consistently improves over BSM, with gains that grow with DRR in both binaural NMSE and interaural cue errors, and are further supported by perceptual evaluation showing a substantial advantage over both baselines across low and high DRR conditions.

0.26

Audiovisual joint learning for end-to-end hearing aids

eess.AS ·2026-10-06 ·You-Jin Li, Yu Tsao, Borching Su, Kuan-Chung Ting, Fan-Gang Zeng
similarity 0.61
Abstract

Speech understanding in noise remains challenging for hearing-aid users, particularly in the presence of competing speakers. Conventional hearing aids typically perform speech enhancement (SE) and hearing-loss compensation in separate stages, which may cause enhancement errors and signal distortions to carry over to the amplification stage. Moreover, audio-only SE often provides limited benefits under competing-speech conditions because the target and interfering speech share similar acoustic characteristics, making them difficult to separate. To address these limitations, we propose AV-NeuroAMP, an end-to-end audiovisual framework that integrates noisy speech, target-talker video, and the listener's audiogram to jointly perform SE, personalized amplification, and dynamic-range compression. We further introduce audiogram-conditioned feature-wise linear modulation (AC-FiLM) to effectively incorporate listener-specific hearing profiles. Objective evaluations showed that AV-NeuroAMP outperformed conventional amplification, the audio-only NeuroAMP model, and two-stage systems on an in-domain English test set, and that these improvements were retained on an unseen Mandarin test set. Listening tests involving normal-hearing participants under simulated hearing loss and listeners with hearing loss further demonstrated improvements in speech quality and intelligibility, with the greatest benefits observed under competing-speech conditions. These findings support end-to-end audiovisual personalized amplification as a promising approach for improving hearing-aid performance in challenging acoustic environments.

0.14

ABSE-NET: A Lightweight Neural Model for Active Binaural Speech Enhancement in Open-Fit Hearing Aids

cs.SD eess.AS ·2026-09-01 ·De Hu, Xue Du, Qingying Zhao, Qintuya Si
top venuecode/data similarity 0.61
Abstract · Accepted to INTERSPEECH 2026. 10 pages, 3 figures

Open-fit hearing aids have attracted growing attention due to their superior wearing comfort. However, the open-fit design inevitably causes acoustic leakage into the ear canal, degrading the performance of existing binaural speech enhancement (BSE). To this end, we propose ABSE-NET, an active BSE framework integrating active noise control (ANC) with BSE to jointly enhance target speech and suppress acoustic leakage. The ABSE-NET pipeline cascades a binaural MVDR (BMVDR) with a lightweight neural network (LNN). The former achieves a coarse BSE, whereas the latter simultaneously cancels acoustic leakage and compensates for BMVDR-induced distortion. The LNN uses an encoder-decoder with a feature fusion module, which includes frequency-time dependency learning and convolutional attention blocks. Unlike traditional BSE+ANC solutions via adaptive filtering, ABSE-NET needs no in-ear microphone in practical deployment. Experiments validate its superiority over state-of-the-art methods. Code repository: https://github.com/Bream101/ABSE-NET.

0.54

Active noise cancellation extends the benefit of AI speech enhancement in real hearing-aid recordings

eess.AS ·2026-10-12 ·Andrew T. Sabin, Linkai Li
similarity 0.60
Abstract · 8 pages, 5 figures, 1 table

Active noise cancellation (ANC) is common in headphones but rare in hearing aids, and its value for understanding speech in noise is not established. Algorithms that improve signal-to-noise ratio (SNR) act only on the hearing aid's amplified path, while unprocessed sound also reaches the eardrum directly through the vent, limiting the SNR improvement that is realized. Our earlier simulations predicted that attenuating this direct path with ANC would substantially increase the benefit of such processing. We tested that prediction in three hardware conditions: a standard receiver-in-canal (RIC) hearing aid, and an ANC-capable RIC with ANC off and on. Each was fit to NAL-NL2 targets for an N3 hearing loss and recorded on KEMAR with an AI speech-enhancement algorithm (AI-SE) off and on, in seven realistic scenes above 70 dB SPL. Speech and noise spectra were estimated from each recording with a binary time-frequency mask derived from the clean speech. AI-SE produced a similar peak benefit of about 4 dB in the high frequencies in all three conditions, but that benefit stayed at or above 1 dB only above 1500, 600, and 130 Hz for the standard RIC, ANC RIC (ANC off), and ANC RIC (ANC on), respectively, closely tracking the measured attenuation of the direct path. With AI-SE on, ANC RIC (ANC on) improved SII-weighted SNR by 5.1 dB relative to the open ear, compared with 3.3 dB for ANC RIC (ANC off) and 1.6 dB for the standard RIC. With AI-SE off, enabling ANC added only 0.2 dB. ANC is therefore an enabling technology: on its own it added little, but by attenuating the direct path it allowed SNR-enhancing processing to deliver substantially greater benefit at the eardrum, extending that benefit to much lower frequencies.

0.21