CS Paper Radar

ABSE-NET: A Lightweight Neural Model for Active Binaural Speech Enhancement in Open-Fit Hearing Aids

De Hu, Xue Du, Qingying Zhao, Qintuya Si

cs.SD eess.AS ·submitted 2026-09-01 09:21 UTC · v1
top venuecode/data

Abstract

Open-fit hearing aids have attracted growing attention due to their superior wearing comfort. However, the open-fit design inevitably causes acoustic leakage into the ear canal, degrading the performance of existing binaural speech enhancement (BSE). To this end, we propose ABSE-NET, an active BSE framework integrating active noise control (ANC) with BSE to jointly enhance target speech and suppress acoustic leakage. The ABSE-NET pipeline cascades a binaural MVDR (BMVDR) with a lightweight neural network (LNN). The former achieves a coarse BSE, whereas the latter simultaneously cancels acoustic leakage and compensates for BMVDR-induced distortion. The LNN uses an encoder-decoder with a feature fusion module, which includes frequency-time dependency learning and convolutional attention blocks. Unlike traditional BSE+ANC solutions via adaptive filtering, ABSE-NET needs no in-ear microphone in practical deployment. Experiments validate its superiority over state-of-the-art methods. Code repository: https://github.com/Bream101/ABSE-NET.

Comment
Accepted to INTERSPEECH 2026. 10 pages, 3 figures

Closest papers

Nearest neighbours by meaning across the whole index.

Active noise cancellation extends the benefit of AI speech enhancement in real hearing-aid recordings

eess.AS ·2026-10-12 ·Andrew T. Sabin, Linkai Li
similarity 0.67
Abstract · 8 pages, 5 figures, 1 table

Active noise cancellation (ANC) is common in headphones but rare in hearing aids, and its value for understanding speech in noise is not established. Algorithms that improve signal-to-noise ratio (SNR) act only on the hearing aid's amplified path, while unprocessed sound also reaches the eardrum directly through the vent, limiting the SNR improvement that is realized. Our earlier simulations predicted that attenuating this direct path with ANC would substantially increase the benefit of such processing. We tested that prediction in three hardware conditions: a standard receiver-in-canal (RIC) hearing aid, and an ANC-capable RIC with ANC off and on. Each was fit to NAL-NL2 targets for an N3 hearing loss and recorded on KEMAR with an AI speech-enhancement algorithm (AI-SE) off and on, in seven realistic scenes above 70 dB SPL. Speech and noise spectra were estimated from each recording with a binary time-frequency mask derived from the clean speech. AI-SE produced a similar peak benefit of about 4 dB in the high frequencies in all three conditions, but that benefit stayed at or above 1 dB only above 1500, 600, and 130 Hz for the standard RIC, ANC RIC (ANC off), and ANC RIC (ANC on), respectively, closely tracking the measured attenuation of the direct path. With AI-SE on, ANC RIC (ANC on) improved SII-weighted SNR by 5.1 dB relative to the open ear, compared with 3.3 dB for ANC RIC (ANC off) and 1.6 dB for the standard RIC. With AI-SE off, enabling ANC added only 0.2 dB. ANC is therefore an enabling technology: on its own it added little, but by attenuating the direct path it allowed SNR-enhancing processing to deliver substantially greater benefit at the eardrum, extending that benefit to much lower frequencies.

0.21

Directivity-Conditioned Low-Latency Neural Filtering for Speech Enhancement in Hearing Aids

eess.AS ·2026-09-14 ·Lennart Uphaus, André Merboldt, Markus Hofbauer, Timo Gerkmann
similarity 0.67
Abstract · Accepted at the International Workshop on Acoustic Signal Enhancement (IWAENC) 2026

Latest advances in neural directional filtering show exceptional results in adapting the direction and shape of directivity patterns during the inference phase. However, in the existing methods for adapting directivity patterns during inference, important real-world constraints have been disregarded. Particularly for hearing devices, scenarios are often much more dynamic, microphone positions vary with head diameter and hearing aid placement, head-shadow effects occur, and strict latency constraints apply. In this work, we propose a novel low-latency (10 ms) deep neural network (DNN) taking the above requirements of hearing devices into account. As in recent work, we use feature-wise linear modulation (FiLM) to steer the directivity patterns during testing. To preserve the desired directivity pattern, a loss function is proposed that maintains the spectral cross-channel relationships. Interestingly, we are able to achieve similar results to methods with relaxed latency constraints.

0.23

Frequency-Weighted Soft-Constrained Spatially Selective Active Noise Control for Open-Fitting Hearables

eess.AS eess.SP ·2026-09-30 ·Tong Xiao, Reinhild Roden, Matthias Blau, Simon Doclo
similarity 0.64
Abstract · 5 pages, 4 figures, 1 table. Submitted to IEEE ICASSP 2027

Recently, soft-constrained spatially selective active noise control (SSANC) for open-fitting hearables has been proposed to enable a flexible trade-off between noise reduction and desired-speech preservation using a scalar trade-off parameter across all frequencies. This paper generalizes soft-constrained SSANC by introducing frequency-dependent weighting of the desired-speech preservation error while maintaining low-delay time-domain active control. Frequency-dependent weightings based on the speech intelligibility index (SII), the modified intermediate reference system (MIRS), the long-term average speech spectrum (LTASS), and a signal-dependent oracle are compared with scalar weighting using measured acoustic impulse responses in a multi-speaker scenario. The results show that LTASS and oracle weightings provide the largest speech-intelligibility improvements and comparable speech-quality improvements at substantially lower distortion levels than those obtained with SII and MIRS weightings. While the oracle weighting provides the best overall perceptual trade-off, LTASS weighting provides a comparable perceptual trade-off without requiring the clean desired-speech signal.

0.23

Audiovisual joint learning for end-to-end hearing aids

eess.AS ·2026-10-06 ·You-Jin Li, Yu Tsao, Borching Su, Kuan-Chung Ting, Fan-Gang Zeng
similarity 0.64
Abstract

Speech understanding in noise remains challenging for hearing-aid users, particularly in the presence of competing speakers. Conventional hearing aids typically perform speech enhancement (SE) and hearing-loss compensation in separate stages, which may cause enhancement errors and signal distortions to carry over to the amplification stage. Moreover, audio-only SE often provides limited benefits under competing-speech conditions because the target and interfering speech share similar acoustic characteristics, making them difficult to separate. To address these limitations, we propose AV-NeuroAMP, an end-to-end audiovisual framework that integrates noisy speech, target-talker video, and the listener's audiogram to jointly perform SE, personalized amplification, and dynamic-range compression. We further introduce audiogram-conditioned feature-wise linear modulation (AC-FiLM) to effectively incorporate listener-specific hearing profiles. Objective evaluations showed that AV-NeuroAMP outperformed conventional amplification, the audio-only NeuroAMP model, and two-stage systems on an in-domain English test set, and that these improvements were retained on an unseen Mandarin test set. Listening tests involving normal-hearing participants under simulated hearing loss and listeners with hearing loss further demonstrated improvements in speech quality and intelligibility, with the greatest benefits observed under competing-speech conditions. These findings support end-to-end audiovisual personalized amplification as a promising approach for improving hearing-aid performance in challenging acoustic environments.

0.14

"What's That Sound?": A Versatile, Robust, and Lightweight Convolutional Transformer for Environment Sound Recognition

cs.SD cs.LG ·2026-08-16 ·Julia Huang
published similarity 0.63
Abstract · 6 pages, 7 figures, published to IYRC Conference Proceedings

The conventional hearing aid is both costly and lim- ited in usage, as it is not intended to detect non-speech audio. Our objective is to develop a machine learning solution to provide a more accurate and affordable mechanism to identify surrounding sounds to improve the safety of the hearing impaired, i.e., if a car is honking behind pedestrians, or a gunshot is fired, and they need to move away from the source. By adding randomized augmentations to audio, concatenating a Mel-Frequency Cepstral Coefficients (MFCCs) diagram and a log-mel Spectrogram, and including Convolutional Neural Networks (CNNs) in a Trans- former architecture, the Randomized Audiomentational Layered Convolutional Transformers (RALCT) model efficiently extracts features from diversified audio representations. In addition, RALCT is small enough, with only approximately 310,000 parameters, to be deployed into mobile devices. Experimental results on the UrbanSound8K dataset resulted in an accuracy consistently over 93% for all variations of RALCT with the highest at 94.56%, reaching state-of-the-art levels. To leverage the capabilities of this technology, a mobile app is developed to be integrated with the model to provide real-time safety control. RALCT thus represents a robust, lightweight, affordable, and versatile deep learning tool to aid the navigation and safety of the hearing impaired.

0.23

A Novel Binaural Cue Preservation Loss for DNN-Based Binaural Speech Enhancement

eess.AS cs.SD ·2026-08-17 ·Jayteerth Amble, Thomas Haubner, Hendrik Schröter, Christoph Hoog Antink, Henning Puder
similarity 0.61
Abstract · Accepted at the International Workshop on Acoustic Signal Enhancement (IWAENC) 2026

Binaural speech enhancement for hearing aids aims to reduce noise while preserving the interaural cues needed for spatial localization. Although deep neural network-based methods achieve strong noise reduction, they often distort the rela- tionship between the left and right signals. In this paper, we propose two novel binaural cue preservation losses. First, a binaural reconstruction error loss that directly penalizes masking-induced distortion in the relationship between the left and right spectra, providing a more direct measure of the binaural consistency than conventional separate interaural level differences (ILD) and interaural phase differences (IPD) errors as in prior work. Second, a binaural cue loss that jointly models ILD and IPD to better preserve the binaural structure. Experimental results show that both proposed losses maintain strong noise reduction performance and reduce masking- induced distortion compared to the state-of-the-art baseline cue loss, while the second proposed joint binaural cue loss also outperforms the baseline in ILD preservation.

0.22

Beyond Model Size: Redesigning LiSenNet for embedded speech enhancement

eess.AS cs.LGcs.SD ·2026-09-24 ·Clément Laroche, Rasmus Kongsgaard Olsson
similarity 0.61
Abstract

Deploying real-time speech enhancement on resource-constrained devices requires meeting strict latency, memory, and energy constraints. Microcontroller NPUs can accelerate neural inference under these constraints, but only through a restricted set of operators in static, integer-quantized graphs. Recent speech-enhancement networks have reduced parameter counts and MACs to levels nominally suitable for microcontrollers, but their operators and execution patterns often remain incompatible with restricted NPUs. We address this gap by redesigning LiSenNet, a 37k parameter sub-band dual-path model, for the STM32N6570-DK Neural-ART accelerator. We replace its recurrent bottleneck with convolutional frequency and temporal mixers, reformulate unsupported operations as static int8-compatible primitives, and use bounded decoder activations to preserve quality after quantization. On VoiceBank-DEMAND, the final NPU-compatible model matches or exceeds the recurrent LiSenNet baseline, reaching PESQ 3.08 versus 3.01 in FP32 and 3.01 versus 2.93 in int8. Deployed on a microcontroller, it processes each 16 ms input hop in 4.83 ms, corresponding to a real-time factor of 0.30. Stateless receptive-field recomputation is an order of magnitude slower at the same frame rate despite higher accelerator utilization. These results show that parameter count and operator compatibility, quantization range, and persistent streaming state must be co-designed to achieve efficient real-time speech enhancement on restricted NPUs.

0.15

Subjective Evaluation of DNN AND Auditory-Model-Based Hearing-Loss Compensation

eess.AS ·2026-09-23 ·Chuan Wen, Brent Nissens, Nele De Poortere, Morgan Thienpont, Matthias Inghels +3 more
similarity 0.61
Abstract

Outer-hair-cell (OHC) loss is a primary deficit of sensorineural hearing loss (SNHL), impairing cochlear amplification and frequency selectivity and thereby elevating hearing thresholds. Biophysically-inspired DNN-based hearing-aid (HA) algorithms have been proposed to compensate for OHC deficits and have shown clear benefits in objective speech intelligibility and quality metrics (e.g. HASPI, HASQI). However, comprehensive subjective validation of these benefits in human listeners is still missing. In this work, we present a subjective evaluation of a biophysically-inspired HA model targeting OHC deficits. The cochlear module of an auditory model was individualized based on each listener's pure-tone audiogram and integrated into a trainable system, which includes the personalized model and a normal-hearing reference model, and the resulting trained HA was evaluated using a Matrix test comparing intelligibility scores for unprocessed and HA-processed noisy speech. The results revealed a significant benefit of the HA model over the unprocessed condition in the range of +1 to +27%, providing behavioral confirmation of the efficacy of the HA model. This study closes the gap between objective and perceptual evidence for this new generation of DNN-based HA algorithms, paving the way for their integration into next-generation DNN-accelerated chips for hearables and hearing aids.

0.30