CS Paper Radar

Insight into Parametric First-Order Ambisonics Upscaling in Time and Frequency Domains

Oded Levi, Boaz Rafaely

eess.AS cs.SD ·submitted 2026-11-17 17:11 UTC · v1

Abstract

First-Order Ambisonics (FOA) offers a practical representation for spatial audio, but its low-order reproduction introduces distinct perceptual limitations during binaural rendering. While recent methods synthesize Higher-Order Ambisonics (HOA) from FOA to overcome these constraints, the importance and potential of early reflections in the upscaling process have not yet been comprehensively explored. To bridge this knowledge gap and understand the potential of incorporating early reflections in FOA upscaling, this work develops an experimental framework that selectively upscales these reflections. By utilizing an energy-based criterion, the proposed framework identifies and selectively renders the strongest early reflections with high-order resolution. The selection mechanism is evaluated across three domains: long-term temporal, short-time temporal, and the joint time-frequency (TF) domain. Validation through objective metrics, including the Scale-Invariant Signal-to-Distortion Ratio (SI-SDR), the clarity index ($C_{50}$), and Interaural Cross-Correlation (IACC), alongside listening tests, demonstrates that the TF-domain provides superior performance. By exploiting the sparsity of speech, the TF approach maintains robust performance in complex multi-talker scenarios, providing insights into the benefits of early reflections for the design of future parametric and data-driven FOA upscaling methods.

Comment
10 figures. Listening-test data: https://doi.org/10.5281/zenodo.24257365

Closest papers

Nearest neighbours by meaning across the whole index.

Array-Agnostic High-Order Ambisonics Encoding via Diffusion Posterior Sampling

eess.AS eess.SP ·2026-10-29 ·Amit Milstein, Nir Shlezinger, Boaz Rafaely
similarity 0.62
Abstract

Ambisonics provides a device-independent representation of three-dimensional sound fields; however, extracting these coefficients from practical microphone array recordings remains strongly dependent on array geometry, acoustic transfer functions, and spatial sampling resolution. This pipeline is typically divided into two distinct tasks: encoding, which entails reconstructing the physically resolvable spatial components, and upscaling, which requires generating higher-order coefficients. Model-based approaches often fail on irregular arrays due to ill-conditioned transfer matrices, whereas conventional neural encoders remain strictly tied to the specific acquisition setup seen during training. Furthermore, existing upscaling methods usually assume ideal low-order inputs, ignoring device-specific array artifacts. We present ADEUPS, a unified framework that couples an array-independent Ambisonic diffusion prior with an explicit physical acquisition model during inference, enabling zero-shot encoding and upscaling across diverse array configurations. We train our prior using an order-masking strategy and perform inference via score blending, which we demonstrate to be crucial for solving the underlying inverse problem. Through extensive numerical evaluations across various array geometries, we show that ADEUPS consistently outperforms task- and array-specific baselines. Finally, we conduct subjective listening tests to confirm the perceptual effectiveness of our approach.

0.12

An Efficient Parametric Codec for Low-Bitrate First-Order Ambisonics

eess.AS cs.SD ·2026-09-27 ·Wei-Ting Lai, Amy Bastine, Lachlan Birnie, Thushara D. Abhayapala, Prasanga N. Samarasinghe
similarity 0.61
Abstract

Driven by the rapid growth of immersive teleconferencing and generative spatial audio, efficient low-bitrate coding of first-order Ambisonics (FOA) has become increasingly important. In this work, we develop a lightweight parametric FOA codec that retains the standard Directional Audio Coding analysis and synthesis while redesigning the spatial metadata quantization scheme. Rather than quantizing direction-of-arrival (DOA) and diffuseness independently, we combine them into a 3-D directivity vector and jointly quantize these vectors across frequency bands via residual vector quantization (RVQ). The RVQ codebooks are optimized within minutes via stage-wise k-means without backpropagation, yielding a constant-bitrate (CBR) representation that decouples metadata rate from the number of frequency bands. Evaluations show that our approach outperforms low-bitrate perceptual codecs in FOA reconstruction, remains competitive with neural codecs on the downstream Sound Event Localization and Detection task, and maintains robust performance when paired with different external monaural codecs. Given its lightweight training, strong performance, and CBR design, we consider the proposed method to be a favorable and reproducible baseline for future FOA codec research.

0.23

Generative Learning for Ambisonic Upscaling

eess.AS eess.SP ·2026-09-20 ·Amit Milstein, Nir Shlezinger, Boaz Rafaely
top venue similarity 0.61
Abstract

Ambisonics Upscaling (AU) aims to enhance the spatial resolution of sound fields by estimating high-order Ambisonics (HOA) components from low-order observations. While deep learning and model-based strategies have been considered for AU, both approaches exhibit significant performance degradation in realistic scenarios, where reverberant sound fields violate the directional sparsity inherent to discriminative mappings. In this work, we address AU as a generative task rather than a deterministic reconstruction, expanding generative modeling to specifically target the recovery of spatial information in reverberant speech. We investigate two dominant continuous-time generative paradigms, adapting both Score-based Generative Model and Flow Matching to these complex acoustic settings. We provide an extensive numerical study comparing our methods against state-of-the-art baselines in various acoustic scenarios. Additionally, we conduct subjective listening tests to evaluate the perceived quality and spatial accuracy of the proposed generative framework across various reverberant scenarios. The studies reveal that Flow Matching consistently outperforms both its discriminative counterparts and Diffusion-based paradigms in all reverberant settings.

0.56

Ambisonics Encoding of Room Impulse Responses using a Device-Agnostic Diffusion Model

eess.AS cs.SD ·2026-08-14 ·Eloi Moliner, Christoph Hold, Juan Azcarreta Ortiz, Sebastian Prepelita, Ishwarya Ananthabhotla +3 more
similarity 0.58
Abstract · IWAENC 2026

We address the problem of encoding room impulse responses (RIRs) into high-order Ambisonics (HOA) representations from arbitrary and potentially insufficient or incomplete microphone array measurements. This task is fundamentally ill-posed for microphone arrays with limited spatial capture capabilities, such as irregular or sparse arrays, as classical linear methods fail to reconstruct high-order spatial detail. We introduce a diffusion-based generative framework that models the statistical properties of HOA RIRs. This enables device-agnostic encoding from arbitrary microphone arrays, potentially unseen during data measurement. Our approach incorporates a posterior sampling procedure that enforces consistency between the estimated signals and the measurements while plausibly reconstructing spatial information that is unobservable from the limited measurements alone. Experiments on simulated data demonstrate that our method outperforms linear and neural baselines, achieving accurate HOA RIR estimation up to 12th order. A listening test with binaural renderings, including both simulated and measured RIRs, further confirms that the proposed method yields higher perceptual similarity to reference Ambisonics RIRs than all baselines. The flexibility and accuracy of the proposed framework opens new possibilities for scalable acoustics simulations.

0.39

HRTF Upsampling and Personalization via Neural Operators

cs.SD cs.LG ·2026-10-14 ·Meenakshi Krishnan, You Zhang, Ramani Duraiswami, C. Phillip Brown
similarity 0.57
Abstract

Head-related transfer function (HRTF) spatial upsampling from sparse measurements is a fundamental challenge in personalized binaural audio. We formulate HRTF upsampling as an operator learning problem and introduce SPARO, a neural operator that maps sparse measurements at a set of spatial directions to the HRTF field, a continuous representation that can be queried at arbitrary positions. SPARO projects the measurements onto a learned set of spherical latent modes, processes the latent modes with Transformer blocks, and decodes the resulting representations at query directions. On SONICOM, using the LAP24 measurement configuration, SPARO attains the best log-spectral distortion (LSD) and interaural level difference (ILD) error at every sparsity level from 3 to 100 measurements (18% lower LSD and 43% lower ILD error than the strongest baseline at 100 measurements) while remaining competitive in interaural time difference (ITD) error. It reconstructs a dense HRTF 15 to 373 times faster than the leading neural-field method RANF. We further validate the approach on HUTUBS, demonstrating its effectiveness across HRTF datasets.

0.11

BAmbiDrop: Array-Agnostic Speech Enhancement with Zero-Shot binaural Reproduction using Multi-center Ambisonics Encoding

eess.AS ·2026-11-07 ·Michael Tatarjitzky, Vladimir Tourbabin, Boaz Rafaely
similarity 0.57
Abstract · Submitted for ICASSP 2027

Array-agnostic enhancement networks generalize across microphone geometries but produce a single channel, while binaural enhancement methods preserve interaural cues only for one fixed device geometry. We propose BAmbiDrop, a framework that extends array-agnostic enhancement to binaural reproduction without retraining. Exploiting the fact that the Ambisonics origin is an arbitrary point in space, a frozen single-channel AmbiDrop network estimates a complex mask at multiple centers across the array; these masks are fused by a complex mean into a single shared mask, applied to the microphone signals and rendered by fixed BSM filters. Across 21 free-field and rigid-sphere arrays, the fused mask improves enhancement and interaural cue reproduction simultaneously, whereas masking each microphone independently degrades the latter without a consistent gain in the former.

0.25

Mask-Based Speech Enhancement for Spatial Audio: A Comparison of Ambisonics, Beamforming, and Microphone Channels

eess.AS ·2026-09-16 ·Sheli Hendel, Boaz Rafaely, Dorothea Kolossa
similarity 0.55
Abstract

Mask-based speech enhancement is widely used for suppressing noise and interference, but its performance in spatial audio algorithms with multichannel output has not been studied extensively. In such settings, speech enhancement must improve speech quality while preserving spatial cues that are essential for localization, spatial awareness, and spatial release from masking. In this work, we systematically compare time frequency masking applied to three signal representations: microphone signals, beamformer outputs, and Ambisonics signals. Performance is evaluated in terms of speech quality, intelligibility, binaural cue preservation, and reverberation preservation. Results reveal a clear trade-off between enhancement and spatial fidelity: beamformer-domain masking achieves the highest speech enhancement scores, while Ambisonics-domain masking better preserves the spatial attributes of the residual interference. All methods preserve the target's localization cues.

0.15

Spatial Audio Generation and Editing with Executable Plans and Self-Distilling Transfusion

cs.SD cs.AI ·2026-10-25 ·Kam Man Wu, Yu Zhang, Pengjun Fang, Mingfei Chen, Weijia Chen +2 more
code/datacommunity pick similarity 0.55
Abstract · 28 pages, 2 figures

Spatial audio authoring coordinates source decisions with soundfield synthesis. For editing, plans specify what to change and retain, while reference audio supplies further acoustic detail. We present AMBIT, a unified framework for native First-Order Ambisonics (FOA) generation and editing with three connected contributions. First, a shared Transfusion backbone couples autoregressive planning with rectified-flow rendering through executable ScenePlans that specify source content, activity, and 3D trajectories. For editing, the plan describes the full target scene, including retained sources; contrastive features ground the requested change, while full reference latents supply residual acoustic detail for preservation. Second, an FOA-aware continuous VAE combines shared-decoder omnidirectional recovery, asymmetric grouped KL, and spatial covariance matching to preserve content and inter-channel structure. Third, joint Transfusion on-policy self-distillation (OPSD) refines planning on student-generated prefixes and rendering at sampling states reached under student plans, with target-conditioned teachers supervising both modes of the shared student. Evaluations across speech, music, and sound show improved generation content and spatial quality, acoustic fidelity during instruction-guided editing, and unedited-source preservation over the compared baselines. Project page: https://wjm-george.github.io/AMBIT/

0.27