CS Paper Radar

Quantizer Redundancy and Exact Index Recoding in a Frozen Neural Audio Codec

Youssef Hemimy

cs.SD eess.AS ·submitted 2026-10-14 16:49 UTC · v1

Abstract

Residual scalar quantization can produce highly dependent indices when successive stages reuse identical, unscaled grids. This case study examines that behavior in a frozen neural audio codec and develops an exact recoding procedure that reduces storage while preserving its decoder inputs. Analysis of the repeated-grid recurrence identifies predictable stage sequences; finite-precision checks motivate preserving every original index through explicit exception records. The model weights and reconstruction graph remain fixed. On the studied 62- and 512-file cohorts, a predictor, adaptive coding and explicit file selection reduce complete stored bytes. The final 512-file rate is 129.591 kbps including headers, 25.943% below an original-index XZ control. Original recorded CPU runs establish bitwise latent and waveform preservation; independent readers verify exact index and metadata recovery in subsequent saved-stream experiments. Later coding choices use previously studied cohorts, and fresh encoding requires generating three alternatives. The engineering result is implementation-specific storage reduction with exact index recovery; the underlying lossy reconstruction quality remains unchanged.

Comment
18 pages, 1 figure; includes an 8-page technical supplement. Submitted to the Journal of the Audio Engineering Society on 13 October 2026

Closest papers

Nearest neighbours by meaning across the whole index.

ZipCodec: Ultra-Low-Frame-Rate Streaming Speech Coding

cs.SD cs.AIcs.LG ·2026-09-10 ·Luca Della Libera, Cem Subakan, Mirco Ravanelli
code/data similarity 0.67
Abstract · 5 pages, 1 figure

Neural audio codecs are a fundamental component of modern speech generation systems. While recent codecs achieve increasingly low bitrates, reducing frame rate remains challenging, as each token must preserve more information while maintaining reconstruction quality. We present ZipCodec, a streaming neural speech codec operating at 6.25 Hz and 0.80 kbps with a theoretical latency of 160 ms. Our approach combines large-scale WavLM distillation with a redesigned transformer-based architecture, a scalar spherical quantizer, and a latency-aware streaming decoder. Experiments show that ZipCodec substantially outperforms existing streaming codecs at comparable bitrates in both reconstruction and downstream tasks, while operating at a significantly lower frame rate. Despite its 842M parameters, ZipCodec achieves real-time single-stream inference on a consumer-grade CPU. Demo samples, code and checkpoints are available at https://lucadellalib.github.io/zipcodec-web/.

0.36

Tracing the Origins: Legacy Codec Identification in Neural Audio Transcoding

cs.SD cs.MMeess.AS ·2026-09-14 ·Wonje Heo, Shinee Youn, Yooshin Kim, Chuck Chae, Donghoon Shin
top venue similarity 0.66
Abstract · 5 pages, 2 figures, to appear Interspeech

Residual Vector Quantization (RVQ)-based neural audio codecs (NACs) enable high-fidelity audio distribution at unprecedentedly low bitrates through discrete token-based representations. However, this shift disrupts traditional forensics, as non-linear neural transcoding obscures the underlying traces of legacy compression. This study defines the forensic gap and proposes a Transformer-based framework designed to leverage the hierarchical and temporal dependencies inherent in RVQ sequences. By modeling inter-layer causal relationships and dynamic forensic significance, our model effectively disentangles superimposed artifacts from legacy-to-neural transcoding. Experimental results achieve 97%+ accuracy for codec identification and robust joint identification performance across 32-128 kbps. These results demonstrate that traditional codec traces persist even after neural transcoding, supporting the feasibility and necessity of neural-codec-aware audio forensics.

0.46

AKDQ: Adaptive Key-Delta Quantization for Low-Bitrate RVQ-based Neural Speech Codecs

eess.AS ·2026-10-29 ·Xiao-Hang Jiang, Yang Ai, Zhen-Hua Ling, Ji Wu
published similarity 0.66
Abstract · SPL accepted

Low-bitrate speech coding is essential for bandwidth-constrained channels and on-device applications, but conventional residual vector quantization (RVQ)-based codecs employ a fixed number of codebooks per frame regardless of content. Adjacent speech frames often exhibit strong similarity, and quantizing them independently with the same scheme results in redundant encoding, thereby wasting bitrate. Therefore, we propose Adaptive Key-Delta Quantization (AKDQ), which replaces the fixed per-frame VQ allocation of RVQ with an adaptive frame-wise key-or-delta policy. AKDQ is a plug-in strategy that requires no modification to the codec's encoder-decoder backbone. Specifically, in AKDQ, each frame is classified as either a key-frame (K-frame) quantized with all VQs, or a delta-frame (D-frame) quantized with only a few VQs on a delta correction from the previous reconstruction, thereby reducing bitrate. The K/D decision is made by a budget-aware causal scheduler through a bit-bank mechanism, keeping the system streamable with no additional lookahead. Experiments on both causal and non-causal codec families demonstrate that AKDQ effectively reduces bitrate redundancy. For instance, on DAC, using AKDQ at 1.5 kbps achieves reconstruction quality comparable to using the baseline RVQ at 2.0 kbps, saving 25% of the bitrate.

0.23

Latent Audio Watermarking for Robustness to Neural Codec Resynthesis

cs.SD ·2026-09-22 ·Lovro Brulec, Sahil Karawade, Leonard Kinzinger
similarity 0.65
Abstract

Existing waveform-domain audio watermarks are robust to many conventional distortions but can degrade substantially under neural codec resynthesis. We investigate whether continuous neural codec latents provide a more suitable embedding space using a restricted formulation built around frozen pretrained EnCodec. To test this, a feedforward embedder maps a multi-bit payload to an additive latent perturbation decoded through the unchanged codec decoder. Compared with AudioSeal and WavMark, our latent watermark formulation degrades more gradually under repeated and low-bitrate EnCodec resynthesis, transfers to unseen DAC, and retains high detection under most waveform distortions. Substantial EnCodec robustness emerges even without codec-resynthesis supervision, indicating that this behavior is inherent to our latent formulation and is further strengthened by codec-aware training. Learned perturbations are also preserved more strongly through codec cycling than equal-norm random controls, with preservation depending more on channel-specific allocation than temporal structure. End-to-end perceptual quality remains close to that of the frozen EnCodec reconstruction, indicating that much of the observed degradation originates from the codec carrier itself. Overall, these results show that continuous neural codec latents provide a promising embedding space for watermarks that remain robust to neural codec resynthesis.

0.10

HE-NAC: High-Efficiency Neural Audio Coding via Hierarchical and Residual Multi-Band Quantization

eess.AS cs.SD ·2026-10-11 ·Darius Pétermann, Wootaek Lim, Seungkwon Beack, Minje Kim
top venuecode/data similarity 0.65
Abstract · Published online as Early Access in IEEE/ACM Transactions on Audio, Speech, and Language Processing (TASLP). Code: https://github.com/darius…

High-fidelity audio coding requires faithful reconstruction of perceptually important high-frequency components, which are often lost due to the aggressive temporal decimation and quantization of neural codecs. State-of-the-art neural audio coding methods rely on adversarial learning to hallucinate these missing components, yielding perceptually plausible but imprecise reconstructions; a limitation for high-fidelity music applications. As a remedy, we propose high-efficiency neural audio coding (HE-NAC), which provides a multi-band extension of the popular residual vector quantization (RVQ) scheme. HE-NAC introduces an effective training mechanism that harmonizes a bitrate-free blind upsampling process, residual learning to compensate it with minimal bitrate, and a multi-head decoder architecture for band-specific reconstruction. As a result, HE-NAC achieves a balanced trade-off, reliably reconstructing the core-band ($\leq$ 3 kHz) component while preserving overall sound fidelity. Subjective listening tests and objective evaluations demonstrate that HE-NAC outperforms both the DAC baseline and MPEG HE-AAC at equal or lower bitrates.

0.51

Neural Audio Codec for Robust Audio Deepfake Detection

cs.SD cs.MM ·2026-09-30 ·Jungwoo Kim, Joonyong Park, Junyoung Koh, Jong-Seok Lee
code/data similarity 0.65
Abstract · 5 pages, 7 figures

Audio deepfake detectors are typically evaluated on uncompressed audio, although real-world audio often undergoes low-bitrate coding. In this work, we investigate how audio coding affects deepfake detection across codecs, bitrates, and detectors, finding higher errors at lower rates. A mixed-pair protocol isolates codec-induced changes in bona fide and spoof audio, revealing asymmetric, codec-dependent failures: low-rate DAC and EnCodec mainly degrade bona fide detection, whereas X-Codec shows a stronger spoof-side limitation. Motivated by these, we propose a forensic-preserving neural audio codec (FP-NAC), which fine-tunes a pretrained codec using a detector-guided objective while preserving its native hard quantization path and bitrate. On ASVspoof 2019 LA, FP-NAC reduces EER by up to 49.8~pp compared with the original DAC at 0.5~kbps while maintaining comparable reconstruction quality. Although supervised by only one detector, FP-NAC improves performance across multiple detectors, highlighting forensic transparency as a codec design objective alongside perceptual quality. Our codes are available at https://github.com/kjungwoo03/FP-NAC.

0.20

Narrowband Voice Communication Using Streaming Neural Compression

cs.SD ·2026-09-21 ·Dahong Luo, Anannya Trehan, Aritrik Ghosh, Nirupam Roy
similarity 0.64
Abstract

Low-bitrate speech communication on resource-constrained edge devices remains challenging due to stringent computational, memory, and bandwidth constraints. We present TinyCall, a lightweight neural audio codec designed for real-time speech communication on low-power platforms such as the ESP32 microcontroller and Raspberry Pi. The proposed system targets emergency communication and other bandwidth-limited scenarios while preserving speech intelligibility, speaker identity, and vocal expressiveness. To enable efficient deployment, we propose a minimal neural audio codec architecture together with a framework for converting a causally trained codec into a truly streamable codec through pseudo-lookahead decoding and decoder-input caching. We further replace conventional residual vector quantization (RVQ) with Residual Finite Scalar Quantization (RFSQ) to reduce inference complexity on edge processors and employ a progressive three-stage training strategy for stable optimization under latent quantization. An MFCC-based perceptual loss encourages preservation of speaker characteristics, including harmonic structure and vocal timbre. Experimental results demonstrate real-time operation on a Raspberry Pi 3 while achieving intelligible speech reconstruction at bitrates as low as 2.3 kbps. The proposed approach demonstrates that practical neural speech communication is feasible on highly resource-constrained edge devices.

0.16

LACE: Layer-Wise Compression for Dynamic Frame Rate Codecs

cs.SD cs.AIcs.CL ·2026-09-15 ·Thanapat Trachu, Samuele Cornell, William Chen, Shinji Watanabe
publishedcode/data similarity 0.63
Abstract · Accepted to SLT 2026. 8 pages, 5 figures

Neural audio codecs are a key component in speech language modeling. However, their high frame rates lead to long sequence lengths, increasing computational costs. Dynamic frame rate codecs mitigate this by reducing the effective frame rate using a compression step to merge multiple frames together. However, most prior methods either operate on single-codebook codecs or apply a single compression step before multi-layer quantization. This forces all quantization layers to share the same segmentation boundaries, despite the residual embeddings at different quantization layers exhibiting different rates of change over time. We propose LACE (Layer-Adaptive Codec Encoding), a dynamic frame rate codec that applies an independent compression step at each quantization layer, enabling layer-specific segmentation boundaries. To use LACE tokens in downstream text-to-speech (TTS), we further introduce union alignment and boundary anchor mechanisms to make durations consistent across layers while preserving compression benefits. Experiments on LibriTTS show that LACE offers a better rate-quality tradeoff than prior dynamic frame rate methods on the reconstruction task and improves TTS inference efficiency while maintaining competitive synthesis quality. Our code is released as part of the ESPnet3 codec recipe.

0.40