CS Paper Radar

AMemOps: Evaluating Memory Updates in Long Audio Streams

Peiwei Chang, Li Zhang, Mark D. Plumbley, Philip J. B. Jackson, Wenwu Wang

cs.SD cs.AIcs.LG ·submitted 2026-10-13 16:16 UTC · v1
benchmark

Abstract

Audio memory must recognise recurring sources across acoustic changes while keeping their attributes up to date. Overall accuracy does not reveal how these demands interact. We introduce Audio Memory Operations (AMemOps), a benchmark for latest-attribute retrieval, preservation of unchanged attributes and absent-source rejection in long audio streams. Paired controls vary recordings, training identities and earlier events to examine their effects on updating. Training on other recordings of the test instruments increases a reference delta-rule memory's update advantage over matched attention by 13.4 percentage points. At retrieval delays of 120\,s, matched histories separate benefits of consistent repetition from costs of conflicting earlier values, while additive-memory controls identify benefits of erasure.

Closest papers

Nearest neighbours by meaning across the whole index.

RefTAG: Reflection-aware Temporal Audio Grounding in Large Audio Language Models

eess.AS ·2026-10-12 ·Yuhang Dai, Xin Shu, Zhennan Lin, Mingchen Shao, Chengyou Wang +3 more
code/data similarity 0.65
Abstract · 5pages, 2 figures, 3 tables

Temporal audio grounding requires locating every occurrence of a queried target, making both occurrence counts and interval boundaries essential. We propose RefTAG, a post-training framework for verifiable temporal self-correction. Its Reflection Chain-of-Thought (Re-CoT) protocol combines draft intervals, discrete count and boundary actions, and revised intervals in one response. Supervised fine-tuning on these trajectories (Re-CoT-SFT) teaches the protocol; reinforcement learning (RL) jointly rewards final localization and revision outcomes. Across 8,228 queries, Re-CoT-SFT improves query-weighted mIoU over final-answer SFT by 1.74 percentage points. RL further improves TAG-Bench multi-target mIoU by 0.40 points and R@0.5 by 0.86 points. The final model achieves 38.50% TAG-Bench mIoU and 51.01% mean mIoU across three datasets, outperforming the evaluated open-source MOSS-Audio and FireRedAudio models and the commercial Gemini-3.1-Pro on both measures. These results demonstrate that explicitly coupling count- and boundary-aware revision with joint reward optimization provides an effective and verifiable path to temporal audio grounding.

0.20

When Sounds Turn Heads: Recovering Event Timing That Audio LLMs Fail to Say

cs.SD cs.CL ·2026-11-19 ·Jongbhin Woo, Jaewook Jeong, Joon Son Chung
benchmark similarity 0.64
Abstract · 11 pages, 4 figures

Audio LLMs can know when a queried sound occurs but fail to say it. We find a few attention heads that track the queried sound: they shift when the query changes, follow the sound when it moves, and removing or blocking them often hurts the model's timestamps. Yet the answer does not express what they track. Intervals read directly from these heads outperform the model's original answer and recover events that its timestamps miss. To bring this timing into the answer, we introduce head-guided decoding, which needs no training. It keeps the model's decision on whether the sound is present and rescores its candidate timestamps with a prior from these heads. Heads selected on development data transfer unchanged to new recordings and free-text queries, where head-guided decoding improves the timestamps of three audio LLMs, scores above their head readout on every benchmark, and, in most settings, outperforms random-head guidance. On recordings released after the models were trained, head-guided decoding raises mIoU from 0.07-0.36 to 0.48-0.56. These results suggest that poor temporal answers in audio LLMs partly reflect a failure to say rather than to hear, and that the model's own attention can close part of this gap.

0.22

Post-fusion Event-aware Time-Frequency Refinement for Audio Moment Retrieval

cs.SD ·2026-11-16 ·Feiyang Xiao, Li'ang Luo, Kejia Zhang, Qiaoxi Zhu, Guangjun He +2 more
publishedbenchmark similarity 0.63
Abstract · Accepted by NCMMSC 2026.Demo Page: https://littleflyingsheep.github.io/etr-amr-demo/

Audio Moment Retrieval (AMR) aims to locate the temporal segment within a long audio recording that semantically corresponds to a natural-language query. This task is particularly challenging because query-related audio events are often sparsely distributed and embedded in complex acoustic environments, where background sounds and acoustically similar events can easily obscure the target moments. Existing methods mainly strengthen audio-text semantic interaction to enhance the query-related information in audio frames, but seldom refine the resulting audio representations before temporal localization, limiting their ability to accurately distinguish target events from irrelevant acoustic interference. To address this issue, we propose ETR-AMR, a post-fusion event-aware time-frequency refinement framework for AMR. Specifically, an event-aware time-frequency refinement (ETR) strategy is proposed after audio-text interaction to refine fused audio frame features before temporal moment localization. ETR consists of two complementary mechanisms: time frame gating and frequency refinement. The time frame gating mechanism estimates frame-wise event relevance to emphasize query-related audio frames and suppress redundant temporal responses, while frequency refinement applies learnable time-frequency reweighting to enhance discriminative event-related frequency patterns and reduce background-dominant interference. By jointly refining fused audio frame features in the temporal and frequency domains, ETR produces more discriminative event-aware features for accurate temporal boundary prediction. Experiments on four AMR datasets demonstrate that ETR-AMR achieves superior performance compared with existing methods, especially on the high-quality CASTELLA benchmark. Duration-wise analysis further confirms the effectiveness of ETR for medium and long audio moments.

0.25

VoxMem: Benchmarking Multimodal Memory in Large Audio Language Models

eess.AS cs.SD ·2026-09-26 ·Yang Xiao, Vidhyasaharan Sethu, Eun-Jung Holden, Ting Dang
position similarity 0.63
Abstract · working in process

Spoken conversational systems must recover information from prior interactions (i.e., memory), yet relevant information in speech extends beyond what was said to who said it, how it was spoken, and what was audible, information that exists only in the audio signal and cannot be recovered from a transcript. Beyond what to remember, memory also demands diverse operations: retrieving a single fact, integrating evidence across turns, tracking an evolving state. Real interactions further unfold across sessions, meaning information accumulates across distinct episodes rather than a single continuous recording. Existing benchmarks fall short on all three dimensions: they focus primarily on lexical content, adopt limited and ad hoc memory operations, and treat memory as a single-session problem. We argue that principled memory evaluation requires jointly characterizing the acoustic evidence to be retained and the operations applied to it, and introduce a taxonomy along these two axes. Building on this taxonomy, we present VoxMem: 3,196 evaluation instances over 34,743 spoken sessions (177 hours) crossing four acoustic evidence types (speech semantics, speaker identity, paralinguistic cues, environmental sound) with four memory operations (information extraction, multi-session reasoning, temporal tracking, and answer refusal), grounded in multi-session histories and stratified across context budgets from 8K to 64K tokens. Evaluating 15 LALMs, no model exceeds 40% at 32K. Models retain what was said far better than who said it, how, or what was audible, a gap that widens for complex operations, grows with history length, and manifests as qualitatively distinct failure modes across evidence types. VoxMem aims to provide a foundation to measure and drive progress on the full scope of spoken conversational memory.

0.21

REVE: Efficient Hallucination Correction for Large Audio-Language Models via Reused Encoder States

cs.SD cs.AI ·2026-09-22 ·Hongjin Song, Jiasheng Kuang, Xinyu Yang, Qiuyu Fang, Ziyu Wu +2 more
similarity 0.63
Abstract

Large audio-language models may mention acoustic events that are absent from the input. A separate audio event detector can verify these mentions, but doing so requires a second audio encoder and a separate forward pass. We propose Reused Encoder States for Verifying Events (REVE), a lightweight method that uses states already computed by the target model. One readout summarizes class scores across audio frames, while another uses pooled states from four consecutive frame intervals. Class-aware score fusion combines their outputs to verify generated event mentions without encoding the audio again. On AudioSet, REVE removes 92.9% of label-unsupported mentions under a faithful-mention recall constraint. With fewer added parameters and no second audio-encoding pass, REVE achieves a reduction comparable to those of CED-Tiny and CED-Base. Its complete verification latency is about 1/18 of the CED-Base path. Results on controlled DESED mixtures and different target-model architectures further confirm the effectiveness of encoder-state reuse.

0.30

Parameter isolation with domain-specific experts for incremental audio classification

eess.AS cs.LGcs.SD ·2026-09-13 ·Jongyeon Park, Do-Hyeon Lim, Sang-won Park, Hong Kook Kim, Kyungdeuk Ko +2 more
similarity 0.62
Abstract · 5 pages, 3 figures, 3 tables. Accepted to the Detection and Classification of Acoustic Scenes and Events (DCASE) Workshop 2026

To successfully deploy a model in time-varying environments such as streaming data prediction and sensing control, domain-incremental learning (DIL) has attracted attention since it aims to adapt a previously trained model to newly arriving domains, while reserving knowledge from earlier domains without accessing their data. Incremental learning across domains can be regarded as a recurrent update, in which the current model is obtained by updating the model carried over from previous domains. Conventional DIL approaches that rely on domain-invariant feature learning and weight regularization gradually overwrite or constrain parameters learned in previous domains, leading to catastrophic forgetting. Instead, this paper proposes a new domain-specific parameter-isolation architecture that retains all past domains. The proposed architecture mitigates catastrophic forgetting through a full-order recurrent update, constructing a new expert using domain-specific data conditioned on all previously frozen models. To achieve this, we incorporate data-free generative replay to reconstruct previous-domain data and cross-domain feature generation to recover later expert features missing from earlier domain samples. Finally, we apply the proposed model architecture to domain-agnostic incremental learning for audio classification, as defined in the DCASE 2026 Challenge Task 7. Consequently, we achieve micro and macro accuracies of 78.4% and 78.9%, respectively, representing increases of 33 and 25 percentage points over the Challenge baseline. Ablation studies are conducted to examine the effectiveness of each processing component in terms of classification accuracy.

0.22

Beyond the Stability--Plasticity Frontier in Streaming Target Speaker Extraction

eess.AS ·2026-09-17 ·Yuesheng Ma, Linyang He, Nima Mesgarani
code/data similarity 0.61
Abstract

Streaming target speaker extraction must maintain a representation of whom to extract while the target may fall silent, be masked by interference, or drift acoustically away from enrollment. Existing systems typically hold this state as a stored embedding updated by hand-designed rules. Across 22 configurations, including confidence-gated and oracle-activity-gated updates, we show that this family lies on a stability-plasticity frontier: even perfect target-activity information cannot combine robustness to target absence with adaptation to enrollment-mixture mismatch. We therefore meta-train speaker-state dynamics through the closed streaming loop, exposing the updater to its own contaminated evidence. Our proposed 41k-parameter anchored fast-weights (AFW) memory moves beyond the measured heuristic frontier, gaining 3.0 dB over the best heuristic under severe mismatch while staying within 0.9 dB of static enrollment after 30 s of absence, at under 5% runtime overhead. A gated recurrent unit (GRU) control confirms that the gain is not AFW-specific, while AFW is smaller and more interpretable: under severe mismatch, its write residual grows and aligns with the target rather than the interferer. Code is publicly available at https://github.com/ym2976/anchor-fast-weight.

0.39

Steer Unified Audio Editing Toward Reliable Source Selection

cs.SD ·2026-11-07 ·Youquan Fu, Dongnuan Cai, Yu Pan, Henghui Du, Yujia Zhang +6 more
benchmarkcode/data similarity 0.61
Abstract

Unified audio editors support diverse operations, yet often struggle to modify the intended source while preserving non-target content. We show that source selection in audio editing is shaped by early denoising states, while incorrect early selections can hardly be corrected during subsequent denoising. Motivated by these findings, we propose Source-selection Training via Early-trajectory Emphasis and Recovery (STEER), a two-stage post-training algorithm for reliable selective audio editing. STEER-Emphasis prioritizes training on task-critical early states to reduce source-selection errors, while STEER-Recovery learns corrective denoising from self-generated rollout states to recover from early deviations. To systematically assess source-selection reliability, we introduce the Selective Audio Editing Benchmark, enabling large-scale automatic evaluation of target manipulation and non-target preservation. Applying STEER to our pretrained multitask base editor gives SteerSound, a unified audio editor with leading selective editing performance among open-source audio editors. Experiments on our base editor, UNISON, and SAM Audio demonstrate that STEER consistently improves both selective editing capability and overall editing performance. Early-trajectory emphasis further brings substantial improvements in transcript-conditioned target speaker removal and separation, highlighting the broader applicability of this training principle.

0.24