CS Paper Radar

Toward accurate speaker inventories for online speaker diarization

Youngki Kwon, Hee-Soo Heo, Minjae Lee, Bong-Jin Lee

eess.AS ·submitted 2026-10-08 05:41 UTC · v1

Abstract

An online speaker diarizer maintains a growing inventory of speakers, and applications read it directly: a live transcript enumerates them as they appear. Existing systems either fix the inventory's size in advance, so speakers beyond that size are merged, or grow it on distance alone, so noise and turn boundaries split speakers. Scored on speaker count over nine public datasets, both failures appear where DER reports neither: on VoxSRC-23 the fixed-capacity systems return 44-46% too few speakers and the unbounded tracker with the lowest DER 63% too many. We build on that embedding-based tracker and change only what triggers a registration. A candidate speaker pool keeps unregistered embeddings apart and registers a speaker only once one candidate has gathered enough mutually compatible embeddings; commit-gated label assignment holds an output while its candidate is undecided, so no speaker appears in the output before it is confirmed. Across the nine datasets the macro speaker-count error falls from 225% to 21% of the reference and macro DER from 17.59% to 15.95%, at a mean added latency of 0.109s over all outputs; on VoxSRC-23 the tracker obtains both the lowest DER and the most accurate speaker count of the systems compared.

Comment
5 pages, 1 figure, submitted to ICASSP 2027

Closest papers

Nearest neighbours by meaning across the whole index.

Slots Are Not Speakers: Training-Free Speaker Linking for Frozen Streaming Diarizers

cs.SD cs.AI ·2026-10-20 ·Aditya Srinivas Menon, Kumarmanas Nethil, Kavya Manohar
similarity 0.73
Abstract · Submitted to ICASSP 2027

This paper presents a causal, training-free linker that allows a frozen streaming diarizer to track more speakers than it has output slots. Streaming end-to-end diarizers such as Streaming Sortformer emit a fixed number N of concurrent speaker slots. Above N speakers, they keep detecting speech but reuse slots across people, because N bounds how many speakers the model can express at once, not how many it will meet. The linker uses only the host's slot posteriors and the audio. It keeps the frames where one slot is active, splits each slot where the speaker changes, embeds each piece with an off-the-shelf speaker encoder, and matches it against a bank of speakers seen so far. It only relabels, so missed speech and false alarm are identical to the host's and every gain is a reduction in speaker confusion. On 1418 sessions from eight corpora, it reduces the 4-slot host's DER above its cap from 36.8% to 22.6%, and barely moves it below the cap. The same constants transfer to the host's 8-slot successor and to two streaming EEND hosts from a different model family. On courtroom hearings with up to 23 speakers it reduces cpWER from 82.7% to 24.4%.

0.15

Unsupervised Cross-device Speaker Diarization And Linking Through Spatiotemporal Structure

eess.AS ·2026-10-11 ·Suhas BN, Nikitha K
similarity 0.70
Abstract · 5 pages, 1 figure

Linking speakers across independently recorded rooms is difficult because microphone, room, and codec mismatch distort speaker embeddings. We study visitor linking without visitor enrollment or identity labels in simulated outpatient workflows. The proposed method combines per-device embedding centering, concurrency constraints, room-type-specific visit constraints, and a clustering threshold calibrated from concurrent visitors and identified staff voices. On simulated days built from VOiCES recordings, with companions, return visits, walk-ins, silent visits, and acoustic leakage, it improves B$^3$-F1 over the best label-free baseline for all 11 extractors, by 5.3-14.6 points (all 95\% intervals exclude zero). ECAPA-TDNN reaches 86.6, compared with 76.4 for the best label-free baseline and 87.7 for the best oracle-informed baseline. Gains persist across the reported workflow perturbations, but depend on removing duplicated cross-room speech: without this step, the method wins for only 2 of 11 extractors. Single-visit patients also remain vulnerable to false merges. These results support structure-aware linking while identifying preprocessing and identity-safety limitations that must be addressed before clinical deployment.

0.14

Target Speaker Identification: A Low-Latency Streaming Pipeline

cs.SD ·2026-08-18 ·Patrick S. Burke, Satyam Raj, Sean Kinahan
benchmarkcode/data similarity 0.70
Abstract · 8 pages, 4 figures

We present a real-time pipeline of open source, pretrained models for streaming identification of a target speaker, motivated by hearing-aid applications where latency as low as 10 ms can be perceptible. We formulate a two-step approach in which incoming audio is first segmented by speaker using low-latency streaming diarization, followed by speaker verification against a registered target speaker. To emulate conversational speech while minimizing overlap, we use the This American Life Podcast Transcripts dataset and select the host as a consistent target speaker. We benchmark offline diarization with Pyannote and LIUM using diarization error rate (DER) and select Pyannote based on baseline performance and compatibility with streaming. We then evaluate speaker verification using Pyannote and TitaNet-Large and generate ROC curves to select an operating region. We integrate Diart and tune clustering parameters to reduce DER while maintaining real-time operation. We pair Diart with Pyannote verification and evaluate system-level performance by converting predicted and ground-truth speech regions into 100 ms binary masks. Across 17 evaluation episodes, the system achieves greater than 0.90 median accuracy with high specificity (0.95-0.98) at cosine distance thresholds of 0.7-0.75, demonstrating a practical proof of concept for downstream low-latency selective amplification.

0.27

eDIAR: Training-Free Open-Speaker Streaming Diarization for Long Meetings and Large Archives via e-means

cs.SD ·2026-11-13 ·Kyudan Jung, Jihwan Kim, Minwoo Lee, Eunji Shin, Jaegul Choo
code/data similarity 0.69
Abstract · Demo & Project page: https://kyudan1.github.io/eDIAR/

Meetings last up to an hour, and nobody knows in advance how many people will speak. Neural diarizers with a fixed number of output slots, such as Streaming Sortformer and Nemotron-3, are accurate and run online, but they cannot name more speakers than they have slots. We remove this limit without retraining. The slot head keeps deciding who is active and when speech overlaps, and speaker embeddings are clustered inside a single slot, with the official e-means algorithm, only after every slot has been used. In the streaming system, re-clustering is confined to a bounded window, labels are revised for a few seconds, and identities are anchored to the slots of the head. The resulting system has no limit on the number of speakers and stays accurate when speakers outnumber the slots, where fixed-slot models break down. On meetings that grow to an hour it stays far more accurate than Streaming Sortformer, which accumulates errors, and its cost per update does not grow with the meeting. The same recipe turns the four-slot Streaming Sortformer into an open-speaker system. Across recordings, e-means makes it affordable to link the speakers of a large archive frame by frame, a task that arises when conversational corpora are prepared, and it reaches the clustering quality of Lloyd's $k$-means several times faster. The method needs little tuning, and e-means runs with its default recipe from a few hundred to more than a million embeddings.

0.37

Beyond DER: Speaker Counting in Crowded End-to-End Diarization

eess.AS ·2026-09-23 ·Lahiru Samarakoon, Hongxu Zhu
publishedhot direction similarity 0.69
Abstract · Accepted to IEEE SLT 2026

Speaker diarization must solve two problems: counting how many speakers are present in a given conversation, and assigning speech to each one. This becomes harder in crowded conversations with five or more speakers. We evaluate how end-to-end neural diarization models count speakers, and find systematic under-counting in crowded recordings. The standard diarization error rate (DER) hides this failure, because it is duration-weighted and barely penalizes the dropped, low-activity speakers. We therefore also report the Jaccard error rate (JER) and explicit counting metrics. We propose a gated loss that couples speaker existence with frame activity. This loss can be computed in two ways, duration-weighted or speaker-weighted, and the speaker-weighted variant, used as a regularizer, reduces the under-count. On real-world crowded recordings our method clearly improves the counting metrics, lowering JER by about 6% relative and DER by about 9% relative, while leaving sparse recordings unharmed.

0.37

OpenSVBench: Diagnosing Speaker Verification Generalization Across Deployment Scenarios

eess.AS cs.SD ·2026-10-09 ·Qituan Shangguan, Bosen Xu, Shuai Wang
publishedbenchmarkcode/data similarity 0.68
Abstract · 15 pages, 3 figures, 3 tables. Accepted at NCMMSC 2026. Code: https://github.com/shangguanqituan/opensvbench-submission-kit ; leaderboard: h…

Speaker verification (SV) systems now achieve very low error rates on standard public protocols, but those results do not fully describe deployment generalization. VoxCeleb-style evaluations remain useful anchors for English in-the-wild media speech. Scenario-specific evaluations and challenges are also valuable. However, neither setting directly shows whether the same system remains reliable across diverse deployment scenarios. We introduce OpenSVBench, a public benchmark for cross-scenario SV evaluation. Its fixed public core contains 26 trial sets organized into 11 scenario groups, with 9.15M verification trials, frozen target/non-target keys, and metadata for scenario-wise reporting. Around this core, the online leaderboard acts as a diagnostic evaluation interface rather than only a ranked list. It accepts score packages from new systems, evaluates them with frozen keys and common scoring, and reports trial-set breakdowns, scenario-group metrics, macro EER/minDCF, model-wise profiles, and scenario-wise comparisons. We evaluate 18 public checkpoints spanning x-vector, ECAPA-TDNN, CAMPPlus, ResNet and ERes variants, ReDimNet2, SAM-ResNet, and pretrained Transformer speaker recipes. The results show clear cross-scenario gaps: systems with similar macro EER can have different scenario profiles, strong performance in one scenario does not guarantee strong performance in another, and same-scenario trial sets can still expose source and protocol gaps.

0.40

When Forgetting Looks Like Improvement: Metric Masking in Streaming Diarizer Adaptation and the Price of Rehearsal

cs.CL cs.SDeess.AS ·2026-09-26 ·Mo Yu, Yang Liu, Jing Qian
similarity 0.68
Abstract · 5 pages, 4 figures

Small-data adaptation can improve speech detection while degrading speaker attribution. We study this discrepancy in a released streaming diarizer adapted on 7.5 h of two-party conversation and evaluated across six corpora. Adaptation substantially improves in-domain diarization performance and transfers to an independent corpus. However, this improvement is not consistent across evaluation scenarios as the additional confusion is mainly associated with impaired temporal identity consistency rather than speaker-count errors. A local-remapping diagnostic reveals different patterns of identity degradation across corpora, indicating that adaptation may alter how streaming models maintain speaker assignments over time. Rehearsal reduces the observed degradation but reduces the cross-domain transfer performance. These results highlight the need to jointly evaluate detection accuracy, identity consistency, and retention behavior when adapting streaming diarization systems.

0.24

Attendance-guided speaker enrollment across meetings

eess.AS ·2026-11-03 ·Minjae Lee, Hee-Soo Heo, Youngki Kwon, Bong-Jin Lee
benchmark similarity 0.68
Abstract · 5 pages, 2 figures, submitted to ICASSP 2027

Speaker-aware applications rely on enrolled voice profiles, yet collecting enrollment speech from every participant is impractical. Meeting archives provide an alternative source of enrollment speech, comprising recordings of recurring participants paired with expected attendance lists. We investigate whether expected attendance lists can be cross-referenced with diarized anonymous speaker clusters to identify reliable enrollment speech for recurring participants. The resulting task, multi-participant automatic enrollment, must accommodate two types of attendance noise: a listed participant may not speak, and a speaker may be unlisted. To this end, we introduce a benchmark for the task: a synthetic corpus calibrated to an in-house meeting corpus, paired with a contamination-sensitive evaluation protocol. The reference method we introduce scores cross-session cluster chains against expected-attendance patterns and abstains when the evidence is insufficient. We quantify the reference method's benchmark performance as a trade-off between enrollment success and contamination. Applying synthetic-development thresholds unchanged to real meetings exposes a synthetic-to-real gap that remains an open challenge. We will release the benchmark generator and evaluation code.

0.11