CS Paper Radar

Attendance-guided speaker enrollment across meetings

Minjae Lee, Hee-Soo Heo, Youngki Kwon, Bong-Jin Lee

eess.AS ·submitted 2026-11-03 06:29 UTC · v1
benchmark

Abstract

Speaker-aware applications rely on enrolled voice profiles, yet collecting enrollment speech from every participant is impractical. Meeting archives provide an alternative source of enrollment speech, comprising recordings of recurring participants paired with expected attendance lists. We investigate whether expected attendance lists can be cross-referenced with diarized anonymous speaker clusters to identify reliable enrollment speech for recurring participants. The resulting task, multi-participant automatic enrollment, must accommodate two types of attendance noise: a listed participant may not speak, and a speaker may be unlisted. To this end, we introduce a benchmark for the task: a synthetic corpus calibrated to an in-house meeting corpus, paired with a contamination-sensitive evaluation protocol. The reference method we introduce scores cross-session cluster chains against expected-attendance patterns and abstains when the evidence is insufficient. We quantify the reference method's benchmark performance as a trade-off between enrollment success and contamination. Applying synthetic-development thresholds unchanged to real meetings exposes a synthetic-to-real gap that remains an open challenge. We will release the benchmark generator and evaluation code.

Comment
5 pages, 2 figures, submitted to ICASSP 2027

Closest papers

Nearest neighbours by meaning across the whole index.

Unsupervised Cross-device Speaker Diarization And Linking Through Spatiotemporal Structure

eess.AS ·2026-10-11 ·Suhas BN, Nikitha K
similarity 0.73
Abstract · 5 pages, 1 figure

Linking speakers across independently recorded rooms is difficult because microphone, room, and codec mismatch distort speaker embeddings. We study visitor linking without visitor enrollment or identity labels in simulated outpatient workflows. The proposed method combines per-device embedding centering, concurrency constraints, room-type-specific visit constraints, and a clustering threshold calibrated from concurrent visitors and identified staff voices. On simulated days built from VOiCES recordings, with companions, return visits, walk-ins, silent visits, and acoustic leakage, it improves B$^3$-F1 over the best label-free baseline for all 11 extractors, by 5.3-14.6 points (all 95\% intervals exclude zero). ECAPA-TDNN reaches 86.6, compared with 76.4 for the best label-free baseline and 87.7 for the best oracle-informed baseline. Gains persist across the reported workflow perturbations, but depend on removing duplicated cross-room speech: without this step, the method wins for only 2 of 11 extractors. Single-visit patients also remain vulnerable to false merges. These results support structure-aware linking while identifying preprocessing and identity-safety limitations that must be addressed before clinical deployment.

0.14

Using the enrollment gallery as evidence: affine-invariant score calibration for target speaker tagging

eess.AS ·2026-10-08 ·Hee-Soo Heo, Minjae Lee, Youngki Kwon, Bong-Jin Lee
benchmark similarity 0.72
Abstract · 5 pages, submitted to ICASSP 2027

Target speaker tagging (TST) assigns enrolled speaker identities to diarized segments of multi-speaker recordings. The enrollment utterances of the other registered speakers are an attractive source of evidence for calibrating each verification decision: they share the deployment domain of the test material and require no external cohort set. We show, however, that they cannot serve as the cohort of conventional score normalization. Enrollments tend to cluster by recording session, so the per-speaker cohort statistics reflect enrollment proximity to the rest of the gallery rather than impostor behavior, and normalization then rejects entire speakers. We propose gallery affinity verification, a cohort-free calibration that exploits the gallery through two affine-invariant terms, score dispersion and enrollment-profile deviance, and is therefore immune to such per-speaker shifts. On a synthetic benchmark and an in-house meeting corpus, it improves tagging accuracy with and without conventional score normalization and adds further gains when combined with it.

0.24

Who Said What, and Will It Be Remembered? Evaluating Persistent Speaker Attribution Across Meetings

cs.SD cs.AI ·2026-09-30 ·Shantanu Vispute, Aditya Mishra, Siddhartha Saxena
publishedbenchmark similarity 0.68
Abstract · A short version is accepted at IEEE SLT 2026, Demo Track

Speech transcripts used as long-term memory must preserve both words and stable speaker identities. Existing meeting-transcription metrics either ignore speakers or remap anonymous speakers independently in each recording, so they cannot measure whether the same person retains one identity across meetings. We evaluate persistent speaker attribution with Speaker Identified cpWER (SI-cpWER), which scores a corpus under one global speaker-ID assignment. The benchmark covers five commercial diarize-then-identify cascades, two open academic baselines, and ThyVoice on the full 129-meeting CHiME-8 NOTSOFAR evaluation set in clean and noiseaugmented form, plus CHiME-6. ThyVoice is our end-to-end reference system; it repairs overlap and gates the evidence used to create and update voiceprints. Requiring persistent identity changes the commercial ranking: ThyVoice records lower SI-cpWER than every evaluated commercial cascade in all three conditions and the lowest mean in the full panel, 47.13 versus 54.75 for the next system. Complementary lexical, diarization, per-recording attribution, and speaker-clustering diagnostics characterize upstream error surfaces in the final attributed record. These results show why persistent attribution must be evaluated directly in systems that reuse conversations across time.

0.25

Toward accurate speaker inventories for online speaker diarization

eess.AS ·2026-10-08 ·Youngki Kwon, Hee-Soo Heo, Minjae Lee, Bong-Jin Lee
similarity 0.68
Abstract · 5 pages, 1 figure, submitted to ICASSP 2027

An online speaker diarizer maintains a growing inventory of speakers, and applications read it directly: a live transcript enumerates them as they appear. Existing systems either fix the inventory's size in advance, so speakers beyond that size are merged, or grow it on distance alone, so noise and turn boundaries split speakers. Scored on speaker count over nine public datasets, both failures appear where DER reports neither: on VoxSRC-23 the fixed-capacity systems return 44-46% too few speakers and the unbounded tracker with the lowest DER 63% too many. We build on that embedding-based tracker and change only what triggers a registration. A candidate speaker pool keeps unregistered embeddings apart and registers a speaker only once one candidate has gathered enough mutually compatible embeddings; commit-gated label assignment holds an output while its candidate is undecided, so no speaker appears in the output before it is confirmed. Across the nine datasets the macro speaker-count error falls from 225% to 21% of the reference and macro DER from 17.59% to 15.95%, at a mean added latency of 0.109s over all outputs; on VoxSRC-23 the tracker obtains both the lowest DER and the most accurate speaker count of the systems compared.

0.20

OSynth: A Longitudinal, Context-Sensitive Evaluation of Meeting Understanding

cs.CL ·2026-10-21 ·Ian Porada, Suraj Tripathi, Chaitanya Animesh, Leda Sari, Chaitanya Narisetty +6 more
top venuedataset similarity 0.67
Abstract · EMNLP 2026

Meeting notetakers are increasingly deployed in settings where longitudinal context is available including calendar metadata, prior meetings, emails, slides, and speaker enrollments; however, meeting corpora remain limited in terms of size, domain coverage, and available context. We present OSynth, a large-scale synthetic dataset of simulated organizations comprising 570 hours of meeting audio together with associated artifacts and a latent ground-truth state. In a pairwise comparison, human raters distinguish OSynth meetings from real meeting transcripts with only 56% accuracy based on excerpts of 150-300 words. Across state-of-the-art LLMs, additional context yields significant gains on several entity-sensitive dimensions, demonstrating limitations of contemporary approaches and the potential usefulness of OSynth in developing better-contextualized meeting-understanding systems.

0.34

Speech Language Models for Full-Meeting Speaker Diarization: Capabilities and Limitations

eess.AS ·2026-09-19 ·Jialu Li, Jinchuan Tian, Shinji Watanabe
published similarity 0.66
Abstract · Accepted to IEEE SLT 2026

Recent advances in Speech Language Models (SpeechLMs), which integrate large language models with speech foundation models, have enabled unified sequence modeling of speech processing tasks. However, many SpeechLM-based approaches to speaker diarization (SD) are tightly coupled with automatic speech recognition (ASR) and evaluated using word-level metrics, making it difficult to assess SD performance independent of ASR accuracy. In this work, we investigate ESPnet-SpeechLM as a token-based backbone for generating SD hypotheses, formulating SD as autoregressive generation of structured tokens conditioned on acoustic input. We systematically compare two output representations: an event-based representation that explicitly models speaker turn onset and offset timestamps, and a frame-based representation that predicts frame-level speaker activity. To provide structured conversational cues, we further incorporate auxiliary tasks including speech activity detection, overlapped speech detection, and speaker turn counting within the output sequence. Across multiple meeting datasets, we find that event-based representations produce more stable and consistent SD outputs than frame-based representations. Our analysis shows that outputs generated by SpeechLMs encode useful temporal SD structure, but full-meeting SD remains limited by recording-level speaker tracking and overlap-related misses. Explicit speaker-linking post-processing substantially reduces speaker confusion, suggesting that robust SpeechLM-based SD requires persistent speaker tracking and overlap-aware generation.

0.33

Label-Free Role-Aware Segmentation of Classroom Recordings: Recovering the Teacher from Transferable Structure

cs.LG cs.SD ·2026-11-17 ·Nitin Jain, Tim Fütterer, Babette Bühler, Çağatay Yıldız
top venue similarity 0.66
Abstract · Accepted at the AAAI track is IAAI (Innovative Applications of AI)

How much the teacher talks and when are among the most informative and costly-to-measure signals of teaching. Yet, classroom audio is among the most sensitive data an AI system can process: recordings capture minors and fall under GDPR, preventing raw audio from leaving the recording device. We present a teacher-student segmentation pipeline designed to meet exactly this constraint: label-free at inference, running on a recording device, and requiring no per-session calibration. The pipeline chains a lightweight voice activity detector, compactly pretrained speaker embeddings, agglomerative clustering, and a small logistic regression that assigns the teacher role based on session-relative structure rather than voice identity. We show that our structural cue transfers across classrooms, languages, and acoustic conditions. In leave-one-teacher-out cross-validation on 292 recordings from 63 teachers across English and German corpora, our pipeline achieves teacher F1 of 0.81 and 0.87, respectively, outperforming both enrollment-based baselines and a 30-second supervised enrollment ceiling. On a commodity smartphone, its performance matches pyannote, a strong neural diarization tool, while running 4.8x faster at comparable peak memory at a real-time factor of ~0.18. We also find that transcribing filtered audio only marginally increases teacher's word error rate from $0.26$ to 0.28. Residual failures concentrate where teacher and student voices are not acoustically separable. However, we show how to flag these sessions label-free at inference so that they can be routed to human review, enabling selective, privacy-preserving deployment.

0.29

Unified Target-Speaker ASR with Text and Enrollment Speech Cues

eess.SP cs.SDeess.AS ·2026-09-27 ·Yuxiang Mei, Yuchen Yan, Dongxing Xu, Jiaen Liang, Yanhua Long
similarity 0.66
Abstract · Submitted to the ICLR 2027

Target-speaker automatic speech recognition (TS-ASR) aims to recognize a designated speaker while suppressing interfering speech in multi-talker environments. Conventional TS-ASR typically relies on an enrollment utterance, whereas text-guided methods use known lexical content, such as a wake word, to identify the target speaker from the observed mixture. These two cues provide complementary information but are usually studied separately. We propose a Unified Dual-Cue TS-ASR framework that supports text cues, enrollment speech, or both within a single model. Text cues interact with the mixture representation to extract target-speaker information conditioned on known lexical content, while an independent enrollment utterance provides complementary speaker information. Cross-attention cue-conditioning modules are integrated into shared Conformer blocks, and negative-cue sampling provides cue-validity supervision during dual-cue training. Experiments on 30,000 two-speaker mixtures across five recording/domain conditions and four oracle text-cue lengths show that, with five-character text cues, the concatenated dual-cue method achieves 8.80% CER, compared with 17.32% for text-only and 29.06% for enrollment-only inference. It also outperforms parallel dual-cue fusion (9.49% CER) and yields lower dual-cue CER across all five evaluation subsets. These results demonstrate the benefit of jointly exploiting complementary lexical and speaker information for target-speaker ASR.

0.28