CS Paper Radar

Measuring Public Participation in Local Government: An Audio-Based Instrument for Classifying Speakers in City Council Meetings

Menglin Liu, J. Sophia Wang, Ge Shi

cs.CY ·submitted 2026-11-13 11:55 UTC · v1

Abstract

How much does the public actually speak in local government? Despite sustained interest in democratic participation, we lack large-scale measures of public voice at the local level, because the underlying record -- audio of open meetings -- has been prohibitively costly to convert into data. We contribute a measurement instrument: an audio-analysis pipeline that classifies each speaker in a city council meeting as an elected official or a member of the public directly from audio, and aggregates these labels into validated measures of participation such as floor-time share. The pipeline first segments each recording by speaker, then classifies each speaker with a multimodal large language model (MLLM), using the meeting's own temporal structure (neighboring speakers, iterative refinement, and a majority vote across three classifiers) as supporting components. We collect 1,618 recordings from seven U.S. cities (2017-2023); after diarization and quality control, the analytic corpus is 1,565 meetings from five cities. The task is non-trivial: audio quality is uneven, members of the public speak informally and are often introduced only briefly, and speech frequently overlaps. Even so, segmentation reaches 89% accuracy (mean pairwise kappa=0.84), and the classifier reaches 87.7% accuracy (kappa=0.736, public-participant F1=0.832) on ground truth from five cities (N=1,761 speakers) -- agreement comparable to that among the human annotators themselves. At scale, members of the public receive a median of just 10% of floor time, the top speaker is an official in over 94% of meetings, and public participation declined over COVID-19 without recovering by 2023. The instrument measures participation directly from audio, without transcription or linkage to external voter or property records, establishing audio-as-data as a practical approach for the study of local politics.

Closest papers

Nearest neighbours by meaning across the whole index.

Public Perceptions of Public Participation in Local Government Mediated through a GenAI Tool

cs.CY ·2026-11-05 ·Jason Yan, Yanna Krupnikov, Nicholas Weber, Sabina Tomkins
hot direction similarity 0.66
Abstract

City council meetings offer a site for direct democracy through public comment periods. Yet, this process can lack transparency, and research on local political participation has documented that people who speak at public meetings are often unrepresentative of their communities. However, a lack of representation does not establish whether other people view these participants as advocates for their concerns. In fact, little is known about how they are perceived. To address this gap and improve transparency into this democratic process, we built a generative AI (GenAI) tool, VoicesView, that allows individuals to retrieve and read public comments. We deploy this tool, along with a novel survey, in a study with 72 participants in Ann Arbor, Michigan, and Seattle, Washington, in order to understand people's perceptions of public commenters. We find that participants generally viewed public commenters favorably, and that these views became more positive after using VoicesView. For example, the share who agreed that public commenters advocate for issues they care about rose from 49% to 62%. Knowledge of how public comment works in their city was moderate and also improved: the proportion of participants who correctly answered a procedural knowledge question about council meetings increased from 53% to 74%. Participants viewed public commenters more favorably than local government, yet their sense of being heard by local government also grew: agreement that their own voice is heard rose from 28% to 46%. These findings suggest that giving residents opportunities to engage with and learn from records of local political participation can foster positive civic outcomes.

0.32

Attendance-guided speaker enrollment across meetings

eess.AS ·2026-11-03 ·Minjae Lee, Hee-Soo Heo, Youngki Kwon, Bong-Jin Lee
benchmark similarity 0.62
Abstract · 5 pages, 2 figures, submitted to ICASSP 2027

Speaker-aware applications rely on enrolled voice profiles, yet collecting enrollment speech from every participant is impractical. Meeting archives provide an alternative source of enrollment speech, comprising recordings of recurring participants paired with expected attendance lists. We investigate whether expected attendance lists can be cross-referenced with diarized anonymous speaker clusters to identify reliable enrollment speech for recurring participants. The resulting task, multi-participant automatic enrollment, must accommodate two types of attendance noise: a listed participant may not speak, and a speaker may be unlisted. To this end, we introduce a benchmark for the task: a synthetic corpus calibrated to an in-house meeting corpus, paired with a contamination-sensitive evaluation protocol. The reference method we introduce scores cross-session cluster chains against expected-attendance patterns and abstains when the evidence is insufficient. We quantify the reference method's benchmark performance as a trade-off between enrollment success and contamination. Applying synthetic-development thresholds unchanged to real meetings exposes a synthetic-to-real gap that remains an open challenge. We will release the benchmark generator and evaluation code.

0.11

Technology-Supported Moderation in Online Deliberation

cs.CY ·2026-10-13 ·Lodewijk L. Gelauff, Ashish Goel, Sukolsak Sakshuwong
published similarity 0.58
Abstract · Book chapter, to appear in "Democratizing Tech Governance: Empowering the Public's Voice Through Deliberative Democracy" (preliminary title)…

Mini-publics are an increasingly popular family of democratic innovation to broaden citizen engagement. However, convening a representative sample in person is resource-intensive, and requires significant organizational efforts to coordinate travel, venue logistics, and facilitator training. This chapter outlines the design and development of the Stanford Online Deliberation Platform, initially tailored for Deliberative Polling, building upon insights from in-person moderation in that setting, but supporting a broader range of methods and approaches in the deliberative space. The platform supports structured conversations for small groups (ideally 8-15 people) with automated moderation functionality, enabling the groups to assume ownership of the deliberative conversation rather than relying on a moderator persona. We describe the platform's architecture and key design decisions, including the deliberation flow, room assignment, speaker management, speaking nudges, agenda management, outcome development, and real-time monitoring. Between its inception in 2018 and 2025, the platform has facilitated over 30,000 unique participants engaging in deliberations; we catalog 54 deliberations with at least 100 participants up to December 2025, spanning a diverse range of languages, countries, and topics. Finally, we establish a standardized vocabulary for future analyses by defining the terminology employed in the platform's data and documenting significant software revisions over time.

0.26

Who Speaks Matters: Authority-Aware Multi-View RAG over Italian Parliamentary Proceedings

cs.AI ·2026-08-13 ·Mirko Tritella, Riccardo Pozzi, Matteo Palmonari
top venue similarity 0.57
Abstract · Accepted at ISWC 2026 In-Use Track. Please cite the ISWC version

Parliamentary proceedings are a primary record of democratic deliberation, yet their volume and fragmentation make multi-perspective access difficult for citizens, journalists, and researchers. Applying Retrieval-Augmented Generation (RAG) to parliamentary transcripts introduces three specific risks: dominance of the most frequent speakers, inability to weight speakers according to topical expertise, and citation misattribution in politically sensitive text. We present ParliamentRAG, a RAG system for the Italian Chamber of Deputies that addresses these risks jointly. Its core contribution is a topic-dependent authority model that estimates each speaker's authority as a function of the current query, combining interpretable components such as profession, education, and previous interventions. Given a user query, the system retrieves relevant speech chunks, identifies topic-relevant experts across parliamentary groups, and generates a summary synthesizing their perspectives, accompanied by supporting quotations. ParliamentRAG is evaluated against Google NotebookLM on 15 policy topics via a two-level protocol combining automated metrics and blind A/B human evaluation by six domain experts. The system achieves higher coverage across political groups (0.97 vs. 0.95), perfect quotation faithfulness (1.00 vs. 0.95), and stronger expert preferences on source-related dimensions, while NotebookLM remains stronger on prose-oriented dimensions.

0.35

Speech event detector: Timestamped detection of non-verbal vocalizations and background sounds for rich speech transcription

cs.SD ·2026-10-09 ·Ayush Kumar Bar, Promila Ghosh
code/data similarity 0.56
Abstract · Code is available at https://github.com/theboringai-work/surround-sound and pre-trained weights are hosted at https://huggingface.co/thebori…

Automatic speech recognition produces words but drops the non-verbal content of a recording: the breaths, coughs, sighs and laughter of the speaker, and the horns, music or barking around them. We describe Speech Event Detector, a 5.8-million-parameter convolutional-Transformer model that outputs probabilities every 40 ms for 54 event classes (20 speaker vocalizations and 34 background sounds), together with post-processing that converts these probabilities into timestamped events and a procedure for merging the events into a Whisper transcript. The model is trained on material drawn from seven public datasets whose labels come in incompatible forms: single detected regions, free-text tags with timestamps, temporally strong ontology labels, symbols embedded in transcripts, isolated recordings and environmental recordings. We unify them using rule-based label mapping, recover timestamps for transcript-embedded labels through CTC forced alignment, locating 6,360 of 8,620 sounds, and combine about 168 hours of real annotated training recordings with synthetic mixtures and augmentation using real background noise. Per-class thresholds for two operating profiles are tuned on real held-out recordings, and performance is estimated using two-fold cross-validation. On whole held-out recordings from four datasets, considering the 48 classes with at least ten held-out events, the sensitive profile detects an average of 49 percent of a class's events, covering 67 percent of all 9,367 events, at a macro precision of 0.30. The balanced profile achieves a macro recall of 0.37 at a precision of 0.36, with macro event F1 scores of 0.32 and 0.33, respectively. The detector runs about 39 times faster than real time on eight CPU threads.

0.26

Speech Language Models for Full-Meeting Speaker Diarization: Capabilities and Limitations

eess.AS ·2026-09-19 ·Jialu Li, Jinchuan Tian, Shinji Watanabe
published similarity 0.56
Abstract · Accepted to IEEE SLT 2026

Recent advances in Speech Language Models (SpeechLMs), which integrate large language models with speech foundation models, have enabled unified sequence modeling of speech processing tasks. However, many SpeechLM-based approaches to speaker diarization (SD) are tightly coupled with automatic speech recognition (ASR) and evaluated using word-level metrics, making it difficult to assess SD performance independent of ASR accuracy. In this work, we investigate ESPnet-SpeechLM as a token-based backbone for generating SD hypotheses, formulating SD as autoregressive generation of structured tokens conditioned on acoustic input. We systematically compare two output representations: an event-based representation that explicitly models speaker turn onset and offset timestamps, and a frame-based representation that predicts frame-level speaker activity. To provide structured conversational cues, we further incorporate auxiliary tasks including speech activity detection, overlapped speech detection, and speaker turn counting within the output sequence. Across multiple meeting datasets, we find that event-based representations produce more stable and consistent SD outputs than frame-based representations. Our analysis shows that outputs generated by SpeechLMs encode useful temporal SD structure, but full-meeting SD remains limited by recording-level speaker tracking and overlap-related misses. Explicit speaker-linking post-processing substantially reduces speaker confusion, suggesting that robust SpeechLM-based SD requires persistent speaker tracking and overlap-aware generation.

0.33

Label-Free Role-Aware Segmentation of Classroom Recordings: Recovering the Teacher from Transferable Structure

cs.LG cs.SD ·2026-11-17 ·Nitin Jain, Tim Fütterer, Babette Bühler, Çağatay Yıldız
top venue similarity 0.56
Abstract · Accepted at the AAAI track is IAAI (Innovative Applications of AI)

How much the teacher talks and when are among the most informative and costly-to-measure signals of teaching. Yet, classroom audio is among the most sensitive data an AI system can process: recordings capture minors and fall under GDPR, preventing raw audio from leaving the recording device. We present a teacher-student segmentation pipeline designed to meet exactly this constraint: label-free at inference, running on a recording device, and requiring no per-session calibration. The pipeline chains a lightweight voice activity detector, compactly pretrained speaker embeddings, agglomerative clustering, and a small logistic regression that assigns the teacher role based on session-relative structure rather than voice identity. We show that our structural cue transfers across classrooms, languages, and acoustic conditions. In leave-one-teacher-out cross-validation on 292 recordings from 63 teachers across English and German corpora, our pipeline achieves teacher F1 of 0.81 and 0.87, respectively, outperforming both enrollment-based baselines and a 30-second supervised enrollment ceiling. On a commodity smartphone, its performance matches pyannote, a strong neural diarization tool, while running 4.8x faster at comparable peak memory at a real-time factor of ~0.18. We also find that transcribing filtered audio only marginally increases teacher's word error rate from $0.26$ to 0.28. Residual failures concentrate where teacher and student voices are not acoustically separable. However, we show how to flag these sessions label-free at inference so that they can be routed to human review, enabling selective, privacy-preserving deployment.

0.30

VoxMem: Benchmarking Multimodal Memory in Large Audio Language Models

eess.AS cs.SD ·2026-09-26 ·Yang Xiao, Vidhyasaharan Sethu, Eun-Jung Holden, Ting Dang
position similarity 0.55
Abstract · working in process

Spoken conversational systems must recover information from prior interactions (i.e., memory), yet relevant information in speech extends beyond what was said to who said it, how it was spoken, and what was audible, information that exists only in the audio signal and cannot be recovered from a transcript. Beyond what to remember, memory also demands diverse operations: retrieving a single fact, integrating evidence across turns, tracking an evolving state. Real interactions further unfold across sessions, meaning information accumulates across distinct episodes rather than a single continuous recording. Existing benchmarks fall short on all three dimensions: they focus primarily on lexical content, adopt limited and ad hoc memory operations, and treat memory as a single-session problem. We argue that principled memory evaluation requires jointly characterizing the acoustic evidence to be retained and the operations applied to it, and introduce a taxonomy along these two axes. Building on this taxonomy, we present VoxMem: 3,196 evaluation instances over 34,743 spoken sessions (177 hours) crossing four acoustic evidence types (speech semantics, speaker identity, paralinguistic cues, environmental sound) with four memory operations (information extraction, multi-session reasoning, temporal tracking, and answer refusal), grounded in multi-session histories and stratified across context budgets from 8K to 64K tokens. Evaluating 15 LALMs, no model exceeds 40% at 32K. Models retain what was said far better than who said it, how, or what was audible, a gap that widens for complex operations, grows with history length, and manifests as qualitatively distinct failure modes across evidence types. VoxMem aims to provide a foundation to measure and drive progress on the full scope of spoken conversational memory.

0.21