CS Paper Radar

SPADE: A Multilingual Dataset for Speech Partial Deepfake Detection and Localization

Yuan Tseng, Aishwarya Fursule, Andrew Zijun Ma, Vamshi Nallaguntla, Anderson Avila, Shruti Kshirsagar, David Harwath

eess.AS ·submitted 2026-09-21 17:46 UTC · v1
publishedbenchmarkcode/data

Abstract

Recent improvements in voice-cloning speech generation systems raise concerns about misuse by malicious actors to impersonate others and spread misinformation. Detecting such tampering is difficult, since deepfakes in the wild may be created by different generative models in a wide range of languages. Furthermore, the speech audio may also only be partially modified, presenting a different and potentially more challenging task than detecting fully-synthetic speech waveforms. To enable further research in this direction, we propose a multilingual dataset for detection and localization of partially edited speech samples. Our dataset includes speech in 12 languages, generated by up to five systems per language, and includes both a training set as well as an evaluation benchmark. To showcase the utility of our proposed dataset, we train localization models of existing architectures and study generalization across three axes: across different languages, across different speech synthesis and editing systems, and across different acoustic environments. Our results show that localization models almost always generalize poorly to speech edited by systems not seen during training. On the other hand, generalization to edited speech in unseen languages still degrades performance but to a lesser extent. We also augment our testing sets with noise to evaluate generalization across acoustic environments, and find that performance of localization models degrade significantly when tested on different acoustic conditions. All together, our results imply that existing deepfake speech detection methods are insufficient for reliably detecting edit-based speech deepfakes in various scenarios unseen during training. SPADE is publicly available on HuggingFace.

Comment
Accepted to SLT 2026; Dataset: https://huggingface.co/datasets/rogertseng/spade

Closest papers

Nearest neighbours by meaning across the whole index.

MADBench: A Benchmark for Modality-Aware Audio Deepfake Detection

cs.SD cs.AI ·2026-08-10 ·Yanqiu Li, Yang Xiao, Jisheng Bai, Bin Chen, Hong Jia +1 more
benchmark similarity 0.74
Abstract · 11 pages, 1 figure

Recent advances in speech synthesis and audio generation have made high-fidelity acoustic forgery low-cost and difficult to attribute, enabling a realistic attack scenario in which speech and background audio are independently manipulated over otherwise authentic video. Yet existing research either focuses on visual manipulation, addresses speech detection in isolation, or conflates speech and non-speech audio as a single undifferentiated audio stream, overlooking the distinct forensic challenges posed by background audio. This conflation is consequential: the two acoustic components arise from fundamentally different generative mechanisms, exhibit distinct artifact profiles, and pose different challenges to detection systems. We introduce MADBench, the first benchmark that treats speech and environmental audio as distinct acoustic components, enabling component-aware evaluation of audio deepfake detection across independently manipulated forgery sources. We benchmark representative state-of-the-art detectors and multimodal large language models under a unified protocol. Our experiments reveal that environmental audio manipulation is more detectable than synthetic speech across general-purpose encoders, while existing pretrained detectors fail on both acoustic components, and manipulated environmental audio asymmetrically degrades speech deepfake detection, findings entirely invisible under the single-label paradigm of prior benchmarks. MADBench establishes a rigorous foundation for future research into robust, component-aware audio deepfake detection.

0.10

GLAD: Global-Local Adaptive Detector for Robust Speech Deepfake Detection

cs.SD cs.AI ·2026-09-28 ·Zelin Zhao, Guanjie Huang, Danny Hin Kwok Tsang, Li Liu
similarity 0.72
Abstract

Recent advances in AI-based speech synthesis have enabled highly realistic speech, increasing the importance of speech deepfake detection (SDD) in preventing misuse. While mainstream Self-Supervised Learning (SSL)-based detectors achieve strong performance, they suffer from poor generalization to unseen domains and often overlook fine-grained signal artifacts due to a bias towards global semantic consistency. In this paper, we conduct the first detailed empirical and visual analysis to validate these limitations explicitly. Our investigation reveals two critical architectural vulnerabilities: (1) a systemic failure to capture localized spoofing traces, and (2) a severe lack of adaptability to domain-driven shifts in SSL layer importance, rendering static aggregation strategies prone to overfitting. To address these vulnerabilities, we propose the Global-Local Adaptive Detector (GLAD). Specifically, to capture localized forgeries, GLAD employs a Hierarchical Global-Local (HGL) backbone that explicitly bridges the granularity gap by fusing global linguistic and acoustic features with fine-grained local signal details. To counter layer importance shifts in out-of-distribution (OOD) scenarios, we introduce a Hierarchical Adaptive Gating (HAG) mechanism that dynamically recalibrates layer-wise focus in a sample-specific manner. Finally, to address shortcut learning induced by environmental biases, we introduce SaniBoost, a composite data augmentation strategy for robust signal standardization and noise sanitization. Extensive experiments demonstrate that GLAD significantly outperforms state-of-the-art methods, particularly on unseen domain cases.The code will be released upon publication.

0.16

Detector-Guided Vocoder Training for Adversarial Text-to-Speech Synthesis

cs.SD ·2026-11-03 ·Jiayan Cui, Davide Salvi, Paolo Bestagini, Stefano Tubaro
published similarity 0.71
Abstract · Accepted at WIFS 2026

In this paper, we propose a novel Text-to-Speech (TTS) framework designed to perform adversarial attacks against speech deepfake detection systems. Given a TTS model and a detector capable of classifying its outputs as synthetic, our approach fine-tunes the vocoder by integrating the detector into the TTS training loop. Unlike traditional adversarial methods that rely on post-hoc perturbations or external noise injection, our approach enables the generation of synthetic speech that is intrinsically adversarial, capable of lowering the target detector's reliability without requiring additional post-processing steps. We evaluate the framework in both white-box and black-box settings, showing its effectiveness against known and unseen detectors. We also propose a multi-detector strategy that allows the system to simultaneously learn to deceive multiple detectors, showcasing the scalability of the method. Experimental results confirm the efficacy of our approach, which has the potential to impact forensic security and detection countermeasures. At the same time, subjective listening tests reveal that the proposed method also enhances synthesized speech quality.

0.24

Tracking the Trend in How Speech Synthesizers Deceive People

cs.CR cs.SDeess.AS ·2026-08-20 ·Milan Šalko, Anton Firc, Kamil Malinka, Vojtěch Staněk, Martin Perešini +2 more
top venuebenchmark similarity 0.70
Abstract · Accepted at the 6th Symposium on Security and Privacy in Speech Communication (SPSC 2026)

Advances in speech synthesis have made deepfake audio highly realistic. Earlier studies reported 70-80% human detection accuracy, but relied primarily on older synthesizers. We compare human detection for three selected voice synthesis tools released in 2019, 2022, and 2024 with 82 IT professionals, and benchmark humans against six pretrained detectors on the same material. For fully synthetic speech (full spoofs), the F1 score drops from about 90% for RTVC and YourTTS to 48% for ElevenLabs, although listeners were explicitly warned that deepfakes were present. For partial spoofing, where only one sentence of an utterance is altered, strict accuracy falls to 9%, and listeners classify the synthetic sentence as bona fide 77% of the time. Humans and detectors fail in complementary ways, and neither reliably localizes short manipulations. Additionally, listeners increasingly mislabel bona fide speech as fake, eroding trust in unmanipulated audio. These findings show that human perception alone is unreliable for the selected modern and partial-spoof conditions and motivate procedural verification, provenance, watermarking, and segment-level detection.

0.21

ReplayFactor: Replay-Aware Dual-Attribute Learning for Practical Speech Deepfake Detection

cs.SD eess.AS ·2026-11-02 ·Miao He, Jiayi Zhou, Yu Zheng, Zhongjie Ba, Peng Cheng +4 more
similarity 0.70
Abstract · 5 pages, 3 figures. Submitted to ICASSP 2027

Advances in synthesis techniques have made deepfake speech increasingly realistic, amplifying voice security threats, including unauthorized voice cloning and impersonation. Although existing deepfake detectors perform well on digitally generated speech, their performance often degrades on physically replayed deepfakes, which constitute a low-cost and practical threat to real-world systems. Existing research focuses primarily on dataset construction and evaluation, while available dataset diversity and replay-aware detector design remain limited. We propose ReplayFactor, a dual-attribute framework that enhances replay evidence by fusing self-supervised learning (SSL) representations with an acoustic prior, while leveraging multi-task learning and supervised contrastive learning to promote robust, generalizable representations. We also construct ReplayFake-CN, a large-scale, multi-device Chinese dataset comprising approximately 185k replayed synthetic utterances to complement public benchmarks. Our framework improves macro-F1 from 60.99% to 76.44% and reduces EER from 11.86% to 3.05% on the EchoFake open set over the baseline, demonstrating strong potential for practical replayed deepfake detection.

0.17

BabelFake: A Multilingual Audio-Visual DeepFake Benchmark

cs.CV ·2026-10-05 ·Carlotta Segna, Joel Tschesche, Anna Rohrbach
benchmark similarity 0.70
Abstract

Reliable and practical audio-visual DeepFake detection requires benchmarks that reflect diverse linguistic contexts and modern data synthesis pipelines for visual as well as audio manipulations. However, existing datasets predominantly contain footage of English-speakers, often include outdated manipulation types, or overlook the audio modality. Further, many datasets feature individuals who did not consent to be used in DeepFake creation. We introduce BabelFake, a multilingual audio-visual DeepFake benchmark recorded with consenting participants. BabelFake contains 399k clips (1,323 hours) from 496 individuals spanning five languages (English, German, Italian, French, Spanish). Our modular data generation pipeline pairs 11 modern video manipulation methods with 4 voice cloning engines, distinguishing visual-only (face swapping) and joint audio-visual manipulations (lip synchronization and portrait animation). By benchmarking state-of-the-art detectors, we show that detection difficulty depends on the audio-visual generation pairing, with substantial performance degradation when authentic audio is preserved. Cross-language/demographic evaluation reveals sensitivity varying across detector architectures and training data, while human evaluation reveals that perceived realism and machine-detection difficulty do not necessarily align.

0.16

The landscape of AI speech generation and deepfake detection

cs.SD ·2026-11-18 ·Simon King
survey similarity 0.69
Abstract · 80 pages including definitions of all acronyms, a glossary of technical terms, and a bibliography

Synthetic speech has reached a level of naturalness that can deceive listeners into believing they are hearing natural speech. This opens up the possibility of using synthetic speech deepfakes to mislead both human listeners and automated systems. One response to this threat is to attempt to automatically detect deepfakes. This report offers the reader a self-contained overview of the landscape of synthetic speech generation and speech deepfake detection. It is intended to be accessible to a general audience and therefore includes brief histories of both fields, definitions of all acronyms, a glossary of technical terms, and an extensive bibliography for those who wish to read more.

0.16

Robust Workflow Generation via Adversarial Learning for Audio Deepfake Detection

cs.SD cs.AI ·2026-09-17 ·Xiang Li, Pin-Yu Chen, Wenqi Wei
similarity 0.67
Abstract

The rapid advancement of speech synthesis and voice conversion technologies has made audio deepfakes increasingly realistic, posing serious security risks in practical applications. While existing detection methods achieve strong performance under controlled conditions, they often fail to generalize under real-world perturbations and corruptions. In this paper, we propose ROGUE, a framework that dynamically constructs robust detection workflows by orchestrating multiple detection tools. ROGUE formulates workflow generation as a sequential decision-making problem and introduces a dual-agent paradigm, where a perturbation agent generates audio perturbations and a policy agent learns to select and execute detection tools under perturbed conditions. Through adversarial learning, ROGUE enables perturbation-aware tool selection, adaptive execution strategies, and improved robustness to distribution shifts. Extensive experiments across multiple datasets and real-world corruptions demonstrate that ROGUE consistently outperforms strong baselines in both robustness and generalization. Our results highlight the effectiveness of adversarially optimized workflow generation for building reliable audio deepfake detection systems in real-world deployment settings.

0.14