CS Paper Radar

Audio Deepfake Detection Using Temporal Coherence Analysis

Justin D. Norman, Sarah Barrington

cs.CR cs.SD ·submitted 2026-09-08 22:08 UTC · v1

Abstract

The proliferation of AI-generated audio (so-called "deepfake" audio) poses significant threats to information integrity, from voice cloning fraud to synthetic music copyright disputes. We present a temporal coherence analysis framework built upon Contrastive Language-Audio Pretraining (CLAP) embeddings that spans speech, instrumental music, and music with vocals. By computing pairwise cosine similarities between audio segment embeddings and extracting statistical features from the resulting distributions, we train lightweight ensemble classifiers that reliably distinguish authentic from synthetic audio. Our work provides an interpretable, computationally efficient alternative to common deep learning methods while still achieving competitive performance across speech and music domains. Further, we reveal two notable empirical findings about audio deepfakes: (1) a feature-label inversion phenomenon in which 21 of 29 statistical features reverse their discriminative direction between training and in-the-wild deployment, and (2) a speech--music direction reversal in which entropy discriminates in opposite directions for speech and music deepfakes.

Closest papers

Nearest neighbours by meaning across the whole index.

AT-ADD: A Benchmark and Challenge for Robust and All-Type Audio Deepfake Detection

cs.SD ·2026-08-24 ·Yuankun Xie, Haonan Cheng, Jiayi Zhou, Xiaoxuan Guo, Tao Wang +10 more
benchmark similarity 0.75
Abstract

Recent audio generation models can synthesize high-fidelity speech, environmental sound, singing voice, and music, creating new risks for multimedia trust. Existing audio deepfake detection (ADD) benchmarks remain predominantly speech-centric and often under-represent realistic channel variation and heterogeneous audio types. This paper presents all-type ADD (AT-ADD), a large-scale benchmark and challenge designed to evaluate both robust speech deepfake detection and all-type audio deepfake detection. Track 1 evaluates speech deepfake detection under unseen synthesizers, diverse recording conditions, signal perturbations, and replay effects. Track 2 evaluates type-agnostic deepfake detection over speech, sound, singing, and music when the audio type is unknown at test time. We detail the dataset construction, evaluation protocol, and reproducible baselines, and analyze the final systems submitted to the ACM Multimedia 2026 Grand Challenge. The strongest official baseline obtains 76.73% and 79.47% Macro-F1 on the Track 1 and Track 2 evaluation sets, respectively, whereas the winning challenge systems reach 90.71% and 96.10%. Beyond the aggregate rankings, sample-level analysis of the top five submissions reveals concentrated failures on specific synthesis methods and replayed speech. It also quantifies cross-system error complementarity and separates stable leaderboard gaps from statistically ambiguous adjacent ranks. Our results show that large-scale self-supervised representations, condition-aware augmentation, multi-crop inference, and structured fusion or routing are central to generalization, while generator-specific robustness and balanced detection across heterogeneous audio types remain unresolved.

0.30

ToolDF: Tool-Integrated Reasoning for Mixed-Authenticity Audio Deepfake Detection

eess.AS cs.AIcs.SD ·2026-09-03 ·Taewoo Kim, Young Han Lee, Nam In Park, Chanwoo Kim
top venuebenchmarkcode/data similarity 0.71
Abstract · To appear in Findings of the Association for Computational Linguistics: EMNLP 2026

Audio deepfake detection is commonly formulated as clip-level binary classification of single-domain audio. However, real-world manipulated audio can exhibit mixed authenticity, where genuine and manipulated cues coexist across temporal transitions, overlapping sources, or both. This setting requires not only detecting manipulated audio but also localizing the components that provide evidence for the decision. We propose ToolDF, a tool-integrated reasoning framework for mixed-authenticity audio deepfake detection. ToolDF employs an audio large language model as an orchestrator trained with supervised tool-use trajectories. It adaptively analyzes the audio scene, selectively performs source separation, routes components to domain-specific experts, and aggregates their evidence into an interpretable verdict. We further introduce a mixed-authenticity ADD benchmark covering temporal transitions, acoustic overlaps, and hybrid mixtures. Experimental results show that ToolDF achieves the best overall performance on composite-type detection, achieving macro-F1 gains of 3.72 and 14.39 points over the strongest monolithic baseline and a fixed pipeline, respectively, while providing interpretable evidence localized to temporal regions and acoustic sources. Our source code and dataset are publicly available online.

0.35

MADBench: A Benchmark for Modality-Aware Audio Deepfake Detection

cs.SD cs.AI ·2026-08-10 ·Yanqiu Li, Yang Xiao, Jisheng Bai, Bin Chen, Hong Jia +1 more
benchmark similarity 0.71
Abstract · 11 pages, 1 figure

Recent advances in speech synthesis and audio generation have made high-fidelity acoustic forgery low-cost and difficult to attribute, enabling a realistic attack scenario in which speech and background audio are independently manipulated over otherwise authentic video. Yet existing research either focuses on visual manipulation, addresses speech detection in isolation, or conflates speech and non-speech audio as a single undifferentiated audio stream, overlooking the distinct forensic challenges posed by background audio. This conflation is consequential: the two acoustic components arise from fundamentally different generative mechanisms, exhibit distinct artifact profiles, and pose different challenges to detection systems. We introduce MADBench, the first benchmark that treats speech and environmental audio as distinct acoustic components, enabling component-aware evaluation of audio deepfake detection across independently manipulated forgery sources. We benchmark representative state-of-the-art detectors and multimodal large language models under a unified protocol. Our experiments reveal that environmental audio manipulation is more detectable than synthetic speech across general-purpose encoders, while existing pretrained detectors fail on both acoustic components, and manipulated environmental audio asymmetrically degrades speech deepfake detection, findings entirely invisible under the single-label paradigm of prior benchmarks. MADBench establishes a rigorous foundation for future research into robust, component-aware audio deepfake detection.

0.10

What Do Deepfake Audio Detectors Hear? From Self-Supervised Models to Audio LLMs

cs.SD ·2026-10-13 ·Wanying Ge, Xin Wang, Junichi Yamagishi
publishedcode/data similarity 0.70
Abstract · Accepted to SLT 2026

Real-world audio recordings for deepfake detection rarely consist solely of speech; they often are mixtures of speech and background events that themselves may also be synthetic. When processing these mixtures, it remains unclear whether detectors attend to the speech or the background event, and where their spectral evidence lies. We study this on the CompSpoof V2 database, whose mixtures pair independently real-or-fake speech and event components. Our results show that all systems detect fake events more accurately than fake speech. However, this performance gap is largely driven by poor generalization to an unseen fake speech domain, rather than a true difference in difficulty. Furthermore, sub-band probing reveals that the front-end encoder dictates where a countermeasure focuses its frequency evidence. To enable reliable deployment in real-world scenarios, future countermeasures should address out-of-domain generalization failures and carefully select front-end encoders aligned with their target acoustic evidence. Our source code and checkpoints are released at https://github.com/nii-yamagishilab/AntiDeepfake-AudioLLM.

0.25

Trajectory Dynamics in Self-Supervised Learning Latent Space for Audio Deepfake Detection

eess.AS cs.LGcs.SD ·2026-08-13 ·Tomás Andrade Weber
similarity 0.70
Abstract · 5 pages, 1 figure

Human speech production is constrained by physiology, giving rise to characteristic temporal structure on acoustic signals. We hypothesise that these constraints manifest as structured trajectory dynamics in the latent space of Self-Supervised Learning (SSL) models, and that synthetic speech violates them detectably. To test this hypothesis, we train a causal Long Short-Term Memory (LSTM) next-frame predictor on bonafide speech only (Stage 1), using the deepfake-specialised SSL backbone Wav2Vec2-Large-AntiDeepfake, and compare against a static global-average-pooling baseline using identical features, thus isolating the contribution of temporal modelling. A supervised Stage 2, which trains a Multi-Layer Perceptron on the frozen LSTM internal states using labelled data, is included to characterise the role of spoof supervision. Our system achieves competitive or state-of-the-art performance across six benchmarks: ASVspoof 2019/2021, Codecfake, In-the-Wild, MLAAD-EN, and Deepfake-Eval-2024, including best published EER on ASVspoof 2021 (0.75\%) and, notably, Stage 1 trained on bonafide speech only surpasses the published supervised baseline from the same backbone on DE2024 (30.35\%). On near-domain benchmarks, static and dynamic approaches perform comparably. On harder cross-corpus benchmarks with diverse synthesis methods, trajectory dynamics provide substantial gains, confirming that temporal physiological constraints carry detection signal beyond utterance-level statistics.

0.07

Adaptive Compression and Targeted Perturbation: A Unified Framework for Generalized Audio Deepfake Detection

cs.SD cs.AI ·2026-10-12 ·Jianqiao Cui, Haoxian Zhu, Hanbiao Meng, Yanghao Zhang
top venue similarity 0.70
Abstract · 10 pages,1 figure, Accepted by NeurIPS 2026

Existing audio deepfake detection (ADD) methods frequently exhibit limited generalization, as fixed bottleneck architectures fail to adapt to the heterogeneous information densities inherent in diverse spoofing artifacts. Moreover, standard augmentation often leads to a pathological robustness-accuracy trade-off. In this paper, we propose a unified ADD framework to address these challenges. Our approach integrates: (1) an Adaptive Audio Feature Integration (AAFI) module that dynamically selects optimal latent dimensions via a compression pool and a stochastic exploration mechanism, ensuring flexible representations across multiple types of dynamic attacks; and (2) a Mel-Band Adversarial Perturbation (Mel-BAP) strategy that applies targeted regularization in the mel-spectrogram domain to encourage the model to learn invariant decision boundaries. Extensive evaluations on benchmarks including ADD 2022, FakeOrReal, and In-the-Wild demonstrate that our model based on Whisper-small achieves a state-of-the-art average Equal Error Rate (EER) of 1.51%. Compared to existing SOTA models such as the 3-billion-parameter Resemble-Detect-3B-Omni, our framework reduces the parameter count by up to 91.6% while achieving a 60.7% improvement in average EER. In a separate cross-lingual evaluation, the Whisper-base configuration achieves EERs of 3.91-5.65% on five unseen languages. The framework also maintains strong resilience to real-world acoustic distortions. This work provides a scalable and efficient "plug-in" solution for enhancing trust and security in the audio Web ecosystem.

0.23

ReplayFactor: Replay-Aware Dual-Attribute Learning for Practical Speech Deepfake Detection

cs.SD eess.AS ·2026-11-02 ·Miao He, Jiayi Zhou, Yu Zheng, Zhongjie Ba, Peng Cheng +4 more
similarity 0.70
Abstract · 5 pages, 3 figures. Submitted to ICASSP 2027

Advances in synthesis techniques have made deepfake speech increasingly realistic, amplifying voice security threats, including unauthorized voice cloning and impersonation. Although existing deepfake detectors perform well on digitally generated speech, their performance often degrades on physically replayed deepfakes, which constitute a low-cost and practical threat to real-world systems. Existing research focuses primarily on dataset construction and evaluation, while available dataset diversity and replay-aware detector design remain limited. We propose ReplayFactor, a dual-attribute framework that enhances replay evidence by fusing self-supervised learning (SSL) representations with an acoustic prior, while leveraging multi-task learning and supervised contrastive learning to promote robust, generalizable representations. We also construct ReplayFake-CN, a large-scale, multi-device Chinese dataset comprising approximately 185k replayed synthetic utterances to complement public benchmarks. Our framework improves macro-F1 from 60.99% to 76.44% and reduces EER from 11.86% to 3.05% on the EchoFake open set over the baseline, demonstrating strong potential for practical replayed deepfake detection.

0.17

A Training-Free Proactive Defense Against Partial Speech Manipulation via Self-Embedding Steganography

cs.SD cs.AI ·2026-08-26 ·Yigitcan Özer, Zhe Zhang, Wanying Ge, Xin Wang, Junichi Yamagishi
top venuebenchmark similarity 0.70
Abstract · 6 pages; 4 figures; 1 tables; accepted at Interspeech 2026; audio samples available at https://nii-yamagishilab.github.io/self-embedding-aud…

Partial deepfake speech, where only limited segments of an utterance are synthesized or manipulated, poses a significant challenge to existing deepfake detection systems. As the proportion of spoofed regions decreases, passive detectors become increasingly unreliable, and accurate detection and restoration remain challenging. In this paper, we revisit audio steganography from a new perspective and propose its use as a proactive defense against partially deepfaked audio. In particular, we consider a self-embedding strategy in which a clean speech signal embeds a compressed representation of itself, enabling post-hoc extraction of reference content. We demonstrate how existing audio steganography methods can be repurposed to support detection of partial deepfakes through codec-based restoration. Experiments on a benchmark dataset show that the proposed approach complements passive defenses. Remarkably, the proposed method operates without any training, providing a robust and data-efficient alternative for partial deepfake detection.

0.50