CS Paper Radar

Privacy-Preserving Deep Joint Source-Channel Coding with In-Loop Concept Erasure

Rami Eid, Maria Slim, Mariette Awad, Hadi Sarieddeen

eess.SP cs.LG ·submitted 2026-09-11 18:02 UTC · v1

Abstract

Deep joint source-channel coding (DeepJSCC) transmits learned semantic features efficiently but can leak sensitive attributes such as gender, race, or speaker identity. We propose LEAPSC (LEACE-in-the-loop privacy for semantic communication), whose core contribution is the integration of in-loop least-squares concept erasure (LEACE) within a variational information bottleneck (VIB) encoder. By periodically refitting the projection operator during training, LEAPSC couples the encoder dynamics to the erasure mechanism, driving attribute-conditional mean differences toward zero within each task-label group on the fitting sample. Additional components, namely conditional value-at-risk (CVaR) tail-sensitive privacy, feature-wise linear modulation (FiLM) signal-to-noise ratio conditioning, and Lagrangian dual ascent, improve robustness across channel conditions and over the high-leakage tail of samples. On CelebA, FairFace, and Google Speech Commands, LEAPSC reaches task accuracy of 0.862, 0.755, and 0.925 respectively, with attacker accuracy at or below the label-only floor on CelebA (0.548 vs. floor 0.580) and within 2 percentage points (pp) of chance elsewhere, improving over an information-bottleneck adversarial baseline (IBAL) at a matched 52-epoch budget by +3.6, +2.5, and +1.3 pp (Welch's t-test, p=0.019 on CelebA).

Closest papers

Nearest neighbours by meaning across the whole index.

Decaf: A privacy preserving speech codec using speaker disentanglement and canonical voice conversion

eess.AS ·2026-09-16 ·Md Shakhrul Iman Siam, Dushyant Sharma, Stanislav Yu. Kruchinin, Peter Skala
similarity 0.69
Abstract · In Proc. of IWAENC 2026

We present DECAF, a privacy preserving neural speech codec that obfuscates a speaker's voice while preserving linguistic content while maintaining automatic speech recognition (ASR) performance at very low bitrates, inspired by decaffeination. At the transmitter end, speech is encoded into speaker independent content embeddings, which are compressed using residual vector quantization and transmitted without any speaker related information. At the receiver, a canonical speaker embedding, shared a priori between endpoints, is used for waveform reconstruction, enabling deterministic and consistent obfuscation of a speaker's voice. The proposed framework leverages an information bottleneck applied to self supervised representations, along with a separate speaker embedding branch, to achieve effective speaker content disentanglement. We further incorporate a CTC-based auxiliary objective, encouraging content representations that are well aligned with downstream ASR tasks. We show that DECAF operating at a bit rate of 0.5 kbps achieves an Equal Error Rate (EER) of up to 43.5% for a speaker verification system, while maintaining competitive ASR performance, yielding a relative reduction in word error rate of 33.2% compared to a state of the art method.

0.12

Towards Privacy-Preserving ASR: Speaker-Level Machine Unlearning

eess.AS ·2026-10-12 ·Seaone Ok, Seungu Han, Eungbeom Kim, Kyogu Lee
top venue similarity 0.67
Abstract · 6 pages, 2 figures, Accepted for Interspeech 2026

Automatic Speech Recognition (ASR) systems are commonly designed to transcribe linguistic content independently of speaker identity. However, speaker-specific information can persist within the latent representations of ASR models, raising privacy concerns when training data includes sensitive individuals. While machine unlearning has been studied in several speech tasks, applying speaker unlearning to ASR remains challenging due to the strong entanglement of linguistic and speaker-related information in speech representations. In this work, we conduct a layer-wise analysis of Self-Supervised Learning (SSL) based ASR representations, revealing that speaker identity is primarily encoded in intermediate encoder layers. Based on this observation, we explore a feature-level unlearning strategy that targets these identity structures. Experiments on the VCTK corpus show that the proposed approach removes the influence of specific speakers while maintaining robust transcription performance.

0.21

CLEAR: Online Speech Content Leakage Estimation through Cross-ASR Disagreement

cs.SD eess.AS ·2026-09-24 ·Bhawana Chhaglani, Tanvi Kandepuneni, Jeremy Gummeson, Prashant Shenoy
similarity 0.67
Abstract

Signal-level speech privacy mechanisms suppress linguistic content while preserving acoustic information needed by downstream sensing applications. However, their privacy settings are typically evaluated/selected offline and remain fixed during deployment, even though speech-content leakage can vary substantially across utterances and speakers. Adapting privacy protection at runtime requires estimating how much speech remains recoverable, but conventional measures such as WER or PER require ground-truth transcripts and therefore cannot be computed online. We present CLEAR, a reference-free approach for estimating speech-content leakage at runtime using disagreement among heterogeneous ASR systems. Our key insight is that independently trained ASRs exhibit consistent behavior when linguistic content remains recoverable and increasingly disagree as privacy transformations obscure speech. Using configurable speech-suppression mechanism, we show that cross-ASR disagreement closely tracks transcript-grounded leakage across privacy operating points, achieving a correlation of 0.8. We further characterize the latency-accuracy trade-off of heterogeneous ASR subsets and use their hypotheses to identify potentially exposed words. These capabilities enable privacy to be treated as a runtime property rather than a fixed configuration: CLEAR can communicate residual speech exposure to users and provide feedback for dynamically adjusting privacy aggressiveness while retaining acoustic utility for downstream sensing tasks.

0.23

Component-Aware Differential Privacy for Federated Multilingual Speech-LLMs

cs.CL ·2026-09-10 ·Jordi Luque, Fernando López, Aleix Sant
published similarity 0.64
Abstract · Accepted in SLT2026

Per-layer differential privacy (DP) clipping improves gradient fidelity in federated learning by allocating per-matrix clipping budgets proportional to parameter count. We show that this recipe breaks for speech large language models (speech-LLMs), when the acoustic encoder and the language decoder differ by an order of magnitude in update norm. Single-pool per-layer methods suffer \emph{cross-component budget collapse}, dragging word error rate (WER) far from flat global clipping or collapsing training entirely. When the norm imbalance is milder, adaptive single-pool methods partially recover, confirming that collapse severity scales with the inter-component norm ratio. We empirically diagnose the root cause across six per-layer methods and three speech-LLM architectures. We then propose \emph{$α$-split}, a two-pool allocation that normalises encoder and LLM parameters into independent pools, and show that joint $\ell_2$ sensitivity and the original $(\varepsilon,δ)$-DP guarantee are unchanged. At architecture-calibrated $α$, our method recovers WER utility compared to flat DP, while granting the encoder $4.47{\times}$ tighter per-component noise protection against speaker voice-based gradient-inversion attacks at only $+2.6\%$ LLM noise overhead.

0.39

Leakage-Aware Encrypted Semantic Communication with Key-Conditioned Signal Perturbation

cs.CR eess.SP ·2026-11-02 ·Omar Kaaki, Maria Slim, Mariette Awad, Hadi Sarieddeen
published similarity 0.64
Abstract · Accepted for presentation at IEEE GLOBECOM 2026

Encrypted semantic communication systems protect transmitted messages by inserting key-conditioned modules into a deep-learning semantic communication pipeline, but existing designs rely on a fixed adversarial weight and operate mainly at the semantic-feature level. We propose two changes to the encrypted semantic communication system (ESCS): a key-conditioned signal-domain perturbation layer, called learned adversarial perturbation injection (LAPI), and an information noise-contrastive estimation (InfoNCE)-guided adaptive training rule that {adds an adaptive leakage penalty alongside the fixed adversarial objective} using an InfoNCE-based proxy for mutual information leakage. Experiments on the Europarl corpus over additive white Gaussian noise (AWGN) and Rayleigh fading channels show that InfoNCE-guided training is the primary source of eavesdropper suppression, reducing the eavesdropper's bilingual evaluation understudy (BLEU) score to approximately 0.07 at high signal-to-noise ratio (SNR). Adding LAPI to the InfoNCE-guided objective further lowers the InfoNCE-estimated leakage by 5-9% with low measured inference overhead, while its impact on legitimate-receiver BLEU depends on channel conditions. The two mechanisms operate on different parts of the pipeline and can be jointly applied.

0.45

Machine Unlearning for Speech Question Answering in Large Audio-Language Models

cs.LG cs.CLcs.SDeess.AS ·2026-08-13 ·Zhe Liu
similarity 0.62
Abstract

Large Audio-Language Models (LALMs) have recently shown strong capabilities in speech understanding and question answering (QA), but they also inherit privacy risks from large-scale training data, including the unintended memorization of sensitive information. In this work, we study machine unlearning for speech QA in LALMs, a setting that is more challenging than prior work on text-based Large Language Models (LLMs) or Automatic Speech Recognition (ASR) due to the tight coupling between acoustic perception and factual knowledge. We present and evaluate multiple unlearning strategies, including gradient ascent, task arithmetic, and alignment-based fine-tuning methods that enforce safe refusal responses, to remove private knowledge while still preserving performance on core capabilities. Through extensive experiments on speech QA datasets, we show that these unlearning methods can reduce the privacy leakage rate by up to 80% while maintaining near-neutral performance on non-private speech QA and general speech understanding benchmarks.

0.04

The Unseen Threat: How Complementary Combinations of Acoustic Features Compromise Speech Privacy

eess.AS ·2026-11-12 ·Jule Pohlhausen, Jörg Bitzer
top venue similarity 0.62
Abstract · Accepted at SPSC 2026 - 6th Symposium on Security and Privacy in Speech Communication

The interaction of acoustic features in leaking private information remains underexplored. This work focuses on time-domain features, i.e., broadband RMS level and pitch, alone and combined with temporally smoothed and subsampled Mel spectrograms. Privacy is assessed using informed automatic speech recognition and speaker-verification attacks, while utility for conversational analysis is evaluated using voice activity detection and speaker diarisation. RMS level and pitch alone exposed limited information, but fusion with 500 ms-smoothed Mel spectrograms enhanced utility while substantially increasing speech-content leakage. Specifically, the combination with RMS level showed that fine-grained temporal dynamics can amplify leakage from coarsely sampled spectral features. Speaker-identity protection remained limited for all tested features. Overall, these findings indicate that privacy assessments performed on individual acoustic features may underestimate leakage after feature fusion.

0.24

Revisiting Privacy Risks in Quantized Deep Neural Networks

cs.CR ·2026-10-27 ·Jie Fu, Ren Li, Wendy Hui Wang
publishedcode/data similarity 0.62
Abstract · Accepted by 2026 IEEE International Conference on Big Data

Model quantization has become a fundamental technique for enabling the efficient deployment of large-scale deep neural networks (DNNs) in data-intensive applications by reducing model size. Despite its widespread adoption, the impact of quantization on training data privacy --- particularly vulnerability to membership inference attacks (MIAs) --- remains largely unexplored. In this paper, we present the first comprehensive empirical study of privacy risks in quantized DNNs, evaluating eight state-of-the-art MIA attacks across six representative quantization methods. Our results reveal that practical quantization levels (4-8 bits) have limited impact on membership privacy risks, with quantized models remaining largely as vulnerable as their full-precision counterparts. Although aggressive low-bit quantization (e.g., 2-bit) can provide meaningful privacy gains, these benefits come at the cost of substantial utility degradation. To understand the underlying causes of these observations, we systematically analyze how different quantization strategies affect model behaviors and outputs, and how these changes influence privacy vulnerability. Beyond evaluating attack risks, we further investigate whether existing privacy defenses developed for full-precision DNNs can effectively protect QNNs. Our results show that these defenses provide only limited additional privacy protection when applied to quantized models, while further amplifying utility degradation. Our study provides practical guidance for privacy-aware deployment of quantized DNNs and motivates the development of new attack and defense mechanisms for QNNs. Our code and datasets are publicly available at https://github.com/JeffffffFu/Priv-QDNN.

0.43