CS Paper Radar

SSM-Attack: Learning Where to Perturb for Black-Box Face Privacy

Tunahan Parlayıcı, Erchan Aptoula, Yücel Saygın

cs.CV cs.CRcs.LG ·submitted 2026-10-12 19:28 UTC · v1
benchmark

Abstract

Face recognition systems are deployed widely enough that a photograph shared online can be matched without its subject's consent. Generator-based protection methods rely on heavy pretrained backbones, multi-stage inference or visible semantic edits. We introduce SSM-Attack, a black-box face privacy framework built on Selective State Space Models. Rather than synthesizing the perturbation, the state-space model predicts in one linear-time pass a spatial allocation of the perturbation budget, marking which facial locations carry identity evidence; a short projected refinement then sets the values within this per-pixel budget. Against published benchmark results, SSM-Attack attains the highest average black-box attack success rate on all three evaluation sets: 87.10 percent on CelebA-HQ, 86.90 on LADN and 81.15 on FFHQ, the last from a network that never saw that dataset. The allocation network holds 5.73 million parameters, or 16.13 million with the frozen networks used by its quality terms. Under the same cap of 0.10 on the maximum per-pixel change, allocation improves structural similarity by 0.044, peak signal-to-noise ratio by 1.62 decibels and Frechet Inception Distance by 11.2 points over a uniform map while spending 37 percent of its capacity. Strong facial privacy protection therefore does not require a large generative prior; it requires knowing where to spend a small one.

Comment
25 pages, 3 figures, 6 tables. Preprint submitted to Pattern Recognition

Closest papers

Nearest neighbours by meaning across the whole index.

Toward Interpretable Privacy Guarantees in Face-Swapping Anonymization

cs.CR ·2026-08-26 ·Vishnu Bondalakunta, Arman Zareian Jahromi, Shuangqing Wei, George Amariucai
hot direction similarity 0.74
Abstract · 18 pages, 15 figures, 5 tables

Face-swapping has emerged as a promising approach to facial privacy protection, replacing a target individual's appearance with that of a donor while preserving non-facial context. The resulting images visually resemble the donor, and face recognition systems tend to suppress the target's match scores -- ostensibly satisfying privacy requirements. Empirical evaluation across a range of face-swapping models, however, reveals that significant target identity leakage still occurs. This raises a deeper question: why does leakage occur, and can it be predicted? We propose a linear stochastic model that treats face-swappers as transformations on the space of identity embeddings, providing an interpretable account of the leakage mechanism. The model is fit to empirical observations and used to derive testable predictions. The aim is to ground privacy assessments in principled, interpretable analysis, thus making formal privacy guarantees explainable -- and perfectible -- rather than purely observational.

0.53

How Private is Private? A Comparative Study for Face De-Identification

cs.CV ·2026-10-07 ·Hui Wei, Hao Yu, Hui Kuurila-Zhang, Guoying Zhao
top venuebenchmarkcode/datahot direction similarity 0.71
Abstract · Accepted to NeurIPS 2026. Project Page: https://cv-ac.github.io/hifd/

Face de-identification (FDeID) has emerged as a critical privacy-preserving technology, yet its evaluation remains fundamentally fragmented. Existing protocols rely on inconsistent metrics, heterogeneous datasets, and partial annotation coverage, so methods targeting different utility dimensions, such as landmark versus expression preservation, are reported on different benchmarks under different metrics, rendering cross-method comparison infeasible. We revisit FDeID evaluation from both the data and metric perspectives. On the data side, we introduce UtilFace, a curated, demographically balanced benchmark with high identity diversity, assembled from four large-scale face datasets through identity-aware cleaning, resolution enhancement, and stratified filtering. On the metric side, we propose HiFD, a Hierarchical Face De-identification metric that unifies identity suppression, multi-level utility preservation, and image quality under a single consistency-based paradigm: every component is computed from pretrained estimators' outputs on the original face and its de-identified counterpart, directly quantifying how much identity is suppressed and how much downstream-perceivable utility survives. HiFD organizes facial signals into a three-level utility hierarchy spanning macro cues (L1), micro cues (L2), and imperceptible cues (L3), and aggregates the five resulting components into a single interpretable score via weighted harmonic mean, with configurable application-specific profiles. Using this unified protocol, we conduct a comprehensive comparative study spanning adversarial, GAN-based, and diffusion-based methods, surfacing trade-offs and failure modes that remain invisible under existing protocols. We release the benchmark and evaluation toolkit to foster systematic and reproducible research in privacy-preserving human face analysis.

0.45

GhostFaceMPC: Privacy-Preserving Face Recognition with Multi-Party Computation

cs.CR ·2026-10-21 ·Davide Carnemolla, Dario Catalano, Mario Di Raimondo
code/data similarity 0.71
Abstract

Facial recognition has become central to modern authentication systems, security frameworks, and personalized services. However, its widespread adoption is hindered by severe privacy risks, particularly surrounding unauthorized access to sensitive biometric data. To address these challenges, this paper presents GhostFaceMPC, the first end-to-end, privacy-preserving face recognition system to evaluate a deep convolutional neural network extractor and matching under secure multi-party computation. GhostFaceMPC enables secure processing of facial data across all stages of a face recognition process, from feature extraction and storage to matching, without exposing raw images or biometric embeddings at any stage. We introduce a multi-party computation-friendly quantization of the GhostFaceNet model and circuit optimizations that make face recognition under multi-party computation practical. The code is available at https://github.com/crypto-unict/GhostFaceMPC.

0.25

Wrong Input, Wrong Threshold: Re-evaluating Adversarial Facial Privacy

cs.CV cs.CRcs.LG ·2026-11-16 ·Tunahan Parlayıcı, Erchan Aptoula, Yücel Saygın
similarity 0.70
Abstract · 10 pages main text and 15 pages supplementary material. Submitted to IEEE Transactions on Information Forensics and Security. This work has …

Published face-privacy methods report protection above 90% against black-box recognition models. That number comes from an evaluation protocol with two independent defects, reported in a measure that cannot size either one. The input is wrong: methods published since 2022 feed the network the whole photograph, or a detector's box, where deployed systems detect, align to a landmark template and crop. Re-scoring the same files through a realistic front end costs a standard transfer attack 56% of its dodging protection and a budget-allocating attack 7%, with no image regenerated. The measure is wrong: that loss is a difference of success rates, so it saturates, moving by 8.2x over a budget sweep while the effect beneath it moves by 1.34x. We therefore report the paired cosine shift, in which both thresholds cancel algebraically, so the input defect holds without committing to a calibration. Measured that way it is 2.2x and 10.7x larger on three unseen recognizers, and at matched budget it reverses the L-infinity transfer ranking: the one surveyed method that aligns at evaluation gains 0.18 where MI-FGSM loses 0.05. The cause is a resampling mismatch, not perturbation lost outside the crop; optimising through a differentiable alignment operator largely removes it, adding +24 to +58 points on unseen recognizers and a commercial system, with a residual on one of them. The thresholds are wrong: the four constants inherited since 2021 hold the false-accept rate near its nominal 1% on aligned crops but admit 2.3-5.4% on whole images, with an opposite-signed bias across dodging and impersonation, the two tasks reported side by side. Differentiable alignment and gallery scoring are not new: the first generation of this literature used both, and we show what dropping them cost.

0.33

Not All Minorities Are Equal: Fairness Outcomes in Private Image Generation

cs.CV cs.CY ·2026-10-12 ·Cohen Archbold, Usman Hassan, Sen-ching Samson Cheung
hot direction similarity 0.69
Abstract · 5 pages, 3 figures, Under review for ICASSP 2027

Differential privacy (DP) provides formal guarantees for protecting individuals in artificial intelligence systems, but its consequences for fairness in generative models remain poorly understood. We investigate how privacy-preserving training affects the representation of demographic groups and visual attributes in generative models of human faces. Across two facial image datasets and multiple privacy levels, we find that stronger privacy can disproportionately degrade the quality and utility of some underrepresented groups, suggesting that privacy can amplify existing disparities. However, representation alone does not explain these effects: several well-represented attributes also experience substantial degradation, while some underrepresented attributes remain comparatively robust. These findings show that the impact of privacy-preserving learning cannot be explained solely by data representation and may depend on the characteristics and interactions of the information being learned. Our results highlight the need to consider both privacy and fairness when evaluating generative models.

0.23

Tomatoes, Potatoes, and Onions: Questioning the Need for Faces in Face Presentation Attack Detection

cs.CV ·2026-08-20 ·Guray Ozgur, Fadi Boutros, Naser Damer
similarity 0.67
Abstract · Accepted at ACM Multimedia 2026 Workshops

Face presentation attack detection (PAD) is traditionally formulated as a face-specific problem, although many of the visual artifacts introduced by print, replay, and recapture processes are not inherently tied to facial appearance. In this work, we investigate whether transferable PAD representations can be learned without using faces during downstream PAD training. To this end, we introduce TPO, a controlled face-free presentation attack dataset consisting of bona fide, print, and replay recordings of, almost randomly chosen, tomatoes, potatoes, and onions acquired under protocols that closely mirror conventional face PAD datasets. Using a foundation-model-based PAD architecture, we demonstrate that a detector trained on TPO achieves an average AUC of 92.70% across four standard cross-dataset face PAD benchmarks, outperforming training on synthetic faces and remaining competitive with models trained on real face datasets. Conversely, models trained on face PAD datasets transfer consistently above chance to TPO, suggesting that the learned representations capture characteristics of the presentation process rather than object semantics. Furthermore, incorporating TPO into conventional face PAD training consistently improves cross-dataset performance under fixed optimization budgets, indicating that face-free data provides complementary information rather than simply additional training samples. Finally, representation and frequency analyses provide further evidence that transferable PAD representations cannot be explained by a single spectral artifact but instead encode richer presentation cues shared across object categories. Together, these results provide empirical evidence that transferable presentation attack representations can be learned independently of facial content, opening new opportunities for privacy-preserving and identity-independent PAD development.

0.19

A Bag of Tricks for Defending Deep Learning-based Face Recognition Pipelines against Backdoor Attacks

cs.CV cs.AIcs.CRcs.LG ·2026-11-04 ·Quentin Le Roux, Marti Jimenez, Yannick Teglia, Philippe Loubet Moundi
similarity 0.67
Abstract

The protection of mature, multi-stage Face Recognition Systems is crucial to system providers. This technical report targets the threat model most relevant to industry practitioners: although model training is performed in-house, training data is sourced from possibly-compromised third parties. Given this setting, we present three practical defense "tricks" against data poisoning-based backdoor attacks. The first operates at training-time, detecting poisoned identities during training or prior to a pipeline's deployment. The last two operate at inference-time, filtering suspicious inputs before an answer can be sent back to a potential attacker. Each method is presented with their motivation, algorithms and computational impact, and evaluated on realistic face recognition workloads. Our goal is to provide actionable hardening techniques to system providers.

0.17

Location Matters: Sparse Latent-Variance Attacks for Face Privacy against Diffusion Personalization

cs.CV cs.CR ·2026-11-10 ·Shuting Yang, Shuxin Li, Linyan Dai, Xinwei Zhang, Dong Huang +2 more
code/data similarity 0.66
Abstract

Text-to-image diffusion models can learn and reproduce an individual's identity from only a few reference photos, creating risks of unauthorized and misleading content generation. Proactive defenses address this risk by adding adversarial perturbations to facial images before release, thereby disrupting identity acquisition during downstream personalization. However, existing methods typically rely on dense $L_\infty$-bounded perturbations, whose accumulated distortions can become visible around facial structures. Recent methods improve protected-image quality, but often at the cost of reduced protection effectiveness. In this paper, we propose Sparse Encoder Attack on Latent Variance (SEAL), a sparse adversarial protection framework that modifies only 5% of the spatial locations in a facial image. SEAL combines a differentiable top-k masking scheme that jointly optimizes perturbation locations and values with a variance-oriented objective over the frozen VAE encoder's latent distribution to disrupt identity-consistent representations. A saliency-guided dispersion penalty further discourages perturbation concentration in perceptually salient facial regions. Experiments on CelebA-HQ and VGGFace2 show that SEAL achieves the best protected-image quality among the compared methods, exceeding 41 dB PSNR with 0.97 SSIM and at most 0.07 LPIPS, while reducing identity similarity to at most 0.15 after diffusion personalization. These results demonstrate that sparse location optimization enables strong identity protection while preserving high visual quality. Code and data are available at https://github.com/Lvxan/SEAL

0.21