CS Paper Radar

SynthFaces: Balanced Large Scale Human Dataset

Sergey Karpukhin, Denis Bobkov, Kirill Morozov, Aibek Alanov

cs.CV ·submitted 2026-10-29 14:17 UTC · v1
publishedcode/data

Abstract

The pursuit of fair and equitable AI in human-centric tasks is critically hindered by the inherent biases and privacy risks of real-world datasets, which offer imbalanced representations of identity and often contain sensitive personal information. Consequently, state-of-the-art generative models trained on this skewed data exhibit unfair performance, particularly in identity-preserving generation. In this work we propose a pipeline for controllable generation of dataset with diverse faces for generative models training and finetuning. We achieve this by expanding foundational Arc2Face diffusion model with a flow matching generative model in the manifold of identity embeddings. We demonstrate that such approach achieves both goals of generative fidelity and label diversity not present in real human-centric datasets. As a result we collect and further contextualize a highly balanced identity-centric dataset of humans named SynthFaces. By design, our released images and identities are synthetic, reducing direct reliance on real personal images; we empirically check identity separation with a nearest-neighbor non-leakage metric. We purpose it for training and tuning of fair models for various downstream tasks, and showcase its usability in finetuning an existing adapter-based method for personalized generation.

Comment
accepted at BMVC 2026 github: https://github.com/ControlGenAI/SynthFaces

Closest papers

Nearest neighbours by meaning across the whole index.

IDPrior: Decoupling Identity Generation from Facial Image Synthesis

cs.CV ·2026-11-05 ·Fadi Boutros, Meriç Demirörs, Darian Tomašević, Vitomir Štruc, Naser Damer
publishedcode/data similarity 0.77
Abstract · Accepted at WACV 2027

Large-scale identity-labeled face datasets are essential for training modern face recognition models, yet collecting and annotating such data is costly and raises privacy and ethical concerns. Existing synthetic face generation methods typically derive identities from predefined class labels, unconditional image generation, or predefined latent sampling strategies, without explicitly modeling the underlying distribution of facial identities. In this paper, we propose IDPrior, a two-stage framework that decouples identity generation from image synthesis. We first learn a diffusion prior in the embedding space of a pretrained face recognition model to model the distribution of facial identities and generate novel identity embeddings. These embeddings are then used to condition a pretrained identity-conditioned diffusion model (DM), enabling the synthesis of multiple images of the same identity. This modular design provides a scalable approach to identity generation while remaining compatible with existing and future identity-conditioned DMs. Experiments indicate that IDPrior generates diverse and well-separated identities while maintaining a closer correspondence to the distribution of authentic identities. Face recognition models trained on datasets generated using IDPrior-generated identities achieve competitive or improved performance compared with existing synthetic-data-based methods across multiple face verification benchmarks. Code and generated datasets are publicly available at https://github.com/fdbtrs/IDperturb.

0.38

No Way To Steal My Face: Proactive Defense Against Identity-Preserving Personalized Generation

cs.CV cs.CR ·2026-10-14 ·Lizhi Xiong, Jun Li, Ziqiang Li, Weiwei Jiang, Zhangjie Fu
top venuecode/data similarity 0.76
Abstract

Recent advances in diffusion models have enabled high-fidelity, identity-preserving image generation for personalized applications such as digital avatars and virtual try-on systems. However, their reliance on sensitive facial reference images raises growing privacy concerns. Existing defense mechanisms are primarily designed for training-based personalization and struggle to generalize to emerging training-free approaches, due to fundamental differences in their identity integration paradigms. To bridge this gap, we propose IDGuardian-the first generalizable and model-agnostic identity protection framework capable of defending against both training-based and training-free personalization methods. IDGuardian abstracts the personalization process into two critical stages: identity extraction and identity injection. It then introduces crafted adversarial perturbations to simultaneously disrupt both stages. Specifically, it degrades the identity features extracted by external encoders and establishes an adversarial conceptual bridge that misdirects the generative trajectory away from the target identity. Extensive experiments show that IDGuardian effectively protects identity across various personalization pipelines and model architectures, while remaining robust to post-processing, adaptive attacks, and cross-dataset generalization. Our code is available at https://github.com/OpenAscent-L/IDGuardian.git.

0.68

Not All Similarity Is Leakage: Distribution-Aware Identity Leakage Assessment in Synthetic Face Datasets

cs.CV cs.AI ·2026-10-19 ·Paweł Borsukiewicz, Daniele Lunghi, Wendkûuni C. Ouédraogo, Fadi Boutros, Jacques Klein +1 more
similarity 0.74
Abstract

Synthetic data provides the means for more ethical and less biased facial recognition. Nonetheless, the limitations of generative models and the conditioning process they rely on pose a risk of identity leakage, resulting in synthetic samples that are overly similar to those seen in a generator's training set, mitigating privacy benefits. In this work, we show that previously proposed approaches are affected by two key limitations: high classifier dependence and a tendency to identify leakage with respect to real datasets that were not used in training. To address these shortcomings, we propose a novel metric called Scaled Overlap (SO), which adapts a non-parametric data-copying test for facial recognition and extends it to the identity level. By leveraging a multi-model setup that compensates for errors in individual classifiers, it provides insights into the true strength of identity leakage. Our analysis identifies leaking datasets (CemiFace, SFace2, IDNet) with the most acute leakage introduced by generative model conditioning in CemiFace, demonstrates that real datasets do not contribute equally to generator training, and shows that the majority of synthetic images are drawn from multiple identities rather than relying on a single identity. Ultimately, our study provides new insights into the quality of synthetic datasets and gives more reliable techniques to help the community ensure the trustworthiness of synthetic data.

0.19

Not All Minorities Are Equal: Fairness Outcomes in Private Image Generation

cs.CV cs.CY ·2026-10-12 ·Cohen Archbold, Usman Hassan, Sen-ching Samson Cheung
hot direction similarity 0.70
Abstract · 5 pages, 3 figures, Under review for ICASSP 2027

Differential privacy (DP) provides formal guarantees for protecting individuals in artificial intelligence systems, but its consequences for fairness in generative models remain poorly understood. We investigate how privacy-preserving training affects the representation of demographic groups and visual attributes in generative models of human faces. Across two facial image datasets and multiple privacy levels, we find that stronger privacy can disproportionately degrade the quality and utility of some underrepresented groups, suggesting that privacy can amplify existing disparities. However, representation alone does not explain these effects: several well-represented attributes also experience substantial degradation, while some underrepresented attributes remain comparatively robust. These findings show that the impact of privacy-preserving learning cannot be explained solely by data representation and may depend on the characteristics and interactions of the information being learned. Our results highlight the need to consider both privacy and fairness when evaluating generative models.

0.23

WithEveryone: Unified Planning and Identity Grounding for Group Image Generation

cs.CV ·2026-08-20 ·Hengyuan Xu, Qixun Wang, Yiji Cheng, Miles Yang, Zhao Zhong +3 more
benchmarkcode/data similarity 0.69
Abstract · Project Page: doby-xu.github.io/WithEveryone/ ;Code will be released: github.com/Doby-Xu/WithEveryone/ ; References updated

Identity-preserving image generation becomes increasingly unreliable when a scene must contain many specified people. Beyond retaining each identity, the model must bind every reference to a distinct person and location, while training-time identity losses must establish correspondence among several noisy predicted faces. We introduce WithEveryone, a unified framework for generating group images up to ten reference identities. WithEveryone injects each selected identity as an addressed token, predicts a structured identity--layout plan, and renders the plan as a visual condition. Its key objective, Layout-Grounded ID Loss, uses annotated face regions to supervise the intended identities directly, avoiding unstable embedding-based face matching; ID Representation Forcing additionally trains a prediction for each identity before image synthesis. On an identity-disjoint benchmark, WithEveryone achieves the highest target-context identity similarity, improving face similarity from 0.462 for GPT-Image-2 to 0.499, while reducing copy-paste artifacts from 0.169 to 0.055. It further covers 97.3\% of the requested identities with a duplicate rate of only 2.8\%. These results show that explicit identity--layout grounding enables identity-preserving generation to scale to larger groups without relying on direct reference-face copying.

0.38

How Private is Private? A Comparative Study for Face De-Identification

cs.CV ·2026-10-07 ·Hui Wei, Hao Yu, Hui Kuurila-Zhang, Guoying Zhao
top venuebenchmarkcode/datahot direction similarity 0.68
Abstract · Accepted to NeurIPS 2026. Project Page: https://cv-ac.github.io/hifd/

Face de-identification (FDeID) has emerged as a critical privacy-preserving technology, yet its evaluation remains fundamentally fragmented. Existing protocols rely on inconsistent metrics, heterogeneous datasets, and partial annotation coverage, so methods targeting different utility dimensions, such as landmark versus expression preservation, are reported on different benchmarks under different metrics, rendering cross-method comparison infeasible. We revisit FDeID evaluation from both the data and metric perspectives. On the data side, we introduce UtilFace, a curated, demographically balanced benchmark with high identity diversity, assembled from four large-scale face datasets through identity-aware cleaning, resolution enhancement, and stratified filtering. On the metric side, we propose HiFD, a Hierarchical Face De-identification metric that unifies identity suppression, multi-level utility preservation, and image quality under a single consistency-based paradigm: every component is computed from pretrained estimators' outputs on the original face and its de-identified counterpart, directly quantifying how much identity is suppressed and how much downstream-perceivable utility survives. HiFD organizes facial signals into a three-level utility hierarchy spanning macro cues (L1), micro cues (L2), and imperceptible cues (L3), and aggregates the five resulting components into a single interpretable score via weighted harmonic mean, with configurable application-specific profiles. Using this unified protocol, we conduct a comprehensive comparative study spanning adversarial, GAN-based, and diffusion-based methods, surfacing trade-offs and failure modes that remain invisible under existing protocols. We release the benchmark and evaluation toolkit to foster systematic and reproducible research in privacy-preserving human face analysis.

0.45

A Data-Interventional Framework for Auditing Privacy and Fairness in Generative Medical Imaging

cs.CV ·2026-09-22 ·Mischa Dombrowski, Bernhard Kainz
top venuecode/data similarity 0.68
Abstract · Accepted for publication at the Journal of Machine Learning for Biomedical Imaging (MELBA) https://melba-journal.org/2026:026

Diffusion-based synthetic data generation offers a promising route for sharing medical imaging data without releasing sensitive patient records. However, generative models face a fundamental tension between privacy and fairness: they may memorize rare training samples, leading to privacy risks, or fail to reproduce underrepresented features, resulting in unfair synthetic distributions. While prior work has largely focused on either memorization or fairness in isolation, their interaction remains insufficiently understood. In this work, we introduce a data-interventional framework to systematically analyze privacy and fairness in diffusion models. We discuss synthetic anatomical fingerprints (SAFs), rare and manually injected image features, as controlled probes to study whether models generalize sensitive attributes across identities, memorize training samples, or suppress rare signals entirely. Across multiple conditioning modalities, we observe a consistent behavior: models either forget these fingerprints or memorize the entire image in which they appear, but do not generalize them to novel images. To support large-scale auditing where explicit sample extraction is infeasible, we further introduce the indicator metric t', which estimates a model's susceptibility to memorization by exploiting the internal structure of the diffusion process. By comparing conditioning signals of varying surprisal, we reveal a clear relationship between conditioning rarity and memorization behavior. Highly surprising conditioning signals act as retrieval keys that amplify memorization, whereas low-surprisal conditioning signals systematically suppress rare features, even when these appear repeatedly in the training data. Our findings provide actionable insights and concrete mitigation strategies for safe and fair synthetic medical data sharing. Code is available at https://github.com/MischaD/Privacy.

0.50

ALADDIN: Adversarial Latent Diffusion Distillation for One-Step Identity Swapping

cs.CV ·2026-10-27 ·Ivan Shchekotov, Timofei Gritsaev, Viacheslav Pirogov, Dmitry Vetrov
published similarity 0.68
Abstract · Accepted to WACV 2027. 15 pages, 5 figures, 4 tables, including references and appendices

Face swapping transfers a source identity onto a target image while preserving the target's pose, expression, hair, and lighting. The most realistic current methods are diffusion-based, but they need many network evaluations per image. GAN-based methods are fast, yet trained from scratch they must learn realism from the adversarial signal alone and produce visibly weaker results. We present ALADDIN (Adversarial Latent Diffusion Distillation for One-Step Identity Swapping), a one-step face swapper obtained by initialising the generator from a pre-trained multi-step flow-matching model and training it adversarially under identity and target-preservation constraints. We expose a controllable trade-off between identity transfer and target-attribute preservation by training a student model with triplet pseudo-label supervision. We additionally identify a bias in the standard evaluation protocol, which favours methods trained against the same identity encoder used to score them, and re-evaluate all baselines with an independent one. ALADDIN is state-of-the-art among one-step methods in realism and target preservation at comparable identity transfer, and matches the best multi-step model with 28 times fewer network evaluations and an order of magnitude fewer training pairs.

0.33