CS Paper Radar

The Nearest Target Is the Wrong One: Target Separation in Arc2Face Identity Unlearning

Zeynel Tok

cs.CV ·submitted 2026-08-30 23:22 UTC · v1
published

Abstract

Unlearning an identity from a face-conditioned generator by redirecting its conditioning embedding can silently fail if the redirected output is still verified as the original person. We show that this failure depends on a controllable choice of how far the redirection target lies from the forget identity in recognition space, and that the most intuitive target, the nearest neighbour, is the one most likely to cause it. We audit Arc2Face with a locked ArcFace protocol and a projection adapter that redirects identity conditioning before generation. On a hard-neighbour stress test built from the hardest 0.5% of eligible identities, four target-selection policies show a monotonic response: clean forgetting rises from 9/30 groups under the nearest hard target to 30/30 under the least similar one. Mean forget-identity re-identification falls from 51.9 to 0.0 while mean retention stays flat. This reflects successful redirection rather than outputs becoming unverifiable: 710 of 720 least-sim-hard generations arrive at the chosen target, with no leakage to unrelated identities. Re-verifying identical images with an independent recogniser (AdaFace) preserves that trend, correlating at r=0.94, arguing against a verifier artefact. Target separation is thus a first-order, reportable design variable for identity unlearning.

Comment
16 pages, accepted at the ECCV 2026 Workshop on Unlearning and Model Editing (U&ME)

Closest papers

Nearest neighbours by meaning across the whole index.

RA$^2$: Retain-Anchored Attraction Preserves Forget-Adjacent Utility in LLM Unlearning

cs.LG ·2026-10-14 ·Zinan Ling, Yang Xiao, Ruimeng Ye, Sijia Liu, Bo Hui
code/data similarity 0.59
Abstract

Large Language Model (LLM) unlearning is often evaluated primarily by target suppression and clean retain utility. These metrics can miss a local failure mode: a model may perform well on standard retain prompts, yet degrade on benign reasoning when prompts lie near the forgotten domain. This phenomenon is termed forget-adjacent utility loss. A fundamental question is: where should forget representations go after they leave the removed behavior? Successful unlearning depends not only on whether forget-conditioned representations depart from the harmful computation, but also on the direction in which they are redirected afterward. We propose RA$^2$ (Retain-Anchored Attraction), a latent-space unlearning method that redirects forget-token states toward nearby retain-supported representations while preserving higher-layer behavior on retain data. On WMDP and MUSE, RA$^2$ yields higher forget-adjacent utility than strong unlearning baselines at similar broad-utility levels. These results show that the migration direction of forget representations materially affects benign behavior near the forget boundary. Our code is available at https://github.com/ZacLing/retain_anchored_attraction.

0.23

Identity Is More Than Recall: A Benchmark for Persistent Identity in Deployed AI Agents

cs.AI cs.MA ·2026-09-12 ·Zhenyu Zhao, Roy Zhao
benchmark similarity 0.58
Abstract · 27 pages, 2 figures, including 18 pages of supplementary material

Persistent agents need evaluations that distinguish identity facts they can recall from those they express and enact. We introduce PAI-Bench, a provider-neutral benchmark for fidelity to a versioned, update-governed identity contract. It separates recall, composition, behavioral enactment, resistance, persistence, lineage, and role-conditioned updates while keeping scoring oracles outside the target process. Two frozen campaigns cover sixteen synthetic profiles, thirty-two probes, and three independently initialized target configurations, yielding 1,536 retained responses. A judge-independent literal audit finds direct-parent identifiers in 48/48 atomic responses but only 1/48 implicit self-portraits. On eight profiles, explicit field cues increase joint presence of three identity identifiers from 0/8 to 7/8 under the same four-sentence instruction. A separate startup body-label substitution increases full-designation presence from 1/8 to 7/8 while parents remain absent. These contrasts reveal prompt-dependent component selection and component-specific sensitivity to startup cues in the tested deployments. Replaying identical factorial responses also yields a Claude headline mean 12.5 percentage points below Astra's, demonstrating evaluator sensitivity separately from target behavior. The studies use single target samples per condition, with post-hoc audits and follow-ups. PAI-Bench provides a reproducible evaluation protocol for measuring factual availability, identity expression, and behavioral enactment as distinct aspects of identity-contract fidelity.

0.32

Damnatio Memoriae: Adversarially and Selectively Forgetting Identities in the Embedding Space of Face Recognition Models

cs.CV ·2026-09-22 ·Ünsal Öztürk, Vedrana Krivokuća Hahn, Sushil Bhattacharjee, Sébastien Marcel
similarity 0.57
Abstract · 15 pages, 7 figures, 5 tables. This work might be submitted to the IEEE for possible publication

A face recognition model links two images of a person recorded on separate occasions when their embedding similarity exceeds an operating threshold. We consider making chosen identities unlinkable across separate occasions while the model remains in service for the rest of the population. Deleting their images and retraining does not achieve this, since the model recognises identities never observed in training. Therefore, the embedding space must be altered against these identities, the process of which we call open-set adversarial forgetting. We propose three loss functions, one that disperses an identity's embeddings from their centroid, and two that map each image onto its own near-orthogonal target, learnt with the classifier head or fixed in advance as an almost-orthonormal frame. Each is fine-tuned alongside the classification objective on a subset of each identity's images. We evaluate them against four methods from prior work in verification and identification, at two forget scales and three backbones. Every loss acting on the embedding geometry makes the forget identities nearly unidentifiable. The orthonormal frame alone achieves strong forgetting, which holds wherever an image of that subset enters the comparison and leaves distinct forget identities unlinkable. It also surpasses a concurrent unsupervised method at a higher retain rate.

0.26

Verification Trap: Understanding Test-Time Selection Failures under False Premises in Code Generation

cs.CL ·2026-10-04 ·Feng He, Hejia Wang, Linghao Meng, Ming Gao, Qiankun Li
top venue similarity 0.57
Abstract · EMNLP 2026

Test-time compute has become a central way to improve code generation: systems sample multiple candidate programs and use verifier-visible evidence to select the final output. This paradigm implicitly assumes that the verifier provides a corrective signal independent from the generator. We challenge this assumption under misleading task premises. When the generator and verifier share a false premise, they become coupled through a mistaken belief: the generator produces premise-consistent shortcuts, while the verifier supplies evidence that fails to expose them. Consequently, the selector may choose a hidden-test-wrong candidate even when a hidden-test-correct program exists in the pool. We call this failure mode Verification Trap. Across three code-generation benchmarks and five code models, false premises consistently degrade first-sample correctness, reduce selector-chosen correctness after 64-sample test-time selection, and amplify recoverable mis-selection. Mechanistically, verifier-written tests inherit the premise-level blind spot, reshaping verifier-visible candidate space away from hidden-test correctness. These traces make Verification Trap predictable before hidden execution: a lightweight gold-free predictor using verifier-visible features reaches 0.846 AUROC. Our results identify decoupled evidence as a key mitigation axis: coupled scaling provides limited recovery, whereas premise-agnostic robustness auditors recover substantial oracle headroom.

0.42

Counterfactual Feature Steering for Adjacency-Preserving Identity Unlearning

cs.CV ·2026-11-16 ·FNU Shivam, Layth Sliman, Ramana Reddy, Donald Adjeroh, Binod Bhattarai +1 more
similarity 0.56
Abstract

Identity unlearning in text-to-image diffusion models is increasingly important for privacy, safety, and ethical deployment, as it aims to remove the model's ability to generate specific individuals. In this work, we study identity unlearning with \emph{adjacency preservation}, where the goal is to remove a target individual's identity while preserving related non-target identities. We propose Counterfactual Feature Steering (CFS), a method requiring one-time SAE training that performs bounded inference-time concept editing in the latent space of a sparse autoencoder attached to the text encoder. CFS derives a target direction by contrasting forget and retain distributions, then applies a per-token projection proportional to each token's current alignment with this direction, yielding counterfactual representations that suppress the target while limiting collateral damage. To evaluate adjacency preservation, we introduce the Adjacency-Weighted Unlearning Score (AWUS), which measures the tradeoff between target erasure and collateral damage, and extend it to face-specific identity similarity using ArcFace. Across 4,000 prompts derived from Celeb-100, spanning 100 celebrity identities, CFS achieves the strongest evaluated tradeoff between identity erasure of target identities and preservation of adjacent and generic concepts, while remaining stable across intervention strengths where other steering-based methods degrade. Independent ArcFace evaluation and a 10,000-caption COCO study further examine identity and general preservation. Quantitative nudity-suppression benchmarks and qualitative artistic-style examples provide evidence of transfer beyond identities on SD~1.4.

0.23

Cross-Person Identity Change Under Source-Coupled Multi-Human Generation: A Controlled UniPortrait Study

cs.CV ·2026-10-15 ·Edrees Al-Edreesi, Ghaleb Al-Gaphari, Muneer Alsurori, Mohammed Edrees, Fouad Al-Hamdani
code/dataunusual similarity 0.56
Abstract · 16 pages. Project website and code: https://github.com/edrees2022/face-identity-repair

Multi-human personalization must account for identity changes in people outside the nominal target of a generative operation. We characterize this behavior in UniPortrait, without introducing a new preservation algorithm. Two sequential studies use 119 accepted two-person baseline scenes and three fixed output seeds, measuring continuous reference-based facial similarity with InsightFace \buffalo. In Study~1, source-coupled single-ID target-only generation (A1) produces negative mean transitions for both the protected person ($-0.567904$) and target ($-0.236904$), indicating broad perturbation rather than established successful repair with collateral loss. In Study~2, native dual-ID generation (A2) yields more positive paired contrasts: protected $+0.512918$ (95\% case-cluster CI $[0.471434,0.550259]$) and target $+0.158853$ ($[0.118903,0.197523]$), across 287 jointly measurable pairs from 111 cases. Shared baseline similarity cancels algebraically in these paired contrasts, leaving a matched POST comparison. Validation designed after the primary result freeze finds directional agreement with an independent identity evaluator and blinded human judgments; within-\buffalo\ nuisance models provide supportive sensitivity evidence. The findings characterize intact native regimes, whose references, routing, adapters, and token counts co-vary. They do not establish component causality, categorical identity preservation, successful localized editing, or generalization beyond the evaluated implementation and measurable cohort.

0.36

Where Identity Lives: Localized, Retain-Free Identity Unlearning in Multimodal Large Language Models

cs.CL cs.CV ·2026-08-31 ·Kangwook Ko, Jaehyuk Jang, Wonjun Lee, Hee-Seon Kim, Changick Kim
top venue similarity 0.56
Abstract · Accepted to Findings of the Association for Computational Linguistics: EMNLP 2026

Removing a specific individual's information from multimodal large language models (MLLMs) is often needed after deployment, but existing methods rely on a retain set, which is hardest to obtain at that point, and rebuilding it recreates the privacy exposure that unlearning aims to remove. Forgetting from the forget set alone instead damages the shared visual-language computation, harming perception. We cast retain-free unlearning as a localization problem: causal tracing, weight transplant, and Fisher overlap all point to early-to-mid decoder MLPs as the layers where identity information is stored and, unlike other module families, can be modified without substantially disrupting vision. We turn this into Pathway-Aware Visual-attribute Anchoring (PAVA), which confines updates to these layers and pairs a forget loss with a visual-attribute anchor that preserves image-grounded behavior by distilling the model's own pre-unlearning answers from the forget images alone. On MLLMU-Bench and ReMem, PAVA gives the strongest forget-retain trade-off among forget-set-only methods and remains competitive with retain-based baselines.

0.53

What Does It Mean to Forget a Person? Individual-Level Unlearning in Vision-Language Models

cs.CR cs.CY ·2026-09-27 ·Xiongtao Sun, Hui Li, Tiantong Wu, Jiaming Zhang, Fuyao Zhang +1 more
benchmark similarity 0.55
Abstract

Erasing individual identities from Vision-Language Models (VLMs) is uniquely challenging because personal data is entangled across modalities rather than stored as isolated attributes. However, existing multimodal unlearning benchmarks primarily evaluate attribute-centric forgetting, overlooking the more critical objective of individual-level unlearning: eliminating a model's ability to access, link, and reconstruct target-related information across modalities. To address this gap, we propose IDUnlearn-Bench, the first benchmark for individual-level multimodal unlearning in VLMs. It represents each individual as connected multimodal evidence and evaluates four task families: attribute access, identity access, identity binding, and identity reconstruction. Experiments on representative VLMs and unlearning methods show that successful attribute-centric forgetting often leaves substantial identity-level knowledge intact and can be non-monotonic: reducing one form of risk may amplify another. Models may suppress selected information while still identifying the target, linking records, or reconstructing the individual. These findings reveal a fundamental gap between forgetting information about a person and forgetting the person as a whole.

0.16