CS Paper Radar

UnAct: Gradient-Free Unlearning via Targeted Activation Intervention

Saeed Abdul Muizz, Aayat Rafiq, Iqra Altaf Gillani, Janibul Bashir

cs.LG cs.CV ·submitted 2026-10-03 10:37 UTC · v1
code/data

Abstract

Machine unlearning seeks to remove the influence of designated training data from a trained model without retraining from scratch. Retrain-free methods such as Selective Synaptic Dampening (SSD) and its label-free variant LFSSD avoid full retraining but still require backpropagation and parameter importance computed over the entire dataset. We ask: what happens when a deletion request arrives with only a few images of the class to be forgotten? To answer this question, we introduce UnAct, a gradient-free class-unlearning method that needs only forward passes over the forget images. UnAct scores late-layer units by their responses, attenuates the most responsive connections, and repeats this for up to 20 rounds using no gradients, no labels, and no retained data. On ResNet-18 trained with CIFAR-10, CIFAR-20, and CIFAR-100, UnAct is competitive with SSD and LFSSD when forgetting entire classes and, unlike them, never collapses the network when forget data is scarce. On ResNet-18, across all tested sizes, UnAct's retain accuracy stays within 2.5 points of retraining, while SSD and LFSSD, at their full-class operating points, lose up to 86 points on some classes. With five forget images on CIFAR-10, UnAct's distance to retraining is 0.21 points, against 67 for LFSSD and 90 for SSD, and re-selecting SSD's threshold at each size with an oracle does not close the gap. In preliminary transfer to ViT-B/16, UnAct's distance to retraining is 11.5 against 33.7 for SSD, and a request is 19x faster than SSD when SSD computes its importance at request time. The code is available at https://github.com/abdulmuizz0903/UnAct

Closest papers

Nearest neighbours by meaning across the whole index.

Unmerge: Efficient Machine Unlearning via Task Arithmetic

cs.LG cs.AI ·2026-09-30 ·Haoran Tang, Andrew Tan, Rajiv Khanna
similarity 0.71
Abstract

Approximate machine unlearning seeks to remove the influence of a forget set from a trained model without full retraining. Existing gradient-based methods require data-dependent hyperparameter search, struggle when forget and retain knowledge are entangled, and offer little insight into where unlearning actually happens inside the network. We recast unlearning through the lens of task arithmetic: if finetuning produces a merged task vector $τ_m$ that combines learning on forget and retain sets, unlearning is the inverse operation that subtracts a learned forget component $τ_F$ to recover the retain task vector $τ_R$. The forget signal is concentrated: at every layer, forget activations lie in a subspace spanned by a handful of dominant directions, so we factorize $τ_F$ in a low-rank forget basis, which is faithful up to a small tail-eigenvalue residual and limits how far the correction can perturb retain. We then optimize three intuitive goals (match the merged vector inside the forget span, suppress leakage into the retain span, and bound the correction size) that provably bound forget leakage and retain damage in activation space. The resulting algorithm, Unmerge, is fast and powerful: on class-level unlearning with ResNet-50 on CIFAR-100 and Tiny ImageNet, it improves Tug-of-War by up to ~24% over a baseline of comparable runtime and by up to ~18% over stronger baselines that run ~5x slower, keeps membership-inference exposure at the level of retraining, and shrinks the feature-distribution gap to the retrained model, where relabeling methods leave forget features cleanly separable. Further studies show that Unmerge also applies to ViT-S/16 and scales to Llama-3.2-3B. The per-layer basis geometry that drives the algorithm also serves as a layerwise diagnostic for when and where unlearning becomes structurally hard.

0.26

Learning with Unlearning: Synaptic Role Partitioning for Simultaneous Continual Learning and Machine Unlearning

cs.LG ·2026-11-18 ·Mohammad Ali Vahedifar, Behzad Bozorgtabar, Qi Zhang
similarity 0.70
Abstract

Continual Learning (CL) asks a model to acquire new knowledge without forgetting what it already knows. Machine Unlearning (MU) asks it to destroy specific knowledge without damaging the rest. The challenge is integrating them in the same system; the two objectives pull on the same weights, and an edit made for one harms the other. To address this, we propose Learning with Unlearning (LwU), a unified framework that resolves the conflict where it occurs: at the individual synapse. LwU scores every weight twice, once against the data to keep and once against the data to delete, and partitions the network into four zones with separate update rules. Weights that only require retention are frozen. Weights that only require deletion are cleared and returned to the network as free capacity. Contested weights take a gradient-ascent step projected orthogonally to the retention gradient. The remaining capacity learns the incoming task by a privileged teacher via self-distillation. Deletion is served post hoc, and no retraining is required. Across both CL and MU jobs, LwU outperforms or matches the strongest baseline. Its real-world practicality has been demonstrated on the Physical Artificial Intelligence (PAI) dataset we conducted. Additionally, in the image generation task and in removing dangerous content, it outperformed the closest baseline, as shown in Fig.1.

0.13

On-the-go Forgetting without Explicit Unlearning via ERASE

cs.LG stat.ML ·2026-09-05 ·Kushal Chakrabarti, Mayank Baranwal
publishedbenchmark similarity 0.69
Abstract · https://openreview.net/forum?id=PIXVov5LQq

Existing unlearning approaches typically rely on post hoc weight adaptation or distillation, leading to duplicated memory costs, degraded generalization, and limited scalability. In this work, we introduce ERASE, Erasure via Reconstructive Adversarial Signal Editing, a framework for on-the-go forgetting that suppresses the observable influence of private data without modifying model weights. ERASE leverages structured, class-conditioned input perturbations to induce selective forgetting during inference, eliminating the need for retraining, fine-tuning, or model copies. We rigorously characterize sufficient conditions when ERASE provably achieves functional forgetting of designated subclasses while preserving predictions across other subclasses within the same superclass. This analysis offers a principled foundation for inference-time forgetting under mild regularity assumptions. Across diverse architectures and benchmark datasets, ERASE maintains the best observed balance between forgetting efficacy, computational efficiency, and retention fidelity over recent unlearning-based methods. By reimagining data removal as forgetting without unlearning, our work establishes a scalable, regulation-aligned pathway for continual, privacy-conscious learning.

0.41

Why Forget-Only Unlearning Needs Memorization

cs.LG cs.ITstat.ML ·2026-10-07 ·Luka Radić, Vikrant Singhal, Amartya Sanyal
similarity 0.68
Abstract

Machine unlearning asks for a deletion algorithm whose output is close to retraining from scratch without the selected forget examples. In this work, we study forget-only unlearning, where the deletion algorithm receives only the trained model and the examples to forget, with no retained data or extra training information. We ask whether forget-only unlearning is always possible. We first show that this depends on the learning method: different datasets can produce the same trained model but require very different outputs after the same examples are removed. Using this observation, we derive lower bounds on how accurately unlearning can match retraining and instantiate them for several standard learning algorithms. We then ask what must be true when forget-only unlearning succeeds. To this end, we derive lower bounds on what an algorithm must memorize about the training data to handle arbitrary deletion requests. For simple threshold learners, the required information can be as large as the entire dataset, even though ordinary training keeps only one boundary point. Overall, our results show that information discarded during ordinary learning may be needed later for deletion, so models designed for forget-only unlearning may need to retain more information than standard training does.

0.18

Do Influence-Derived Data Perturbations Enable Machine Unlearning? A Controlled Study of Three Plausible Roles

cs.AI ·2026-09-11 ·Chenkai Wu, Chrispine Kambimbi, Qinyang Zeng, Jun Yan
similarity 0.67
Abstract

We evaluate Deep Perturbation Learning (DPL), which perturbs training images and labels along influence-derived directions, in three roles in which prior work has positioned it for machine unlearning: a direct deletion signal (the strongest claim), a utility-preserving regularizer, and a warm start for adversarial unlearning. Evidence for the weaker roles has been used to support the stronger one, so we test each role separately under a matched protocol with exact-seed retraining baselines. An audit of the public implementation identifies two correctness issues: image directions are computed on augmented, normalized tensors but applied to raw images, and the label perturbation falls below float32 resolution, leaving labels unchanged. After correcting the image-perturbation pipeline, DPL fails the direct-deletion criterion on CIFAR-10/ResNet-18 in all three paired seeds. Its utility effects are inconsistent in sign across seeds, and once direction-computation time is counted it underperforms simple warm-start baselines. A one-seed Tiny ImageNet check likewise does not favor DPL as a regularizer or warm start; preprocessing inconsistencies in the released code make the direct comparison there inconclusive. These results cover random instance deletion only and do not rule out influence-based methods in other deletion regimes. We release a role-matched evaluation protocol and an audit checklist for perturbation-based deletion claims.

0.28

Memory-Coupled Unlearning for Data-Absent Identity Forgetting

cs.CR ·2026-10-09 ·Chun-I Fan, Lu-En Chen, Ming-Feng Tsai, Tomohiro Morikawa
similarity 0.67
Abstract

Machine unlearning aims to reduce the influence of designated training data from a trained model while preserving its utility on retained data. Identity-level unlearning is particularly challenging when only limited target-specific information is available when a forgetting request is processed. In this work, we propose Memory-Coupled Unlearning (MCU), a framework that augments a host model with a Modern Hopfield Network (MHN) serving as a controllable external identity-prototype memory. During prototype construction, feature representations associated with each training identity are aggregated into identity prototypes and stored in the MHN. A Prototype Adaptation Network (PAN) transforms the retrieved memory representation, while a Dependency Gating Unit (DGU) controls its contribution to downstream prediction. When an identity is requested to be forgotten, MCU first removes the corresponding prototype from the external memory and subsequently readapts the model using limited support information associated with the target identities together with retained data. Importantly, in this work, ``data-absent'' specifically refers to the absence of the complete original forget set during request-time gradient-based readaptation; retained data may still be used to preserve downstream-task utility. Experiments based on the identity-partitioning and one-shot support construction of 1-SHUI show that MCU achieves a favorable empirical trade-off among model utility, membership distinguishability, and request-time computational efficiency compared with the evaluated baselines. These results provide empirical evidence for the potential of controllable external memory in machine unlearning, rather than a formal guarantee of complete information removal.

0.26

On the Plasticity Collapse in Continual Machine Unlearning

cs.LG cs.AI ·2026-08-30 ·Yingdan Shi, Xiang Xu, Kaize Ding, Alfred O. Hero, Ren Wang
code/data similarity 0.66
Abstract

Machine unlearning enables deep neural networks to selectively remove the influence of specific data in response to privacy and regulatory requirements. While prior work largely studies single-shot unlearning, real-world systems must accommodate continual unlearning, where multiple unlearning requests occur sequentially over time. In this work, we identify a fundamental limitation of this setting: plasticity collapse, a progressive breakdown in a model's ability to effectively forget. Through theoretical analysis of continual unlearning dynamics, we show that continual unlearning operations accumulate geometric constraints in parameter space, leading to saturated subspaces that restrict future updates. This structural effect induces two distinct failure modes: (1) Forward failure -- diminishing forgetting quality for subsequent tasks, and (2) Backward failure -- spontaneous re-memorization of previously forgotten information. Extensive experiments across multiple architectures, datasets, and methods in image classification confirm that plasticity collapse is not an artifact of specific implementations, but a pervasive phenomenon inherent to continual unlearning. Our findings reveal a critical barrier to the long-term reliability of machine unlearning systems and motivate the development of plasticity-preserving unlearning algorithms. Our code is available at https://github.com/TIML-Group/Continual-Machine-Unlearning-Plasticity-Collapse

0.22

Forgetting Only What Matters: Layer-Selective Unlearning toward Robust LLMs

cs.LG cs.AI ·2026-09-09 ·Ravi Ranjan, Olivera Kotevska, Agoritsa Polyzou
highly cited similarity 0.66
Abstract · 22 pages, 6 figures, 11 tables, AACL-IJCNLP 2026, conference paper

Large Language Models (LLMs) can memorize and reproduce sensitive, copyrighted, or otherwise undesirable training content, creating privacy, safety, and regulatory concerns. Machine unlearning offers a practical alternative to full retraining, but many existing methods apply broad or fixed parameter updates that can degrade utility and remain brittle under deployment changes such as post-training quantization, where forgotten knowledge may partially re-emerge. We propose Forgetting Only What Matters via Unlearning Layers (FOM-UL), a layer-level unlearning framework that selects transformer layers using a forget-to-retain significance score. This score identifies layers with high influence on the forget set and low sensitivity to the retain set, allowing FOM-UL to concentrate updates where they are most effective while leaving most of the model unchanged. This targeted update strategy improves the forgetting-utility trade-off and provides an empirical path toward quantization-resilient unlearning by reducing the chance that small, diffuse updates are erased by low-bit rounding. Across TOFU, KnowUnDo, and MUSE-style evaluations, FOM-UL reduces residual memorization compared with strong GA, NPO, KLD, SURE, ReLearn, and LUNAR-based baselines while preserving retain-set utility close to the vanilla model. Under 8-bit and 4-bit post-training quantization, FOM-UL maintains stronger memorization suppression and utility preservation than competing methods, and adversarial prompt evaluations show lower recovery of forgotten content. Overall, FOM-UL provides an efficient unlearning strategy that improves targeted forgetting, utility preservation, and deployment robustness without claiming formal guarantees of erasure.

0.43