CS Paper Radar

A Rank-One Bypass of Fixed Linear Erasure

Jai Kumar Sharma, Vanshaj Khattar

cs.LG ·submitted 2026-11-18 05:32 UTC · v1
published

Abstract

Linear concept erasure can make a target unreadable to a specified family of linear classifiers applied to a model's current internal features. In an open-weight release, however, an attacker can modify the layers that produce those features. We study a fixed affine erasure layer whose upstream model is allowed to change. If the layer transmits even one direction visible to an allowed downstream classifier, the attacker can write a target score into that direction and carry it through the layer. We show that this condition is also necessary, and that for a nonconstant linear target score the required change is a rank-one affine update. Across six CIFAR-10 class-pair tasks, the route restores nearly all pre-erasure target performance and can leave a deployed benign linear head's scores unchanged to numerical tolerance. We then price the attack: with eight target labels and 2,000 unlabelled calibration examples, a 10% mean relative feature edit yields 79.8% test accuracy, which is 91.2% of the fitted score's gain over chance; tighter edit limits sharply reduce success. Linear guardedness therefore remains a valid guarantee about the current representation, but it is not by itself a tamper-resistance guarantee for an editable open-weight model.

Comment
Accepted at the NeurIPS 2026 Workshop on Child Safety in AI (poster). 10 pages, 3 figures

Closest papers

Nearest neighbours by meaning across the whole index.

Bait-and-Recover: Poisoning Internal Refusal Signals to Defend LLMs against White-Box Editing Jailbreaks

cs.CR cs.AIcs.CL ·2026-09-05 ·Tian Gao, Zhipeng Xie, Yuhao Wu, Junhua Liu, Xin Fang
publishedcode/data similarity 0.66
Abstract · 13 pages, 3 figures. Code: https://github.com/SparkShieldLab/bait-and-recover

Open-weight large language models face a low-cost white-box threat from representation engineering attacks. Attackers can estimate refusal directions and search for projection-matrix edits that suppress safety alignment while preserving general capabilities, within minutes on a single GPU and without gradient-based training. We propose Bait-and-Recover, a weight-level defense that places a bait adapter where attackers read activations and a paired recovery adapter at the subsequent layer. Trained via gradient routing, this decouples the observation path from the behavior path. By actively poisoning the residual signal used for measurement, Bait-and-Recover disrupts the attacker's edit search, while the recovery layer restores clean downstream computation. Across four open-weight models, our defense raises the minimum refusal rate against white-box edit searches from 16.25% to 71.75% under a strict behavior-preservation budget (KL <= 0.10), with negligible impact on general benchmarks. By invalidating the core measurement assumption of these attacks, observation-path poisoning offers a practical complement to behavior-level safety training.

0.48

Rethinking Backdoor Repair Evaluation: Distinguishing Aggregate Clean Utility from Benign Performance Preservation

cs.CR ·2026-09-22 ·Baogang Song, Changtian Song, Jian Chen, Fan He, Junwei Zhou +2 more
similarity 0.65
Abstract

Backdoor repair aims to suppress malicious behavior in compromised models while preserving benign task performance. Existing studies typically evaluate these objectives using Attack Success Rate (ASR) and Overall Clean Accuracy, but aggregate clean accuracy can obscure substantial degradation concentrated in a small portion of the label space. We revisit benign-performance evaluation from a preservation perspective by distinguishing aggregate clean utility from the preservation of previously available class-wise performance. We define class-wise preservation loss by comparing clean performance before and after repair and show that aggregation can hide localized degradation through localized-loss dilution and cross-class compensation. To complement Overall Clean Accuracy, we characterize localized preservation loss using Worst-Class Preservation Loss and Tail Preservation Loss. We conduct a systematic empirical study across representative backdoor attacks, repair methods, datasets, attack targets, and model architectures, with additional validation under clean-label attacks. Results show that effective attack suppression and favorable aggregate clean performance do not necessarily imply uniform preservation of previously available benign performance across classes. Substantial localized preservation losses can remain, and their severity and class-wise structure vary across repair conditions. These findings motivate preservation-oriented class-wise evaluation alongside ASR and Overall Clean Accuracy.

0.20

Client-Side Probing of Deleted Ridge Statistics in Federated Unlearning

cs.CR cs.LG ·2026-09-03 ·Yijun Quan, Giovanni Montana
similarity 0.63
Abstract

Federated unlearning aims to remove a client's data from a shared model without retraining from scratch. Some efficient systems make deletion exact by storing compact, additive summaries of the training features and broadcasting an updated linear classifier after every accepted change. We show that these broadcasts can also reveal the hidden summaries. A malicious client can submit known changes, use the returned classifiers to identify the server state, and compare states immediately before and after an isolated deletion. This exposes the deleted sample, class, or client summary and can enable its reinsertion. We characterize exactly when the observations contain enough independent information, give a matching optimal construction for unrestricted probes, and derive a more realistic estimator based on additions formed from the attacker's own data. On MNIST and CIFAR-10, high-precision broadcasts permit exact label recovery for every tested sample deletion with both probe types. Lower-precision broadcasts sharply reduce fine-grained recovery, and insufficiently diverse responses prevent identification altogether. Unrestricted probes are readily detected by their size; most individual attacker-data additions resemble honest batches, although we do not claim that the complete sequence is inconspicuous. The results identify a concrete privacy and integrity risk, its algebraic cause, and practical limits involving broadcast precision, update verification, response rate, and concurrent activity.

0.32

Removing Information Content Does Not Certify Tamper Resistance in Open-Weight Models

cs.LG cs.CR ·2026-10-06 ·Domenic Rosati, Alessa Carbo, Ali Dadsetan, Hong Huang, Matthew Young +3 more
published similarity 0.63
Abstract · Under submission AISTATS 2026

Does removing harmful information make open-weight models resistant to fine-tuning attacks? We show that mutual information at release alone cannot universally certify slow recovery. Function-preserving reparameterizations leave information unchanged while altering gradient-descent geometry, so an invariant certificate is bounded by the fastest reachable parameterization. We apply this principle to weight--data mutual information under training-data filtering and label--representation mutual information under capability removal. Training order can change recovery time at fixed weight--data information, while exact representation-level independence can preserve the entire parameter Jacobian. An explicit construction has both information quantities equal to zero and recovers in one gradient step. Controlled experiments illustrate order-dependent recovery and parameterization-dependent attack speed. These results identify the missing requirement for certification: constraints on attack dynamics beyond mutual information at release.

0.28

Beyond Vector Hiding: Breaking and Mitigating Shared-Direction Weight Obfuscation in TEE-Offloaded Large Language Models

cs.CR ·2026-08-27 ·Menghui Zhang, Aoying Zheng, Guoxiao Liu, Zizhuang Deng, Jiejing Wen +2 more
similarity 0.62
Abstract

Trusted Execution Environment (TEE)-shielded partitioning of Large Language Models (LLMs) accelerates on-device inference by offloading obfuscated linear layers to an untrusted accelerator while retaining only a small correction inside the TEE. However, earlier lightweight obfuscation schemes preserved weight-vector directions and were broken by ArrowMatch. To defend against this attack, ArrowCloak injects scalar multiples of the same hidden direction into all weight vectors, enabling lightweight trusted correction. We show that this reuse leaves a rank-one relation across the complete accelerator-visible matrix. For the released real-valued scheme, we propose SpectralLeak, which estimates and removes the shared component. Across 12 task settings, its surrogates achieve $87.98\%$ mean accuracy versus $89.85\%$ for the victims. In our defense-favorable mod-$Q$ realization of ArrowCloak's published modular security formulation, mod-$Q$ arithmetic suppresses this spectral signal but retains the algebraic rank-one relation modulo $Q$. We therefore propose LatticeLeak, which exploits the resulting hidden lattice. In our BERT-Base and GPT2-Base experiments, it reconstructs every protected fixed-point parameter exactly; across all evaluated architectures, the reconstructed models retain victim-level task accuracy without victim queries, labels, or fine-tuning. These findings identify shared rank-one reuse as the root cause of the leakage exploited by our attacks. Guided by this insight, we design ButterflyCloak, a keyed maximal-rank butterfly mask that replaces the reused direction with distinct mask rows while retaining fast trusted correction...

0.36

StableJudge: A Lightweight Adversarial Defense Framework for LLM-as-a-Judge

cs.CR cs.AI ·2026-10-12 ·Anning Hu, Yangyu Li, Yuqi Huang, Shengchao Hu, Junchi Yan
similarity 0.61
Abstract · 12 pages, including references and appendix

Large language models (LLMs) are increasingly used as judges to assign scalar scores that drive downstream decision-making, including tasks such as paper review and recommendation. However, LLM-as-a-Judge pipelines are vulnerable to score-hijacking prompt injection embedded in evaluated content, which corrupts scoring signals and distorts induced rankings. Existing defenses often enforce robustness by modifying the judge model itself, making safety dependent on a specific model's scale and training and requiring repeated effort when the judge is updated or replaced. We propose StableJudge, a lightweight defense framework that externalizes prompt-injection mitigation as a reusable pre-scoring capability. StableJudge centers on a compact 0.13B Sentry that localizes injected spans and performs targeted sanitization prior to evaluation, and is trained using adversarially evolving attacks to improve robustness beyond a fixed prompt set. To stabilize measurement, StableJudge derives deterministic, fine-grained scalar scores from the judge's output distribution. Across held-out scientific abstracts and out-of-distribution attack variants, StableJudge improves score recovery under attack while largely preserving general-purpose performance, demonstrating that robust LLM-based judging can be achieved by decoupling safety from judge scale via reusable defense capabilities. Code and data will be released.

0.31

On the Vulnerabilities and Data-Free Removal of Watermarks in Batch Normalization Layers

cs.CR ·2026-10-20 ·Elena Rodríguez-Lois, Fernando Pérez-González
similarity 0.60
Abstract · \c{opyright} 2026 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current…

Batch Normalization (BN) layers have become a common choice for embedding white-box watermarks in Federated Learning (FL), providing a simple and effective mechanism for model ownership verification and participant identification. However, previous works have focused on robustness against generic model modifications, leaving the security implications of this design largely unexplored. In this paper, we identify fundamental vulnerabilities of BN-based watermarking schemes and propose a practical data-free attack that exploits them. Experimental results on two representative FL watermarking schemes show that the proposed attack consistently removes the watermark with minimal impact on model performance, even without access to representative training data. More broadly, our findings highlight the importance of evaluating DNN watermarking schemes under threat models consistent with Kerckhoffs's principle, rather than relying on security through obscurity.

0.12

Poison-then-Hide: Finetuning-Activated Backdoor Attack on Pretrained Vision Encoders

cs.LG cs.CRcs.CV ·2026-11-11 ·Qixuan Jin, Abinitha Gourabathina, Vinith Menon Suriyakumar, Walter Gerych, Marzyeh Ghassemi
similarity 0.60
Abstract

Backdoor attacks threaten the integrity of machine learning models by allowing attackers to control model behavior through triggers. Models can be compromised during finetuning by backdoor attacks through either poisoned data or adversarial training objectives. Because existing finetuning-activated attacks assume limited domain shift or frozen encoder layers, they often fail under full-model finetuning. We target a stealthy finetuning-activated attack where a dormant backdoor is implanted in a pretrained encoder, and later activated by finetuning on clean downstream data. We propose Poison-then-Hide, a novel attack that remains effective when the entire model is finetuned in a domain transfer. Our approach consists of three components: trigger optimization, base encoder poisoning, and targeted unlearning to conceal the backdoor. We evaluate our method on six datasets and three model architectures, and achieve state-of-the-art (up to 97%) attack success rates. We find that jointly learning the benign target task alongside the poisoning is critical to the attack's success. We demonstrate standard detection and mitigation defenses cannot fully remove the backdoor, which can reappear after benign finetuning.

0.14