CS Paper Radar

Harmful Partial Interventions and Benign Leakage in Concept-based Models

Gregory Verghese, Christopher J. Soelistyo, Mateo Espinosa Zarlenga, Alessandro Barp, Christopher R. S. Banerji

cs.LG cs.AI ·submitted 2026-10-15 12:06 UTC · v1
theory

Abstract

Artificial intelligence systems deployed in high-risk settings increasingly require models that support meaningful human oversight. Concept-based models (CMs) offer a promising framework by enabling users to inspect and intervene on intermediate, human-understandable concepts. However, concept interventions can counterintuitively worsen downstream predictions, a failure mode we formalise as intervention harm. We distinguish harm after all concepts have been intervened on from harm under partial interventions, the more likely practical use case. Modern CMs often achieve high accuracy by encoding, or leaking, additional task-relevant information in their intermediate concept representations. Recent work argues that such leakage can be benign with respect to intervention harm when concept representations satisfy two conditions: sufficiency and localisation. We prove that these criteria are not sufficient to guarantee that concept interventions improve downstream performance. Across controlled and real-world settings, we find that concept supervision incompleteness is associated with increased intervention harm. Importantly, optimising the model when all concepts are intervened on, a previously proposed leakage mitigation objective, reduces endpoint harm but does not reliably prevent harmful partial corrections. Hence, we introduce a complementary monotonic intervention objective that directly penalises increases in task loss along partial intervention trajectories. Combining endpoint and monotonic intervention training substantially reduces the prevalence and magnitude of partial intervention harm while preserving concept and task predictive performance.

Closest papers

Nearest neighbours by meaning across the whole index.

ConceptGuard: Benchmarking Context-Sensitive Unlearning in Large Language Models

cs.CL ·2026-08-20 ·Sahil Kale, Ian Harris
benchmarkpositioncode/datahot direction similarity 0.69
Abstract · 17 pages, 9 figures

Large Language Models (LLMs) often require selective removal of harmful or sensitive knowledge, called unlearning, yet existing benchmarks evaluate this capability incompletely. They rely on disjoint forget and retain sets of independent facts, and measure success by simple factual recall. This misses a key requirement of unlearning from a safety perspective, namely eliminating harmful behaviours while preserving benign and beneficial knowledge. We argue that effective unlearning must operate at the level of concepts, removing unsafe applications while maintaining correct and useful usage. To evaluate this, we operationalize dual-use concepts, i.e., concepts that can be used in both harmful and benign contexts, and construct ConceptGuard, a benchmark of 2,583 harmful--benign pairs over 68 concepts, where forget and retain sets are explicitly paired in concept usage. Evaluation is intent-sensitive and centres on contextual separation, which measures how well a model suppresses harmful behaviour while preserving benign usage of the same concept. Across four unlearning methods and four models, we find that current techniques reduce harmful memorization but show strong forgetting--utility trade-offs and limited contextual separation, well below a retain-only reference. Separation trends also depend more on the concept and model than on the method. Our results point to the need for unlearning approaches that better align with real-world safety requirements. Our dataset is publicly available.

0.69

Residual Stochastic Concept Bottleneck Models

cs.LG ·2026-10-16 ·Stephen Marcou, Mateo Espinosa Zarlenga, Sonia Laguna, Moritz Vandenhirtz, Julia E. Vogt +1 more
similarity 0.67
Abstract

Concept-based models (CMs) learn human-interpretable intermediate representations that enable users to understand the model's reasoning and steer its predictions through test-time concept interventions. However, in realistic settings, their performance can be limited by concept incompleteness, where the training concepts fail to capture all task-relevant information. Adding an unconstrained residual pathway to the model's predictive process can overcome the consequences of incompleteness. However, this introduces an inscrutable, leak-prone prediction pathway that may weaken concept interventions, as these often leave residual variables unchanged. To address this limitation, we introduce Residual Stochastic Concept Bottleneck Models (Res-SCBMs), a family of concept-based models that jointly model concept and residual variables through a multivariate Gaussian distribution. Res-SCBMs learn a covariance structure that explicitly captures dependencies between the known concepts and learnt residual variables, allowing interventions to propagate to statistically dependent residual variables through Gaussian conditioning. This extends the intervention capabilities of previous CMs without requiring concept completeness during training. Our results show that, across multiple concept-incomplete benchmarks, Res-SCBMs improve predictive performance over competing baselines while retaining effective test-time concept interventions.

0.37

Are Concept Bottleneck Models Effective as Decision-Support Systems?

cs.HC cs.AI ·2026-08-26 ·Alessandro Bogani, Nicola Debole, Emanuele Marconato, Andrea Pugnana, Katya Tentori +1 more
similarity 0.65
Abstract

Concept Bottleneck Models (CBMs) are interpretable-by-design neural networks that detect human-understandable concepts from the input and use them to generate predictions. By allowing users to inspect the concepts underlying a prediction and explore how predictions change under alternative concept configurations, CBMs have emerged as one of the most prominent approaches to supporting human-AI collaboration. However, user studies investigating their actual effectiveness as decision-support systems remain limited. We present two large-scale user studies (N participants = 705, N observations = 6,959) evaluating how concept-based explanations and user interventions on the model's concepts affect the performance of the human-AI team in two distinct binary classification tasks. Our results show that CBMs, and particularly their interactive component, can improve human-AI team accuracy relative to both unaided human performance and performance with non-interpretable AI support. However, these benefits emerge only under certain conditions: classification tasks perceived as difficult, easily identifiable concepts, and active interaction with the model. We also discuss how inaccurate concept detection may undermine users' trust in the model. Overall, this work provides practical guidance for the deployment of CBMs as effective decision-support tools.

0.43

Adaptive Localization of Knowledge Negation for Continual LLM Unlearning

cs.LG ·2026-11-02 ·Abudukelimu Wuerkaixi, Qizhou Wang, Sen Cui, Wutong Xu, Bo Han +3 more
top venuecode/datahighly cited similarity 0.61
Abstract · 24 pages. Published at ICML 2025. Code: https://github.com/zaocan666/ALKN

With the growing deployment of large language models (LLMs) across diverse domains, concerns regarding their safety have grown substantially. LLM unlearning has emerged as a pivotal approach to removing harmful or unlawful content while maintaining utility. Despite increasing interest, the challenges of continual unlearning, which is common in real-world scenarios, remain underexplored. Successive unlearning tasks often lead to intensified utility degradation. To effectively unlearn targeted knowledge while preserving LLM utility, it is essential to minimize changes in model parameters by selectively updating those linked to the target knowledge, thereby ensuring other knowledge remains unaffected. Building on the task vector framework, we propose a new method named ALKN (Adaptive Localization of Knowledge Negation), which uses dynamic masking to sparsify training gradients and adaptively adjusts unlearning intensity based on inter-task relationships. Comprehensive experiments across three well-established LLM unlearning datasets demonstrate that our approach consistently outperforms baseline methods in both unlearning effectiveness and utility retention under continual unlearning settings.

0.70

REINS: Refusal-Enhanced Inhibitory Steering with Sparse Autoencoder Features

cs.AI ·2026-08-28 ·Kai-Xuan Ding, Hao-Xiang Xu, Ji-Hua Peng, Zi-Qi Chen, Jiaqi Wang +1 more
top venue similarity 0.60
Abstract · Accepted at EMNLP 2026 Main Conference

Steering with Sparse Autoencoders (SAEs) offers a lightweight inference-time path for adapting the behavior of large language models without retraining. By exposing sparse and interpretable features, SAE steering provides a promising interface for safety control that guides harmful continuations toward refusal. However, we observe that complex wrappers can still undermine existing SAE steering methods on harmful prompts. To evaluate this failure mode systematically, we construct Generalized Undercover Instruction Safety Evaluation (GUISE), a dataset of harmful prompts with complex wrappers. Existing single direction SAE steering methods do not reliably produce refusals on harmful prompts, suggesting that refusal enhancement alone can be too weak when the harmful continuation path remains active. This motivates us to propose Refusal-Enhanced INhibitory Steering (REINS), which suppresses harmful continuation features and enhances safe refusal features in the same SAE feature space. Experiments on GUISE and other datasets show that prior methods either intervene too weakly or achieve only apparent safety through collapse, while REINS substantially reduces harmful responses, markedly improves safe refusals and largely preserves general capabilities.

0.64

What should AI forget? A design and taxonomy for negative space governance in AI memory

cs.HC ·2026-10-20 ·Eryue Xu, Ke Ding, Ting-Yu Chang, Tianshi Li, Masooda Bashir
similarity 0.60
Abstract

AI memory personalizes assistants, yet users want some disclosed information forgotten. We characterize the negative space of AI memory, where protection should take precedence over potential future personalization benefits. From 179 incidents reported by 146 users, we developed a taxonomy of 15 information categories users want excluded from memory, revealing sensitivity concerns beyond conventional taxonomies. We operationalize this taxonomy in FENCE, a policy-level memory gate that automatically applies user-selected category-based rules to subsequent conversations, preventing information from protected conversations from entering AI memory by default while allowing case-by-case overrides. In a within-subjects study (N=15), participants described FENCE as making memory governance more usable, interpretable, and trustworthy than conventional post-hoc management. Our findings demonstrate how taxonomy-based rules can potentially support user-defined memory boundaries. We call for collective efforts to establish policies governing what AI should not remember, rather than leaving protection to individual retrospective management.

0.20

Causal Concept Explanations for Deep Neural Models

cs.LG cs.AI ·2026-11-10 ·Josh Rountree, Pulkit Verma, Oswin So, Chuchu Fan, Julie A. Shah
top venuecode/data similarity 0.60
Abstract · Accepted at NeurIPS 2026. 28 pages, 1 figure, 9 tables. Code: https://github.com/squarebush/ccw-net-neurips2026

Concept-based explanations in neural models are important for the development, certification, and adoption of autonomous systems. Yet, interpretability remains correlational. Without methods to guide and quantify causal relationships between explanations, a concept explanation can be provided without being necessary or sufficient for the observed output thereby appearing correct while misleading. We introduce Causal Concept Wrapper Network (CCW-Net), the first use of interventional mediation analysis from fields such as economics, medicine, and epidemiology as a training objective to align causal effects of concept-based explanations with a model's output. We further apply mediation analysis as an evaluation tool to quantify, per concept label combination, how much of any model's output is causally attributable to each concept explanation. As a training objective, CCW-Net shapes a model's concept representations such that they are both necessary and sufficient for the outputs they are credited with and independent of other concepts' contributions. Counterfactual concept representations anchor a set of causal alignment criteria: necessity, sufficiency, and independence. Necessity drives irrelevant concept effects toward zero, sufficiency ensures relevant concept representations fully reconstruct a model's output, and independence enforces the total effect decomposes linearly across concepts. We evaluate CCW-Net across aircraft control, driving, and image classification. CCW-Net provides a step toward concept-based explanations that are not merely interpretable, but quantifiably causal, supporting safer evaluation, certification, and deployment of transparent and trustworthy neural models.

0.51

Example-Based Spatial Guidance for Training-Free Concept Erasure in Diffusion Models

cs.CV ·2026-11-16 ·Younghwan Kil, Giung Nam, Joonhyeong Park, Jinwoo Shin, Juho Lee
published similarity 0.60
Abstract · Accepted at NeurIPS 2026 (poster). 51 pages, including references, checklist, and appendix

Text-to-image (T2I) diffusion models often synthesize policy-violating content, necessitating robust safeguards and rigorous evaluation frameworks. However, current approaches remain largely monolithic in both execution and evaluation, limited to coarse-grained categorization within broad unsafe concepts. Such approaches fail to account for the specific constituent factors of a violation, resulting in weak learning signals and allowing model collapse to bypass safety checks. To overcome these limitations, we propose Example-Based Spatial Guidance (EBSG), a training-free method that utilizes user-editable exemplar packs to provide granular spatial guidance for precise concept steering. Unlike existing methods that coarsely steer the entire image away from a broad concept, EBSG decomposes these categories into specific sub-concepts via exemplar text-image pairs, providing explicit, localized steering signals for more robust safety control. Furthermore, we introduce a vision language model (VLM)-based evaluation protocol that provides a fine-grained assessment, avoiding the pitfall of conventional binary evaluators that permit model collapse to bypass safety checks. Empirically, EBSG achieves state-of-the-art erasure performance across diverse safety datasets and concept categories, spanning four nudity benchmarks, seven I2P harmful-concept slices, and four MJA metaphor categories, while supporting multi-concept removal and transferring to a modern backbone, SD3.

0.40