CS Paper Radar

Forgetting to Improve: Principled Data Removal in Active Learning

Manuel Wendl, Erik Englesson, Andreas Krause, Carl Henrik Ek

cs.AI cs.LG ·submitted 2026-11-02 22:50 UTC · v1

Abstract

The uncertainty of a statistical model is most commonly factorized into an aleatoric and an epistemic part. This factorization changes how predictions are interpreted in downstream decision tasks. Importantly, except for the idealistic scenario with no model mismatch, the quantification is a characteristic of the model and not the data generating process. In this paper, we propose Forgetting to Improve a method that reduces this discrepancy by incorporating the task into the modeling framework. Our key insight is to acknowledge that in scenarios of model mismatch, data can have a detrimental effect on the modeling for a specific task. Based on this insight, we propose an influence function for Gaussian process models that allows for principled removal of detrimental data samples. We showcase the flexibility of this approach by demonstrating significant improvements across a range of tasks, including Bayesian optimization, model-based reinforcement learning, and transductive learning.

Closest papers

Nearest neighbours by meaning across the whole index.

Predictively Oriented Gaussian Process Posteriors

cs.LG stat.ML ·2026-10-02 ·Callum Lau, Jeremias Knoblauch, Louis Sharrock
similarity 0.63
Abstract

Gaussian Processes (GPs) are a powerful tool for modelling and quantifying uncertainty in functional relationships. However, they require practitioners to make a number of design decisions, such as the choice of the kernel and the observation model. Suboptimal choices can produce misspecified models that do not capture the underlying data generating process. We introduce Predictively Oriented Gaussian Processes (PrO-GPs), which treat predictive uncertainty as the primary inferential target and provide a robust alternative to standard GPs. Although direct computation of a PrO posterior for nonparametric models is intractable, we derive a reduced formulation and practical sampling scheme for efficient computation. Through synthetic and real data experiments, we show that PrO-GPs produce better calibrated predictive distributions under model misspecification compared to standard GP approaches.

0.05

Learn Only Where You Can't Compute: Structured Corrections to Gaussian Process Posteriors

cs.LG ·2026-10-15 ·Tobias Weber, Carla Sagebiel, Philipp Hennig
similarity 0.62
Abstract · 25 pages, 12 figures, including appendices

Physics-informed Gaussian processes (GPs) provide a principled framework for scientific inference by expressing uncertainty over empirical measurements, boundary conditions, and governing equations. However, they struggle to scale to larger datasets and nonlinear information. Conversely, diffusion models excel at learning complex distributions over functions from massive datasets, but struggle in scientific contexts where training data are scarce and physical or observational information must be incorporated. We propose a structured posterior-correction framework that combines a tractable Gaussian reference with data-driven corrections via generative learning. After a mean correction and covariance calibration, a diffusion model learns a targeted correction in a low-dimensional subspace selected to capture discrepancies from the reference, while the complement retains its Gaussian law. Across three scientific inference tasks, these corrections recover hidden modes, improve sample structure, and reduce prediction error relative to the Gaussian reference. Comparisons with full-field generative models reveal task-dependent benefits and tradeoffs between predictive accuracy and computational cost.

0.13

Parameter Learning in Active Inference via Variational Message Passing

cs.AI cs.LG ·2026-10-22 ·Benedikt Waldeck
similarity 0.62
Abstract · Accepted for publication in the proceedings of the 7th International Workshop on Active Inference (IWAI 2026), Springer CCIS

Active inference casts perception, learning and planning as variational inference, where the Expected Free Energy (EFE) defines the planning objective. Recent work showed that EFE minimization can be written as Variational Free Energy (VFE) minimization on a generative model augmented with epistemic priors, and that the resulting entropy-corrected objective is minimized by sum-product message passing (SP-MP) on modified factor functions. These formulations treat model uncertainty as inference over a finite set of known models. We extend them to planning during learning, where beliefs over the observation and transition parameters $θ_A$, $θ_B$ are Dirichlet priors. Closed-form Dirichlet updates are usually obtained by a structured mean-field factorization between parameters and states, but under exactly that factorization the information-gain corrections vanish. We resolve this by giving every factor its own parameter prior, frozen during a rollout. Conjugacy then makes each factor belief a product of a marginal over the categorical variables and a one-step Bayes posterior of the parameters. This reintroduces the datum-dependent parameter posteriors that mean-field factorization destroys. We show that evaluating the epistemic priors under the resulting Bethe joint recovers the information gains. Stationarity of this single closed-form objective yields SP-MP whose factor functions are augmented by the information-gain weight. Experiments on two grid POMDPs, in which only a planner that learns the model finds the safe route, show that the method plans to efficiently learn $θ_A$ and $θ_B$. It reaches the goal on every episode, whereas SP-MP planners with the same inference update plateau or take the risky route.

0.33

Bayesian transfer for offline Q-learning with multi-task Gaussian processes

cs.LG stat.ML ·2026-11-07 ·Wenxing Guo
similarity 0.61
Abstract · 88 pages, 13 figures

Offline reinforcement learning can benefit from data collected in related tasks, yet indiscriminate knowledge transfer may lead to negative transfer under task mismatch and limited target-data coverage. We propose a target-aware Bayesian transfer framework for offline Q-learning based on multi-task Gaussian processes that jointly model task-specific Q-functions. Source tasks are assessed using target predictive evidence and maximum mean discrepancy (MMD), while posterior uncertainty and behavior-supported action sets guide conservative Bellman updates and policy extraction. To account for the dependence induced by Bellman bootstrapping, we distinguish Bellman innovations from bootstrap errors and establish stability and predictive source-selection results under explicit regularity conditions. We further derive selection-adaptive posterior contraction and offline policy-performance bounds, identifying the structural conditions under which source information can be pooled to achieve minimax-optimal rates in a shared-function Gaussian sequence experiment. Simulation studies with heterogeneous source tasks demonstrate improved target Q-function estimation and reduced material negative transfer compared with indiscriminate source pooling. An application to human decision-making data further shows higher average expected immediate payoff than both target-only learning and matched-size random source selection. These results highlight the benefits of combining target-aware source selection with uncertainty-aware conservative learning for reliable and sample-efficient offline reinforcement learning.

0.13

Local Support Learning

cs.LG cs.AI ·2026-10-01 ·Assaf Ben-Kish, Akarsh Kumar, James Glass, Raja Giryes
similarity 0.60
Abstract · Website and code: https://assafbk.github.io/lsl

We explore catastrophic forgetting in the context of large pre-trained models. By considering forgetting as a geometric problem in the input space of each weight matrix, we uncover a natural retention objective under which updates produced by gradient-based optimizers are suboptimal. Following this observation, we propose Local Support Learning (LSL), a general-purpose framework that augments gradient-based training for retention of prior capabilities without access to prior data. During a new learning phase, LSL pairs two components with distinct roles: a standard weight adapter, trained as usual to minimize the loss, and a gating function that enables the adapter only on input activations from its own training distribution, making the update local to that distribution. The key challenge is that this gate must route data from all learning phases while training only on data from the current one. We address this with a gate based on a Gaussian Mixture Model (GMM), whose likelihood decays rapidly away from its training data, giving it a natural tendency to stay closed on data from prior phases. We show that this post-training approach can resolve forgetting in LLMs of up to 7 billion parameters, retaining both pretrained and finetuned capabilities across multiple training phases, while being efficient in memory and compute, robust to hyperparameter choice, and showing scaling potential.

0.28

Evaluation of Active Feature Acquisition Policies with Tabular Foundation Models

cs.LG ·2026-10-05 ·Yuta Kobayashi, Divyam Madaan, Shalmali Joshi
similarity 0.60
Abstract

Active feature acquisition learns policies that sequentially acquire features to maximize information about a target variable. We study how to learn and evaluate such policies from finite offline data using prior-data fitted networks (PFNs), which are off-the-shelf models that output posterior predictive distributions without task-specific training. We show that under the imbalanced coverage of offline data, using total predictive entropy as a reward creates an epistemic bias that penalizes acquiring sparsely observed features. Specifically, this reward conflates epistemic uncertainty (arising from lack of offline data) with aleatoric uncertainty (arising from uninformative features). To address this, we target the posterior expected (aleatoric) entropy instead of the total predictive entropy output by a PFN for evaluating feature acquisitions. Empirical evaluations on synthetic and real-world datasets demonstrate that our approach consistently reduces value estimation bias and yields credible intervals with strong empirical coverage, which can translate to improved downstream policy selection.

0.32

GP--RL--MPC: Uncertainty-Aware Reinforcement Learning-Augmented Model Predictive Control with Gaussian Process Postdecision State-Value Functions

cs.LG eess.SY ·2026-11-17 ·Jon Li, Jay H. Lee
similarity 0.60
Abstract · 19 pages including references and appendices, 3 figures, 2 tables. Submitted to L4DC 2027

We propose Gaussian Process Reinforcement Learning-Augmented Model Predictive Control (GP--RL--MPC), which uses a learned critic as the MPC terminal penalty to account for costs beyond the prediction horizon. Gaussian process regression (GPR) enables data-efficient critic learning from limited, noisy rollouts within optimistic approximate policy iteration (OAPI), with soft updates mitigating policy chattering. A tunable risk coefficient weights GP epistemic uncertainty in the terminal cost, regulating sensitivity to uncertain critic predictions. The postdecision state formulation shifts disturbance averaging offline to account for aleatoric uncertainty from the terminal transition onward. Under matching objectives and admissible actions, a one-step horizon recovers greedy value-space RL. After fitting GP hyperparameters by exact marginal likelihood, we construct a sparse deterministic training conditional (DTC) posterior using inducing points selected by pivoted Cholesky (PC), reducing online evaluation costs as the training dataset grows. In a continuous stirred-tank reactor (CSTR) subject to exogenous disturbances, GP--RL--MPC with an appropriately selected risk coefficient and policy screening reduces the sum of squared errors (SSE) by an average of 80.52% relative to an otherwise matched baseline MPC across five random seeds (sample standard deviation: 3.48%).

0.28

Online Adaptive Kernel Mixing for Gaussian Process Decision Making

cs.LG ·2026-09-17 ·Kavin Aravindan, Mani Tej Sriram, Gautam Dasarathy, Tejas Bodas
published similarity 0.60
Abstract · 35 pages, 9 figures. Accepted as a full paper at IFIP Performance 2026

Gaussian Processes (GPs) are widely used as surrogates for black-box functions in sequential decision-making problems such as Bayesian optimization (BO), level set estimation (LSE), and Bayesian active learning (BAL). GP performance critically depends on kernels, and standard kernels can lead to suboptimal decisions under misspecification. To address this, we introduce HACK GPs (Hedge Adaptive Cumulative Kernels), a method that views kernel selection as an online learning with expert advice problem. HACK treats each candidate kernel as a GP "expert" and updates a distribution over experts online using AdaHedge, based on a loss received as a proxy for their ability to fit the function and align with the task objective. We provide two variants of HACK: (i) Mixture of Gaussians (MoG) and (ii) categorical sampling. We establish general guarantees showing that, under a loss-gap condition, the weight concentrates on the best kernel and the resulting acquisition function is close to that of the best expert. Empirically, we observe robust performance across BO, LSE, and BAL compared to standard kernels such as Squared Exponential and Matern-5/2, as well as simple ensemble baselines.

0.33