CS Paper Radar

Beyond One Epoch: Uncertainty-Weighted Sensitivity Regularization for Recommendation Models

Richard Lettich, Shagun Gupta

cs.LG ·submitted 2026-09-28 01:20 UTC · v1

Abstract

Recommendation models with sparse embeddings and a shared consumer often exhibit the one-epoch phenomenon: a second epoch lowers training loss while sharply degrading generalization. We present a view based on the violation of the prequential principle. On the first epoch, an example's label has not affected the embedding rows used to score it. On later epochs, those rows contain a displacement induced by the labels earlier update. This creates an incentive for the shared consumer to exploit this displacement in subsequent epochs, which fails to generalize. We call this self-influence asymmetry. Using an exact scalar model and local influence analysis, we connect this mismatch to the uncertainty in the embeddings and the consumers incentive to exploit it in subsequent epochs. We verify this hypothesis using an embedding-consumer-update interventions in deep recommendation models and propose uncertainty-weighted sensitivity regularization (UWSR) which counteracts this mismatch by augmenting the loss function to penalize the consumer for relying on uncertain embeddings. Unlike existing remedies, UWSR preserves the learned embeddings and across three benchmarks, four-epoch UWSR reduces test cross-entropy by 1.38%-6.78% and improves AUC by 0.0058-0.0231 relative to one-epoch training.

Closest papers

Nearest neighbours by meaning across the whole index.

Mitigating Popularity Bias in Sequential Recommendations with Sparse Prior-Regularized Objective

cs.IR ·2026-10-13 ·Anton Pembek, Anton Klenitskiy, Alexander Savchenko, Alexey Vasilev
top venuebenchmark similarity 0.67
Abstract

Popularity bias is a well-known challenge in recommender systems, often leading to the over-exposure of popular items and reduced catalog coverage. In this work, we propose PR-entmax, a sparse prior-regularized objective that makes popularity correction context-dependent. We derive our method from a generalized variational principle in which an item popularity prior is incorporated directly into a sparse probability mapping, yielding a popularity-aware active support: the set of items receiving non-zero probabilities and hence non-zero gradients is determined jointly by contextual relevance and item popularity. As a result, the prior affects not only item probabilities, but also which items participate in gradient updates during training. The method requires no architectural changes and can be integrated into existing training pipelines by modifying only the loss function. At inference time, the prior term is removed, and items are ranked by the learned contextual relevance alone. Experiments across multiple benchmark datasets and sequential recommendation architectures show that PR-entmax consistently reduces popularity bias while preserving or improving recommendation relevance, advancing the relevance-debiasing trade-off.

0.33

SURF: Subtractive Updates for Recommender Forgetting

cs.IR ·2026-09-16 ·Filippo Betello, Antonio Purificato, Nicola Tonellotto, Fabrizio Silvestri
top venuecode/data similarity 0.66
Abstract

The increasing demand for user privacy and compliance with regulations such as GDPR has made machine unlearning a fundamental requirement for modern recommender systems. However, Sequential Recommender Systems (SRS) pose unique challenges for unlearning due to their reliance on temporal interaction patterns. Existing approaches either require computationally prohibitive full retraining or fail to account for the sequential nature of user behavior. We propose SURF (Subtractive Updates for Recommender Forgetting), a lightweight framework for approximate machine unlearning in SRS. SURF operates in three stages: (i) identifying the neighborhood of the item to forget in the embedding space, (ii) training an auxiliary model on this compact local subset, and (iii) subtracting the auxiliary model's scores from the original model at inference time. Experiments against five baselines on 7 datasets show that SURF achieves unlearning effectiveness comparable to full retraining while substantially reducing computational cost, yielding up to a 32% improvement in NDCG@20 while requiring just 2% of the original retraining baseline time budget. We share our code at https://github.com/FilippoBetello/SURF.

0.36

Unbiased Recommender Systems with Implicit Feedback

cs.IR ·2026-08-17 ·Md Aminul Islam
top venue similarity 0.66
Abstract

Recommender systems typically rely on implicit feedback (e.g., clicks) to infer user preferences. However, such data is inherently prone to various biases, including position bias and popularity bias. Position bias occurs when higher-ranked items receive more interactions regardless of true relevance. Popularity bias reinforces frequent exposure of popular items while under-recommending relevant, yet less popular ones. Directly learning from such data fails to capture true user preferences, leading to suboptimal recommendations. This research focuses on mitigating position bias and popularity bias in recommender systems. Specifically, I address position bias in learning-to-rank (LTR) systems and popularity bias in collaborative filtering (CF) models and social recommender systems based on graph neural networks. My work develops methods that overcome the limitations of existing approaches to mitigating position bias and popularity bias, enabling more relevant and personalized recommendations that align with users' preferences.

0.37

Pathwise Alignment for Collaborative Filtering: Bilateral Conditional Flow Regularization

cs.IR ·2026-10-11 ·Zhiwei Li, Guodong Long, Chengqi Zhang, Jing Jiang
theory similarity 0.65
Abstract · 25 pages, 13 figures. Includes appendices

Collaborative filtering predicts user-item interactions from implicit feedback. We focus on matrix factorization (MF) trained with the Bayesian Personalized Ranking loss. This loss updates each user embedding only through that user's observed items and sampled negatives, so sparse feedback gives MF few constraints. To constrain each user through other users, we propose Pathwise Alignment for Collaborative Filtering (PACF), which trains MF with one network shared by all training pairs. At points between user and item embeddings, a flow matching term trains the network to predict the user-to-item step, reversed and scaled for sampled negatives. A trajectory consistency term pulls each user's predicted path to end at the observed item. Both terms update the embeddings, so the network is discarded after training, thereby maintaining the efficiency of MF at inference. We prove that a network with bounded sensitivity to its input point accurately fits users who share an item only if their embeddings are close, so these users constrain one another. Against eleven discriminative and generative baselines, PACF achieves the highest performance on all seven datasets. Its relative gains over MF are largest where feedback is sparsest. The shared network thus supplies the missing constraints through other users.

0.24

Alternatives to Shallow Autoencoders for Collaborative Filtering

cs.LG cs.IR ·2026-11-04 ·Mario Mallea, Angela Nebot, Francisco Mugica
published similarity 0.64
Abstract

Collaborative filtering (CF) is a cornerstone of recommender systems and plays a relevant role in many modern applications. CF uses user-item interaction data to discover future preferences. Although deep learning models have shown promise in CF, Embarrassingly Shallow Autoencoders for Sparse Data (EASE) has gained attention for its outstanding ranking accuracy provided by its closed-form solution. EASE relies primarily on relationships among items to fit a full-rank high-dimensional linear mapping. We hypothesize that this design limits its capacity to capture similar but not equivalent fine-grained user relationships, consequently limiting its recommendation accuracy. This paper introduces an alternative formulation based on EASE that takes advantage of user-user information. Furthermore, we propose a hybrid model that combines both user-user and item-item adjacency distributions. Our experiments reveal that the proposed models outperform EASE on well-known recommendation benchmarks, highlighting the significance of including the user alternative in shallow autoencoder studies for CF.

0.31

Closing the Long-Short View Gap in Sequential Recommendation without Cached History

cs.IR ·2026-09-05 ·Lingfeng Shi, Chengkai Huang, Lina Yao, James Caverlee
top venue similarity 0.63
Abstract · Accepted at CIKM 2026

Sequential recommenders are typically trained on long user histories to capture rich behavioral signals, yet serving with training-length sequences is often impractical due to real-time efficiency constraints. Directly using only recent behaviors leads to a severe performance drop. To bridge this gap, existing approaches compress user histories into persistent per-user states, storing and retrieving them at inference time; while effective, they impose non-trivial infrastructure overhead and offer little remedy in cold-start scenarios. In this paper, we empirically identify two structural flaws rooted in geometric properties and dataset sparsity, and propose a novel two-stage framework to close the long-short-view performance gap. Specifically, in the first stage, we replace the commonly used dot-product with angular similarity scoring and leverage a modified softmax to counter prefix position bias. In the second stage, we fine-tune only bias and LayerNorm components, which are universal to standard sequential backbones, for further improvement. Both stages are guided by carefully designed learning objectives. Extensive experiments on two representative backbones across three public datasets demonstrate the effectiveness of our proposed framework.

0.27

On the Regularization Landscape for the Linear Recommendation Models

cs.AI ·2026-09-10 ·Dong Li, Zhenming Liu, Ruoming Jin, Hao Zhou, Zhi Liu +2 more
similarity 0.62
Abstract

Recently, a wide range of recommendation algorithms inspired by deep learning techniques have emerged as the performance leaders on several standard recommendation benchmarks. While these algorithms were built on different DL techniques (e.g., dropouts, autoencoder), they have similar performance and even similar cost functions. This paper studies whether the models' comparable performance are sheer coincidence, or they can be unified under a single framework. We find that all linear performance leaders effectively add only a nuclear-norm based regularizer, or a Frobenius-norm based regularizer. The former ones possess a (surprising) rigid structure that limits the models' predictive power but their solutions are low rank and have closed form. The latter ones are more expressive and more efficient for recommendation but their solutions are either full-rank or require executing hard-to-tune numeric procedures such as ADMM. Along this line of finding, we further propose two low-rank, closed-form solutions, derived from carefully generalizing Frobenius-norm based regularizers. The new solutions get the best of both nuclear-norm and Frobenius-norm world.

0.14

Mitigating Popularity Bias in Recommendation with Global Listwise Learning and Progressive Bi-Weighting

cs.IR ·2026-09-28 ·Tianyu Zhu, Jiandong Ding, Yansong Shi, Guoqing Chen, Jian-Yun Nie
top venuetheory similarity 0.62
Abstract · Accepted at ACM TOIS

In recommender systems, user feedback typically follows a long-tail distribution, which leads many recommendation algorithms to exacerbate popularity bias by disproportionately favoring popular items. To mitigate this issue, recent studies have employed Inverse Propensity Scoring (IPS) to rebalance training data via reweighting user-item interactions. However, the effectiveness of IPS-based approaches is often constrained by locally unbiased objectives and inaccurate propensity estimation. In this paper, we propose Multinomial Likelihood with Bi-Weighting (Mult-BiW) to address these limitations. First, we introduce a debiasing framework, termed Mult-IPS, which integrates multinomial likelihood with IPS to capture global and unbiased user preferences over the entire item set. Second, we develop a Bi-Weighting (BiW) strategy that jointly leverages propensity scores and a collection model, incorporating a smoothing mechanism to enhance the robustness of propensity estimation. We further provide theoretical analyses that establish an upper bound on the empirical bias and characterize the optimal form of the collection model. Third, to mitigate the adverse effects of aggressive reweighting on representation learning, we design a Progressive Bi-Weighting strategy that gradually transitions from discriminative representation learning to popularity debiasing. Extensive experiments on real-world datasets show that Mult-BiW consistently outperforms state-of-the-art baselines.

0.36