CS Paper Radar

What is the goal of unsupervised machine learning?

Aapo Hyvärinen

cs.LG ·submitted 2026-10-08 11:07 UTC · v1
unusual

Abstract

Unsupervised learning is one of the main branches of machine learning. Here I argue that unlike the other branches of machine learning (supervised and reinforcement learning), unsupervised learning is a rather heterogenous field that can serve several different goals. It seems futile to try to define one single goal for unsupervised learning. I identify four different goals for unsupervised learning: 1) Estimating the distribution, 2) Generating new data points, 3) Extracting features for downstream tasks, and 4) Understanding the data.

Closest papers

Nearest neighbours by meaning across the whole index.

Position: Unlabeled IS NOT Equal to No Human Supervision in Visual Learning

cs.CV cs.LG ·2026-09-02 ·Dong Lao
publishedposition similarity 0.55
Abstract · ICML 2026

This position paper argues that the absence of labels does not imply the absence of human supervision in visual learning, and urges the research community to identify sources of supervision more explicitly. Many recent methods in computer vision build upon representations learned from large-scale unlabeled data, and are therefore grouped under the same umbrella term ``unsupervised.'' However, different data curation schemes and training objectives embed substantially different human priors on which models rely, and we argue that one ``unsupervised'' umbrella term is no longer capturing these distinctions. This ambiguity makes it harder to compare unsupervised learning research conducted under different assumptions, coinciding with a sharp decline in papers titled with ``unsupervised'' in flagship computer vision conferences since 2021, despite continued growth of the field. While we fully embrace pre-training as a strong foundation for modern computer vision, we advocate for a community-level effort toward greater conceptual clarity: authors are encouraged to disclose priors in data selection and learning objectives, and to specify which components of a learning pipeline depend on which assumptions. Standardized disclosure practices can improve academic communication, ensure fairer comparisons, and preserve methodological diversity in unsupervised learning.

0.47

Towards Truly Unsupervised Evaluation of Feature Selection -- Extended Version

cs.LG ·2026-08-12 ·Hafiz Saud Arshad, Muhammad Rajabinasab, Arthur Zimek
published similarity 0.53
Abstract · The paper is extended with Appendix. We have added the results for all the datasets and the ranking produced by each of the proposed metrics

Feature selection is one of the most important and fundamental tasks in data mining, tackled by a family of methods with an established set of evaluation techniques to measure the quality of a specific method. Most of the methods commonly used for the unsupervised evaluation of feature selection algorithms suffer from critical design flaws which question their unsupervised nature. In this paper, we provide a critical discussion on the established allegedly unsupervised evaluation techniques, and shed light on the reasons why they are not truly unsupervised but, at best, supervised evaluation under an unsupervised downstream task. We also propose a novel, truly unsupervised evaluation framework to measure the quality of the feature selection algorithms without any form of information about the labels. The proposed framework utilizes unsupervised Principal Component Analysis, and optimal transport to measure the quality of the feature selection methods in a truly unsupervised manner.

0.23

Training Foundation Models with a Single Algorithm

cs.LG ·2026-10-29 ·Yuda Song, Alvin Zhang, Fahim Tajwar, Daman Arora, Jeff Schneider +5 more
similarity 0.47
Abstract · 67 pages, 31 figures, 22 tables

Foundation models are typically trained in stages: pretraining, midtraining, supervised fine-tuning (SFT), and reinforcement learning (RL), with changes in both data and learning algorithms. We show that this algorithmic separation is not fundamental. By deriving suitable RL objectives as maximum-likelihood fitting to model-generated target distributions, we place all of these stages under a single likelihood-based training algorithm, unified training. This perspective turns the foundation-model training lifecycle into a problem of choosing which data to train on, in what proportions, and when, with conventional staged training as a special case. We instantiate this framework in a unified training system that mixes static data and RL generations within a single trainer. Across controlled and real-world experiments, unified training improves the accuracy-compute tradeoff and task coverage while preserving broader capabilities, and provides a stronger starting point for continual learning. This work lays the groundwork for a unified science of foundation-model training, connecting mathematical principles, training infrastructure, and the development of model capabilities.

0.28

Learning to Unlearn: Machine Unlearning via Learning the Unlearning Behaviors

cs.LG cs.AI ·2026-08-17 ·Hang Zhang, Kaifeng Zhang, Yixiao Ma, Weijie Xu, Ye Zhu +1 more
similarity 0.47
Abstract

Various machine unlearning techniques have been developed in response to privacy legislation requirements, enabling individuals to exercise their legal right to have their data $D_f$ removed from a machine learning model. This process is typically accomplished via the use of an unlearning function denoted as $U$. Existing methods focus on designing an intricate $U$ to unlearn $D_f \subset D$ from a previous model $A(D)$, so that the unlearned model performs as closely as possible to the retrained model $A(D \setminus D_f)$. However, these methods often suffer from high computational costs when dealing with massive training data, as the complex structures of $U$ become a bottleneck even for models with fewer parameters. Inspired by Learning to Optimize, we introduce the first learning-based model-agnostic approach, Learning-to-UnLearn (L2UL). Our core insight is to shift from manually designing $U$ to learning the unlearning behaviors from a distribution perspective, thereby acquiring a simple and efficient $U$ via learning. Our experimental results demonstrate that the accuracy achieved by L2UL is comparable to that of retraining while exhibiting impressive efficiency, particularly in data-intensive scenarios. Furthermore, we validate the performance and scalability of our method on larger models ResNet.

0.15

A Data-Driven Approach to State Construction in Markov Models

cs.LG stat.MLmath.PRstat.ME ·2026-08-21 ·Linde Van Gestel, Marie-Anne Guerry, Evy Rombaut
similarity 0.46
Abstract

A Markov chain is a widely used stochastic process modelling random events over time. These models are built on subsets of the entire dataset, referred to as states, which are considered to be homogeneous regarding transition probabilities. However, the creation of these states is often disregarded or based on prior assumption, potentially violating the homogeneity requirement and thus decreasing the validity and predictive power of the model. In order to fill this gap, this paper combines supervised feature selection with unsupervised learning techniques for data-driven state construction. Density-based clustering, spectral clustering, and Kohonen self-organizing maps are examined for their ability to identify latent groups without prior assumptions. The contribution of this study is twofold. First, the paper presents a methodological framework for state construction incorporating suitable unsupervised learning techniques, with appropriate measures both for classification performance and Markov model accuracy. Secondly, the framework is tested on an application, resulting in a comparative analysis showing that spectral clustering and Kohonen self-organizing maps are best at capturing inherent structure. These results serve as a cornerstone in providing theoretical and methodological guidance for improving state definition in applied Markov modelling.

0.25

Epiplexity Guided Data Selection and Generation for Out-of-Distribution Generalization

cs.LG cs.CL ·2026-08-12 ·Ellen Su, Andres Potapczynski, Shikai Qiu, Edward Hughes, Andrew Gordon Wilson
code/data similarity 0.44
Abstract · Code available for EpiSelect (https://github.com/eysu35/EpiSelect) and EpiGen (https://github.com/eysu35/EpiGen)

Modern systems are increasingly expected to transfer across tasks not specified during training. What data facilitates generalization to unanticipated settings? One hypothesis is that data with more structural information could contain shared circuits and subprograms that may be recycled in a wider array of downstream settings. Epiplexity, a recently proposed measure of the structural information a compute-bounded learner can extract from data, offers a mechanism to reason about this relationship. As existing estimators for epiplexity only operate on data already seen during training, we show how to operationalize epiplexity as an online training signal by greedily maximizing its stepwise gain in two settings. For data selection, we fit scaling laws to the training loss curves of natural data domains to predict the expected epiplexity gain as a function of training tokens, and use this signal to adaptively determine the sampling weights over domains during training. For synthetic data generation, where no domain structure exists, we reward a generator with the change in epiplexity of a learner's synthetic training data and use REINFORCE policy gradients to guide the generator toward an epiplexity-maximizing distribution. In both cases, we find higher epiplexity predicts improved downstream performance on zero-shot and fine-tuning tasks, supporting the hypothesis that data rich in structural information yield representations that better transfer across domains.

0.24

Modeling Deletion Requests in Machine Unlearning

cs.LG cs.CRcs.CY ·2026-10-03 ·Christian Cianfarani, Aloni Cohen
similarity 0.44
Abstract · 17 pages, 7 figures

Machine unlearning is seen as a promising approach to enable users to exercise the "right to erasure" in the context of AI models. We ask how users might influence the behavior of models when exercising this right. We define two types of behaviors that users might adopt when requesting the deletion of their data: adaptivity and collectivity. Drawing connections between the goals of users in this context and results in stochastic optimization, we demonstrate theoretical gaps between the potential effects of groups of users who do and do not display these behaviors. We then show how techniques from data valuation might be used to design deletion requesters that can significantly alter model behavior in realistic settings. In experiments on computer vision tasks, we demonstrate the differential effects of different models of user behavior and attempt to isolate the impacts of adaptivity and collectivity.

0.29

Harnessing Vision-Language Models for Federated Unsupervised Visual Recognition

cs.LG cs.CV ·2026-11-09 ·Jian Liang, Kuangpu Guo, Ran He, Tieniu Tan
code/data similarity 0.43
Abstract

Federated learning (FL) is a promising research paradigm that enables collaborative model training across distributed clients without data sharing. To handle unlabeled data in local clients, existing unsupervised FL research has primarily focused on tasks like clustering and self-supervised representation learning, leaving the critical problem of unsupervised visual recognition largely unexplored. To bridge this gap, we introduce FedUR, the first framework designed for unsupervised recognition by leveraging the zero-shot capabilities of pre-trained vision-language models like CLIP. Specifically, FedUR addresses data heterogeneity through a pseudo-labeling mechanism with a class-aware data selection strategy. This strategy first estimates a global pseudo-label distribution, then allocates pseudo-labeled data to clients accordingly. To handle the uncertainty of conventional positive pseudo-labels, FedUR further incorporates class weights and a negative pseudo-labeling strategy. Furthermore, FedUR proposes a personalized prompt aggregation protocol where general visual prompts are aggregated on the server, and personalized text prompts encoding local knowledge are retained on clients. Extensive experiments demonstrate that FedUR consistently outperforms baseline methods under both label-level and feature-level distribution shifts. We believe this work paves the way for further advances in federated unsupervised recognition. Code will be available at {github.com/tim-learn/FedUR/}}.

0.45