CS Paper Radar

Integrated Imputation-Classification for Supervised Learning with Missing Data

Yue Liu, Ben Liang, Ali Tizghadam, Ilijc Albanese

cs.LG ·submitted 2026-10-03 03:59 UTC · v2
publishedtheory

Abstract

We study supervised classification problems with missing feature values. Existing approaches often decouple imputation from classification, producing imputations that may be plausible but uninformative for prediction. Instead, we propose the Integrated Imputation and Classification Network (IICN), which jointly trains an imputer and an ${(n{+}1)}$-class discriminator adversarially with a single class-supervised objective, where the discriminator learns to distinguish among the $n$ true classes and an additional ``imputed'' class. We prove that it admits a global optimum where the imputer and discriminator together implement marginalization over missing coordinates and yield a Bayes-optimal classifier. We evaluate IICN on FashionMNIST, CIFAR-10, and tabular datasets with naturally occurring missingness. IICN outperforms classical impute-then-classify pipelines and recent generative baselines, showing strong robustness and accuracy in challenging settings.

Comment
camera ready

Closest papers

Nearest neighbours by meaning across the whole index.

ProCTI: Prototype-Refined Global Conditioning for Diffusion-Based Time Series Imputation

cs.LG cs.AI ·2026-09-29 ·Fariza Rashid, Duc Van Le, Rahat Masood, Gustavo Batista, Aruna Seneviratne +1 more
benchmark similarity 0.66
Abstract

Time series imputation has progressed from statistical and deep learning approaches to diffusion-based models, which have shown strong recent performance. Existing diffusion-based methods typically condition the reverse process using local contextual information from the current or neighbouring windows. Meanwhile, global dataset-level structure often remains implicit, limiting performance when local observations are sparse, noisy, or unrepresentative. To address this issue, we propose ProCTI, a diffusion-imputation framework that augments local conditioning with retrieved global dataset-level priors through learned prototypes. A hybrid conditioning mechanism integrates this global context with local signals during reverse diffusion, enabling more accurate reconstruction under varying missingness scenarios. Experiments across multiple benchmark datasets show that ProCTI outperforms strong baselines overall under random missingness, while remaining competitive under attribute-wise missingness. Furthermore, we use a latent-regime data model to characterise the precise conditions under which prototype-derived global conditioning provably improves imputation. We support this with a general theoretical analysis of local-global conditioning.

0.18

Marformer: A Transformer for Predicting Missing Data Distributions

cs.LG stat.ME ·2026-10-08 ·Prabhav Singh, Xiheng Tom Wang, Haojun Shi, Jason Eisner
similarity 0.66
Abstract · 45 pages, 19 figures; presented in part in a COLM 2026 keynote

Real decisions are made under incomplete information. If we observe only some of the random variables we need, we can predict the others. The \textbf{conditional marginals} over the missing variables are the key ingredient for computing Bayes risk and Value of Information (VOI), the expected gain from acquiring one more observation before deciding. We present the Marformer, a Transformer trained to directly predict conditional marginals given any set of observed values. Like BERT, which is trained to predict missing words from context, the Marformer constructs a hidden-vector representation for each distribution $p(X_i)$ and iteratively refines it through attention to other distributions $p(X_j)$. Unlike generative approaches, the Marformer does not model the full joint distribution, requires no domain knowledge of the data-generating process, and makes all predictions in a single forward pass. We evaluate across three synthetic domains with missing data---Bayesian networks, discretized multivariate Gaussians, and structured annotation data. The Marformer can match or outperform classical missing-data methods, even when those methods are given the true model family and prior that generated the synthetic data. We also evaluate on a real annotation dataset, where the Marformer outperforms the evaluated baselines at the largest training size. In both cases, the Marformer is substantially faster than the evaluated generative baselines.

0.32

When Does Task-Aware Imputation Matter? An Analysis of Robustness Under Train-Test Mismatch

cs.LG ·2026-10-14 ·Maria V. Salvador, Ana F. Sequeira, Jaime S. Cardoso
similarity 0.65
Abstract

Missing data is a pervasive challenge in machine learning, often degrading predictive performance and complicating model deployment. Most existing approaches treat imputation as a preprocessing step independent of the downstream task, a separation that becomes problematic under train-test mismatches in missingness mechanisms or rates. We introduce a unified framework that treats missing entries as variables in a task-conditioned optimization problem. We instantiate this framework in two settings: an amortized approach for scenarios with missing data at both training and test time, and a latent optimization approach for settings where only training data is incomplete. The evaluation is conducted on 98 tabular datasets, and robustness under train-test mismatches is assessed on 36 datasets. Results reveal that reconstruction error constitutes a poor proxy for predictive utility (Spearman rho approximately 0.16 to 0.32), and that task-aware imputation becomes critical in scenarios where classical methods fail to recover complete-data performance, achieving win rates of up to 80% in challenging train-test mismatch scenarios.

0.17

Beyond Imputation: Mask-Adaptive Conformal Prediction via Tree Embeddings on General Missing Data Mechanisms

cs.LG stat.ML ·2026-11-01 ·Jiarong Fan, Juhyun Park, Thi Phuong Thuy Vo, Nicolas J-B. Brunel
similarity 0.64
Abstract

Conformal prediction (CP) provides a distribution-free framework for uncertainty quantification with finite-sample marginal coverage guarantees. Yet, missing values introduce distributional heterogeneity, where marginal coverage can hide systematic undercoverage for certain missingness patterns. At the same time, existing methods based on distances or smoothness require a continuous representation of partially observed covariates. Although imputation restores continuity, it obscures the missingness patterns and introduces task-dependent overhead. We develop a new CP framework that is \emph{adaptive to missingness patterns}, without imputing covariates. We rely on data augmentation by feature embedding that accounts for missingness. Our framework represents each input with missing values by a tree embedding together with its raw missingness mask, and then learns a regularized quantile score function. Varying the choices for the penalty function and the augmentation enables us to characterize three practically relevant validity levels: exact coordinate-wise validity, exact validity on observed masks, and approximate missingness-conditional guarantees for general mechanisms (MCAR, MAR, MNAR). Experiments on synthetic and real datasets support these guarantees and show that the proposed method reduces miscoverage risk while maintaining informative intervals, especially in non-MCAR settings. These results highlight the practical value of uncertainty quantification that adapts to missingness.

0.18

TWIN-MoE: Two-stage Intra-Expert Imputation Network for Incomplete Multimodal Learning

cs.LG cs.AI ·2026-10-15 ·Liang Nathan Zheng, Jiyuan Liu, Wenhao Liang, Wei Emma Zhang, Weitong Chen +1 more
similarity 0.63
Abstract

Recent Mixture-of-Experts (MoE) approaches address missing modalities either by routing each missingness pattern to a dedicated expert, which scales poorly as the number of modalities grows, or by imputing the missing modality from representations contributed by multiple experts, which requires a costly retrieval mechanism. We show that routing induces expert-conditioned representation neighborhoods and derive an imputation-error bound controlled by their local feature dispersion. When these expert-conditioned neighborhoods are more compact than the global representation space, intra-expert aggregation admits a tighter worst-case reconstruction bound than unconstrained global aggregation. Guided by this analysis, we propose TWIN, a two-stage intra-expert imputation framework. Pre-imputation generates a context-conditioned estimate in the encoder embedding space and radially soft-clips it using modality-specific statistics maintained in an observed FIFO queue, this ensures that the pre-imputed modality stays in empirically calibrated region of the target modality space. Post-imputation refines the missing representation after MoE by aggregating expert-conditioned observed representations within the same sample. Across four missing-modality benchmarks under natural, random, and asymmetric dropout protocols, TWIN achieves state-of-the-art performance while using substantially fewer parameters than competing MoE-based imputation methods.

0.09

Hybrid Methods for Robust Tabular Data Imputation

cs.LG ·2026-09-30 ·Jinwei Li, Michelle Bruch, Daniel Tenbrinck
benchmark similarity 0.61
Abstract · 35 pages, 8 figures

Missing data are a fundamental challenge in statistical analysis and machine learning, as the choice of imputation method substantially impacts downstream inference. In this work, we propose two hybrid imputation methods called NuclearForest and SoftForest, which combine nuclear-norm-based low-rank initialization using Singular Value Thresholding (SVT) and SoftImpute, respectively, with a non-iterative Random Forest refinement. For the SVT-based component, we further introduce an adaptive step-size rule, prove adaptive step-size bounds, and establish convergence for the corresponding zero-initialized iteration. The low-rank initialization provides a structured warm start that captures the global covariance patterns in the data, while the subsequent Random Forest step recovers residual nonlinear signals encoding local dependencies. We conduct an extensive benchmark on diverse datasets from different application domains, comparing the proposed methods with seven established imputation methods under the Missing Completely at Random (MCAR), Missing at Random (MAR), and Missing Not at Random (MNAR) mechanisms across varying missingness rates. Our results demonstrate that NuclearForest and SoftForest match or exceed the imputation fidelity of state-of-the-art iterative methods such as MissForest, while significantly reducing computational cost. In particular, they achieve speedups of approximately 5.81 times and 9.52 times over MissForest by replacing iterative cycles with a single refinement step. Our approach effectively exploits the low-rank structure of real-world tabular data and accommodates mixed-type variables, providing an efficient and robust solution for data imputation in bioinformatics, economics, and beyond.

0.29

ITU-DESC: a missingness-aware machine-learning dementia risk score in the UK Biobank cohort

cs.LG ·2026-10-19 ·Abdullah Kavaklı, Faik Boray Tek
similarity 0.61
Abstract

Background: Existing dementia risk scores rely on fixed, pre-selected features and linear models, or ignore bias from informative missingness in cohort data. Here we introduce the Istanbul Technical University Dementia Risk Score (ITU-DESC), an explainable machine-learning score for incident all-cause dementia that combines broad feature selection across roughly 8,000 candidate features with explicit control of missingness bias. Results: We studied approximately 500,000 UK Biobank participants with a median follow-up of 16.6 years. We withheld missingness patterns from the model using unconditional random hot-deck imputation. We stabilized feature selection by repeating BorutaSHAP-based feature selection with ten random-imputation seeds and selecting predictors chosen in at least eight of the ten seeds, which yielded 38 predictors. We trained a CatBoost gradient-boosted decision-tree model and evaluated it on a holdout test set of 50,041 participants. For all-cause incident dementia, ITU-DESC achieved an average precision of 0.138 (95% confidence interval 0.123-0.156) and an area under the receiver operating characteristic curve (ROC AUC) of 0.853 (0.844-0.863), significantly outperforming other risk scores, including the strongest comparator, the UK Biobank Dementia Risk Prediction model (0.118 and 0.841). We identified dementia risk predictors absent from established scores, including leukocyte telomere length and liver enzymes. Conclusions: ITU-DESC is the first dementia risk score to combine broad feature selection with explicit control of missingness bias.

0.23

A Deep Latent Variable Framework for Jointly Modeling Missingness, Measurement Error, and Heterogeneity

cs.LG stat.ML ·2026-08-30 ·Yasin Khadem Charvadeh, Grace Y. Yi, Mithat Gönen, Pouya Faroughi
similarity 0.60
Abstract · 24 pages, 4 figures

Missing data, measurement error, and population heterogeneity are pervasive challenges in analyzing data arising from modern observational studies and machine learning applications. Although these problems frequently coexist and interact, they are often treated separately in existing works. We propose a unified probabilistic framework that jointly addresses these issues utilizing deep latent variable representation. The proposed method integrates a novel hierarchical tree-routed variational autoencoder with pattern-aware latent representations and calibration-based denoising. The framework accommodates missing data mechanisms, including MCAR, MAR, and MNAR, while simultaneously learning subgroup-specific and globally shared latent structure. The introduced reconvergent routing mechanism enables selective parameters to be shared across related subpopulations, which offers flexibility as well as improved statistical efficiency. Simulation studies demonstrate substantial improvements over existing deep generative imputation approaches under complex heterogeneous missingness and measurement-error settings. The proposed framework provides a principled approach for learning from noisy and incomplete data in modern healthcare and other high-dimensional applications.

0.17