CS Paper Radar

Predicted Class Correction: Conformal Thresholds for Classes with No Calibration Data

One Octadion, Novanto Yudistira, Yuta Nakashima

cs.LG cs.CV ·submitted 2026-10-18 15:45 UTC · v1

Abstract

Class-conditional conformal prediction calibrates a separate score threshold for every class and therefore requires labelled calibration examples of every class. In long-tailed deployments these are exactly what is missing: rare classes are left with few or no held-out examples, and existing methods are either undefined for them or fall back to the shared marginal threshold. We ask whether the per-class correction can instead be predicted from descriptors of the class that require no labels at all. We propose Predicted Class Correction (PCC), which regresses the offset between the per-class and the marginal conformal quantile on geometric descriptors of the classifier head, shrinks the prediction by a scalar selected on the labelled classes, and refits a single marginal offset. The descriptors depend only on the model's weights, so a class with zero calibration examples still receives its own threshold, and since only one scalar is calibrated, split conformal's marginal guarantee is retained. On ImageNet, PCC improves worst-class coverage on such classes by 0.058 [0.006, 0.110] at matched average set size, 42% of an oracle that sees the test labels. Of eight baselines on the same splits, six change the statistic by exactly zero and the best reaches 0.016 with an interval containing zero, so PCC is the only one distinguishable from no improvement; the gain persists with only 100 labelled classes out of 1000. A second finding concerns evaluation: worst-class coverage becomes unmeasurable before it becomes unachievable. Below roughly 35 test examples per class no improvement can be shown even for an oracle, so a null result there reflects the evaluation set, not the method. Restricting evaluation above this depth recovers a significant improvement on Pl@ntNet-300K, where the effect had appeared absent.

Comment
23 pages including supplementary material, 4 figures

Closest papers

Nearest neighbours by meaning across the whole index.

Does Marginal Coverage Guarantee Class-Conditional Safety for Zero-Shot VLMs Under Shift?

cs.CV cs.AI ·2026-08-19 ·Jai Kumar Sharma, Amartya Dutta
published similarity 0.67
Abstract · Accepted at the ECCV 2026 Workshop on Uncertainty Quantification for Computer Vision (UNCV). 34 pages (16 main + 18 supplementary), 10 figur…

Split-conformal prediction provides marginal coverage under exchangeability and is increasingly used as an abstention layer for zero-shot vision-language models (VLMs). We audit this practice under deployment shift for CLIP, OpenCLIP, and SigLIP across ImageNet and non-ImageNet settings. Marginal coverage can remain relatively high while class-conditional tail coverage collapses: on ImageNet-Sketch, worst-class coverage falls to $\approx 0$ and 10-12% of classes lie below a finite-sample null floor, despite marginal coverage of about 0.86. The failure is aligned with target-domain class accuracy but is not predicted by the source-domain diagnostics we test. Source-side Mondrian calibration improves the in-distribution tail but does not transfer, while clustered conformal and Conf-OT improve marginal or average metrics without recovering the worst-class tail. Target-side class calibration substantially lifts the tail, but requires labels for every class and remains set-size-intensive. We further identify a 2-3$\times$ cross-family efficiency gap and show that native SigLIP sigmoid scores remove APS's probability-mass interpretation. The findings persist across the tested model scale, pretraining corpus, prompt, miscoverage level $α$, and shifted non-ImageNet settings. Marginal conformal coverage should therefore be treated as an average reliability statistic, not as a safety guarantee for the class tail.

0.62

How Synthetic Labels Improve Conformal Prediction: A Perspective on Conditional Coverage

cs.LG stat.ME ·2026-09-27 ·Qianyi Chen, Bo Li
similarity 0.65
Abstract

Conformal prediction provides distribution-free finite-sample marginal coverage, but post-hoc calibration data may be too scarce to learn how uncertainty varies across inputs. Meanwhile, abundant covariates can often be labeled cheaply by domain models or general-purpose language models. We study whether these synthetic labels can improve conditional coverage when only a small trusted sample is available. Building on score-quantile regression, we introduce prediction-powered quantile learning: a synthetic-labeled pool estimates pinball risk, paired trusted and synthetic outcomes correct its bias, and an independent trusted split performs final conformalization. Profiling pinball risk over scalar corrections reveals that population conditional-coverage error is its functional gradient; the corresponding Hessian removes global shifts and weights remaining shape error by boundary density. Composing this geometry with prediction-powered learning yields a three-resource expansion and a benefit--cost rule for synthetic power. Across eight regression benchmarks, synthetic-powered quantile learning substantially improves downstream conditional coverage while preserving marginal validity and producing more compact prediction sets. A human-rating study finds similar gains from external LLM labels and exposes a quality--quantity--cost tradeoff.

0.20

Sharp training-conditional coverage for conformal prediction under covariate shift

cs.LG stat.MLmath.STstat.ME ·2026-09-27 ·Mehrdad Pournaderi
theory similarity 0.65
Abstract

Weighted split conformal prediction reweights calibration scores by the likelihood ratio between the test and training covariate distributions and guarantees marginal coverage under covariate shift. We study its coverage conditional on the calibration data. An elementary argument, based on a single concentration inequality at a fixed population quantile, gives explicit training-conditional bounds without unspecified constants, and shows that the relevant scale is not the supremum of the likelihood ratio but a variance proxy built from the chi-squared divergence of the shift and from the average of the ratio over the part of the test population, of probability equal to the miscoverage level, where it is largest. A two-point lower bound shows that the root-m rate and the chi-squared contribution are intrinsic to the shift. Run at an explicitly inflated level, the weighted quantile becomes a deterministic PAC prediction set. We compare it with randomized rejection sampling and with importance-weighted learn-then-test and, through a certified choice of a clipping level for the likelihood ratio, map the regime in which each gives the narrower valid set. The analysis extends to estimated likelihood ratios and to tail functionals estimated from an unlabeled source sample, which yields a fully finite-sample certificate.

0.45

Scaling Full Conformal Image Classifiers

cs.LG cs.CV ·2026-09-29 ·Julio Silva-Rodríguez, Ender Konukoglu
publishedcode/data similarity 0.65
Abstract · NeurIPS 2026. Code: https://github.com/jusiro/T-FCP

Conformal prediction provides set-valued predictions with distribution-free coverage guarantees, making it attractive for high-stakes image classification. However, split conformal prediction is data-inefficient, while full conformal prediction (FCP), despite its stronger statistical efficiency, is computationally prohibitive at scale because it requires candidate-specific model refits at test time. We address this limitation by leveraging zero-shot vision-language models (VLMs) to guide scalable FCP in large label spaces. We introduce Targeted Full Conformal Prediction (T-FCP), which uses a lightweight inductive conformal predictor to prune unlikely labels and applies FCP only to the remaining candidates, reducing computation while retaining the formal guarantee of the combined conformal procedure. We further propose Stabilized Online LDA (SO-LDA), an efficient VLM adaptation solver based on rank-one inverse-covariance updates. Across multiple benchmarks, including ImageNet, T-FCP enables practical full-conformal image classification with modest test-time overhead, yielding efficient prediction sets and more stable empirical coverage than split conformal alternatives.

0.39

Backward Conformal Prediction under Covariate Shift

cs.LG stat.ML ·2026-10-12 ·Amin Meghrazi, Yuanyuan Li
benchmarktheory similarity 0.63
Abstract · 26 pages, 2 figures, 19 tables

Backward conformal prediction (BCP) fixes the size of a prediction set and estimates its miscoverage, but assumes calibration and test data are exchangeable. We extend BCP to covariate shift through importance weighting, preserving its guarantees when the density ratio is known. We show that BCP's tight miscoverage report depends only on the relative weight assigned to missed calibration examples, explaining why a better domain classifier can sometimes produce a worse estimate. Motivated by the fact that BCP depends only on nonconformity scores, we propose estimating density ratios on sorted score vectors rather than high-dimensional inputs. We prove that this approach recovers the same report asymptotically under an interpretable condition and characterize the resulting bias otherwise. Experiments with simulated and benchmark distribution shifts show that the proposed method improves BCP's miscoverage estimates, while score-based weighting consistently yields more accurate estimates than embedding-based weighting.

0.24

Candidate-Label Saliency for Efficient Conformal Pruning of Text Classifiers

cs.LG cs.CL ·2026-08-24 ·Ibne Farabi Shihab, Joyanta Jyoti Mondal, Adria Binte Habib
hot direction similarity 0.63
Abstract · 32 pages, 4 figures

Recalibration can preserve conformal coverage after pruning while enlarging prediction sets and leaving more alternatives for downstream decisions. Directly searching for deletions that keep sets small is costly. We study whether a small candidate-gradient budget can approximate this search at lower cost. Candidate-Saliency Pruning (CSP) estimates the squared first-order change in two plausible labels' scores when each parameter is deleted. CSP averages these changes, combines the resulting importance with a base pruner, and selects the mixing weight under validation accuracy and coverage requirements. Independent split-conformal calibration supplies marginal coverage under exchangeability. We evaluate CSP on three classification tasks, two base pruners, and three sparsity levels against a squared true-label control that separates candidate choice from gradient squaring, under fixed recovery updates, equal total time, and restricted access to the training corpus. Before recovery, CSP reduces mean set size by 13.2\% to 19.2\% against the reference pruner and by 7.7\% to 13.2\% against the control in all six encoder cells at 50\% sparsity. The advantage over the control shrinks with recovery to practical equivalence in all six cells at 1,000 updates, and at equal total time the cheaper reference pruner is practically equivalent in 18 of 18 settings. When recovery is limited to the retained pruning pool, CSP keeps a meaningful advantage in all six cells. Its value therefore depends on the time and data available for recovery.

0.39

Adaptive Conformal Redistribution for Inter-class Transitional Uncertainty in Medical Image Classification

cs.CV cs.AIcs.LGstat.CO+1 ·2026-09-10 ·Saibal Ghosh, Samarup Bhattacharya, Sanjoy Kumar Saha, Umapada Pal, Tapabrata Chakraborti
benchmarkcode/data similarity 0.63
Abstract

Medical image classification is frequently complicated by transitional categories whose feature distributions overlap those of adjacent classes, producing ambiguous decision boundaries. Conformal prediction returns uncertainty-aware prediction sets, but these are not directly actionable in clinical screening, where a single decision is required. This work proposes adaptive conformal redistribution (AdaConRed), a label-free post-conformal decision rule that converts ambiguous prediction sets into refined class assignments. A five-stage pipeline is developed. Vision-language generative augmentation addresses minority-class scarcity; a frozen DermFoundation encoder provides embeddings; a lightweight multi-layer perceptron performs classification; an entropy-modulated, margin-aware nonconformity score constructs adaptive prediction sets; samples predicted as transitional with multi-label sets are reassigned to the most probable alternative class within the set, using only model outputs at inference. Evaluation uses the OSCC oral lesion and ISIC skin lesion benchmarks at a miscoverage level of 0.2. On the 3-class OSCC benchmark, overall accuracy improves from 73.54% to 77.38%, with oral cancer accuracy rising from 64.29% to 82.14% and benign accuracy from 56.57% to 70.20%. Reassignment of transitional samples reduces OPMD accuracy from 84.78% to 80.16%, consistent with the asymmetric cost of missed malignancy. On ISIC, overall accuracy improves from 85.83% to 87.19%, melanoma accuracy rising from 66.04% to 68.34%. AdaConRed outperforms LAC, APS and RAPS under an identical backbone and redistribution rule. Conformal prediction can be extended beyond uncertainty quantification toward actionable decision support where transitional disease categories are present, with gains concentrated in the clinically critical malignant categories. Code repository: https://github.com/saibal436ghosh/AdaConRed.

0.38

ReCIRC: Rectified Conformal Risk Control

cs.LG stat.ML ·2026-09-29 ·Bruno Marcondes e Resende, Helton Graziadei, Thiago Rodrigo Ramos, Rafael Izbicki
similarity 0.63
Abstract · 69 pages, 11 figures

Many applications of black-box predictive models require controlling task-relevant error rates, such as missed lesion pixels in segmentation or missed labels in multilabel classification. Conformal risk control (CRC; Angelopoulos et al., arXiv:2208.02814) gives distribution-free guarantees for such losses, but it calibrates a single threshold shared by all inputs. Because conditional risk varies with the input, this marginal guarantee often overprotects easy cases and underprotects hard ones. We propose ReCIRC (Rectified Conformal Risk Control), which inverts each input's estimated local risk curve to reparameterize the calibrated threshold as a risk budget $a$ representing a common target conditional risk, and then applies CRC unchanged to the resulting family. ReCIRC retains CRC's finite-sample marginal guarantee regardless of the accuracy of the estimated curves, while accurate curves yield approximate conditional risk control and, under additional conditions, asymptotically exact conditional risk control; they also support a risk-calibration diagnostic. Across three synthetic and five real-data settings spanning segmentation, multilabel and multiclass classification, and regression, ReCIRC attained the lowest average worst-group risk and mean positive group excess in every setting, while maintaining marginal risk close to the target, whereas changes in prediction size were application-dependent.

0.54