CS Paper Radar

Finite-sample Calibration Guarantees for Binary Segmentation

Bruno Viti, Martin Holler

cs.CV cs.LG ·submitted 2026-11-06 07:58 UTC · v1
top venue

Abstract

Machine learning based segmentation models are frequently miscalibrated, producing for each pixel a label prediction whose associated confidence does not accurately reflect the true underlying probability. This limitation is particularly critical in high-stakes applications such as medical image segmentation. Although several calibration methods have been proposed, only a small subset of them provides theoretical guarantees on the calibration error, i.e., they do not typically ensure that, with high probability, the model exhibits a low calibration error. In this work, we develop a post-hoc calibration method for which formal calibration guarantees can be established. To achieve this, we derive from standard concentration inequalities an improved theoretical bound that explicitly accounts for the spatial dependence inherent in binary segmentation outputs. We empirically demonstrate that the proposed calibrator satisfies the derived guarantee and outperforms existing post-hoc calibration approaches.

Comment
Published in Transactions on Machine Learning Research 10/2026

Closest papers

Nearest neighbours by meaning across the whole index.

CalCErt: Bin-wise Certification of Confidence Calibration in Medical Image Classification

cs.CV ·2026-10-02 ·Leo Fillioux, Stergios Christodoulidis, Stergios Christodoulidis, Maria Vakalopoulou, Jose Dolz
top venuecode/data similarity 0.67
Abstract

Deep neural networks remain vulnerable to adversarial perturbations, which can distort not only predictions but also confidence scores, undermining uncertainty calibration. While existing certification methods focus on preserving the predicted category, providing guarantees on how calibration behaves under adversarial attacks remains overlooked. In this work, we introduce CalCErt, a simple and efficient post-hoc strategy that certifies bin-wise confidence calibration for any pretrained differentiable classifier. Our approach combines empirical calibration estimates, statistical concentration bounds, and local Lipschitz estimates of the confidence function to derive data-dependent upper bounds on worst-case miscalibration within an $ell_2$-ball of radius R. We evaluate CalCErt across 11 medical image classification tasks and multiple adversarial perturbations, demonstrating substantially higher certified coverage than baseline strategies while maintaining competitive tightness. Our code is available at https://github.com/leofillioux/calcert.

0.45

Segmentation Confidence for Arbitrary CNNs

cs.CV ·2026-10-27 ·Baris Oguz, Xing Yao, Claudia Tawil, Alison Pouch, Brett Byram +3 more
published similarity 0.66
Abstract · 5 pages, 6 figures, published in ISBI 2026

Convolutional neural networks (CNN) are widely used for medical image segmentation. However, they lack an inherent measure of confidence. Existing methods for estimating prediction uncertainty often require a custom network architecture and/or dedicated training setups and need to be trained from scratch. In addition, the deduction of a single confidence score per input from the uncertainty maps is not always straightforward. In this study, we explore voxel-based uncertainty estimation methods that can be added to existing segmentation pipelines without training from scratch. We also propose a novel method to estimate a single confidence score from the uncertainty maps. Our evaluations on different ROIs and modalities suggest that our confidence score can predict segmentation performance in unseen data.

0.25

Uncertainty of Vision Medical Foundation Models

cs.LG ·2026-08-31 ·Haoxu Huang, Narges Razavian
similarity 0.65
Abstract

Accurate uncertainty estimation is essential for machine learning systems de- ployed in high-stakes domains such as medicine. Traditional approaches primarily rely on probability outputs from trained models (point predictions), which provide no formal guarantees on prediction coverage and often require additional calibra- tion techniques to improve reliability. In contrast, conformal prediction (region prediction) offers a principled alternative by generating prediction sets with finite- sample validity guarantees, ensuring that the ground truth is contained within the set at a specified confidence level. In this study, we explore the impact of pre-training approach, dataset scale and domain on both point and region-level uncertainty quantification, by studying domain-specific vision medical foundation models vs. general domain vision foundation models. We conduct a comprehensive evaluation across foundation models trained on retinal, histopathological, and Chest X-Rays data, applying various calibration techniques. Our results demonstrate that (1) pre-training on higher-quality domain-specific datasets along with self-supervised learning leads to better-calibrated point predictions than general domain pre-training, (2) stan- dard re-calibration methods alone cannot fully mitigate uncertainty discrepancies across models trained on different data sources, (3) domain-specific foundation model can lead to more efficient conformal prediction. These findings highlight the importance of careful model selection and the inte- gration of both point and region prediction to enhance the reliability and trust- worthiness of medical AI systems. Our work underscores the need for a holistic approach to uncertainty quantification in recent development of medical vision foundation model, ensuring robust and interpretable AI-driven decision-making.

0.06

Noisy-Label Calibration with Multiple Unreliable Labels

cs.LG stat.ML ·2026-10-14 ·Keisuke Hanada, Futoshi Futami
similarity 0.63
Abstract · 43 pages, 2 figures

In medical annotation and crowdsourcing, multiple annotators assign labels to each example, but these labels may be incorrect. Treating such noisy labels as ground truth can bias calibration evaluation. To address this problem, we propose methods for evaluating and recalibrating a fixed binary predictor under the underlying clean-label distribution, using only noisy annotations. First, given annotator-specific noise probabilities, we propose Noise Corrected ECE (NC-ECE), a binned estimator of expected calibration error (ECE) under class-conditional noise. We derive non-asymptotic bounds on the estimator's bias. Then, we propose practical algorithms for estimating unknown noise probabilities under conditional independence and label-orientation assumptions. We also develop loss-based and histogram post-hoc recalibration with clean-risk and calibration guarantees, respectively. Experiments on semi-synthetic benchmarks show that the proposed ECE estimator closely tracks its clean-label oracle counterpart and that the recalibration methods substantially improve latent-label calibration, whereas uncorrected recalibration against noisy annotations can degrade it.

0.28

Missing Modality-Aware Calibration for Trustworthy Brain Tumor Segmentation

cs.CV cs.AI ·2026-10-08 ·Sol Lee, Hyunji Kim, Sungrae Hong, Donghee Han, Mun Yi
top venue similarity 0.62
Abstract · MICCAI2026 poster

Multimodal brain tumor segmentation typically leverages multiple MRI modalities, yet incomplete modality acquisition is common in clinical practice due to protocol heterogeneity and scan failures. Although recent methods maintain segmentation accuracy under missing modality conditions, they frequently overlook prediction reliability, leading to miscalibrated confidence estimates that hinder clinical adoption. Existing calibration techniques are largely modality-agnostic or assume that prediction difficulty decreases monotonically as additional modalities become available. However, in brain tumor segmentation, prediction difficulty depends primarily on which modalities are absent rather than how many, leading to combination-specific and spatially heterogeneous calibration errors. To address this, we propose Missing Modality-Aware Local Temperature Scaling (MMA-LTS), a post-hoc voxel-wise confidence calibration method. It estimates a spatially adaptive temperature field conditioned on a modality-availability learnable token and a voxel-wise difficulty score. Experiments on BraTS 2020 and FeTS 2024 show that MMA-LTS improves calibration while preserving the segmentation accuracy of state-of-the-art models across diverse missing-modality scenarios, thereby enhancing trustworthiness toward clinical deployment.

0.27

A Decision-Aware Evaluation Protocol for Uncertainty in Medical Image Segmentation

cs.CV ·2026-10-15 ·Saket Maganti, Shreyank N Gowda
similarity 0.60
Abstract

Medical segmentation uncertainty is often evaluated with calibration error or pixel-level error-detection scores, but these metrics do not show whether uncertainty improves downstream review decisions. We propose a decision-aware evaluation protocol that separates uncertainty estimation from deferral policy and evaluates their combination under fixed review budgets: which pixels, regions, or images should be accepted, and which should be reviewed. The protocol reports fixed-budget error reduction, risk-coverage behaviour, region-level review proxies, structural failure modes, probability-derived baselines, and sensitivity to review assumptions. We instantiate it across retinal vessel, skin lesion, and polyp segmentation with multiple backbones and uncertainty estimators. Under matched inference budgets and canonical multi-layer MC Dropout, test-time augmentation (TTA) gives a stronger error-detection signal across matched MC--TTA comparisons, including architecture and dropout-placement checks. However, estimator rankings are not universal. For example, probability-derived scores remain competitive, deep ensembles can outperform TTA, learned deferral helps mainly when the base estimator is weak, and rankings can change with review unit, uncertainty target, correction assumptions, and cost model. Structural and second-observer analyses further show that boundary and thin-structure failures are partly tied to annotation ambiguity. These findings support evaluating uncertainty as an estimator-policy-review configuration, not as a standalone score map. Code: https://anonymous.4open.science/r/Uncertainity-in-Medical-Image-Segmentation/README.md

0.34

ProGeo-RC: Prototype-guided Semantic Reliability and Geometric Error Calibration for Semi-supervised Cardiac Image Segmentation

cs.CV ·2026-11-03 ·Yunfei Zhu, Wenfeng Zhang, Pan Zeng, Xin Huang, Tao Jia +3 more
published similarity 0.59
Abstract · Accepted at IEEE BIBM 2026. 7 pages, 3 figures

Semi-supervised medical image segmentation aims to train reliable segmentation models using limited labeled data and abundant unlabeled data. However, existing methods typically rely on prediction confidence or model consistency to estimate pseudo-label reliability, but these criteria are insufficient to characterize both semantic reliability and geometric plausibility. To address this issue, we propose ProGeo-RC, a prototype-guided semantic reliability and geometric error calibration framework for semi-supervised medical image segmentation. ProGeo-RC consists of two key modules: Prototype-guided Semantic Reliability Estimation (PSRE) and Structured Semantic-Geometric Error Calibration (S-SGEC). PSRE uses probabilistic class prototypes to estimate the semantic reliability of unlabeled predictions and calibrate pixel-wise teacher pseudo labels, while S-SGEC introduces signed-distance-field-based geometric constraints to refine ambiguous boundaries and suppress spurious foreground responses outside target structures. Experiments on the ACDC and MVAA-CT datasets show that ProGeo-RC achieves better performance compared with representative semi-supervised segmentation methods. Under the 10\% labeled setting on ACDC, ProGeo-RC improves DSC and Jaccard by 0.37\% and 0.49\% over the corresponding second-best results, while reducing 95HD and ASD by 0.61 and 0.21, respectively. These results indicate that joint semantic reliability estimation and geometric error calibration can provide more reliable pseudo supervision for semi-supervised medical image segmentation.

0.32

CARD: Calibration via Agreement in Reverse Diffusion for Out-of-Domain MRI Segmentation

cs.CV ·2026-08-26 ·Jiaheng Dai, Weidong Guo, Qingbiao Li, Jie Xu, Yi Guo +2 more
similarity 0.59
Abstract · 10 pages, 5 figures

Probability calibration aligns model confidence with predictive accuracy, enabling clinicians to identify unreliable segmentation regions. This alignment breaks down under domain shift, where artifacts and unseen protocols produce confident errors. Existing post-hoc methods adapt the correction at test time, conditioning on predictive entropy, the logit pattern, or augmentation response, but each proxy is read from the terminal prediction, the very quantity that shift corrupts. This motivates reliability evidence beyond the terminal prediction, which categorical diffusion provides in two ways. First, a generative shape prior keeps a capacity-limited reference intact when appearance is corrupted, so its disagreement with the primary segmentor highlights primary-model errors. Second, every reverse step yields a class distribution, separating persistent disagreement from transient discrepancy. Aggregated over the trajectory, this disagreement correlates with Dice at 0.788, against 0.521 for a matched discriminative control. We therefore propose CARD (Calibration via Agreement in Reverse Diffusion), which maps the temporal aggregate of this disagreement to a temperature field applied per pixel across all classes, so that confidence changes while the segmentation does not. Across cardiac, prostate and brain MRI shifts, CARD lowers calibration error in 45 of 49 comparisons against the strongest baseline in each setting.

0.22