CS Paper Radar

Certify, Report, or Refuse: Harm-Constrained Conformal Prediction for Clinical Actions

Zhiyuan Li, Lequan Yu

cs.LG cs.AI ·submitted 2026-11-09 16:30 UTC · v1

Abstract

Routing a melanoma case to routine follow-up is a serious action error that diagnostic coverage alone does not rule out. We present Harm-Constrained Risk-Optimal Conformal Prediction (HC-ROCP), a training-free framework that uses frozen classifier probabilities and adds a harm penalty when choosing prediction sets and actions. We test the resulting policies on disjoint calibration splits with multiplicity adjustment. HC-ROCP certifies supported claims, reports descriptive fallback policies, and refuses unsupported subgroup claims. We evaluate these outputs retrospectively on COVID-19 radiographs, ISIC 2019 dermoscopy, and EyePACS-to-APTOS retinopathy transfer. Across ten calibration seeds, HC-ROCP reduces COVID-19 severe-action errors from the exact risk-optimal baseline's 4.68\% to 0.54\%. At a 5\% budget, global claims are certified for COVID-19 and ISIC, but the rare squamous-cell-carcinoma claim is refused. At 1\%, all settings receive descriptive reports. For each budget, we also report the minimum severe-case count needed to pass the global harm gate with zero errors, providing a starting point for further data collection.

Comment
Findings of the 6th Machine Learning for Health Symposium (ML4H 2026)

Closest papers

Nearest neighbours by meaning across the whole index.

RadFusion: Towards Threshold-Controllable Radiology Report Generation

cs.AI cs.CLcs.CV ·2026-08-11 ·Ying Jin, Noel C. F. Codella, John Corring, Mu Wei, Dinei Florencio +1 more
similarity 0.65
Abstract

Automated radiology report generation is advancing rapidly in response to the shortage of radiologists, yet unlike a perception model, existing generation models offer no control over the sensitivity-specificity trade-off of their diagnostic content. Such control is essential because clinical scenarios diverge: emergency triage prioritizes sensitivity to reduce missed findings, whereas confirmatory interpretation emphasizes specificity to limit unnecessary interventions. A single fixed report can neither adapt to these scenarios nor support the ROC-based validation widely expected for regulatory clearance. We introduce RadFusion, a framework that equips report generation with threshold controllability. Our method fuses a multi-label classifier, which provides per-disease confidence scores, with a VQA-based report generator, which describes medical findings in detail; an LLM then rewrites the report so that its stated diagnoses follow the classifier's decisions at the selected threshold while staying grounded in the generator's descriptions. On MIMIC-CXR, the performance of RadFusion conforms to the classifier's ROC curve: sweeping the threshold and mapping the reports back to class labels reproduces the classifier's validated ROC performance. This conformance makes generated reports quantitatively evaluable through ROC analysis, strengthening the case for regulatory clearance, and enables operating-point selection that matches report behavior to clinical context. Moreover, combining the two model types improves diagnostic accuracy over uncontrolled generation: sensitivity increases by 6.9% at matched specificity, and specificity by 20.7% at matched sensitivity. These results show that RadFusion makes report generation clinically adaptable, quantitatively verifiable, and diagnostically more reliable.

0.04

Beyond Accuracy: Benchmarking Uncertainty Quantification in Retinal Disease Screening

eess.IV ·2026-11-17 ·Mahvash Jebeli, Mohammad Eslami, Nazlee Zebardast, Lucia Sobrin, Tobias Elze
benchmarkdataset similarity 0.65
Abstract

Reliable uncertainty quantification (UQ) is essential for evaluating retinal AI systems beyond classification accuracy, yet it remains unclear whether existing UQ strategies generalize across diseases, model architectures, and reliability criteria. We introduce a large-scale, multi-disease benchmark comprising over $242,000$ fundus photographs from $32$ datasets covering diabetic retinopathy, age-related macular degeneration, and glaucoma. We evaluate six representative UQ strategies across three Vision Transformer configurations, assessing discrimination, probabilistic calibration, selective prediction, and conformal prediction-set coverage. Our findings reveal no universally superior model or UQ strategy: methods that improve reliability in one disease or architecture may underperform in another, while improvements in discrimination, calibration, and selective prediction do not necessarily align. Notably, computationally intensive methods do not consistently outperform simple confidence scores, and domain-specific foundation models and calibration-oriented loss functions do not guarantee greater reliability. In contrast, conformal prediction provides a complementary framework for achieving near-nominal marginal prediction-set coverage across diverse settings, subject to its statistical assumptions. These findings challenge universal UQ recommendations and highlight the need for disease-specific, model-specific, and multidimensional reliability evaluation before clinical deployment. We release standardized dataset splits, trained checkpoints, evaluation code, and an interactive research demonstration to support reproducible benchmarking.

0.06

TRIAGE: Risk-Controlled Pseudo-Label Admission for Annotation-Efficient Semi-Supervised Retinal OCT Classification

cs.CV ·2026-08-14 ·Md Ashraful Hossen Akash, Shyla Afroge, Abdullah Al Mamun, Md. Kishor Morol, Tze Hui Liew
code/data similarity 0.63
Abstract · 38 pages, 12 figures, 13 tables. Code available at https://github.com/mdahakash/TRIAGE/

The advanced retinal disease diagnosing imaging modality, optical coherence tomography (OCT), encounters a lack of automation because of the high expenses for annotations performed by specialists. The use of SSL solves the problem of insufficient annotations using unlabeled B-scans; however, most of the current techniques for generating pseudo-labels are based on prediction confidence without considering the asymmetry between different types of errors. This paper proposes TRIAGE, a risk-controlled semi-supervised framework for OCT scans classification, which uses the concept of a patient-level conformal risk controller with an asymmetric cost matrix. TRIAGE unites three crucial modules: a hierarchical classifier that is capable of working with partially abnormal supervision of the disease subtypes, a patient-grouped conformal risk controller with primal-dual coverage control, and a context-aware Transformer teacher for cross-slice verification. On the dataset from Noor Eye Hospital (16,822 B-scans, 161 patients, and 554 volumes) with a test set of unseen patients, TRIAGE demonstrates 89.66% scan-level accuracy, 0.8805 macro-F1, 0.9641 macro-AUC, and an 8.34% under-grading rate when using only 20% of the labeled data. With only 5% of the labeled data, TRIAGE keeps 76.88% accuracy and a 0.1656 under-grading rate. Compared with the other six state-of-the-art semi-supervised methods, TRIAGE significantly outperforms them with ablation study demonstrating the contribution of each module in the overall framework performance (by 42.7% in terms of under-grading rate comparing to fixed threshold methods). TRIAGE demonstrates 98.00% accuracy for 3-class classification with 1% labeled data and 95.94% accuracy for 8-class classification with 10% labeled data on the OCT-C8 dataset.

0.30

ReCIRC: Rectified Conformal Risk Control

cs.LG stat.ML ·2026-09-29 ·Bruno Marcondes e Resende, Helton Graziadei, Thiago Rodrigo Ramos, Rafael Izbicki
similarity 0.62
Abstract · 69 pages, 11 figures

Many applications of black-box predictive models require controlling task-relevant error rates, such as missed lesion pixels in segmentation or missed labels in multilabel classification. Conformal risk control (CRC; Angelopoulos et al., arXiv:2208.02814) gives distribution-free guarantees for such losses, but it calibrates a single threshold shared by all inputs. Because conditional risk varies with the input, this marginal guarantee often overprotects easy cases and underprotects hard ones. We propose ReCIRC (Rectified Conformal Risk Control), which inverts each input's estimated local risk curve to reparameterize the calibrated threshold as a risk budget $a$ representing a common target conditional risk, and then applies CRC unchanged to the resulting family. ReCIRC retains CRC's finite-sample marginal guarantee regardless of the accuracy of the estimated curves, while accurate curves yield approximate conditional risk control and, under additional conditions, asymptotically exact conditional risk control; they also support a risk-calibration diagnostic. Across three synthetic and five real-data settings spanning segmentation, multilabel and multiclass classification, and regression, ReCIRC attained the lowest average worst-group risk and mean positive group excess in every setting, while maintaining marginal risk close to the target, whereas changes in prediction size were application-dependent.

0.54

Region-aware error triage for medical image segmentation with groupwise conformal risk control

cs.CV ·2026-11-06 ·Nishi Kanta Paul, Md Shihabul Islam Shovo, Hoileong Lee
similarity 0.61
Abstract · 22 pages, 7 figures, 12 tables. Submitted to Biomedical Signal Processing and Control (BSPC)

Segmentation masks are increasingly reviewed rather than redrawn, which requires knowing where a mask is likely wrong. Learned error maps rank pixels by risk, but the number of pixels to warn is chosen ad hoc, and calibrating one warning threshold over all error pixels may leave errors far from the predicted contour under-flagged. We propose RACE-Triage, a post-hoc error triage framework that leaves the segmentation mask unchanged. A risk network is trained on out-of-fold predictions from backbone-independent inputs. The predicted contour partitions each image into a boundary band and a non-boundary region, and conformal risk control is applied separately in each region to bound the expected fraction of missed errors among cases with errors there. On ISIC 2018 dermoscopy and BUSI breast ultrasound (where patient identifiers are unavailable, so independence across images is assumed), pooled control met its overall target but missed on average 31.8% and 36.4% of non-boundary errors. Regional control reduced these fractions to 19.3% and 20.9% and raised the recall of size-filtered false-negative components from 0.722 to 0.818 and from 0.611 to 0.763. The mean warned area grew by 1.6 and 4.9 percentage points, and its 95th percentile reached 23.5% and 38.5%. The effect held across three seeds and a second backbone, and area-matched heuristics did not reproduce it. On the external PH$^2$ set, frozen thresholds exceeded both targets, whereas recalibration with 100 local images brought the mean over repeated splits below them. Regional calibration improves non-boundary error coverage at a measured increase in warned area.

0.30

A Decision-Aware Evaluation Protocol for Uncertainty in Medical Image Segmentation

cs.CV ·2026-10-15 ·Saket Maganti, Shreyank N Gowda
similarity 0.61
Abstract

Medical segmentation uncertainty is often evaluated with calibration error or pixel-level error-detection scores, but these metrics do not show whether uncertainty improves downstream review decisions. We propose a decision-aware evaluation protocol that separates uncertainty estimation from deferral policy and evaluates their combination under fixed review budgets: which pixels, regions, or images should be accepted, and which should be reviewed. The protocol reports fixed-budget error reduction, risk-coverage behaviour, region-level review proxies, structural failure modes, probability-derived baselines, and sensitivity to review assumptions. We instantiate it across retinal vessel, skin lesion, and polyp segmentation with multiple backbones and uncertainty estimators. Under matched inference budgets and canonical multi-layer MC Dropout, test-time augmentation (TTA) gives a stronger error-detection signal across matched MC--TTA comparisons, including architecture and dropout-placement checks. However, estimator rankings are not universal. For example, probability-derived scores remain competitive, deep ensembles can outperform TTA, learned deferral helps mainly when the base estimator is weak, and rankings can change with review unit, uncertainty target, correction assumptions, and cost model. Structural and second-observer analyses further show that boundary and thin-structure failures are partly tied to annotation ambiguity. These findings support evaluating uncertainty as an estimator-policy-review configuration, not as a standalone score map. Code: https://anonymous.4open.science/r/Uncertainity-in-Medical-Image-Segmentation/README.md

0.34

Uncertainty of Vision Medical Foundation Models

cs.LG ·2026-08-31 ·Haoxu Huang, Narges Razavian
similarity 0.61
Abstract

Accurate uncertainty estimation is essential for machine learning systems de- ployed in high-stakes domains such as medicine. Traditional approaches primarily rely on probability outputs from trained models (point predictions), which provide no formal guarantees on prediction coverage and often require additional calibra- tion techniques to improve reliability. In contrast, conformal prediction (region prediction) offers a principled alternative by generating prediction sets with finite- sample validity guarantees, ensuring that the ground truth is contained within the set at a specified confidence level. In this study, we explore the impact of pre-training approach, dataset scale and domain on both point and region-level uncertainty quantification, by studying domain-specific vision medical foundation models vs. general domain vision foundation models. We conduct a comprehensive evaluation across foundation models trained on retinal, histopathological, and Chest X-Rays data, applying various calibration techniques. Our results demonstrate that (1) pre-training on higher-quality domain-specific datasets along with self-supervised learning leads to better-calibrated point predictions than general domain pre-training, (2) stan- dard re-calibration methods alone cannot fully mitigate uncertainty discrepancies across models trained on different data sources, (3) domain-specific foundation model can lead to more efficient conformal prediction. These findings highlight the importance of careful model selection and the inte- gration of both point and region prediction to enhance the reliability and trust- worthiness of medical AI systems. Our work underscores the need for a holistic approach to uncertainty quantification in recent development of medical vision foundation model, ensuring robust and interpretable AI-driven decision-making.

0.06

Multi-Channel Feature Fusion and Monte Carlo Dropout for Uncertainty-Aware Diabetic Retinopathy Grading

eess.IV cs.CVeess.SP ·2026-08-15 ·Saksham Kumar
similarity 0.60
Abstract

Automated five-stage diabetic retinopathy (DR) grading requires more than high accuracy alone. Medical-grade deployment calls for lesion-aware preprocessing, ordinal predictions, calibrated uncertainty, and explainability to support reliable diagnostic systems. We present a unified pipeline that addresses these requirements using a Ben-Graham-green-channel CLAHE feature representation, an EfficientNetV2-L ordinal regressor, and Monte Carlo dropout for uncertainty-driven referral. Grad-CAM provides visual explanations aligned with clinically relevant lesions. The proposed method achieves a QWK of 91.31% on the APTOS-2019 official test split, placing it within the near-perfect agreement band (>80%). At a 20% referral rate, 293 of 366 images are automatically graded with a QWK of 90.40%. More complex cases are referred for specialist assessment, demonstrating a practical trade-off among grading quality, automation, and patient safety in robust, reliable, and deployment-ready medical diagnostic systems.

0.04