CS Paper Radar

Risk Control Certifies Rates, Not Magnitudes: An Audit of Selective Fetal Biometry

Zhengshuai Yang, Linna Jiang, Changyun Li, Guihu Zhao

cs.AI cs.CV ·submitted 2026-11-04 02:41 UTC · v1
theory

Abstract

Prenatal ultrasound biometry must return a head circumference (HC) and refuse when the image cannot support one. Selective prediction trades coverage against error, but its finite-sample guarantees bound the rate of gross errors, not their magnitude. We audit a tool-grounded biometry pipeline on two separately pre-registered four-way splits of the 999 labelled HC18 images, retraining the segmenter three times on each. Calibrating the contour-completeness threshold by exact Clopper-Pearson risk control returns a certificate indistinguishable from a hand-set floor, yet the pipeline still emits a 271.3 mm error: a small, geometrically complete ring fitted to an image whose true HC is 324.6 mm. The accepted candidate scores 0.94 completeness, so the score cannot see the failure, and one outlier among $\approx 150$ answers leaves the certified risk bound ($α= 0.10$) intact. A second, untouched split reproduces the mechanism on a different image (233.3 mm). Constraining the admissibility region---the anatomical range in which the measurand is defined---removes the family instead: raising the lower bound from 50 to 55 mm costs 0.4 coverage points and cuts the worst-case error on both splits to 11.4 and 11.1 mm, whereas suppressing it through the completeness threshold costs 19.0 and 12.1 points. At an anatomically defined 70 mm the frozen tests give 1.91 and 1.81 mm MAE.

Comment
10 pages, 2 figures, 3 tables

Closest papers

Nearest neighbours by meaning across the whole index.

Bound-Aware Per-Organ Recall Risk Control for Multi-Organ CT Segmentation under Clinical Domain Shift

cs.CV cs.AIcs.LG ·2026-08-18 ·Souraj Adhikary, Negar Chabi, Andre Mastmeyer
published similarity 0.64
Abstract · 12 pages, 4 figures, 2 tables

Distribution-free risk control adds organ-specific recall guarantees to frozen segmentation. We calibrate per-organ thresholds for an AMOS-trained nnU-Net, audit transfer to RAOS, and estimate local re-certification cost using case-level voxel false-negative rate (FNR). The AMOS control passes, but $7/12$ organs exceed $α{=}0.10$ after transfer; smaller calibration sets can mask exceedances with conservative or vacuous thresholds. Risk-Controlling Prediction Sets (RCPS) give high-probability control of population-mean risk, whereas Conformal Risk Control (CRC) gives weaker expectation control. Both require exchangeability; fixed and global thresholds give no per-organ guarantee. The Waudby--Smith--Ramdas (WSR) betting bound re-certifies six Tier-1 organs with 25 local cases, versus 30--40 for Hoeffding--Bentkus (HB). CRC needs 10--15 but has a heavier individual-case tail. No Tier-2 organ meets our illustrative precision criterion with 25 cases.

0.63

Region-aware error triage for medical image segmentation with groupwise conformal risk control

cs.CV ·2026-11-06 ·Nishi Kanta Paul, Md Shihabul Islam Shovo, Hoileong Lee
similarity 0.63
Abstract · 22 pages, 7 figures, 12 tables. Submitted to Biomedical Signal Processing and Control (BSPC)

Segmentation masks are increasingly reviewed rather than redrawn, which requires knowing where a mask is likely wrong. Learned error maps rank pixels by risk, but the number of pixels to warn is chosen ad hoc, and calibrating one warning threshold over all error pixels may leave errors far from the predicted contour under-flagged. We propose RACE-Triage, a post-hoc error triage framework that leaves the segmentation mask unchanged. A risk network is trained on out-of-fold predictions from backbone-independent inputs. The predicted contour partitions each image into a boundary band and a non-boundary region, and conformal risk control is applied separately in each region to bound the expected fraction of missed errors among cases with errors there. On ISIC 2018 dermoscopy and BUSI breast ultrasound (where patient identifiers are unavailable, so independence across images is assumed), pooled control met its overall target but missed on average 31.8% and 36.4% of non-boundary errors. Regional control reduced these fractions to 19.3% and 20.9% and raised the recall of size-filtered false-negative components from 0.722 to 0.818 and from 0.611 to 0.763. The mean warned area grew by 1.6 and 4.9 percentage points, and its 95th percentile reached 23.5% and 38.5%. The effect held across three seeds and a second backbone, and area-matched heuristics did not reproduce it. On the external PH$^2$ set, frozen thresholds exceeded both targets, whereas recalibration with 100 local images brought the mean over repeated splits below them. Regional calibration improves non-boundary error coverage at a measured increase in warned area.

0.30

Cross-Device Transfer of a Hospital-Trained Fetal Head Circumference Model to a Low-Cost Handheld Ultrasound Probe

eess.IV ·2026-10-28 ·Patrick Gao, Lara Hart, Ken Sullivan
benchmark similarity 0.63
Abstract · 24 pages, 5 figures, 4 tables

Automated fetal head circumference (HC) measurement reaches near-expert accuracy on the HC18 benchmark, but published models are trained and evaluated on hospital-grade equipment. We quantify what happens when such a model is carried to a low-cost handheld probe. A lightweight segmentation network (CSM, 0.13M parameters) was trained on HC18 across five independent runs and applied without modification to images from a $1,350 handheld probe (SonoQue C3), acquired at the same visit as the reference hospital scan in 18 patients. In-domain accuracy was 2.34 +/- 0.43 mm; on handheld images the same model gave 16.12 mm against the hospital record, a 6.9-fold degradation. Input preprocessing, a choice rarely reported, accounted for about a third of that gap: direct resizing gave 49.45 mm against 16.12 mm for an aspect-preserving pipeline. A leave-one-out data-efficiency curve showed that fine-tuning on two images significantly worsened accuracy, and that from four images upward error fell by 0.75 mm per doubling of the local dataset (p = 0.0045), although no training-set size up to 27 images significantly improved on the unmodified model. Much of the residual gap belongs to the reference rather than the model: scored against a manual tracing of the same image the model was already within 10.48 mm, and its disagreement with sonographer calipers on the same frame was a near-constant offset, negative in 10 of 10 cases. A single calibration constant estimated leave-one-out reduced that disagreement from 8.12 mm to 3.49 mm, reaching the intraobserver reproducibility reported for experienced sonographers, but did not improve agreement with the hospital record. Cross-device evaluations should therefore report the input preprocessing used and characterise the precision of their reference standard.

0.21

SafeRestore: Detector-Relative Risk Certificates for Selective Industrial Image Restoration

cs.CV stat.AP ·2026-09-03 ·Shaoliang Yang, Jun Wang
similarity 0.59
Abstract · 27 pages, 10 figures, 6 main-text tables, and 17 supplementary tables. Shares the Carinthia-S image identities with arXiv:2607.17401 by the …

Industrial inspection pipelines often restore a measured image before a detector acts on it, yet restoration can suppress detector-supported defect structure or create clean-region activations. We formulate restoration as a selective action problem over the measured display, five restored candidates, and review. SafeRestore ranks candidates with action-specific fitted scores, chooses a gate on threshold-tuning data, and evaluates the fixed gate on a disjoint certification sample with two one-sided exact binomial bounds: one for the positive-conditional evidence-loss incident rate and one for the all-accepted excess-activation incident rate. The guarantee is marginal for one policy fixed before its certification outcomes are observed, under an image-level i.i.d. working model. In a retrospective split-sample study of 4,591 public Carinthia-S images, the protocol yields auditable risk-coverage behavior. The primary all-action policy passes in one of five training repetitions (12.0% +/- 26.9% pass-gated test coverage when failures count as zero), whereas fixed bicubic and reduced-complexity variants pass more often. On reserved morphologies, evidence-loss incidence rises to 81.1-90.3%, and KolektorSDD lacks both detector competence and enough positive certification images for the stated target. The contribution is therefore an auditable, detector-relative framework for deciding when a transformed image may be returned automatically and when review remains necessary -- not a claim that adaptive routing outperforms simpler policies on the present evidence.

0.43

When accuracy hides risk: risk-controlled confirmation of the critical view of safety under expert disagreement

cs.CV ·2026-11-18 ·Bowen Xiang, Xiao Yu, Mingze Xu
theory similarity 0.59
Abstract

Automated assessment of the critical view of safety (CVS) in laparoscopic cholecystectomy has advanced rapidly, but a deployed check must decide, before the first clip, whether to confirm CVS. The standard rule calibrates a threshold so that false confirmations are rare on average. We show that this rule concentrates its errors on disputed videos, which one of three expert raters judged sufficient, and that the failure is structural. When reference labels are votes of a panel sampled from a larger expert pool, the rule that maximizes sensitivity at a fixed average false-confirmation rate also maximizes false confirmation of disputed videos, and for recognizers that recover expert judgement with less noise this rate rises with accuracy. We propose certified confirmation, a risk-controlled monitoring rule whose threshold is selected by multiple testing so that, with probability at least 1 - δ, false confirmation is at most α overall, among disputed videos, within vendor and country groups, and before CVS is visible, using disputed labels only to calibrate. In a pre-specified, hash-locked evaluation on the 300 test videos (18 countries) of the SAGES CVS Challenge, standard calibration falsely confirmed 8 of 39 disputed videos (0.21; one-sided p = 0.037 against α = 0.10), whereas certified confirmation confirmed 2 (one-sided 95% upper bound 0.15). On development data, protection held for vendor and country groups left out of calibration. The cost was confirming 10% rather than 35% of sufficient videos on time. Average accuracy hides this risk; disputed cases must be protected explicitly.

0.23

Interpreting Selective-Review Coverage for Head-and-Neck Auto-Contours

eess.IV ·2026-11-11 ·Jaden Moon, Caiwen Jiang, Songlin Zhao, Wei Liu
unusual similarity 0.58
Abstract · 5 pages, 2 figures, 3 tables. Submitted to IEEE ISBI 2027

Selective-review systems aim to reduce human review by passing candidates predicted to be low risk. However, coverage (the fraction passed) can be misleading when measurements needed to interpret risk are unavailable. We separate structural evaluability (a nonempty mask with all required measurements available), computational-endpoint status, and pass-or-review decisions using archived Mayo records and controlled OpenKBP analyses. In 3,968 Mayo records, all 1,167 recorded empty predictions (predicted target-organ masks containing zero voxels despite nonempty reference masks) were endpoint-negative and passed both modules, accounting for approximately 76% of each pass set. In a held-out OpenKBP test, a logistic-regression router achieved an area under the receiver operating characteristic curve of 0.833 and average precision of 0.449 but passed all 544 non-evaluable candidates, including 542 prespecified empty-mask controls and 2 erosions that became empty. A structural gate removed those passes without changing scores or the threshold, preserved all evaluable passes, and assigned 98.71% of endpoint-positive candidates to review. Under the original preprocessing and in patient-disjoint development evaluation, a router trained only on evaluable candidates and a router modeling an unavailable state likewise passed no non-evaluable candidates, with similar evaluable discrimination and at least 99.0% of endpoint-positive candidates assigned to review in pooled outer-fold predictions. These results show that coverage should be interpreted jointly with candidate eligibility, endpoint scope, missingness handling, and routing rules; they do not establish clinical validity, deployment safety, or natural-failure prevalence.

0.25

A Decision-Aware Evaluation Protocol for Uncertainty in Medical Image Segmentation

cs.CV ·2026-10-15 ·Saket Maganti, Shreyank N Gowda
similarity 0.58
Abstract

Medical segmentation uncertainty is often evaluated with calibration error or pixel-level error-detection scores, but these metrics do not show whether uncertainty improves downstream review decisions. We propose a decision-aware evaluation protocol that separates uncertainty estimation from deferral policy and evaluates their combination under fixed review budgets: which pixels, regions, or images should be accepted, and which should be reviewed. The protocol reports fixed-budget error reduction, risk-coverage behaviour, region-level review proxies, structural failure modes, probability-derived baselines, and sensitivity to review assumptions. We instantiate it across retinal vessel, skin lesion, and polyp segmentation with multiple backbones and uncertainty estimators. Under matched inference budgets and canonical multi-layer MC Dropout, test-time augmentation (TTA) gives a stronger error-detection signal across matched MC--TTA comparisons, including architecture and dropout-placement checks. However, estimator rankings are not universal. For example, probability-derived scores remain competitive, deep ensembles can outperform TTA, learned deferral helps mainly when the base estimator is weak, and rankings can change with review unit, uncertainty target, correction assumptions, and cost model. Structural and second-observer analyses further show that boundary and thin-structure failures are partly tied to annotation ambiguity. These findings support evaluating uncertainty as an estimator-policy-review configuration, not as a standalone score map. Code: https://anonymous.4open.science/r/Uncertainity-in-Medical-Image-Segmentation/README.md

0.34

Certify, Report, or Refuse: Harm-Constrained Conformal Prediction for Clinical Actions

cs.LG cs.AI ·2026-11-09 ·Zhiyuan Li, Lequan Yu
similarity 0.58
Abstract · Findings of the 6th Machine Learning for Health Symposium (ML4H 2026)

Routing a melanoma case to routine follow-up is a serious action error that diagnostic coverage alone does not rule out. We present Harm-Constrained Risk-Optimal Conformal Prediction (HC-ROCP), a training-free framework that uses frozen classifier probabilities and adds a harm penalty when choosing prediction sets and actions. We test the resulting policies on disjoint calibration splits with multiplicity adjustment. HC-ROCP certifies supported claims, reports descriptive fallback policies, and refuses unsupported subgroup claims. We evaluate these outputs retrospectively on COVID-19 radiographs, ISIC 2019 dermoscopy, and EyePACS-to-APTOS retinopathy transfer. Across ten calibration seeds, HC-ROCP reduces COVID-19 severe-action errors from the exact risk-optimal baseline's 4.68\% to 0.54\%. At a 5\% budget, global claims are certified for COVID-19 and ISIC, but the rare squamous-cell-carcinoma claim is refused. At 1\%, all settings receive descriptive reports. For each budget, we also report the minimum severe-case count needed to pass the global harm gate with zero errors, providing a starting point for further data collection.

0.15