CS Paper Radar

Impact of Data Augmentation on Confidence Calibration in Melanoma Classification

Morgan May, Simon Caton, Pierpaolo Dondio

cs.AI cs.CVcs.LGcs.LO ·submitted 2026-10-05 11:18 UTC · v1
published

Abstract

Accurately quantifying the predictive uncertainty or improving model calibration plays an important role in medical image classification, in particular in melanoma diagnosis, where accurate uncertainty quantification can have significant implications for patient care. One of the methods for calibration improvement is data augmentation. In addition, data augmentation as a method for synthetically increasing the size of the dataset has been proven to improve the performance of models trained on imbalanced datasets. However, the impact of data augmentation, as a transformation of a part of the original data, on calibration of models trained on imbalanced datasets, in particular in melanoma classification is under-explored. We train neural networks on SIIM-ISIC 2020 melanoma classification dataset under two conditions: with and without data augmentation, and compare the differences in AUC and expected calibration error (ECE) in both scenarios. Our results shows improvements in uncertainty calibration using different augmentation methods.

Comment
Presented at the 30th Conference on Medical Image Understanding and Analysis (MIUA 2026)

Closest papers

Nearest neighbours by meaning across the whole index.

On Impact of Loss Function on the Performance of Neural Networks in Melanoma Diagnosis

cs.AI cs.CVcs.LG ·2026-10-05 ·Morgan May, Pierpaolo Dondio, Simon Caton
published similarity 0.75
Abstract · Presented at the 30th Conference on Medical Image Understanding and Analysis (MIUA 2026)

Melanoma is the deadliest type of skin cancer, whose early diagnosis is crucial for patients' survival. Image classification using deep learning models has shown promising results for melanoma diagnosis. However, the performance of these models on the melanoma datasets such as SIIM-ISIC melanoma classification dataset is a challenge due to the class imbalance. One of the methods to deal with this challenge is using loss function modifications. In this work, we have investigated the effect of different loss functions on the performance of deep neural networks. We trained these networks using focal loss, logit-adjusted softmax cross-entropy (CE) loss, and weighted softmax CE loss, and we report different metrics for evaluating performance and uncertainty calibration. Our results suggest that focal loss delivers a good combination of performance in terms of AUC and uncertainty calibration in terms of expected calibration error (ECE) simultaneously.

0.26

Estimating Performance for Safe Deployment of Machine Learning Models

cs.LG ·2026-10-21 ·Zeju Li, Konstantinos Kamnitsas, Mobarak I. Hoque, Melanie Roschewitz, Cheng Ouyang +2 more
published similarity 0.69
Abstract · Accepted by Medical Image Analysis; updated author names and affiliations

The main challenge in applying deep neural networks to real-world scenarios lies in distributional discrepancies between training and test datasets, often caused by variations in data acquisition conditions. Assessing the performance of pre-trained models without labels is crucial for safety, but this is complicated by deep models' overly optimistic probabilistic outputs, often exacerbated by class imbalances, leading to unreliable predictions. To address these issues, we introduce MOVAL, a novel MOdel eVALuation tool that estimates performance using unlabeled test data. MOVAL incorporates class-wise confidence calibration to adjust model outputs, allowing for the estimation of metrics beyond accuracy and mitigating the detrimental effects of class imbalance on confidence calibration. Additionally, we propose an ensemble approach that combines multiple calibrated confidence scores for a more robust solution. We validate MOVAL on natural and medical image datasets with various acquisition shifts, covering tasks from classification to segmentation. It achieves state-of-the-art results across evaluation metrics such as accuracy, F1-Score, AUC, and more. Furthermore, we demonstrate that MOVAL provides accurate performance estimations at the class, instance, and pixel levels, ensuring the transparent and safe deployment of machine learning models. We provide MOVAL as a deployment tool for monitoring and estimating performance under a range of domain shifts.

0.26

Uncertainty-driven training for three-dimensional calibrated lung nodule classification

eess.IV cs.CV ·2026-09-17 ·Giuseppe Tripodi, Alessandro De Rosis, Saleh Rezaeiravesh
publishedbenchmark similarity 0.67
Abstract

In this work, we present an uncertainty-driven training framework for three-dimensional computed tomography (CT) lung nodule classification, where validation-based uncertainty estimates guide loss reweighting to enhance predictive performance and probability calibration. Two Uncertainty Quantification (UQ) methods are considered: Monte Carlo Dropout (MCD) and Evidential Deep Learning (EDL). Both provide per-class uncertainty estimates that modulate the loss and encourage focus on hard or unreliable classes. The framework is evaluated with ResNet, DenseNet, EfficientNet, Vision Transformer (ViT), and Swin Transformer backbones on two datasets: the clinical LIDC-IDRI cohort and the NoduleMNIST3D benchmark. Uncertainty-driven training achieves classification performance similar to conventional training while substantially improving calibration, with an expected calibration error (ECE) reduced by up to 65% on LIDC-IDRI. EDL attains competitive performance on shallower architectures with single-pass inference, whereas MCD is more robust on deeper networks. Analysis across architectural families reveals that uncertainty-driven training benefits convolutional backbones more consistently than transformer-based architectures: EDL in particular degrades on ViT, suggesting that the Dirichlet evidence parameterisation may interact unfavourably with attention-based architectures at lower input resolutions. A posteriori temperature scaling proves highly effective across all configurations, indicating that a simple scalar calibration can be competitive even without explicit uncertainty-aware training. Our results indicate that integrating UQ into the training loop can significantly improve probabilistic calibration and support more trustworthy deployment of three-dimensional medical imaging models.

0.19

Evidence-Driven Differential Diagnosis of Malignant Melanoma

eess.IV cs.AIcs.CVcs.LG ·2026-08-28 ·Naren Akash, Anirudh Kaushik, Jayanthi Sivaswamy
top venuecode/data similarity 0.67
Abstract

We present a modular and multi-level framework for the differential diagnosis of malignant melanoma. Our framework integrates contextual information and evidence at the lesion, patient, and population levels, enabling decision-making at each level. We introduce an anatomic-site aware masked transformer, which effectively models the patient context by considering all lesions in a patient, which can be variable in count, and their site of incidence. Additionally, we incorporate patient metadata via learnable demographics embeddings to capture population statistics. Through extensive experiments, we explore the influence of specific information on the decision-making process and examine the tradeoff in metrics when considering different types of information. Validation results using the SIIM-ISIC 2020 dataset indicate including the lesion context with location and metadata improves specificity by 17.15% and 7.14%, respectively, while enhancing balanced accuracy. The code is available at https://github.com/narenakash/meldd.

0.69

CalCErt: Bin-wise Certification of Confidence Calibration in Medical Image Classification

cs.CV ·2026-10-02 ·Leo Fillioux, Stergios Christodoulidis, Stergios Christodoulidis, Maria Vakalopoulou, Jose Dolz
top venuecode/data similarity 0.65
Abstract

Deep neural networks remain vulnerable to adversarial perturbations, which can distort not only predictions but also confidence scores, undermining uncertainty calibration. While existing certification methods focus on preserving the predicted category, providing guarantees on how calibration behaves under adversarial attacks remains overlooked. In this work, we introduce CalCErt, a simple and efficient post-hoc strategy that certifies bin-wise confidence calibration for any pretrained differentiable classifier. Our approach combines empirical calibration estimates, statistical concentration bounds, and local Lipschitz estimates of the confidence function to derive data-dependent upper bounds on worst-case miscalibration within an $ell_2$-ball of radius R. We evaluate CalCErt across 11 medical image classification tasks and multiple adversarial perturbations, demonstrating substantially higher certified coverage than baseline strategies while maintaining competitive tightness. Our code is available at https://github.com/leofillioux/calcert.

0.45

ActiveAugment: Online Active Learning for Augmentation Selection in Deep Learning

cs.CV cs.AI ·2026-08-28 ·Noah Videcrantz, Mostafa Mehdipour Ghazi
publishedbenchmarkcode/data similarity 0.64
Abstract · Accepted at BMVC2026

Data augmentation is a cornerstone of deep learning pipelines, yet existing strategies treat it as a static, model-agnostic preprocessing step, either relying on expensive dataset-specific policy search or applying transformations uniformly at random, regardless of what the model has already learned. We introduce ActiveAugment, a unified framework that treats augmentation selection as an online active learning problem. For each training minibatch, ActiveAugment generates a pool of candidate augmented views and scores each candidate using a combination of the model's predictive uncertainty and the feature discrepancy induced by the augmentation. The augmentation under which the current model is most fragile is selected per sample, and the model is then trained with a joint supervised classification and supervised contrastive objective that enforces intra-class invariance to the selected augmentations while maintaining inter-class separation. We evaluate ActiveAugment on eight benchmark datasets spanning natural and medical imaging, using CNN and transformer architectures across three training regimes (training from scratch, full fine-tuning, and linear probing), and comparing eight active selection strategies for augmentation scoring. ActiveAugment outperforms AutoAugment, RandAugment, and TrivialAugment under controlled augmentation shifts across all domains and budgets, with the most pronounced gains at low labelling budgets. On medical imaging datasets, where data is scarce and domain shift relative to natural-image pretrained models is large, ActiveAugment achieves higher test F1 than all baselines, demonstrating strong cross-domain adaptability. Our analysis reveals that the augmentation selection policy evolves meaningfully during training and that strategy choice has a direct impact on generalisation. Code is available at: https://github.com/noahvide/ActiveAugment.

0.41

Adaptive Conformal Redistribution for Inter-class Transitional Uncertainty in Medical Image Classification

cs.CV cs.AIcs.LGstat.CO+1 ·2026-09-10 ·Saibal Ghosh, Samarup Bhattacharya, Sanjoy Kumar Saha, Umapada Pal, Tapabrata Chakraborti
benchmarkcode/data similarity 0.63
Abstract

Medical image classification is frequently complicated by transitional categories whose feature distributions overlap those of adjacent classes, producing ambiguous decision boundaries. Conformal prediction returns uncertainty-aware prediction sets, but these are not directly actionable in clinical screening, where a single decision is required. This work proposes adaptive conformal redistribution (AdaConRed), a label-free post-conformal decision rule that converts ambiguous prediction sets into refined class assignments. A five-stage pipeline is developed. Vision-language generative augmentation addresses minority-class scarcity; a frozen DermFoundation encoder provides embeddings; a lightweight multi-layer perceptron performs classification; an entropy-modulated, margin-aware nonconformity score constructs adaptive prediction sets; samples predicted as transitional with multi-label sets are reassigned to the most probable alternative class within the set, using only model outputs at inference. Evaluation uses the OSCC oral lesion and ISIC skin lesion benchmarks at a miscoverage level of 0.2. On the 3-class OSCC benchmark, overall accuracy improves from 73.54% to 77.38%, with oral cancer accuracy rising from 64.29% to 82.14% and benign accuracy from 56.57% to 70.20%. Reassignment of transitional samples reduces OPMD accuracy from 84.78% to 80.16%, consistent with the asymmetric cost of missed malignancy. On ISIC, overall accuracy improves from 85.83% to 87.19%, melanoma accuracy rising from 66.04% to 68.34%. AdaConRed outperforms LAC, APS and RAPS under an identical backbone and redistribution rule. Conformal prediction can be extended beyond uncertainty quantification toward actionable decision support where transitional disease categories are present, with gains concentrated in the clinically critical malignant categories. Code repository: https://github.com/saibal436ghosh/AdaConRed.

0.38

Uncertainty-Aware and Explainable Ensemble Deep Learning Framework for Multi-Class Skin Lesion Classification

eess.IV cs.AIcs.CVcs.LG ·2026-08-11 ·Rofiqul Islam, Lilatul Ferdouse
similarity 0.63
Abstract · 5 pages, 3 figures, IEEE AIBThings 2026

Skin cancer diagnosis from dermoscopic images remains challenging due to high intra-class variability, inter-class similarity, class imbalance, and the limited interpretability of deep learning models. This paper proposes an uncertainty-aware and explainable deep learning framework for multi-class skin lesion classification. The framework combines a vision transformer model (MaxViT-Tiny) with CNN-based models (ConvNeXt-Tiny and EfficientNetV2-B0) through deep ensemble learning. Monte Carlo (MC) Dropout estimates predictive uncertainty and identifies unreliable predictions, while Grad-CAM++, an explainable AI (XAI) technique, provides visual explanations by highlighting lesion regions that influence model decisions. Evaluated on the HAM10000 dataset, the framework achieves 96% accuracy and 99% ROC-AUC under uncertainty-aware filtering (entropy < 1.0, confidence >= 0.7), with macro-average precision, recall, and F1-score of 94%, 95%, and 95%, respectively, and 96% weighted-average scores across all three metrics. The results demonstrate accurate, interpretable, and uncertainty-aware skin lesion classification for trustworthy computer-aided diagnosis.

0.07