CS Paper Radar

DDEL: Closed-Loop Mutual Enhancement of Dataset Distillation and Ensemble Learning

Weiqi Zhang, Yongquan Yang

cs.CV cs.LG ·submitted 2026-10-13 05:09 UTC · v1

Abstract

Dataset distillation (DD) reduces a training set to a small set of examples, while ensemble learning (EL) combines predictions from several models. When applied in sequence, the distilled set remains fixed as the ensemble learns. We introduce DDEL, a framework that feeds ensemble behavior back into distillation. Each round trains the ensemble, measures loss and prediction disagreement on the source data, and uses these signals to guide the next data update. We implement this loop with distribution matching and a homogeneous three-member ensemble. Preliminary experiments cover skin lesion classification (ISIC 2019), treatment-response prediction (I-SPY1), and binary two-year prognosis classification (NSCLC-Radiomics). With the same images-per-class budget as the open-loop comparison, DDEL reaches balanced accuracy 0.3471 on ISIC and mean ROC-AUC 0.5120 and 0.5597 on I-SPY1 and NSCLC. The I-SPY1 difference is small, and the ISIC trajectory shows that more rounds do not consistently improve performance. These results offer initial evidence for studying DD-EL feedback, with its computational cost and robustness still requiring controlled evaluation.

Comment
11 pages, 1 figure, 3 tables

Closest papers

Nearest neighbours by meaning across the whole index.

Masked Privileged-Information Distillation for Multimodal Skin Lesion Classification Under Missing Clinical Metadata

cs.CV cs.LG ·2026-10-02 ·Anirban Barua, Md Mahir Abrar Khan, Ayman Iktidar, Md. Sajjatul Islam
similarity 0.63
Abstract

Multimodal skin lesion classification combines clinical images with patient metadata to improve diagnostic accuracy. However, complete metadata available during training may be only partially accessible at deployment, and resource-constrained settings additionally require computational efficiency. We address these challenges with a privileged-information distillation framework in which a multimodal teacher trained on complete metadata supervises a 9.2x smaller student trained with randomly masked clinical fields. Clinical fields are masked as whole groups at a per-sample rate drawn from U(0,1), so one training run covers the full metadata availability range. Fusion is residual, with metadata added as a gated correction to an unconditional image base. On the PAD-UFES-20 dataset, distillation under masked training improves balanced accuracy over cross-entropy training at every availability level, by an average of +4.7 points versus +1.8 points without masking. The masked student loses only 7.8 balanced-accuracy points as metadata decreases from complete to absent, compared with 36.6 points for the same student trained on complete metadata, highlighting the role of masked training in graceful degradation beyond distillation alone. Grad-CAM visualizations further show that the masked student's attention generally remains lesion-centered as metadata is withdrawn. The resulting compact model targets point-of-care settings, where clinical metadata is often incomplete.

0.23

Towards Democratising Low-Data Medical Image Classification: CPU-Trainable ZACH-ViT and Its Geometric Variants

cs.CV cs.LG ·2026-11-12 ·Athanasios Angelakis
similarity 0.60
Abstract · 17 pages, 4 figures

Medical image classification in resource-constrained settings must contend with limited annotations and computational capacity. We evaluate four ZACH-ViT variants with linear, Euclidean prototype, Poincare and spherical heads alongside two compact multiple-instance-learning baselines. All models are trained from scratch with fewer than 260,000 parameters across seven MedMNIST+ datasets and three external benchmarks (PCam, ISIC2017 and Shenzhen tuberculosis radiographs). The low-data protocol uses 50 training images per class and five seeds, with validation-selected checkpoints at 23 and 47 epochs; a full-data reference uses one seed. Compact-TransMIL achieves the highest ten-dataset mean test Macro-F1 at 23 epochs (53.34%) and with full data (73.93%). At 47 epochs, ZACH-Linear narrowly leads (57.76% versus 57.54% for Compact-TransMIL). Extending training improves Macro-F1 in 239 of 300 paired comparisons. On ISIC2017, hZACH-Spherical has the highest 47-epoch low-data Macro-F1 (44.09% with SD 4.09%), whereas ZACH-EuclideanProto leads with full data (51.27%). We additionally conduct 40 CPU-only training runs on ISIC2017 and Shenzhen using four CPU threads; model-only single-image inference requires approximately 4-9 ms, while image decoding and preprocessing substantially increase end-to-end latency. The results demonstrate the feasibility of CPU-only training on the tested platform and the task- and regime-dependence of compact medical imaging under strict resource constraints.

0.18

FedHisto-PAST: Parameter-Efficient Stain-Aware Federated Learning for Cross-Site Lung Histopathology Classification

cs.CV cs.AI ·2026-09-25 ·Muhammad Muhtasim Shahriar, M. M. Golam Hafiz, Saad Aloteibi, Mohammad Ali Moni
similarity 0.60
Abstract · Submitted to Engineering Applications of Artificial Intelligence (Elsevier)

Cross-site lung histopathology classification must account for stain variation, non-IID client data, missing classes, and the cost of adapting large pathology encoders. This study evaluates FedHisto-PAST v2 for three-way classification of adenocarcinoma (ACA), Normal, and squamous cell carcinoma (SCC). FedHisto-PAST v2 combines a frozen HIBOU-B foundation model with parameter-efficient adaptation, stain-conditioned paired-view prediction and feature consistency, reliability-aware prototype learning, and adaptive federated aggregation. Experiments used a five-client, non-IID, raw-data-local simulation with fixed internal evaluation, client-level analysis, component ablations, communication accounting, and a development-influenced exploratory LungHist700 cohort. All principal methods achieved near- ceiling internal performance, which limited discrimination on the fixed split. On LungHist700, FedHisto- PAST v2 achieved a Macro-F1 of 0.728560 and a balanced accuracy of 0.730454. Higher recognition of Normal and SCC was accompanied by lower ACA recall, and calibration remained imperfect. Prediction-level consistency was the only component with a clearly supported independent contribution in the external ablation analysis. Feature consistency and prototype regularization showed no conclusive independent overall gains in Macro-F1. The framework updated 1.253841% of the model parameters. The results provide exploratory cross-dataset evidence for stain-aware, parameter-efficient federation; they do not establish formal privacy, patient-level independence, prospective deployment, or clinical validation.

0.26

Is a CNN-Transformer Ensemble Worth It When a Chest X-Ray Foundation Model Is Available?

eess.IV cs.CV ·2026-11-09 ·Aditya Seelaboyina, Irfan Khan
code/data similarity 0.59
Abstract · 8 pages, 2 figures, 10 tables. Code: https://github.com/adityaseelaboyina/cxr-ensemble-vs-foundation-model

Ensembles of ImageNet-pretrained convolutional networks and Vision Transformers are a common way to improve multi-label chest X-ray classification. We ask whether such an ensemble is still worth its cost when a model pretrained on chest radiographs is available. On the official patient-disjoint split of NIH ChestX-ray14 (25,596 test images, 2,797 patients), we train five ImageNet-pretrained backbones and the EVA-X-S foundation model with validation-only model selection, compare five fusion rules, and estimate every difference with a patient-level paired bootstrap. EVA-X-S alone reached a test macro-AUROC of 0.828; the five-model ensemble reached 0.814 (difference -0.0136, 95% CI -0.0178 to -0.0098) at about 7.8 times the summed backbone latency and ten times the parameters. Averaging the five ImageNet models with EVA-X-S changed AUROC by -0.0056 (-0.0095 to -0.0021). With all five ImageNet backbones trained with the EVA-X-S recipe, their ensemble scored 0.817, still -0.0109 (-0.0142 to -0.0076) below EVA-X-S, and across three seeds EVA-X-S varied by at most 0.001 AUROC. Under the ImageNet models' recipe, however, EVA-X-S fell to 0.798. Our full-stacking configuration never outperformed calibrated averaging. In this setting, the tested ImageNet ensembles did not improve on a single chest X-ray foundation model fine-tuned with its own recipe.

0.38

Uncertainty Identifies Difficult Samples Across Methods: A Multi-Task Study on a Heterogeneous Skin Lesion Dataset

cs.CV cs.LG ·2026-08-14 ·Leon Koole, Jiapan Guo, Matias Valdenegro-Toro
published similarity 0.59
Abstract · 12 pages, 9 figures, UNSURE 2026 @ MICCAI camera ready

Skin lesion classifiers can be confidently wrong on the cases that matter most, so knowing when a prediction should not be trusted is clinically as useful as the prediction. We study uncertainty quantification on a dataset pooled from many ISIC sources, with a shared backbone and two jointly learned heads: a binary malignant versus non-malignant head and a five-class diagnostic head. Five UQ methods (MC Dropout, DropConnect, Flipout, Deep Ensembles, DUQ) are compared on accuracy, calibration, uncertainty decomposition, and risk-coverage. Difficulty is largely method-agnostic: even methods with narrow entropy distributions rank the same samples as hard (per-sample entropy correlations of $0.54$ to $0.91$). The choice of method matters more for calibration and uncertainty decomposition, where Deep Ensembles is the clear winner, than for finding difficult cases. The ranking is also good enough that deferring the most uncertain cases removes a disproportionate share of errors, supporting uncertainty-based selective referral, evaluated here in-distribution only.

0.33

Uncertainty-Aware and Explainable Ensemble Deep Learning Framework for Multi-Class Skin Lesion Classification

eess.IV cs.AIcs.CVcs.LG ·2026-08-11 ·Rofiqul Islam, Lilatul Ferdouse
similarity 0.59
Abstract · 5 pages, 3 figures, IEEE AIBThings 2026

Skin cancer diagnosis from dermoscopic images remains challenging due to high intra-class variability, inter-class similarity, class imbalance, and the limited interpretability of deep learning models. This paper proposes an uncertainty-aware and explainable deep learning framework for multi-class skin lesion classification. The framework combines a vision transformer model (MaxViT-Tiny) with CNN-based models (ConvNeXt-Tiny and EfficientNetV2-B0) through deep ensemble learning. Monte Carlo (MC) Dropout estimates predictive uncertainty and identifies unreliable predictions, while Grad-CAM++, an explainable AI (XAI) technique, provides visual explanations by highlighting lesion regions that influence model decisions. Evaluated on the HAM10000 dataset, the framework achieves 96% accuracy and 99% ROC-AUC under uncertainty-aware filtering (entropy < 1.0, confidence >= 0.7), with macro-average precision, recall, and F1-score of 94%, 95%, and 95%, respectively, and 96% weighted-average scores across all three metrics. The results demonstrate accurate, interpretable, and uncertainty-aware skin lesion classification for trustworthy computer-aided diagnosis.

0.07

Evidence Field for Histopathology: Cross-Centre Evaluation and Replayable Spatial Decisions

eess.IV ·2026-10-15 ·Isarar Siddique, Raiyan Siddique, Arshi Niyaz, Jamyang Ugyen Tshomo, Atish Raza
similarity 0.59
Abstract · 8 pages, 4 figures

Histopathology classifiers must transfer across acquisition sites while exposing evidence that can be checked independently. Evidence Field couples learned image measurements and spatial classification with a native covering-program record. We evaluate this architecture on all 455,954 released CAMELYON17-WILDS patches, using patient-separated source partitions and two held-out hospital centres. Eighteen from-scratch fits compare four Evidence Field objectives with residual convolutional and compact vision-transformer classifiers. Checkpoints, temperatures and review thresholds are selected using source validation. The source-selected paired-cross-entropy ensemble achieves areas under the receiver operating characteristic curve of 0.9356 and 0.9453, with accuracies of 84.46% and 86.77% at the two held-out centres. Corresponding convolutional values are 0.7352 and 0.7656, and transformer values are 0.8502 and 0.7924. Evidence Field has 10,550 trainable parameters per member and additionally returns selected covers, signed region and interaction terms, contextual dependencies and independently reconstructible predictions. Patient-cluster intervals, calibration, source-fixed review policies and recorded training resources characterize the results. A 340-image, three-member audit reconstructs probability mixtures and 6,120 class programs under three rigid views. The evaluation supports cross-centre discrimination and reproducible computation for these implemented models; unequal family tuning, previously examined public targets and limited patient groups constrain broader interpretation.

0.47

BagShift: Measuring How Patch Selection Changes the Evidence Seen by Whole-Slide MIL

cs.CV ·2026-08-16 ·Ruicheng Yuan, Zhenxuan Zhang, Liwei Hu, Anbang Wang, Haijie Xu +2 more
similarity 0.58
Abstract · 18 pages include Supplementary Material. 8 figures

Whole-slide multiple-instance learning (MIL) observes only the patches admitted by its selector. Deployment can alter this selector through compute limits, tissue masking, or regional workflows, even when the patch count is unchanged. We introduce BagShift, a paired protocol that changes the selector for the same case while holding its features and predictor fixed, thereby isolating selector response from case mix. With equal 128-patch budgets, sampling across the tissue or concentrating around one coordinate exposes markedly different evidence: on PANDA, the two views reduce quadratic weighted kappa by 1.57 and 17.96 points, respectively (QWK reported on the $\times100$ scale). On CAMELYON16, lesion annotations withheld from model development show that localized views retain tumor in only 10.0\% of micrometastatic observations, and matched exposure does not consistently recover the loss. The same fixed-count stressor produces a much smaller response on external lung subtyping, although differences in relative coverage make cross-task severity descriptive. When repeated localized observations are available, unioning their patches before one nonlinear MIL pass improves PANDA QWK by 7.87 points over averaging regional predictions. Patch count specifies computation, not observed evidence; deployment evaluations should report both what a selector preserves and how repeated observations are aggregated.

0.10