CS Paper Radar

Self-supervised Pre-training Helps Retinal Disease Progression Modelling Most When Data Is Scarce

Ifeoma Veronica Nwabufo, Julius Gervelmeyer, Sarah Müller, Philipp Berens

cs.CV ·submitted 2026-09-11 13:28 UTC · v1

Abstract

Modelling how a disease progresses over time requires longitudinal imaging cohorts, which are scarce and small, whereas cross-sectional data -- one image per participant -- is abundant. Self-supervised pre-training on such data offers a way to bridge this gap, but it is unclear which strategy best supports progression modelling, or how that answer depends on the amount of labelled longitudinal data. We study this for age-related macular degeneration (AMD), pre-training encoders on the large cross-sectional NAKO cohort and predicting time to late AMD on the longitudinal AREDS dataset. We compare in-house self-supervised encoders against a general-purpose (DINOv2) and a domain-specific (RETFound) foundation model, across contrastive, masked-autoencoding, and self-distillation objectives, under frozen and fine-tuned protocols, and across labelled training sets from 100 to 32,250 examples. Which model performs best depends on how the encoder is used. When the encoder is frozen and labels are few -- the regime typical of longitudinal cohorts -- pre-trained representations reach clinically reasonable discrimination from a few hundred labelled samples, while models trained from scratch do not; this advantage fades under fine-tuning. Transfer is governed by the self-supervision objective rather than corpus scale or domain match, so that an encoder pre-trained on a modest cross-sectional cohort matches or exceeds a far larger in-domain foundation model. Together, these results offer a practical recipe for building progression models where longitudinal data is scarce: a frozen self-supervised encoder with a lightweight survival head.

Closest papers

Nearest neighbours by meaning across the whole index.

Exploring the Latent Structure of AMD Severity Using Retinal Foundation Models

eess.IV cs.CV ·2026-10-09 ·Niccolò Fineschi, Jovana Panic, Atsushi Watanabe, Hideaki Hayashi, Kazuya Nishimura +5 more
published similarity 0.75
Abstract · 7 pages, 5 figures. Accepted for publication at IEEE EMBS BHI 2026

Retinal foundation models have demonstrated strong performance across a wide range of ophthalmic tasks, yet relatively little is known about how disease-related information is represented within their learned embeddings and whether this distribution influences downstream prediction performance. In this work, we use Age-related Macular Degeneration (AMD) as a case study to investigate how disease severity is represented within the embedding space of RET-CLIP, a retinal vision-language foundation model, using bilateral fundus images from the AREDS dataset, benchmarked against non-retinal encoders. We first characterize the distribution of AMD severity within the learned representation space and then evaluate how this structure influences classification performance across three clinically relevant task formulations of increasing granularity. Finally, we analyze the most influential latent dimensions combining feature attribution, statistical phenotype association, and qualitative image analyses. Our results show that RET-CLIP represents AMD severity as a continuous spectrum rather than a set of discrete categories, with healthy and advanced disease stages exhibiting clearer separation than intermediate ones. This organization was reflected in downstream performance, which decreased from a balanced accuracy of 0.844 and 0.814 for binary AMD and Late-AMD detection respectively to 0.446 for six-class grading. Furthermore, the most influential latent dimensions were statistically associated to distinct retinal phenotypes, including drusen accumulation, pigmentary alterations, and advanced retinal lesions. These findings suggest that the structure of retinal foundation-model representations influences the difficulty of clinically relevant prediction tasks and demonstrate the potential of representation-level analysis for understanding disease organization and improving interpretability.

0.18

A Disease-specific Retinal Aging Biomarker for Predicting Progression to Late Age-related Macular Degeneration

cs.CV ·2026-11-12 ·Hexin Dong, Benjamin Hou, Puzhen Wu, Mingquan Lin, Emily Y. Chew +2 more
similarity 0.72
Abstract · 41 pages, 13 figures (including supplementary material)

Age-related macular degeneration (AMD) is a leading cause of irreversible vision loss in older adults. While chronological age is a primary risk factor, it inadequately reflects individual variations in retinal aging, limiting effective AMD risk stratification and timely intervention. Here, we propose and validate fundus-based AMD age (FAA), which quantifies disease-specific retinal aging in AMD and captures accelerated AMD-related retinal aging, thereby improving prognostic precision over chronological age. Trained on 36,782 images from 2,192 participants in the Age-Related Eye Disease Study (AREDS) dataset and validated across AREDS and AREDS2 cohorts, our model demonstrates significant improvements over traditional chronological age-based approaches. Specifically, integrating the proposed FAA into Cox proportional-hazards models significantly enhances prediction performance. FAA improved the C-index from 0.836 to 0.888 and reduced the 5-year Brier score from 0.050 to 0.046 in AREDS, with consistent gains observed in AREDS2 (C-index 0.715 to 0.756; 2-year Brier 0.078 to 0.076). FAA showed improved prognostic performance across the two datasets and under different phenotype definitions (reading center-defined vs. specialist-defined grading), supporting its potential utility for AMD risk stratification. These findings highlight the advantage of incorporating FAA into AMD risk prediction frameworks, supporting further evaluation of FAA for AMD risk stratification.

0.21

SEEC: A Large-Scale Dataset and Benchmark for Self-Supervised External-Eye Representation Learning

cs.CV ·2026-11-19 ·George R. Nahass, Michelle Fleishaker, Sanjay Ganesh, Sasha Hubschman, R. V. Paul Chan +4 more
benchmark similarity 0.68
Abstract · 36 Pages, 10 Figures, 9 Tables

External-eye photography captures clinically relevant periocular anatomy at low acquisition cost, yet no large-scale standardized resource exists for evaluating self-supervised external-eye representations. We introduce a 4.35-million-image Standardized External-Eye Corpus (SEEC) derived from fourteen heterogeneous public face datasets, together with a benchmark for external-eye representation learning. For evaluation, we provide held-out evaluation distributions, landmark-annotated anatomical probe sets, and clinical disease-classification experiments spanning ptosis, thyroid eye disease, and craniofacial syndromes. We evaluate representations at two complementary levels: (a) the representation level, by assessing embedding geometry across multiple data distributions, and (b) the functional level, by assessing anatomical recovery, out-of-distribution anatomical transfer, and disease classification. Our models include ResNet-101 and ViT-B encoders pretrained using InfoNCE, VICReg, and LeJEPA objectives and compared with DINOv2 and MAE. Our results indicate that representation-level and functional-level evaluations are dissociable: broader embedding-space utilization does not consistently correspond to anatomical transfer or disease-classification performance, and no single encoder is universally optimal. Domain-specific external-eye encoders outperform DINOv2 and MAE on cross-dataset anatomical transfer despite being pretrained at substantially smaller scale, while external-eye-pretrained ViT models match or outperform these general-purpose baselines on frozen disease-classification probes. We release benchmark code, evaluation scripts, and preprocessing pipelines to reproduce SEEC at https://monkeygobah.github.io/seec-project/

0.31

Domain-Specific Self-Supervised Representation Learning for Retinal Fundus Classification

cs.CV cs.LG ·2026-08-27 ·Bekzat Nurlanbekova, Fung Fung Ting
similarity 0.67
Abstract

Annotated medical images remain scarce and costly to obtain, which motivates self-supervised learning (SSL). We present a controlled empirical study of two SSL frameworks, SimSiam and SimCLR, for retinal fundus classification, asking whether retina-specific augmentations (Graham's local-mean subtraction and CLAHE) improve representations beyond standard augmentation and grayscale conversion. ResNet-18 backbones are pretrained on 48K unlabeled fundus images and evaluated by linear probing and fine-tuning on EDID (multi-disease) and RetinaMNIST (DR grading). On EDID, grayscale conversion accounts for most of SimSiam's linear-probe gain, and Graham or CLAHE add little on top of it. After fine-tuning, SimSiam with Graham+Gray exceeds an ImageNet-supervised baseline on RetinaMNIST (+3.0 accuracy points) but only matches it on EDID. Retina-specific augmentation therefore helps inconsistently and should be validated per SSL method and task.

0.14

Combining General and Domain-Specific Pretext Tasks for Brain MR Image Segmentation

cs.CV cs.AI ·2026-09-25 ·Tasneem Nasser, Susanne Schmid, Roberto Souza, Naser El-Sheimy
code/data similarity 0.66
Abstract · 7 figures, 5 tables

A key challenge in medical image analysis is the scarcity of large annotated datasets for specific populations and diseases. As deep learning models rely heavily on labeled data, effective transfer learning strategies are needed to reduce the dependence on manual annotations. Self-supervised learning has emerged as a promising approach for developing foundation models by enabling the learning of transferable feature representations from large-scale unlabeled medical imaging datasets. In this study, we investigate voxel-level brain age prediction as a domain-specific self-supervised pretext task and compare it with image inpainting, a widely used non-domain-specific alternative. We further propose a multitask self-supervised pretraining framework that jointly optimizes both objectives to learn complementary neuroimaging representations. The pretrained models are evaluated on three downstream magnetic resonance image segmentation tasks: multiple sclerosis lesion segmentation, ischemic stroke lesion segmentation, and cortical brain structure segmentation. Overall, the proposed multitask pretraining framework consistently outperformed the single-task pretrained models and training from scratch across most experimental settings, demonstrating the benefit of combining domain-specific and general self-supervised learning pretext tasks for the development of generalizable neuroimaging foundation models.\ Code Availability: The source code used in this study is publicly available at https://github.com/TasneemN/Combining-General-and-Domain-Specific-Pretext-Tasks-for-Brain-MR-Image-Segmentation/

0.30

Longitudinal retinal imaging and auxiliary tasks for discrete-time survival prediction of referable diabetic retinopathy and maculopathy: a retrospective cohort study

eess.IV ·2026-10-27 ·Paul Nderitu, Laura Webster, Samantha S. Mann, Christos Bergeles, Timothy L. Jackson
similarity 0.66
Abstract

Deep learning can predict progression to referable diabetic retinopathy (rDR) and referable maculopathy (rM) from screening photographs, but existing models ignore the prior images that screening programmes hold. We assessed whether multi-visit (MV) images and auxiliary objectives that model the future visit improve prognostic prediction. In this retrospective cohort study of macula-centred photographs from the South-East London Diabetic Eye Screening Programme (UK), eyes without referable disease at baseline and with a follow-up at 20-28 months were split into development (72,559 eyes, 36,648 patients) and test (17,917 eyes, 9,071 patients) sets. A time-aware discrete-time survival model predicted incident rDR (R2 or R3), rM (M1), or either (rDR/rM). We compared baseline-only input with up to three visits (MV), alone or with auxiliary supervision of future-visit age (MV+age) or future-visit latent prediction (MV+latent), using the AUROC of 28-month cumulative risk and paired DeLong tests with Benjamini-Hochberg correction. In the test set, 308 eyes (1.7%) developed rDR/rM, 58 (0.3%) rDR, and 275 (1.5%) rM. MV images increased the AUROC for rDR/rM from 0.749 to 0.786 (+0.037, 95% CI 0.012-0.061; q=0.011) and for rM from 0.749 to 0.788 (+0.039, 0.013-0.065; q=0.010). Auxiliary objectives did not improve MV model performance. In eyes with two or three visits, prior visits accounted for 30-38% of positive Grad-CAM attributions. Prior screening images improved prediction of incident maculopathy and the composite rDR/rM outcome, with gains driven by longitudinal images, not auxiliary supervision. External validation and prospective studies are required before the deployment of models to individualise screening.

0.28

Representation Learning for 3D Brain Imaging: A Benchmark

cs.CV ·2026-11-04 ·Pierre Falconnier, Robin Trombetta, Nicolas Duchateau, Carole Lartizien
top venuebenchmarkcode/data similarity 0.65
Abstract

Many deep learning architectures for computer vision tasks rely on features extracted by a pre-trained encoder on large databases of natural 2D RGB images. However, the wide domain gap with medical images hampers the direct transfer of optimized deep learning models from computer vision to clinical applications. Recently, self-supervised pre-training and foundation models emerged as promising alternatives to mitigate these limitations. Yet, constructing such models for medical imaging faces critical challenges, including designing and adapting training strategies for large 3D volume datasets, and the need for standardized benchmarks to evaluate feature extraction performance. In this study, we propose an extensive comparison of vision encoders with a focus on 3D brain MRI. First, we consider 8 fully and self-supervised models pretrained on natural images (e.g. ResNet, DINO, CLIP) and 10 backbone architectures trained on medical images, either based on supervised (e.g. RadImageNet) or self-supervised (e.g. BiomedCLIP, SimCLR) tasks. All pre-trained encoders are evaluated on various downstream tasks to assess the representation power of their latent representations, including age and sex prediction as well as disease classification on the ADNI and PPMI datasets. Our study shows that 2D encoders pre-trained on natural images perform the most favorably, even compared to models pre-trained on more than 100k brain MRI volumes, highlighting the need for more suitable and robust methods for representation learning in 3D medical imaging. The source code for this study is publicly available at: https: //github.com/PierreFalconnier/brain-representation-benchmark.

0.50

Modelling Geographic Atrophy Progression using Implicit Neural Representations

cs.CV cs.AI ·2026-08-11 ·Simone Sarrocco, Paul Friedrich, Florentin Bieder, Christina Bornberg, Philippe Valmaggia +2 more
publishedcode/data similarity 0.64
Abstract · Accepted at MICCAI 2026 Off-Grid Workshop

Age-related Macular Degeneration (AMD) is the major cause of blindness in the Western world. Its late dry phase is characterised by irreversible atrophic areas, namely Geographic Atrophy (GA). Longitudinal Fundus Autofluorescence (FAF) image acquisitions are currently the main tool for assessing lesion growth over time at the image level. However, due to its highly individualised progression, the evolution of late AMD remains poorly understood. In this work, we propose using Implicit Neural Representations (INRs) to model GA progression at the individual level in a low-data setting. Our approach generates both FAF and GA segmentation at both past and future time points. Among the comparison models, our method achieves competitive segmentation quality across different scenarios, yielding the lowest Mean Absolute Error (MAE) for the GA lesion area and the highest DICE score, without sacrificing FAF image quality. The code is available at https://github.com/SimoneSarrocco/ga-progression-with-inrs.

0.38