CS Paper Radar

Learning Pathology-Sensitive Representations for Unsupervised Retinal OCT Anomaly Detection

Tania Haghighi, Sina Gholami, Behafarin Emam, Jennifer I Lim, Theodore Leng, Atalie C Thompson, Hamed Tabkhi, Minhaj Nur Alam

cs.CV cs.LG ·submitted 2026-10-23 16:50 UTC · v2

Abstract

Deep learning approaches for retinal optical coherence tomography (OCT) predominantly rely on predefined disease categories, limiting their ability to detect rare or previously unseen abnormalities. While visual foundation models provide transferable representations, their effectiveness in identifying pathological deviations from normal retinal anatomy remains insufficiently understood. In this work, we introduce an unsupervised framework for retinal OCT anomaly detection and localization that adapts pretrained visual representations to normal retinal anatomy through anatomy-aware reconstruction and contrastive triplet learning guided by synthetic structural perturbations, without requiring disease-specific annotations. We systematically investigate the influence of pretraining strategies, anatomy-aware representation adaptation, and anomaly scoring mechanisms on the detection and localization of unseen retinal abnormalities. Our findings demonstrate that adapting pretrained representations to normal retinal structure improves anomaly sensitivity, while pixel-space reconstruction errors and feature-space deviations provide complementary signals for identifying pathological changes. These findings underscore the importance of anatomy-aware representation learning and offer insights into developing generalizable retinal anomaly detection frameworks beyond predefined disease categories.

Closest papers

Nearest neighbours by meaning across the whole index.

CNN-Based Retinal OCT Classification: Research Gap Analysis and a From-Scratch Baseline, ED126-A

cs.CV cs.LG ·2026-11-01 ·Shkelqim Sherifi
benchmark similarity 0.69
Abstract · 18 pages, 26 figures

Automated classification of retinal pathologies from Optical Coherence Tomography (OCT) images remains challenging because diagnostically relevant morphological patterns may exhibit substantial inter-class similarity. Although high classification accuracies have been reported on benchmark OCT datasets, cross-study comparison is limited by heterogeneous evaluation protocols and by the limited characterization of compact convolutional neural network (CNN) baselines trained from random initialization. This study addresses this gap through two contributions. First, it provides a focused review of CNN-based retinal OCT classification and identifies methodological considerations relevant to controlled benchmarking. Second, it presents ED126-A, a purpose-built four-class CNN baseline trained from scratch under a documented experimental protocol on the Kermany labeled OCT dataset. The model contains 1,241,380 total parameters, including 1,239,460 trainable parameters, and is evaluated using training-dynamics analysis, class-wise performance metrics, confusion-matrix analysis, confidence analysis, t-SNE representation-space visualization, Grad-CAM, hierarchical feature-map analysis, and one-vs-rest ROC/Precision-Recall analysis. On the image-level held-out test set, ED126-A achieved 95.03% accuracy, with a macro-F1 of 0.9205 and a weighted-F1 of 0.9521. The principal class-conditional limitation occurred for Drusen, which attained a recall of 0.9605 but a precision of 0.7493 and an F1-score of 0.8419. The confusion matrix identified Normal to Drusen and CNV to Drusen as major error channels. These results support the use of ED126-A as a documented from-scratch reference baseline and provide an empirical basis for controlled evaluation of transfer learning, increased depth, attention, multi-scale feature fusion, and calibration in subsequent studies.

0.30

CAMEL: Cross-Architecture Masked Embedding Learning for Ultra-Lightweight Retinal Foundation Models

cs.LG cs.AIcs.CV ·2026-10-29 ·Andre Guarnier De Mitri, Giovani Decico Lucafo, Miller Matheus Lima Anacleto Rocha, Diego Furtado Silva
similarity 0.68
Abstract · 8 pages, 6 figures, pre-print

Recent progress in deep learning has substantially advanced automated grading of ophthalmologic pathologies, supported by large annotated datasets and improved representation learning. However, a significant gap remains between state-of-the-art performance and clinical scalability. In particular, foundation models have reshaped image-based medical screening and diagnosis. These large-scale architectures trained on extensive datasets learn rich, generalizable feature representations that can be adapted to a wide range of downstream tasks. In ophthalmology, RETFound, one of the most relevant foundation models for retinal images, performs strongly across classification, segmentation, and risk prediction tasks. Despite their effectiveness, foundation models are often computationally prohibitive, hindering their deployment in real-world clinical environments where hardware constraints and latency are critical. To bridge this gap, we introduce CAMEL (Cross-Architecture Masked Embedding Learning), a novel cross-architecture distillation framework that uses an asymmetric masking strategy during embedding learning to effectively transfer knowledge from massive teacher foundation models to compact, lightweight student models. Our experimental evaluation across four datasets shows that CAMEL preserves accuracy while drastically reducing computational complexity, enabling lightweight yet highly accurate models suitable for resource-limited clinical settings.

0.21

Where to Predict in Retinal OCT? Anatomy-Guided Target Selection for I-JEPA

cs.CV ·2026-11-12 ·Yunchu Feng, Wai Tak Lau, Kaveri A. Thakoor
published similarity 0.66
Abstract · 18 pages. Accepted as a poster at the GenAI for Health Workshop at NeurIPS 2026 (non-archival)

Self-supervised learning aims to learn meaningful representations from unlabeled data, potentially allowing models to capture subtle anatomical variations across patients and reduce the amount of labeled data needed. Image-based Joint-Embedding Predictive Architecture (I-JEPA) learns by predicting target-region representations from visible context rather than reconstructing pixels. However, its uniform target placement does not account for the anatomical structure of retinal OCT, where retinal tissue occupies only part of each B-scan. We investigate anatomy-guided target selection using image intensity and predicted retinal segmentation while retaining the I-JEPA architecture and latent prediction objective. We compare anatomy-guided rectangular targets, anatomy-shaped targets and coverage-constrained placement with uniform masking, and evaluate the learned representations using linear probing for glaucoma classification on FairVision. We found that rectangular targets guided by image intensity achieved the highest linear-probe AUC in our comparisons. We show that anatomically meaningful target selection is promising for retinal OCT representation learning, while greater anatomical specificity did not consistently improve performance.

0.35

Domain-Specific Self-Supervised Representation Learning for Retinal Fundus Classification

cs.CV cs.LG ·2026-08-27 ·Bekzat Nurlanbekova, Fung Fung Ting
similarity 0.66
Abstract

Annotated medical images remain scarce and costly to obtain, which motivates self-supervised learning (SSL). We present a controlled empirical study of two SSL frameworks, SimSiam and SimCLR, for retinal fundus classification, asking whether retina-specific augmentations (Graham's local-mean subtraction and CLAHE) improve representations beyond standard augmentation and grayscale conversion. ResNet-18 backbones are pretrained on 48K unlabeled fundus images and evaluated by linear probing and fine-tuning on EDID (multi-disease) and RetinaMNIST (DR grading). On EDID, grayscale conversion accounts for most of SimSiam's linear-probe gain, and Graham or CLAHE add little on top of it. After fine-tuning, SimSiam with Graham+Gray exceeds an ImageNet-supervised baseline on RetinaMNIST (+3.0 accuracy points) but only matches it on EDID. Retina-specific augmentation therefore helps inconsistently and should be validated per SSL method and task.

0.14

Exploring the Latent Structure of AMD Severity Using Retinal Foundation Models

eess.IV cs.CV ·2026-10-09 ·Niccolò Fineschi, Jovana Panic, Atsushi Watanabe, Hideaki Hayashi, Kazuya Nishimura +5 more
published similarity 0.65
Abstract · 7 pages, 5 figures. Accepted for publication at IEEE EMBS BHI 2026

Retinal foundation models have demonstrated strong performance across a wide range of ophthalmic tasks, yet relatively little is known about how disease-related information is represented within their learned embeddings and whether this distribution influences downstream prediction performance. In this work, we use Age-related Macular Degeneration (AMD) as a case study to investigate how disease severity is represented within the embedding space of RET-CLIP, a retinal vision-language foundation model, using bilateral fundus images from the AREDS dataset, benchmarked against non-retinal encoders. We first characterize the distribution of AMD severity within the learned representation space and then evaluate how this structure influences classification performance across three clinically relevant task formulations of increasing granularity. Finally, we analyze the most influential latent dimensions combining feature attribution, statistical phenotype association, and qualitative image analyses. Our results show that RET-CLIP represents AMD severity as a continuous spectrum rather than a set of discrete categories, with healthy and advanced disease stages exhibiting clearer separation than intermediate ones. This organization was reflected in downstream performance, which decreased from a balanced accuracy of 0.844 and 0.814 for binary AMD and Late-AMD detection respectively to 0.446 for six-class grading. Furthermore, the most influential latent dimensions were statistically associated to distinct retinal phenotypes, including drusen accumulation, pigmentary alterations, and advanced retinal lesions. These findings suggest that the structure of retinal foundation-model representations influences the difficulty of clinically relevant prediction tasks and demonstrate the potential of representation-level analysis for understanding disease organization and improving interpretability.

0.18

Evaluation of Clinically Steerable Retinal Image Generation from Foundation Model Latent Spaces

cs.CV ·2026-08-13 ·Zuzanna A. Wakefield-Skórniewska, Bartłomiej W. Papież
published similarity 0.64
Abstract · MICCAI 2026 Workshop SASHIMI Submission

Medical foundation models learn latent representations of clinically meaningful phenotypes, yet their ability to support controllable image generation remains largely unexplored. We evaluate four retinal foundation models within the representation tokenizer framework and examine whether demographic and clinical information encoded in latent representations from foundation models is preserved during synthetic image generation. We show that generated representations and images faithfully inherit phenotype information when evaluated within their originating foundation models, consistently outperforming conventional latent diffusion on multiple downstream prediction tasks. However, these gains largely disappear when evaluated using classifiers trained on real images, revealing a previously uncharacterised synthetic-to-real representation gap. These findings demonstrate that foundation-model latent spaces provide a powerful substrate for controllable retinal synthesis while highlighting the need to better align synthetic representations with real-image distributions.

0.17

When Does Unsupervised Learning Succeed or Fail? A PoS Perspective on Reconstruction-Based Anomaly Detection

cs.LG ·2026-09-23 ·Mehmet Yamaç, Yagmur Mustu, Muhammad Numan Yousaf, Lei Xu, Marcel van Gerven
similarity 0.64
Abstract · 39 pages, 8 figures, and 27 tables, including appendices

Reconstruction-based unsupervised learning can fail in two opposing ways: a model may reconstruct anomalies too accurately or discard valid nominal variation. Using the Pursuit of Subspaces hypothesis, we characterize these failures through the meet, union, and join geometries induced by the nominal components. Excess learned range produces join blindness, while insufficient capacity produces meet preference and loss of nominal fidelity. We show that the compact nominal union is optimal among nominal faithful ranges and generally requires a nonlinear reconstruction map. Based on this geometry, we introduce Dynamic Push and Pull, which learns from controlled perturbations without anomaly labels, and nested manifold carving, which applies the same principle recursively in latent space. Experiments confirm the predicted changes in latent geometry across every tested Push and Pull configuration. The proposed methods improve reconstruction-based anomaly detection across standard benchmarks and unseen image degradations, while also improving pretrained ECG representations for downstream classification. These results connect reconstruction failures to identifiable geometric conditions and provide practical mechanisms for learning compact representations.

0.25

A Visual Grounding Benchmark for Retinal OCT with Multimodal Large Language Models

cs.CV ·2026-10-22 ·Ronald Fecso, José Morano, Botond Fazekas, Paul Widmann-Sedlnitzky, Dorota Himel +4 more
benchmarkcode/data similarity 0.64
Abstract · Under review at npj digital medicine

Multimodal large language models (MLLMs) can answer clinical questions about medical images, but trustworthy use requires visual grounding: models should localize the pathology they report rather than rely on spurious shortcuts. Whether MLLMs can visually ground clinical findings in retinal optical coherence tomography (OCT), the most widely used imaging modality in ophthalmology, remains largely unexplored. To address this gap, we introduce RetinaGroundBench, the first benchmark for biomarker visual grounding in retinal OCT. It comprises 815 B-scans exhaustively annotated by ophthalmologists with 4,537 bounding boxes covering 13 biomarkers across 8 diseases. Because MLLMs often fragment a single lesion into multiple partial predictions, we propose AggIoU, a metric that evaluates whether the union of overlapping predicted boxes covers the annotated region. Across 27 open-source MLLMs (0.8-38B parameters), we find that visual grounding remains largely unsolved: the best general-purpose model achieves a macro F1 of just 23.7%, with consistently low recall, despite reaching up to 73% balanced accuracy on biomarker presence questions. To narrow this gap, we developed RetGrounder, an MLLM fine-tuned using an OCT-specific strategy integrating grounding and textual information. Despite its relatively small size (8B), RetGrounder improves grounding F1 to 33.9%, a 43% relative improvement. Our benchmark provides a reference point for measuring progress in the field of OCT diagnosis with MLLMs.

0.30