CS Paper Radar

Segment What It Can Draw: Querying a Text-to-Image Generator with Its Own Images for Skin Lesion Segmentation

Hongyu Cao, Guangfu Guo, Jinghan Zhang, Dakshak Keerthi Chandra, Ying Zhang, Kunpeng Liu

cs.CV cs.AI ·submitted 2026-11-04 19:20 UTC · v1

Abstract

Skin lesion segmentation without annotated masks requires the target to be specified through other cues. Existing approaches rely on dermoscopy-specific rules, annotation-derived model priors, or user-selected regions. A general text-to-image diffusion transformer offers a text interface to the target, but its concept-word attention is unreliable on dermoscopy images: in the model's attention-output space, the word-derived skin-to-lesion direction is poorly aligned with the direction separating real skin from real lesions. We show that the direction derived from generated healthy-skin and lesion images is substantially better aligned. Based on this finding, we propose GENIQ, a training-free method that constructs initial skin and lesion prototypes from generated images and refines them using the most skin-like and lesion-like patches from unlabelled dermoscopy images. It scores test patches along the refined direction and applies a gated threshold to suppress contextual spill-over around small lesions. Without inference-time annotations or additional models, GENIQ achieves Dice scores of 0.85-0.88 on three dermoscopy test sets. It outperforms zero-shot and word-based attention methods on all three sets, and a hand-designed dermoscopy pipeline in Dice and IoU on two.

Closest papers

Nearest neighbours by meaning across the whole index.

InstEditSeg: Instruction-Driven Image Editing for Polyp and Skin Lesion Segmentation

cs.CV cs.AI ·2026-09-02 ·Ziquan Liu, Zhewei Zhu, Xuyang Shi
code/data similarity 0.69
Abstract · 21 pages, 11 figures

Accurate segmentation of polyps and skin lesions is pivotal for clinical diagnosis, yet existing methods struggle with low contrast, ambiguous boundaries, and cross-domain distribution discrepancies. Discriminative networks and most diffusion-based segmentation approaches predict standalone binary masks, leaving the visual priors of large-scale pretrained generative models largely unexploited. We propose InstEditSeg, a unified generative framework that reformulates medical segmentation as an instruction-driven image editing problem. Instead of emitting a mask, the model renders a color-coded overlay on the original image, conditioned on a textual instruction, so that the edited output aligns with the natural image distribution learned by latent diffusion models and mitigates the domain gap between natural and medical imagery. To recover fine anatomical structures, we introduce DINOv3 as an auxiliary visual encoder and a DINO Feature Guidance Block that builds a multi-scale feature pyramid. The pyramid is fused into the diffusion U-Net by channel concatenation and zero-initialized convolution so that hierarchical discriminative priors can be injected without perturbing the pretrained weights. A dual-branch classifier-free guidance strategy requiring only two forward passes per denoising step reduces inference cost. On polyp and skin lesion benchmarks the framework achieves accuracy competitive with strong discriminative baselines, and it further demonstrates concrete advantages of the generative formulation: notably better cross-domain generalization on unseen data, more complete multi-lesion segmentation, instruction-conditioned task control, and sampling flexibility. We also analyze the strengths and limitations of the paradigm, including its color sensitivity and unsupported attribute-conditioned selection. Code is available at: https://github.com/wincharm001/InstEditSeg.

0.34

MDSkin-Net: Multi-Task Skin Lesion Analysis Driven by Pattern Analysis Priors and Spatial Alignment Regularization

cs.CV ·2026-09-25 ·Yijian Li, Saad Bedros, Paul Bigliardi, Mei Bigliardi Qi, Vassilios Morellas +1 more
benchmark similarity 0.67
Abstract · 13 pages 4 figures

Reliable skin lesion segmentation and classification are central to dermoscopic computer-aided diagnosis. Existing multi-task frameworks couple the two tasks architecturally without clinical knowledge, while knowledge-injecting approaches rely on the macroscopic ABCD rule, which was not designed for dermoscopy. Dermoscopic diagnosis is grounded in Pattern Analysis, a microscopic framework structured around dermoscopic features. We propose MDSkin-Net, which incorporates cue-level Pattern Analysis priors into a hybrid CNN-Transformer architecture. At its core is a Pattern Analysis-Guided Attention Module (PAGAM) comprising three priors motivated by distinct dermoscopic cues: an improved Efficient Channel Attention (iECA), a Multi-Scale Spatial Attention (MSSA), and a Biased Asymmetry Attention (BAA). We further introduce a multi-scale spatial alignment regularization (MSAR) that uses the segmentation ground-truth mask as hierarchical soft supervision, confining the classification head to lesion-localized evidence and coupling both task pathways through a shared spatial prior. Trained exclusively on the ISIC 2017 training split without external dermoscopy data, the MDSkin-Net ensemble transfers robustly under zero-shot evaluation, reaching a Dice Similarity Coefficient (DSC) of 92.38% and a melanoma AUC of 97.84%on PH2, and a DSC of 88.92% on the ISIC 2018 Task 1 test set. On the in-domain ISIC 2017 benchmark, the ensemble attains a mean Area Under the Curve (AUC) of 91.60% across the two classification tasks (melanoma and seborrheic keratosis vs. rest), and a DSC of 84.72% for segmentation. Classification remains competitive with baselines; in-domain segmentation trails single-task specialists, yet the proposed priors and alignment regularization yield representations that generalize consistently across cohorts of different scales.

0.30

Deep Learning for Biopsy-Free Subtyping of Basal Cell Carcinoma from Dermatoscopic Images

cs.CV cs.AI ·2026-09-07 ·Alexandros Papadopoulos, Chrysa Episkopou, Ioannis Sarafis, Aimilios Lallas, Anastasios Delopoulos
similarity 0.66
Abstract

Basal Cell Carcinoma (BCC) is the most common type of skin cancer, accounting for nearly 80% of skin cancer di- agnoses. Its optimal clinical management is guided by the distinct histopathologic subtype, with aggressive variants requiring more drastic measures. In current clinical practice, subtyping relies on skin biopsies, a procedure both costly and invasive. In this paper, we conduct a preliminary investigation into using deep learning for BCC subtyping, solely from a single dermatoscopic image of the lesion. Given the limited data at our disposal, we employ pre-trained vision transformers (ViTs), a state-of-the-art family of models highly effective for challenging downstream tasks with limited labeled data. Through repeated stratified k-fold cross-validation, we demonstrate that ViTs can achieve superior performance (AUC 0.784 on a dataset of 1271 dermatoscopic images of various BCC subtypes) over standard CNN-based baselines as well as previously-reported human reader perfor- mance, on the task of differentiating aggressive BCCs from other subtype families. These initial findings highlight the potential of combining deep learning and dermatoscopy to provide a biopsy- free alternative for BCC subtyping, thus aiding in improving treatment planning and patient outcomes.

0.09

BruNet: A Cross-Domain Transfer Framework for Bruise Segmentation

cs.CV ·2026-09-10 ·Qiming Wang, Richard J. Motley, Ebube E. Obi, Xianfang Sun, Paul L. Rosin
similarity 0.66
Abstract

Segmenting bruises is a challenging task in medical imaging due to limited data and annotations, diffuse boundaries, and highly variable appearance. In this work, we propose BruNet, a segmentation framework that combines a ViT-based visual encoder (a self-supervised DINOv3 or a pretrained LingBot-Vision backbone) with a SAM-based mask decoder. BruNet is trained on the HAM10000 skin lesion dataset and evaluated on a separate bruise dataset without additional fine-tuning. Although a small number of prior studies have explored machine learning and computer vision for bruise analysis, existing work has primarily focused on detection, classification, or colour analysis rather than pixel-level localisation. To the best of our knowledge, this is the first study to address automatic bruise segmentation. Our results show that BruNet outperforms CNN-based models, state-of-the-art segmentation models, ChatGPT-4o/5-assisted SAM2 zero-shot baselines, and the medical-oriented MedSAM model, demonstrating strong cross-domain generalisation to bruise segmentation.

0.06

Bridging Research and Practice: A Systematic Evaluation of Generalist and Dermatology-Specific Models in Clinical Skin Lesion Classification

cs.CV ·2026-10-02 ·Emanoel dos Santos, Kelvin Cunha, Rodrigo Mota, Fabio Papais, Thales Bezerra +6 more
top venuebenchmark similarity 0.66
Abstract · 10 pages, 1 figure, 3 tables, approved at MICCAI 2026

The application of machine learning to dermatology has grown substantially in recent years, moving beyond proof-of-concept studies toward potential applications. However, clinical dermatology remains a challenging and still open problem. Diagnostic assessment is often ambiguous, and skin lesions exhibit high variability, compounded by differences in acquisition modality, device quality, and patient demographics. These factors hinder the development of robust models suitable for safe and equitable clinical use. To support translation into practice, it is essential to systematically evaluate how contemporary models generalize across heterogeneous data sources. In this work, we benchmark a diverse set of architectures on recent dermatology datasets, spanning dermoscopic images and smartphone-based clinical photographs. We assess the robustness of recent general-purpose and medical vision-language models, as well as foundation models, and compare them against task-specific dermatology classifiers, including embedding-based approaches and convolutional neural networks. Our study provides an evaluation of model performance under distribution shifts, modality changes, and demographic variability. By quantifying the gap between current state-of-the-art models and the requirements of clinical deployment, we aim to contribute to the development of reliable, accessible, and clinically applicable AI systems for dermatology.

0.33

VL-AcneSeg: A Vision-Language Framework for Region-Aware Acne Lesion Segmentation

cs.CV cs.AI ·2026-09-28 ·Sukju Oh, Soo Ick Cho, Dae Hun Suh, Sukkyu Sun
publishedcode/data similarity 0.65
Abstract · Accepted for publication in the IEEE Journal of Biomedical and Health Informatics

Acne assessment is crucial for clinical decision-making, yet traditional grading and counting are subjective and fail to account for lesion size. While area-based assessment has emerged as a promising alternative, acne segmentation has continued to rely on general-purpose architectures. To address this gap, we propose VL-AcneSeg, a multimodal framework for acne lesion segmentation that leverages CLIP and region-level text prompts to incorporate spatial priors, enabling lesions to be localized across the whole face. Because region-level prompts indicate which facial areas contain lesions, we report a single global prompt, which requires no such information, as our primary setting. On our internal clinical dataset, VL-AcneSeg achieves a Dice score of 0.5082 and an IoU of 0.3407 under this protocol, the highest among all compared methods, including recent vision-language segmentation methods that are themselves given region-level prompts; region-level prompting raises these to 0.5296 and 0.3602. Moreover, lesion area measurements derived from our segmentation correlate with IGA scores at a level comparable to expert annotations (Pearson r = 0.719 versus 0.658). Notably, our framework maintains consistent performance across external validation datasets, performing reliably even on uncontrolled smartphone images without requiring additional training or fine-tuning. By pairing a protocol that requires no lesion-location information with area-based severity estimation, this work provides a foundation for objective acne assessment outside the clinic. Our implementation is publicly available at: https://github.com/sukjuoh/VL-AcneSeg

0.36

A Lightweight Attention-Based CNN with Saliency-Map Interpretability for Skin Cancer Detection

cs.CV cs.AI ·2026-10-27 ·Sanjana Islam Kasfia, Md. Ismiel Hossen Abir, AKM Bahalul Haque, Abir Mahmud Shahariar
similarity 0.65
Abstract

Skin cancer is among the most prevalent and life threatening forms of cancer worldwide. Early and accurate diagnosis significantly improves patient survival rates. However, manual dermoscopic assessment remains challenging due to visual similarity. This study proposes an Explainable AI-driven framework for automated binary classification of skin lesions using a lightweight attention-based convolutional neural network (CNN-Attention). The model integrates channel attention to focus on informative regions of dermoscopic images while maintaining low computational complexity. A publicly available dataset comprising 2,637 training and 660 test images was used, with balanced class weights. The proposed model achieved 87 percent accuracy with 1.95 million parameters and 7.45 MB of storage, matching the best baseline (MobileNet) while using approximately half as many parameters and half as much storage. ROC AUC and PR AUC scores reached 0.9475 and 0.9430, respectively. Additionally, saliency maps were incorporated as an explainability technique to visualize regions that influence model decisions. Comparative analysis with VGG16,VGG19,ResNet101,and MobileNet demonstrates the proposed models effectiveness in balancing classification results, computational efficiency, and interpretability.

0.09

Toward Artifact-Free Dermoscopy: Characterization of ISIC 2019 Image Defects for Deep Learning Applications

eess.IV ·2026-11-02 ·Victoria Wegley, Joshua Hoog, Tristan Crawford, Keith Miller, William Fons +8 more
similarity 0.65
Abstract · 9 pages, pre journal publication

The International Skin Imaging Collaboration (ISIC) archive offers the largest publicly available collection of dermoscopic images for skin lesion analysis. However, the ISIC 2019 dataset suffers from several acquisition artifacts such as hair occlusions, vignetting, surgical ink markings, and ruler scales, all of which compromise its utility in deep learning workflows by introducing non-biological visual structures. To address these limitations, we present a rigorously annotated version of ISIC 2019 that includes binary masks for key artifact types. Ink marks and vignetting were annotated through systematic manual review and iterative refinement using professional image editing software. For hair and ruler artifacts, we implemented a semi-automatic pipeline based on our previously developed deep learning architecture, ChimeraNet. Automated predictions were then manually corrected as needed for precision. The resulting dataset supports both the development of our own melanoma detection model and broader community research requiring artifact-mitigated dermoscopic images.

0.28