CS Paper Radar

Single-Annotation, Multi-Hypothesis Segmentation with Semantic-Spatial Residual Banks

Armin Dadras, Robert Sablatnig, Franziska Proksa, Markus Seidl

cs.CV cs.LG ·submitted 2026-10-29 23:32 UTC · v1
published

Abstract

Multi-rater segmentation datasets show that a single image can admit several valid masks, yet most training sets expose only one annotation per image. We study single-annotation multi-hypothesis segmentation: each training image exposes one fixed mask, while the model must return a finite bank that covers held-out human annotations at test time. Our experiments reveal a representation tradeoff. Banks learned directly from images recover variations in local boundaries and extent but lose structural hypotheses, whereas frozen semantic features preserve structural grouping but underrepresent spatial variation. We introduce the Semantic-Spatial Residual Bank (SSRB), which freezes a semantic bank trained with balanced assignments and learns a residual from the image for each hypothesis. Across five medical endpoints, SSRB improves coverage of held-out masks at $K=8$ over a Probabilistic U-Net under the same one-mask constraint. On LIDC and LIDC-hard, it also produces substantially more distinct useful masks. On BSDS, it preserves the semantic bank's performance in grouping regions. Its advantage grows as more masks are returned, showing that residuals learned for corresponding heads improve coverage of local annotation variation while preserving how regions are grouped.

Comment
Accepted at WACV 2027

Closest papers

Nearest neighbours by meaning across the whole index.

Beyond Classification: Structured Supervision Aligns Visual Evidence with Medical Semantics

cs.CV ·2026-09-05 ·Hexiang Bai, Hanyang Xu, Xiaoxue Li, Xiaoliang Wu, Shangde Gao +2 more
published similarity 0.67
Abstract · 22 pages,7 figures,Accepted to BMVC 2026

Vision Transformers (ViTs) have shown immense potential in medical image analysis. However, standard pre-training via global image classification suffers from spatial collapse, where models rely heavily on background shortcuts rather than localising critical foreground lesions. To overcome this limitation and align visual evidence with precise medical semantics, we systematically investigate alternative pre-training paradigms.Specifically, we evaluate three independent forms of structured supervision: topological priors via graph self-supervision, dense pixel-level constraints via segmentation, and cross-modal semantic grounding via image-text pairs. Notably, our empirical analysis reveals that while all three forms of structured supervision successfully alleviate the global pooling bottleneck and steer visual attention towards foreground regions, image-text alignment achieves the most superior performance. By embedding high-dimensional diagnostic logic, the cross-modal approach not only anchors attention on precise visual evidence but also enables profound abstract reasoning. Extensive experiments demonstrate that this semantically enriched pre-training fundamentally enhances the model's feature representation. Consequently, when fine-tuned for downstream clinical classification tasks, our models achieve superior accuracy and yield highly interpretable attention maps focused on true pathological features, vastly outperforming vanilla classification baselines.

0.32

Less Is More in the Long Tail: Stage-Adaptive Sample Selection for Annotation-Efficient Dense Prediction

cs.CV ·2026-09-22 ·Xiaofei Du, Lei Zhang, Shuyu Yan, Manning Wang, Zhijian Song
similarity 0.66
Abstract

Deep learning performance generally improves with increasing training data, yet this scaling is fundamentally constrained by annotation cost in large-scale dense prediction tasks with long-tailed category distributions, where pixel- or voxel-level annotation is prohibitively expensive. We propose SASS (Stage-Adaptive Sample Selection), a stage-adaptive data-selection framework for pool-based active learning in long-tailed dense prediction. SASS combines three components: label-free self-supervised gradient scoring, prior-guided category rebalancing with validation-driven feedback, and stage-adaptive acquisition aligned with model training dynamics. This design avoids candidate ground-truth masks during gradient scoring while making acquisition responsive to long-tail imbalance and evolving representations. We evaluate SASS on a multimodal 3D medical segmentation testbed comprising over 100,000 samples spanning 108 anatomical structures. SASS recovers 98.3% of full-dataset performance with a 40% training-pool annotation budget, outperforming BADGE by 5.1 percentage points. Moreover, SASS exhibits a statistically supported less-is-more pattern, surpassing full-dataset training at the Hard-group level and, at the structure level, for the pancreas and gallbladder. More broadly, SASS shows that annotation-efficient learning depends not only on which samples are selected, but also on how the annotation budget is distributed across categories and when model-derived scores begin to guide selection.

0.18

Sam-Correction: Fully Adaptive Label Noise Reduction for Medical Image Segmentation

cs.CV physics.med-ph ·2026-10-23 ·Takuro Shimaya, Masahiro Saiko
publishedhighly cited similarity 0.66
Abstract · 5 pages, accepted to IEEE International Symposium on Biomedical Imaging (ISBI) 2024

The quality of the teacher label has a dramatic impact on the outcome of the machine learning process. Medical image segmentation often suffers from a noisy ground truth (GT) mask due to difficulties in consistent annotation. Although a recent study has proposed a promising method to curate the noisy GT mask, this requires additional well-curated GT masks that are not always accessible without the aid of multiple medical experts. As an alternative, we propose SAM-Correction, which utilizes a foundation model to correct the noisy GT mask in a fully adaptive manner instead of manually preparing well-curated masks. In our approach, we progressively mitigate label noise via the segment anything model (SAM), where prompts are given from another segmentation network that we train concurrently. Experiments with various types of label noise derived from actual human annotations demonstrate that SAM-C achieves precise noise reduction and high-performing segmentation models trained thereby. Our method paves the way for robust model training even when clean labels are entirely unavailable.

0.48

FoRIS: Progressive Foreground Refinement for Training-Free In-Context Segmentation

cs.CV ·2026-09-03 ·Ming Hu, Jianfu Yin, Mingyu Dou, Miaomiao Zhang, Yao Wang +3 more
code/data similarity 0.65
Abstract

In-Context Segmentation (ICS) aims to precisely segment arbitrary semantic concepts, such as objects or parts, given one or a few annotated visual exemplars. In this paper, we revisit ICS from a more classical segmentation perspective, viewing it as a coarse-to-fine progressive refinement process. Rather than directly predicting the final mask through reference-query matching, we progressively refine the segmentation from coarse and ambiguous foreground responses to precise and complete foreground structures. Building upon this perspective, we propose a training-free in-context segmentation framework, termed FoRIS. Specifically, FoRIS consists of three key stages: Foreground Purification, Foreground Localization, and Foreground Consolidation, which progressively suppress background distractions, localize discriminative target regions, and recover complete foreground structures through semantic aggregation. Experimental results demonstrate that FoRIS achieves SOTA performance across semantic and part segmentation tasks, with average improvements of 4.5 and 4.8 mIoU points over existing approaches in the 1-shot and 5-shot settings, respectively. Code: https://github.com/Xi-Mu-Yu/FoRIS.

0.32

GazeRefine: Expert Gaze as a Test-Time Prompt for Training-Free Medical Image Segmentation

eess.IV cs.AIcs.CVcs.HC+1 ·2026-09-01 ·Mohammed Oussama Benyahia, Marouane Tliba, Mohamed Amine Kerkouri, Taifour Yousra, Bin Wang +8 more
publishedcode/data similarity 0.64
Abstract · 9 pages, 5 figures. Accepted at MICCAI Workshop 2026

Medical image segmentation remains difficult to scale because high-performing methods typically rely on dense expert annotations and task-specific training. We introduce GazeRefine, a training-free framework that uses gaze as an inference-time prompt for zero-shot medical image segmentation. Sparse, duration-weighted fixations are converted into foreground and background priors that initialize semantic prototypes in frozen DINOv3 feature space. These prototypes are iteratively refined through foreground-background discrimination, feature-space affinity propagation, and anchoring to the initial gaze guidance, allowing segmentation to extend beyond directly fixated regions while limiting semantic drift. GazeRefine requires no segmentation masks, fine-tuning, adapters, prompt encoders, or gradient updates. We evaluate the method on gaze-annotated polyp segmentation and prostate MRI segmentation. The results show strong performance on colonoscopy images and competitive performance on prostate MRI, supporting gaze-guided prototype refinement as a promising approach for segmentation-label-efficient, human-in-the-loop medical image segmentation. Our tools and code can be found in the following repository: https://github.com/MohammedOussamaBEN/GazeRefine.git

0.73

CorrCLIPv2: Harnessing Unlabeled Images for Open-Vocabulary Semantic Segmentation

cs.CV ·2026-10-17 ·Dengke Zhang, Fagui Liu, Quan Tang, C. L. Philip Chen
top venuecode/data similarity 0.64
Abstract · 25 pages, 12 figures, 12 tables. Published in International Journal of Computer Vision

Open-vocabulary semantic segmentation aims to classify pixels without being restricted to a fixed set of categories. Although Contrastive Language-Image Pre-training (CLIP) demonstrates remarkable potential for open-vocabulary recognition, its segmentation capability is impaired by noisy inter-class patch interactions. To this end, this work proposes a unified training-free object-centric framework, named CorrCLIPv2, from three complementary aspects of patch correlation, feature representation, and output regularization. Specifically, for patch correlation, we leverage class-agnostic mask generators to explicitly disentangle inter-class interactions. Moreover, we utilize spatially-coherent features from self-supervised models to compute attention scores to mitigate mask inaccuracies. Regarding feature representation, we devise a refinement mechanism to simultaneously strengthen spatial details and semantic representation. Within the output space, we unify predictions via voting in each mask to enhance spatial consistency. To mitigate contextual bias and enhance prediction accuracy of objects, we further extend the framework by incorporating a cross-image object retrieval paradigm, which incurs minimal computational overhead. We build a memory bank of CorrCLIP-enhanced features from unlabeled images and implement a regime of multi-CLIP ensemble. By aggregating semantically consistent object features across diverse contexts retrieved from the memory bank, CorrCLIPv2 mitigates context-induced interference and obtains more robust predictions. Extensive experiments demonstrate that our approach establishes a new state-of-the-art on 10 standard benchmarks. Remarkably, it even surpasses fully supervised open-vocabulary methods across 22 multi-domain datasets. Code is available at: https://github.com/zdk258/CorrCLIP.

0.44

MARS-CLIP: Multi-Resolution and Attention Refined Zero-Shot Image Segmentation

cs.CV ·2026-09-08 ·Nagito Saito, Shintaro Ito, Koichi Ito, Takafumi Aoki
published similarity 0.64
Abstract · Accepted to IEEE International Conference on Image Processing (ICIP) 2026

Contrastive Language-Image Pre-training (CLIP) has demonstrated impressive capabilities in zero-shot transfer but often struggles with dense prediction tasks due to low spatial resolution and the loss of structural information. To address these limitations, we propose MARS-CLIP (Multi-resolution and Attention Refined Segmentation for CLIP), a novel framework for zero-shot semantic segmentation. Our approach introduces two key strategies: (i) a multi-resolution feature extraction module that fuses local fine-grained features with global context to overcome input resolution constraints, and (ii) an attention refinement mechanism that injects spatial and color biases from intermediate layers into the final self-attention block to accurately restore object boundaries. A set of experiments on six public datasets demonstrates that MARS-CLIP significantly outperforms state-of-the-art methods.

0.27

MedPixel: A Unified Pixel-Language Model for Medical Reasoning and Segmentation

cs.CV cs.AI ·2026-08-10 ·Haoyu Yang, Meixing Shi, Zengjie Chen, Haoran Sun, Haitao Leng +3 more
code/data similarity 0.63
Abstract

Reliable medical image understanding requires models to connect clinical language and visual reasoning with pixel-level grounding. Yet medical vision-language models often lack precise localization, whereas medical segmenters typically rely on explicit target categories or precise spatial prompts. This divide is reinforced by a supervision mismatch: segmentation datasets provide precise masks but little language supervision, whereas medical vision-language data rarely pair language with dense spatial annotations. To address this gap, we present MedPixel, a unified medical pixel-language model built around a shared language--mask interface. To provide scalable supervision, we introduce MedPLG-440K, comprising approximately 440K pixel-language task samples constructed through a clinically motivated synthesis process without external LLM annotation. MedPixel is trained with joint multi-task supervised fine-tuning followed by Pixel-Level Preference Optimization, which uses ground-truth masks as offline verifiers to derive response preferences from mask quality. MedPixel supports a broad spectrum of tasks spanning explicit grounding, implicit reasoning, spatial interaction, grounded explanation, and medical VQA. Across this task spectrum, MedPixel achieves strong performance in both pixel-level prediction and response generation, together with effective zero-shot transfer to external grounding benchmarks and robustness to imperfect spatial prompts. Code and model checkpoints will be released at https://github.com/yhy-whu/Medpixel.

0.24