CS Paper Radar

SomaNet: Weakly Supervised Learning for Instance Soma Segmentation in 3D Electron Microscopy with Partial Annotations

Mohammad Khateri, Morteza Ghahremani, Jussi Tohka, Alejandra Sierra

eess.IV cs.CV ·submitted 2026-09-19 13:37 UTC · v1
code/datahot direction

Abstract

Soma instance segmentation, i.e., identifying and delineating individual cell somas as distinct instances, is crucial for cellular analysis and connectomic reconstruction. Three-dimensional electron microscopy (3D EM) provides nanometer-scale resolution for capturing fine-grained soma morphology. However, dense instance-level manual annotation is prohibitively costly, limiting the scalability of fully supervised methods. To address this challenge, we propose SomaNet, a weakly supervised framework for 3D EM soma instance segmentation under partial annotation constraints. SomaNet adopts a teacher--student learning paradigm tailored to partial labels. The teacher is trained using partially annotated data to generate pseudo-labels, while the student jointly learns from the partial ground-truth annotations and the generated pseudo-labels, progressively recovering dense instance segmentations. To accurately delineate cell somas under limited supervision with varying instance counts, SomaNet incorporates affinity learning, which encourages high similarity within instances and low similarity across instance boundaries. Semantic-guided affinity decoding and 2D-to-3D reconstruction then produce volumetrically consistent 3D soma instances while preserving the large receptive fields of 2D backbones. The framework is architecture-flexible and supports diverse backbones, including vision transformers and foundation models, enabling direct transfer of pretrained visual representations to volumetric EM segmentation. Experiments on 3D EM brain datasets demonstrate that SomaNet achieves accurate and robust soma instance segmentation across regions with diverse soma morphologies under partial annotation. Code is available at https://github.com/mkhateri/SomaNet.

Comment
10 pages, 7 figures, 1 table,

Closest papers

Nearest neighbours by meaning across the whole index.

Patch-based Querying Identifies Structures of Interest in Electron Microscopy

cs.CV ·2026-10-05 ·Niels Vyncke, Nicolas Nadisic, Yvan Saeys, Aleksandra Pižurica
published similarity 0.65
Abstract · 41 pages, 20 figures, 5 tables. Accepted for publication in Computers in Biology and Medicine

Volume electron microscopy (vEM) has emerged as an essential sensing technique in biomedical research, allowing the three-dimensional imaging of biological cells and tissues at nanometer-scale resolution. The ability to generate extensive datasets has reached the limitations of downstream analysis processes, which depend significantly on the intervention of human experts for preprocessing and annotation. We propose an efficient and reliable patch-based retrieval framework based on self-supervised learning of local image descriptors to locate self-similar structures in vEM datasets. Given a few manual annotations of a given cellular structure, our method can retrieve similar structures across the EM volume. Our framework is interactive, allowing the human expert to refine the search queries and retrieve relevant image patches quickly and using little labeled data. Experiments on real-world vEM images of biological tissues demonstrate that our framework can reliably identify relevant cellular structures, generalize across different organelles and acquisition modalities, and substantially reduce the search space for downstream analysis.

0.38

Unsupervised Learning of Cell Instances with Generative Routing Pyramids

cs.CV cs.LGq-bio.QM ·2026-08-17 ·Ziwen Liu, Martin Weigert
publishedcode/data similarity 0.65
Abstract · 15 pages, 4 figures; ECCV 2026 Workshop BIC

Identifying and representing object instances such as cells or nuclei is a common task in microscopy image analysis. Established machine learning workflows typically use supervised detection or segmentation followed by feature extraction or classification, which requires manual annotations and treats instance segmentation and cell representation as separate stages. We describe a new unsupervised method for cell instance segmentation and phenotypic classification from unlabeled microscopy images. Our method is based on reconstructing each image using a coarse-to-fine routing pyramid that associates pixels with spatially sparse latent sources. The resulting pixel-to-latent associations yield instance masks, while the source latents encode cell morphology. We demonstrate competitive performance in instance segmentation across diverse cell morphologies and imaging modalities, as well as generative modeling of cellular phenotypes under perturbations. Source code and checkpoints are available at https://github.com/weigertlab/routing-pyramids.

0.49

The microscope is the mask: privileged views and labels from a cryo-ET forward model

cs.CV cs.LGmath.OC ·2026-09-03 ·Bogdan Toader, Kiarash Jamali, Tanmay A. M. Bharat, Sjors H. W. Scheres
benchmark similarity 0.63
Abstract · 18 pages, 7 figures

We explore the use of simulated data for training a model for protein annotation in crowded cryo-electron tomography volumes reconstructed from images collected at limited tilt angles and severely corrupted by the measurement operator. Firstly, we leverage the corruptions imposed by the forward model to generate domain-specific augmented paired views of the exact same scene for an invariance objective integrated into the LeJEPA self-supervised training framework. Secondly, we use additional information from the simulation pipeline such as the positions and identity of proteins in the simulated volumes to inform the architecture of the model and the loss function, so that semantic information is localised at protein positions in the resulting dense feature volume. The resulting model, CARNIVAL, is evaluated without finetuning on classification and detection tasks in real tomograms, using a benchmark dataset containing multiple protein types and two tomogram processing types. We show that CARNIVAL outperforms a state-of-the-art model trained using a contrastive objective on simulated data but without forward model-based paired views or privileged information.

0.25

AutoRASOR: Autonomous Rapid Scanning Electron Microscope Operator

eess.IV cond-mat.mtrl-sci ·2026-09-18 ·Kevin Zhang, Mohammad Taha, Rafael Espinosa Castañeda, Yutong Liu, Yin Zhu +5 more
similarity 0.63
Abstract

Scanning Electron Microscopy (SEM) is a foundational technique for characterizing material microstructure, which dictates many fundamental physical and chemical properties. With the rise of Self-Driving Labs (SDLs), samples are now synthesized in large batches, demanding equally high-throughput, autonomous characterization. Existing automated electron microscopy pipelines are task-specific: they detect pre-defined features, optimize known properties, or require prior knowledge of the sample, restricting each pipeline to the material system it was built for. We introduce AutoRASOR, a task-agnostic autonomous SEM pipeline that captures the multi-scale morphology of an unknown sample without domain-specific pre-training, fine-tuning, or human prompting. AutoRASOR embeds micrographs in real time with a vision foundation model (VFM), DINOv3, and selects regions of interest (ROIs) through two complementary policies: Latent Farthest Point Sampling (LFPS) and active learning on morphological ambiguity, which is the conditional variance of unresolved fine-scale features given lower magnification appearance. Active learning on ambiguity consistently captures more diverse and rare morphologies than random ROI selection, while LFPS reliably recovers the specimen's morphological distribution within a limited capture budget, tested both on real SEM micrographs and synthetic phase-field images. By shifting autonomous characterization from pre-defined, task-specific targeting of features to morphological survey, AutoRASOR generates information-rich multi-scale datasets for diverse downstream tasks. Without the need for domain-specific pre-training or prior material assumptions, this framework can be deployed out-of-the-box to characterize novel materials produced by SDLs.

0.31

MixSIS: Mixed-supervised instance segmentation

cs.CV ·2026-10-27 ·Christian Wilms, Emre Gülsoylu, Sven Schreiber, Simone Frintrop
publishedcode/data similarity 0.62
Abstract · Accepted at BMVC 2026. Code available at https://github.com/chwilms/MixSIS

Weakly-supervised instance segmentation (WSIS) based on either box-, point-, or image-level annotations has been proposed to overcome the labor-intensive annotation process in fully-supervised instance segmentation. Despite decent results, most WSIS methods only focus on a single annotation level, ignoring that some objects are easy to spot from point annotations, while others require box or even mask annotations. Therefore, we propose the Mixed-Supervised Instance Segmentation (MixSIS) task, which allows a flexible mix of annotation levels (image, point, box, and mask) within a dataset. MixSIS has a high practical relevance, as it allows optimizing the annotation process by annotating challenging objects in more detail, while saving time on simpler instances with less detailed annotations. To address MixSIS, we propose AnnotationsMixer (AnnsMixer) as a baseline. AnnsMixer is a truly mixed-supervised instance segmentation system, which generates pseudo mask annotations from all annotation levels, even mixed within a single image. We evaluate AnnsMixer on both WSIS and MixSIS tasks, largely outperforming state-of-the-art methods. In three case studies, we also demonstrate how the unique features of MixSIS and AnnsMixer can substantially reduce the annotation time (-90%) or improve the results (+34%) in instance segmentation with a simple annotation rule or leveraging heterogeneous data. Our code is available at https://github.com/chwilms/MixSIS

0.36

Feature Puzzles: Single-Image Supervision for Multi-View 3D-Aware Segmentation

cs.CV ·2026-11-04 ·Jiahao Ma, Qiang Zhang, Miaomiao Liu, David Ahmedt-Aristizabal, Chuong Nguyen
code/data similarity 0.61
Abstract · Feed Forward 3D Reconstruction and Spatial Understanding, Multi-view Promptable Segmentation, Instance Tracking

Humans can easily recognize the same objects across different viewpoints of a 3D scene, whereas teaching models such multi-view-consistent perception typically requires videos or calibrated multi-view captures, which are considerably more expensive and less diverse than large-scale monocular data.~In this paper, we introduce Feature Puzzles, a novel framework for learning multi-view-consistent segmentation using only single-image semantic segmentation supervision. Specifically, we propose a simple yet effective pipeline that constructs multi-view-consistent segmentation pseudo-labels through image cropping and geometric transformations. We further introduce a lightweight, plug-and-play network that leverages the multi-view-consistent image features learned by existing pretrained 3D foundation models and the generated pseudo-labels to perform multi-view-consistent segmentation. Importantly, our flexible framework supports various types of prompts provided from one or multiple views and can be readily extended to open-vocabulary 3D segmentation. Under this single-image supervision, our feed-forward model achieves the best mIoU on NVOS, remains competitive on SPIn-NeRF and ScanNet++ against baselines using per-scene optimization or in-domain multi-view training, and demonstrates zero-shot transfer to open-vocabulary 3D segmentation. Visual comparisons are available on our project page: https://featurepuzzles.github.io/

0.25

A Fully Automatic Pipeline for 3D Dendrite Instance Segmentation in SBF-SEM

cs.CV ·2026-10-02 ·Zewen Zhuo, Ilya Belevich, Eija Jokitalo, Alejandra Sierra, Jussi Tohka
publishedcode/data similarity 0.61
Abstract · Accepted at 2026 IEEE-EMBS Conference on Biomedical Engineering and Sciences (IECBES)

Accurate three-dimensional (3D) reconstruction of individual dendrites in serial block-face scanning electron microscopy (SBF-SEM) is essential for quantifying structural plasticity in the brain, yet manual annotation at scale is infeasible. We present a fully automatic pipeline for 3D dendrite instance segmentation that unifies YOLOv6-guided Segment Anything Model (SAM) prompting on downsampled slices, iterative two-dimensional mask refinement, random forest 3D instance linking, and instance-aware high-resolution refinement using nnU-Net at native resolution into a single system requiring no manual prompting at inference. Applied to hippocampal CA1 SBF-SEM datasets from a control rat and a pilocarpine- induced epileptic rat, our pipeline reconstructs coherent, well- separated dendrites with high semantic accuracy (Dice 0.93 and 0.91) and strong instance-level performance on control tissue, while analysis of the more challenging epileptic tissue identifies instance recognition in dense regions as the principal remaining limitation. The high-resolution refinement stage recovers thin dendritic protrusions, providing a basis for downstream spine- level analysis. Code is available at https://github.com/ ZE-WEN/dendrite-3d-instance-seg.

0.39

Learning 3D biophysical cell properties from 2D images and cell-population statistics

cs.AI ·2026-09-18 ·Santiago Hernández-Orozco, Hector Zenil
similarity 0.60
Abstract

Inferring 3D cellular properties from 2D microscopy is difficult when a reference instrument reports only population statistics rather than labels for individual cells. Here we develop a population-supervised framework that maps single 2D red-cell images to latent biophysical quantities and aggregates them to mean corpuscular volume, red-cell distribution width and mean corpuscular haemoglobin. The model combines shared local inference, a biophysically structured decoder for volume and haemoglobin, learned instance weighting and device-specific calibration. We formalise conditions under which aggregate observations identify restricted instance predictors, show why population agreement does not by itself identify single-cell properties or 3D geometry, and derive the dispersion penalty induced by subset mean matching. The development dataset comprises 390 specimens and 1,105 acquisitions across six devices, with reported Pearson correlations of 0.86--0.98 against a Sysmex analyser. The framework provides a testable route from 2D images and population supervision to 3D cellular biophysics without claiming explicit 3D reconstruction.

0.19