CS Paper Radar

CellSplat4D: PSF-Aware 4D Gaussian Splatting for Sparse Robotic Live-Cell Imaging

Yingda Tao, Guoyu Lu

cs.CV ·submitted 2026-10-03 01:27 UTC · v1

Abstract

A robotic microscope watching living cells cannot afford to look as often as it would like. Every volume it acquires costs photons the specimen does not get back, and time owed to other wells. What such a platform exists to produce is a record of individual cells through time: which cell is which from one volume to the next, and which cell divided into which two. Sampling sparsely breaks that record exactly where it matters, and the fault lies in the acquisition schedule rather than in the analysis software. We fill the gaps by reconstructing them, fitting a 4D Gaussian model to whatever volumes the hardware could afford. The model is a cloud of light-emitting blobs, each carrying a position, a shape and a lifetime. Being continuous in time, it renders any missing volume on demand, decoupling how often the robot analyses from how often it can afford to look. The microscope's point-spread function is measured from the data rather than inherited from acquisition metadata or left to the optimizer, because metadata inflates it and the optimizer cannot recover it at all: a wider blur around a smaller blob fits the images equally well. Each blob's lifetime is stored in frames rather than as a fraction of the recording, so that it denotes a fixed duration on any sequence. Unmeasured timesteps are supervised at coarse scale by a 3D U-Net that predicts the intermediate volume directly and estimates no motion field, since a dividing cell becomes two and no motion describes that. On two Cell Tracking Challenge sequences, a C. elegans embryo and a Chinese Hamster Ovarian (CHO), with fidelity scored per cell nucleus, our reconstruction holds the highest nucleus fidelity at every distance from an acquired frame, has the flattest decay across the gap, and best recovers focal planes it was never shown with graceful degradation across the gap.

Closest papers

Nearest neighbours by meaning across the whole index.

MicroSplat: Towards Automating 3D Small-Scale Reconstruction

cs.RO ·2026-09-28 ·Roman Mineyev, Robert Azarcon, Suvrat Jain, Chuizheng Kong, Shreyas Kousik
similarity 0.59
Abstract

Reconstructing sub-millimeter detail of small objects underpins tasks such as circuit-board defect detection, specimen digitization, and archiving. 3D Gaussian Splatting (3DGS) makes photorealistic reconstruction possible from posed images alone, but macro optics violate two conditions it depends on. The field of view (FOV) and the depth of field (DoF) shrink to millimeters, so viewpoint error can take a sample out of frame, and only a thin focal plane is sharp in frame. Existing automated macro-capture rigs handle the shallow DoF with per-view focus stacking, but typically rely on classical photogrammetry and offline, human-in-the-loop processing. We present MicroSplat, which generates a 3DGS using a low-cost consumer microscope on a manipulator arm. After a single up-front object-localization step, the system runs unattended. It performs calibration, plans collision-free viewpoints, captures and fuses a focal stack at each viewpoint, segments the object across all views, and optimizes a 3DGS model warm-started with the arm's own poses. We further compare against blur-aware 3DGS methods, which infer sharp scenes from defocused images, and find that measuring sharpness with a controlled focal sweep outperforms inferring it. We provide real and synthetic datasets (including robot and camera parameters) and the fitted 3DGS reconstructions.

0.28

The Neverwhere Visual Parkour Benchmark Suite

cs.RO cs.CVcs.LG ·2026-09-14 ·Ziyu Chen, Henghui Bao, Haoran Chang, Alan Yu, Ran Choi +10 more
publishedbenchmarkcode/data similarity 0.57
Abstract · 9 pages, 14 figures. Accepted to IROS 2026. Project page: https://ziyc.github.io/neverwhere-bench/

State-of-the-art visual locomotion controllers are increasingly capable at handling complex visual environments, making evaluating their real-world performance before deployment increasingly difficult. This work intends to narrow this train/evaluation gap by developing a collection of hyper-photo-realistic, closed-loop evaluation environments - The Neverwhere Benchmark Suite - comprised of over sixty 3D Gaussian Splatting reconstructions of urban indoor and outdoor scenes. Our goal is to encourage large-scale and reproducible robot evaluation by making it easier to create and integrate Gaussian splats-based reconstructions into simulated continuous testing setups. We also underscore the potential pitfalls of relying exclusively on 3D Gaussian-generated data for training, by providing policy checkpoints trained over multiple Neverwhere scenes and their performance when evaluated in novel scenes. Our analysis illustrates the necessity of sourcing diverse data to ensure performance. Code and data are available on the project page: https://ziyc.github.io/neverwhere-bench/.

0.54

Learning 3D biophysical cell properties from 2D images and cell-population statistics

cs.AI ·2026-09-18 ·Santiago Hernández-Orozco, Hector Zenil
similarity 0.57
Abstract

Inferring 3D cellular properties from 2D microscopy is difficult when a reference instrument reports only population statistics rather than labels for individual cells. Here we develop a population-supervised framework that maps single 2D red-cell images to latent biophysical quantities and aggregates them to mean corpuscular volume, red-cell distribution width and mean corpuscular haemoglobin. The model combines shared local inference, a biophysically structured decoder for volume and haemoglobin, learned instance weighting and device-specific calibration. We formalise conditions under which aggregate observations identify restricted instance predictors, show why population agreement does not by itself identify single-cell properties or 3D geometry, and derive the dispersion penalty induced by subset mean matching. The development dataset comprises 390 specimens and 1,105 acquisitions across six devices, with reported Pearson correlations of 0.86--0.98 against a Sysmex analyser. The framework provides a testable route from 2D images and population supervision to 3D cellular biophysics without claiming explicit 3D reconstruction.

0.19

Amortized Anchor Refinement for Deployable Continuous-Time 4D Gaussian Reconstruction

cs.CV ·2026-08-31 ·Jingong Chen, Qingwen Zhang, Sanghyeon Jun, Chulwoo Pack, Kyle Gao +1 more
benchmark similarity 0.57
Abstract · 18 pages, 13 figures

Continuous-time 4D reconstruction remains impractical on standalone XR headsets. Per-scene optimization demands deployment-infeasible compute, and lower budgets cause collapse rather than degrade gradually. Feed-forward prediction is fast, but struggle to recover scene-specific detail. We present Amortized Anchor Refinement, which uses a frozen backbone to predict an initial Gaussian representation and a short optimization to specialize it under a fixed compute budget, with a capacity floor preserving representational density. A training-free stage then applies a persistent-homology constraint to prune unstable Gaussians while preserving topologically persistent structures, and streams the resulting trajectories directly as scene flow. On the Stage-Capture benchmark, Amortized Anchor Refinement achieves 24.31$\pm$2.22dB, while our deployment experiments demonstrate reconstruction within the target budget on a single consumer GPU and playback on a standalone XR headset.

0.21

Rethinking Microscopy Generation: Co-Designed Diffusion for Biologically Interpretable Single-Cell Synthesis

cs.CV ·2026-10-24 ·Shumpei Takezaki, Duway Nicolas Lesmes-Leon, Ryoma Bise, Gillian Lovell, Bianca Migliori +2 more
published similarity 0.57
Abstract · Accepted at BMVC2026

Generative models for microscopy can already produce realistic whole-image fields of view, but offer no principled way to control how each individual cell looks. We address this gap by recasting microscopy generation at the single-cell level and designing the input controls and the model's internal representation jointly, so that both are organized along the same biologically meaningful directions. On the input side, we aggregate a long list of standard cell measurements into a small set of semantic axes (size, elongation, brightness, texture, etc.) that are each named, measurable, and traversable. On the model side, we introduce BioVAE, an encoder-decoder whose latent space is explicitly aligned with the same set of measurements. A key finding is that the benefit of biologically structuring the latent emerges only when conditioning is also moved to the semantic axes, showing that the two ends must be designed jointly. Experiments on LIVECell show that semantic-axis conditioning substantially improves generative fidelity over high-dimensional conditioning, a blind study with biologists confirms the realism of generated cells, and per-axis traversals produce stable and interpretable phenotypic changes.

0.36

LabWorld: Learning Interactive 4D Worlds for Scientific Experimentation

cs.CV ·2026-10-27 ·Suozhi Huang
similarity 0.56
Abstract · arXiv admin comment: This version has been removed by arXiv administrators due to data privacy concerns

Autonomous scientific agents must not only reason about experiments but also interact with the physical environments in which they are executed. Existing laboratory simulators typically rely on manually constructed assets and predefined procedures, while laboratory videos capture real equipment, layouts, and interactions but leave this information implicit in pixels. We introduce LabWorld, an agent-driven Real2Sim framework for reconstructing interactive 4D laboratory worlds from monocular RGB video. LabWorld recovers reusable 3D assets, a metric scene, time-varying object states, and procedure events. Multi-frame observations provide complementary evidence for asset and scene reconstruction, while motion reconstruction combines visual and geometric observations with event constraints encoding the spatial and temporal requirements of laboratory interactions. We evaluate LabWorld on real pipetting and vortexing videos collected in our laboratory against general-purpose agentic and dedicated reconstruction baselines. For pipetting, LabWorld achieves a mean Mask IoU of 0.501, exceeding the strongest baseline by 0.033. For vortexing, it achieves a mean Tube-Length-Normalized PCK-AUC of 0.638, exceeding the strongest baseline by 0.035. On five pipetting videos, event-conditioned reconstruction increases the fraction of modeled physical constraints satisfied from 14.3% (5/35) to 100% (35/35). The resulting worlds expose geometry, motion, articulation, and events as explicit, editable state for replay, inspection, and future embodied scientific systems.

0.22

Cross-Platform Benchmark of Neural 3D Reconstruction for Autonomous Laboratory Robots

cs.RO ·2026-08-26 ·Yongho Kim, Mengjiao Han, Victor Mateevitsi, Silvio Rizzi, Michael E. Papka +1 more
benchmark similarity 0.56
Abstract · This manuscript is peer-reviewed from the committees in the workshop "VAxAutoSci: Visual Analytics in the Age of Autonomous Scientific Disco…

Autonomous robots performing laboratory tasks depend on 3D reconstruction pipelines that can turn raw camera streams into actionable object representations within the latency budget of a physical control loop. Neural 3D reconstruction methods have demonstrated high-quality view synthesis, but their real-time viability across the compute platforms on which laboratory robots actually run remains poorly characterized. In this work, we present a systematic compute-platform benchmark of neural 3D reconstruction methods, evaluating NeRF and 3D Gaussian Splatting training and rendering on GPU-enabled computing devices ranging from single-board computers to server-class nodes, and place Meta's SAM3D single-image reconstruction on the same axes to quantify its latency and fidelity gap relative to per-scene optimization. Our results show that Gaussian Splatting yields higher rendering quality than NeRF at greater GPU cost, and that onboard compute is insufficient for full per-scene optimization at interactive rates. Our preliminary assessment on SAM3D indicates that it delivers plausible object geometry within seconds, but with detail mismatches that can compromise downstream manipulation. Together, these findings motivate tiered pipelines in which lightweight feed-forward reconstruction sustains the real-time perception-and-tracking loop for laboratory robots, while heavier neural reconstruction is scheduled selectively on suitable compute.

0.19

Junction Geometry Survives an Unidentifiable Blur

cs.CV physics.ins-det ·2026-09-30 ·Sohail, Sarkar, Shakuntala Baichoo
unusual similarity 0.56
Abstract · 37 pages, 13 figures, 11 tables

Thin filaments cross in microscopy, and dispersed spectra cross on X-ray detectors. At a junction, where these structures cross or branch, how many straight arms meet, and which way do they go? Unknown blur obscures the answer. At a balanced crossing, distinct Gaussian blurs can even produce the same observation law. We show that the arms can still be recovered. For separated rays of nonvanishing weight and an unknown centered Gaussian-mixture blur, a distance combining arm count, directions, relative weights and supporting lines is bounded by a constant times the Euclidean difference in cell probabilities on a sufficiently fine fixed grid. The bound is uniform over a bounded model class, including unidentifiable blurs. From $n$ positions or photon counts, the minimax error is of order $n^{-1/2}$. This rate is proved for a global estimator. In practice we fit rays and blur by constrained local likelihood, reducing the BIC count when two detectors agree on fewer arms in the microscopy studies. Across three studies with protocols fixed before evaluation, on 360 crossings and branches assembled from measured single-molecule localizations, the fit gets the arm count and every direction within $10^\circ$ 24 to 37 percentage points more often than a tuned fixed-scale Hessian ridge detector, averaged over 25 to 400 localizations. On acute angles with correct counts, mean absolute error is $3.5^\circ$ to $4.3^\circ$, versus $10.4^\circ$ to $15.5^\circ$. On Chandra spectra, 38 of 90 fits at 800 photons find all four arms within $10^\circ$; among these, median error is $0.023^\circ$.

0.29