CS Paper Radar

USB Gym: A Gymnasium Testbed for Closed-Loop Ultrasound View Matching

Thanapong Intharah, Zhichao Gao, Yiliang Liu, Hao Dong

cs.CV cs.LGcs.RO ·submitted 2026-11-05 17:34 UTC · v1
benchmarkcode/data

Abstract

Background and Objective: Our main goal is ultrasound guidance: moving a probe until its image matches a target view. Published methods for each part of a guide are evaluated in separate simulators. We asked whether these methods, combined into one guide, are good enough. Methods: We built USB Gym, a testbed whose standard engine simulates ultrasound from public patient CT, with a network drawing organ textures from CT labels. Its guide has three replaceable parts (slots), an organ segmenter, a controller and a reward, which received 20 published methods, each run on the same 60 episodes of six held-out patients. In each slot, a published method replaced ours only if detectably better on these patients. The better of this combination and our original guide on 20 new CTs is the benchmark baseline. A pose oracle, reading the exact probe pose available only in simulation, set the reference. The results below are exploratory. Results: On these CTs, the probe started on average 16.0 mm from the target view; the pose oracle brought it to 1.5 mm. Of the results known when the combination was fixed, only an image-similarity reward (SSIM) was slightly but detectably better, so the combination was our guide with that reward. This combination was the benchmark baseline, ending at 13.7 mm (16% of the oracle's progress). Conclusions: Combining untuned published methods with ours did not guide the probe near the target view; the task remains under-explored. The testbed accepts new methods; no guide was tested on hardware.

Comment
75 pages; supplementary material (62 pages) as an ancillary file. Submitted to Computer Methods and Programs in Biomedicine. Code: https://github.com/tohnperfect/USBGym

Closest papers

Nearest neighbours by meaning across the whole index.

Cross-Device Transfer of a Hospital-Trained Fetal Head Circumference Model to a Low-Cost Handheld Ultrasound Probe

eess.IV ·2026-10-28 ·Patrick Gao, Lara Hart, Ken Sullivan
benchmark similarity 0.63
Abstract · 24 pages, 5 figures, 4 tables

Automated fetal head circumference (HC) measurement reaches near-expert accuracy on the HC18 benchmark, but published models are trained and evaluated on hospital-grade equipment. We quantify what happens when such a model is carried to a low-cost handheld probe. A lightweight segmentation network (CSM, 0.13M parameters) was trained on HC18 across five independent runs and applied without modification to images from a $1,350 handheld probe (SonoQue C3), acquired at the same visit as the reference hospital scan in 18 patients. In-domain accuracy was 2.34 +/- 0.43 mm; on handheld images the same model gave 16.12 mm against the hospital record, a 6.9-fold degradation. Input preprocessing, a choice rarely reported, accounted for about a third of that gap: direct resizing gave 49.45 mm against 16.12 mm for an aspect-preserving pipeline. A leave-one-out data-efficiency curve showed that fine-tuning on two images significantly worsened accuracy, and that from four images upward error fell by 0.75 mm per doubling of the local dataset (p = 0.0045), although no training-set size up to 27 images significantly improved on the unmodified model. Much of the residual gap belongs to the reference rather than the model: scored against a manual tracing of the same image the model was already within 10.48 mm, and its disagreement with sonographer calipers on the same frame was a near-constant offset, negative in 10 of 10 cases. A single calibration constant estimated leave-one-out reduced that disagreement from 8.12 mm to 3.49 mm, reaching the intraobserver reproducibility reported for experienced sonographers, but did not improve agreement with the hospital record. Cross-device evaluations should therefore report the input preprocessing used and characterise the precision of their reference standard.

0.21

Ultrasound Operator Guidance Using World Modeling and Retrieval Based Action Planning

cs.LG cs.CV ·2026-10-05 ·Noortje I. P. Schueler, Hans van Gorp, Ruud J. G. van Sloun
similarity 0.62
Abstract · 11 pages, 8 figures, 5 tables

Ultrasound is widely used, but acquisition quality is heavily dependent on the operator's knowledge and expertise. With demand for examinations outpacing the supply of trained sonographers, operator-guidance systems aim to close this gap by instructing a less trained user how to move the probe toward a target view. In this paper, we propose a retrieval-induced latent transition model for ultrasound acquisition dynamics, formulating ultrasound operator guidance as multi-step planning and retrieval in a world model. Using a V-JEPA 2.1 backbone, observations are first encoded into a latent space where anatomically related views lie close together. We then retrieve similar views from a reference database containing encoded latent states and corresponding probe positions and orientations. Rather than learning a parametric transition function, we directly use physically executed transitions from the database to establish our nonparametric, retrieval-induced transition model that supports receding-horizon planning. At deployment, guidance is generated from the live ultrasound image feed alone, without any probe tracking hardware. Applied to carotid ultrasound, the proposed planner reaches the target view in 86% of retrospective closed-loop episodes, versus 52% and 43% for representative baselines, outperforming both on every target view, including the challenging longitudinal internal and external carotid artery views. A prospective feasibility study on unseen volunteers, run in real time on a CPU using distillation, reaches 83% target-view reachability. Because planning is driven by proximity to any encodable goal latent, the same world model can navigate back to any previously acquired, patient-specific frame, supporting reproducible longitudinal imaging for e.g. perioperative or follow-up monitoring.

0.20

UltraWorld: Learning Interactive Ultrasound World Models from Untracked Clinical Videos with Acoustic Sampling Map

cs.CV cs.RO ·2026-10-07 ·Keke Yang, Erqi Wang, Sainan Guan, Hongliang Ren
code/data similarity 0.62
Abstract

World models can enable autonomous ultrasound scanning by predicting the outcomes of probe motions from local observations. Learning this action--observation relationship typically relies on synchronized video--pose pairs, which are costly to collect at scale and largely unavailable in routine clinical recordings. Reliable action following further requires modeling ultrasound's cross-sectional sampling geometry. We present UltraWorld, a self-distillation recipe that transfers priors from clinical ultrasound videos into interactive world models without real action annotations. Starting from clinical videos, we adapt a video foundation model into an ultrasound generator conditioned on reference images and anatomical masks. Anatomical masks sampled along programmable trajectories through 3D anatomy provide spatial guidance for synthesizing action--video pairs. We then use these synthetic pairs to self-distill the generator into a world model that predicts future observations from local observations and actions, without requiring anatomical masks or other 3D assets at inference time. To further improve action following, we introduce the Acoustic Sampling Map (AsMap), which represents probe poses and imaging settings as pixel-wise 3D sampling positions, beam directions, and depths. Experiments demonstrate improved prediction fidelity and action following. Across nine simulated closed-loop local planning episodes, UltraWorld reduces the mean final distance to the goal and orientation error by 29\% and 38\%, respectively, compared with visual servoing. Project Page: https://ultraworld-project.github.io/.

0.30

SCoPE-Reg: Efficient Rigid Ultrasound Slice-to-Volume Registration via State-Space Correlation and Closed-Form Pose Estimation

eess.IV cs.CV ·2026-08-28 ·Niklas Schwarz, Jens Kleesiek, Moritz Rempe
publishedcode/data similarity 0.61
Abstract · Accepted to Winter Conference for Applications in Computer Vision 2027, Orlando, Florida

Ultrasound-guided interventions can require localization of an untracked 2D frame within a 3D anatomical reference. Rigid slice-to-volume registration (SVR) estimates this six-degree-of-freedom pose but remains challenging because of limited anatomical context, acoustic artifacts, and view-dependent appearance. Existing methods often use dense cross-attention, whose cost scales with the product of slice and volume token counts, or direct pose regression that lacks explicit correspondence constraints. We introduce SCoPE-Reg, combining dual-input state-space slice-volume interaction, dense 3D coordinate prediction, and parameter-free weighted Kabsch estimation. On SVR tasks from CAMUS and $μ$-RegPro, SCoPE-Reg yields mean target registration errors of $0.73$ mm and $2.27$ mm against $1.24$ mm and $2.63$ mm for the best-performing baseline, reduces peak error on CAMUS by 56% ($12.5\to5.5$ mm), and registers $100%$ and $80%$ of frames within $3$mm. On CAMUS at $128^2$ it retains the lowest error at increasing pose-perturbation magnitude. It holds $6.49$ M parameters independent of resolution, sustaining $51$ FPS at $512^2$. By coupling correspondence-based accuracy with bounded worst-case error and resolution-independent cost, SCoPE-Reg becomes viable at native acquisition resolution during intervention, whereas previous methods had to compromise on accuracy, reliability or frame rate. Supplementary code: https://github.com/TIO-IKIM/SCoPE-Reg.

0.50

Scanning While Imagining: A Scene-Graph World Model for Robotic Ultrasound Navigation

cs.RO cs.AI ·2026-09-26 ·Xuesong Li, Shuai Chen, Feng Li, Zhongliang Jiang, Nassir Navab +1 more
code/data similarity 0.60
Abstract

Ultrasound (US) acquisition depends on the operator's ability to interpret anatomy and anticipate how the view will change with probe motion. Many robotic US navigation methods select actions without explicitly predicting these anatomical changes. We propose SonoGraph-WM, an action- and goal-conditioned world model for anticipatory probe navigation. The model represents anatomy as scene graphs (SGs), capturing visible structures, their geometry, and spatial relationships without synthesizing US images. Given a history of SGs and probe poses, a unified Transformer jointly predicts future SGs and poses. A receding-horizon planner recursively imagines candidate trajectories, selects the shortest predicted path reaching a goal graph, and follows it over a short execution horizon before replanning from new observations. To reduce reliance on tracked and anatomically annotated US sequences, we generate aligned SG--pose training data from computed tomography (CT) label maps along surface-constrained probe trajectories. On four held-out CT cases, spatial relation F1 remains above 93% over 20 prediction steps, and closed-loop navigation achieves 77.50% and 75.00% success for the gallbladder and pancreas, respectively, using annotation-derived SGs. In robot--phantom navigation experiments with label-map-derived SGs, the planner reached the target view in 73.7% of trials. These findings support CT-supervised anatomical world modeling for probe planning and highlight the importance of frequent observation updates for reliable navigation. Project Page: https://noseefood.github.io/us-sonograph-wm/

0.35

Simulating Vein Compressibility for Ultrasound-Guided Femoral Access: Speed, Accuracy, and Phantom Validation

cs.RO ·2026-11-09 ·Amit Manicka, Laurent Coulon, Darwin Ang, Michael Loushin, Timothy M. Kowalewski
similarity 0.60
Abstract · Submitted to the Journal of Medical Robotics Research (JMRR) for the International Symposium of Medical Robotics 2027 (ISMR 2027) 11 Pages (…

In ultrasound-guided femoral access, probe pressure deforms the tissue and changes the vascular anatomy a robot observes. A simulator must reproduce these mechanics fast enough for robotic interaction, yet the computational effort needed for a clinically meaningful signal like vein collapse before artery is unclear. We test the feasibility of a physics-based simulation in NVIDIA's Isaac Sim for Healthcare, with Stable Neo-Hookean tissue solved by Vertex Block Descent on one GPU. A simulation of a low-fidelity cylindrical benchtop phantom with known geometry and mechanical properties, pressed by a robot, predicted its artery within 5 percentage points at every depth. It matched the vein within 3 points through 11 mm of a 25.4 mm press and followed the collapse behavior of the low-fidelity physical phantom. Across element size, time step, and solver iterations, wall clock time per frame scaled linearly with solver work, but accuracy did not improve with more work. The chosen setting reduced solver time per frame by a factor of 976, from 29,700 ms to 30.4 ms, while remaining within 1.03 points of the finest reference setting. The physics solver alone took 30.4 ms per frame, while the full stack, including ultrasound and graphics rendering, took 56.4 ms per frame. Finally, when seeded with realistic literature-based mechanical properties, the digital leg model compressed the vein far more than the artery at every setting, reproducing the expected clinical signal of vein collapse before artery, which is used to distinguish them. In total, these results suggest that the NVIDIA Isaac Sim platform may be suitable for augmenting reinforcement learning training or online simulation for autonomous femoral access with ultrasound.

0.29

MedImageOSWorld: Benchmarking GUI Agents for Medical Image Consoles

cs.AI cs.CV ·2026-10-03 ·Ziyang Long, Xinqi Li, Lujing Xing, Hsin-Jung Yang
benchmark similarity 0.60
Abstract · 11 pages, 3 figures. Supplementary material (8 pages) included as an ancillary file

Graphical consoles offer a practical interface for medical acquisition assistance, allowing agents to work through the controls and visual feedback used by human operators. Reliable assistance requires linking on-screen anatomy to acquisition decisions that determine what image evidence becomes available next. We introduce MedImageOSWorld, a benchmark for evaluating this capability in simulated CT, MR, and ultrasound consoles. Using screenshots and mouse-and-keyboard actions, agents configure protocols, plan acquisitions, inspect the resulting images, and make corrective adjustments across seven capability levels, from console operation to feedback-driven control. Evaluation combines task-specific workflow checks with hidden anatomical ground truth to assess procedural completion and acquisition outcomes separately. A common evaluation protocol specifies episode conditions and interaction budgets, while recorded trajectories support analysis of how agents observe, act, and respond to acquisition feedback. Across eleven open-weight agents, success rates range from 3.0 to 25.0 on a 0-100 scale while workflow-progress rates reach 21.5-74.5: agents complete much of the console workflow but rarely acquire the intended anatomy. Success collapses between perception and planning, from 79-90% at the lowest three levels for the best agent to at most 12% for millimetre-level planning and 0% for closed-loop control among open-weight agents. Two proprietary agents reach success rates of about 34 and exceed the best open-weight agent mainly in acquisition quality (61 versus 42). MedImageOSWorld provides a controlled setting for studying whether general-purpose GUI agents can translate visual observations into effective medical acquisition decisions.

0.37

US-VLA: An Ultrasound Vision-Language-Action Model for Embodied Abdomina

cs.RO cs.CV ·2026-08-17 ·Cheng Zhang, Xingzheng Wu, Guihao Yan, Xifeng Hu, Zhi Liu +2 more
top venuedatasetcode/data similarity 0.60
Abstract

Artificial intelligence-assisted ultrasound scanning enhances diagnostic reliability and efficiency by providing real-time guidance for standardized image acquisition and reducing operator dependence. However, existing reinforcement learning and learning-assisted ultrasound scanning methods typically rely on carefully designed reward functions or extensive interaction data, which limits their generalization ability and stability across different devices, patient populations, and complex clinical scenarios. To address these challenges, we propose an ultrasound vision-language-action model (US-VLA) for automated ultrasound scanning that explicitly encodes clinical semantic goals and generates sequential probe manipulation actions under real-time ultrasound feedback. In particular, we first design an ultrasound-aware expert fusion module to jointly integrate ultrasound observations with auxiliary contextual information, enabling semantic ultrasound feedback to effectively guide the scanning process. Then, we construct US-VLA-Data, a real-world dataset covering liver and kidney examinations, which includes five clinically defined standard planes and comprises 320 expert scanning trajectories with approximately 80,000 synchronized timesteps. Extensive experiments demonstrate that US-VLA achieves competitive performance in ultrasound probe manipulation tasks, indicating its effectiveness and promising generalization within the evaluated abdominal ultrasound setting. The source code is available at https://github.com/VMVLab/US-VLA.

0.51