CS Paper Radar

Scene Graph-Driven Haptic Feedback for Safety Enhancement in Robotic Ophthalmic Surgery via Physically Simulated iOCT

Danial Arbabi, Korab Hoxha, Angelo Henriques, Mirza Imamovic, M. Ali Nasseri

cs.RO cs.CV ·submitted 2026-09-07 18:16 UTC · v1
published

Abstract

Robotic ophthalmic surgery offers high precision but introduces a "sensory gap" by decoupling the surgeon from their instrument, resulting in a loss of tactile feedback. This paper presents a novel haptic feedback system for subretinal injection tasks leveraging Scene Graphs (SG). The system bridges the sensory gap by analyzing a physically simulated intraoperative Optical Coherence Tomography (iOCT) feed to construct a real-time surgical SG. The SG serves as a semantic abstraction layer for the surgical scene, which is then utilized by a deterministic, rule-based engine to generate state-dependent haptic feedback on a robotic input device. The system was evaluated in a user study (N=16) using an anthropomorphic head phantom and a custom-built surgical robot. Results demonstrate that the SG-driven haptic feedback improved surgical precision, reducing needle alignment error by 14% (p = 0.044) and improving System Usability Scale (SUS) scores by 8% (p = 0.015), while maintaining comparable task completion times. A needle trajectory analysis revealed the emergence of a safer "Align-then-Approach" strategy, in which our haptic negative reinforcement prompted users to fine-tune the tool's trajectory before approaching the retinal target. This work suggests that SGs can effectively serve as the direct computational foundation for real-time, safety-enhancing context-aware haptic feedback in robotic microsurgery.

Comment
7 pages, 9 figures, 2 tables. Accepted at the 2026 IEEE RAS/EMBS 11th International Conference on Biomedical Robotics and Biomechatronicsnics (BioRob 2026)
Journal ref
2026 11th IEEE RAS/EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob), Edmonton, AB, Canada, 2026, pp. 1082-1087
DOI
10.1109/BioRob66782.2026.11681904

Closest papers

Nearest neighbours by meaning across the whole index.

An Open-Source ROS 2 Framework for Shared Control in Surgical Teleoperation: Integrating the Haply Inverse3 with the da Vinci Research Kit

cs.RO eess.SY ·2026-11-19 ·Balint Varga, Sebastian Fritz, Gergely Kiss, Kristof Takacs, Peter Galambos
benchmarkcode/data similarity 0.66
Abstract

Surgical teleoperation deprives surgeons of touch, forcing visual inference of tissue forces; the commercial platforms now restoring force feedback remain closed, preventing open research on haptic shared control. This paper provides a system contribution: it presents an open-source, ROS 2-native framework that integrates the compact, three-degree-of-freedom (3-DOF) Haply Inverse3 haptic device with the da~Vinci Research Kit (dVRK). The framework includes a novel mapping strategy that combines 3-DOF translational device input with separately tracked VerseGrip orientation into 6-DOF patient-side manipulator commands. Two virtual-fixture-inspired guidance modes serve as reference implementations to evaluate the framework in a within-subjects user study on a teleoperated hot-wire benchmark task in simulation, and the architecture is further demonstrated on physical dVRK hardware. Relative to unassisted manual control, linear guidance significantly reduced both the number of wire contacts and the task completion time, whereas the progressive-stiffness guidance produced consistent but statistically non-significant trends in the same direction. Subjective ratings of workload, agency, and satisfaction showed favorable descriptive trends for the guidance modes. The results demonstrate the feasibility of haptic shared control with a compact device coupled to the dVRK and provide an openly available, reproducible platform for future research. They further suggest that simpler linear feedback may be more robust for inexperienced operators than more complex profiles.

0.25

MISTac: A Vision-Based Tactile Sensor for Minimally Invasive Surgery

cs.RO ·2026-08-14 ·Robin Koch, Annabella Mascot, Rayan Younis, Martin Wagner, Stefanie Speidel +3 more
code/data similarity 0.64
Abstract · This work has been submitted to the IEEE for possible publication

Minimally invasive and robot-assisted surgery offer many advantages over traditional open surgery, but deprive surgeons of tactile feedback and the ability to palpate tissue with their fingers. To address this lack of tactile feedback, we introduce the MISTac, a high resolution vision-based tactile sensor specifically designed for palpation in MIS. The sensor has a replaceable sensor tip with a diameter of 8 mm which allows it to fit through the trocars used in minimally invasive surgery. Its modular 3D-printed case design allows the use of bulky off-the-shelf illumination and imaging hardware that can easily be exchanged and upgraded. The sensor has an optical resolution of 176.68 $μm$, a tactile resolution of 250 $μm$, and can resolve forces as little as 24.3 mN. An in vivo study with the sensor shows its usability in minimally invasive surgery. We trained a machine learning model with the tactile data collected in the trial on a tissue classification task achieving an aggregate accuracy of ~84% in a leave-one-out cross validation. Tactile sensors have the potential to one day aid surgeons during minimally invasive surgery with tasks such as tissue classification or intra-operative tumor localization; MISTac is a small step towards this vision. We open-source MISTac at https://github.com/lasr-lab/mistac

0.26

Occupancy Network-Guided Autonomous Robotic Partial Nephrectomy

cs.RO cs.CV ·2026-09-14 ·Ethan Kilmer, Pit Henrich, Jiawei Ge, Paul M. Scheikl, Laura Connolly +11 more
top venue similarity 0.63
Abstract

Autonomous soft-tissue cancer surgery has been limited to interventions on organ surfaces, because current systems cannot perceive and adapt to anatomy once it deforms or is cut. We introduce the first vision-guided autonomous system capable of performing complete tumor resections for partial nephrectomy. Our system integrates conditional occupancy networks, trained entirely in a physics-based simulation, that infer full 3-D anatomy (tumor, margin tissue, and kidney) from single-view partial point clouds. These occupancy networks maintain intraoperative tracking even as tissue is cut and deformed, enabling adaptive planning and execution. The surgical platform combines a depth camera for capturing surface point clouds, dual robotic arms for electrosurgical cutting and vacuum-based tissue manipulation, and an autonomous control strategy for tumor resection. In patient-derived hydrogel phantoms under an open partial nephrectomy setting, the robot performed eight consecutive autonomous tumor resections comprising 77 electrosurgical cuts, with all cuts achieving negative surgical margins and 1.61 $\pm$ 0.48 mm mean absolute margin error. This work demonstrates, for the first time, a foundation for supervised autonomous closed-loop, imaging-driven, margin-negative tumor removal in phantoms.

0.30

A Controlled Comparison of Manual and Teleoperated Intraocular Instrument Motion for an Input Device

cs.RO cs.HC ·2026-09-08 ·Korab Hoxha, Mirza Imamovic, Angelo Henriques, M. Ali Nasseri
similarity 0.63
Abstract · This work has been submitted to IEEE Access for possible publication

Input devices for robotic microsurgery are frequently described as preserving the surgeon's trained technique, but the claim is rarely measured. We compared manual and teleoperated intraocular instrument motion with the trocar constraint, the instrument, the eye model and the tracking source common to both conditions, so that the control interface was the only factor varied. Prior comparisons cannot hold the instrument fixed, because a robotic instrument is not the tool used manually. Sixteen participants performed a navigation task on a commercial ophthalmic simulator by hand and through a three-degree-of-freedom input device commanding a five-joint robot. Task outcome was equal but at ceiling: every participant acquired all five targets under both interfaces with no retinal or lens injury. Execution differed on every measure. Teleoperated trials took three times as long at a quarter of the median speed, covered less than half the angular working range, and were broken into 3.5 times as many separate movements. Completion time and movement fragmentation improved substantially across four trials of practice and had not plateaued; the measures set by the configured rate ceiling and joint limit changed the least. Finger activity doubled and pinch variability tripled, so reducing instrument degrees of freedom redistributed manual effort rather than reducing it. The interface preserves the outcome and reshapes the execution.

0.30

SurgiCLAW: An Interpretable Atomic-Skill Agent for Precision Robotic Micromanipulation

cs.RO cs.AI ·2026-10-11 ·Angen Ye, junjie xie, Yujia Song, Junkai Huang, Dapeng Zhang
dataset similarity 0.60
Abstract

Recent multimodal large language models (MLLMs) and vision--language--action models have provided robots with new capabilities for long-horizon task understanding and planning. In surgical robotics, however, end-to-end prediction of low-level actions remains limited by insufficient precision, opaque decisions, and scarce data. This paper presents SurgiCLAW, a hierarchical skill-based framework for precision surgical manipulation. SurgiCLAW confines an MLLM to intraoperative visual understanding, structured explanation, and atomic surgical-skill selection, while delegating physical execution to verifiable deterministic control programs. For brain--machine-interface implantation, a representative high-precision surgical task, we construct a state-to-skill invocation dataset and expand it through visual and state perturbations. The model first learns structured skill invocation through supervised fine-tuning and is then optimized using three rewards for action correctness, operation prerequisites, and format validity, improving action selection, operational safety, and output parseability. We evaluate subtask success, offline model comparison, real-robot performance, and ablations. On a fixed test set, SurgiCLAW achieves 94.6% skill-selection accuracy and 93.0% macro-averaged F1 (Macro-F1). Across 50 closed-loop real-robot implantation trials, it achieves an 88.0% task success rate.

0.18

Language-Conditioned Behavioral Cloning for Autonomous Endoscope Guidance

cs.RO ·2026-10-30 ·Ariel Rodriguez, Lorenzo Mazza, Ortrun Hellig, Alejandra Toscano, Rayan Younis +1 more
published similarity 0.60
Abstract · The Evolving Landscape of Surgical Robotics: Surgical Autonomy for Today and Tomorrow @ IROS 2026

Automated endoscope guidance reduces the need for continuous manual camera control during minimally invasive surgery. However, the desired camera motion depends on the surgeon's immediate intentions and cannot always be inferred from the visual operative scene alone. We propose to automate the task using language-conditioned behavioral cloning; we define 6 clinically relevant guidance behaviors and collect 235 subtask demonstrations. In preliminary phantom experiments, our policy achieves 100% success on focus, overview, and stop, 80% on search-tip and follow-tip behaviors, and 50% on horizon leveling. These results demonstrate the feasibility of learning heterogeneous surgeon-requested camera behaviors with a single visuomotor policy, while the observed failure modes motivate future work on adding temporal context for more demanding guidance tasks.

0.15

OphBiWSSD: Scaling Temporal Action Localization in Ophthalmic Surgeries with Bidirectional Weight-tied State Space Duality

cs.CV ·2026-09-11 ·Yang Liu, Qionghong Ma, Joongwon Chae, Lihui Luo, Yibing Shen +8 more
benchmarkcode/data similarity 0.60
Abstract · Submitted to IEEE Transactions on Image Processing. Under review

High-frequency surgical maneuvers in ophthalmology necessitate high-fidelity temporal modeling, yet characterizing long-range procedural dependencies remains computationally prohibitive for attention-based architectures. Existing models often require aggressive temporal downsampling, which compromises the detection of fine-grained action boundaries and instrument-tissue interactions. To address these scalability constraints, we present OphBiWSSD, a framework that reformulates surgical temporal action localization leveraging Bidirectional State Space Duality. By employing a weight-tied selective scan mechanism that incorporates both preceding and succeeding surgical contexts, our approach facilitates the global synthesis of non-causal temporal cues with linear complexity. This streamlined architecture is well-suited to capture the bidirectional dependencies present in ophthalmic workflows, effectively bridging the gap between local boundary precision and long-range procedural context without incurring the quadratic memory overhead of traditional Transformers. Extensive experiments on the OphNet benchmark demonstrate that OphBiWSSD achieves state-of-the-art temporal localization performance, with mean Average Precisions of 44.42% on phases and 43.08% on operations, surpassing the baselines by 6.80% and 6.66%, respectively. Empirical validation indicates that our approach ensures precise temporal localization and offers a computationally viable pathway for deploying surgical intelligence systems in clinical environments. The code is publicly available at https://github.com/yo3nglau/OphBiWSSD.

0.30

NeuroClick: Preserving Surgeon Autonomy through Hands-Free Earable Tooth-Click Control in Neurosurgery

cs.HC ·2026-09-11 ·Jonas Hummel, Maximilian Burzer, Clara Sayffaerth, Valeria Zitz, Amir El Rahal +4 more
similarity 0.59
Abstract

Neurosurgeons frequently interact with operating room (OR) technologies while sterility and occupied hands constrain control. We introduce earables as a direct, hands-free control platform for neurosurgery using tooth-click input. Formative OR observations and interviews with 10 domain experts grounded the design. Using OpenEarable 2.0 data from 12 participants, we developed a real-time recognition pipeline whose classifier achieved a median macro F1-score of 98.6% under leave-one-subject-out cross-validation. We evaluated the technique with 20 neurosurgeons during a simulated resection task in a neurosurgical OR. Participants reported few focus shifts and rated Earable favorably for workflow integration and perceived safety. Autonomous microscope control was rated significantly higher with Earable than Delegation, whereas Delegation enabled faster task completion under continuous assistant availability. Workload, usability, and task errors showed no significant differences. Preferences depended on training, reliability, context, and assistant availability. Earables thus add a direct, hands-free option for controlling selected functions alongside established workflows.

0.26