CS Paper Radar

Probabilistic Physics Digital Twin for Human-Robot Interaction

Yushen Hu, Harshvardhan Chandirasekar, Apurv Kushwaha, Jonathan King, Chen Zhu-Tian

cs.RO ·submitted 2026-10-14 00:22 UTC · v2

Abstract

Digital twins are increasingly used in robotics to simulate interactions, predict action outcomes, and support planning. However, constructing accurate digital twins requires precise estimates of physical properties such as mass, friction, and center of mass, which are often difficult to obtain in real-world environments. We propose probabilistic digital twins, which represent uncertain physical properties as probability distributions rather than fixed values. By simulating actions across these distributions, the twin predicts a range of possible outcomes and their likelihoods. These probabilistic simulations can guide both human and robot actions by identifying actions that are effective despite physical uncertainty. We demonstrate this framework through human-robot interaction scenarios, including interfaces that communicate simulated outcomes to users and robot planning strategies that explicitly account for uncertainty.

Closest papers

Nearest neighbours by meaning across the whole index.

VPTwin: Real-Sim-Real Video Prediction for Robotic Manipulation Planning

cs.RO cs.AI ·2026-09-27 ·Zhenghao Xiao, Minting Pan, Nantian He, Dongzhan Zhou, Yunbo Wang
similarity 0.71
Abstract

While action-conditioned video prediction provides an intuitive world model for robotics, purely data-driven predictors often suffer from compounding errors and physically implausible hallucinations in long-horizon rollouts, severely undermining downstream action planning. We propose VPTwin, a Real-Sim-Real video prediction framework that anchors real-world future prediction using real-synchronized simulation twins. For a target manipulation task, a VLM reconstructs an executable digital twin from a real demonstration episode. To accommodate the ill-posed estimation of unobserved physical properties, Isaac Sim simulates multiple forward dynamic rollouts across randomized physical configurations under candidate action trajectories. Using these rollouts as in-context references, VPTwin harmonizes both domains, using simulation dynamics to enforce physical plausibility while capturing unmodeled contact interactions from real video. Furthermore, we establish a predictive planning loop using VPTwin to visually verify VLM-proposed actions and guide reliable real-world execution. Evaluations show substantial reductions in physical hallucinations during video prediction and marked improvements in manipulation planning performance.

0.43

TwinBridge: Digital Twin-Driven Policy and Reward Transfer for Real-World Reinforcement Learning in Laboratory Robotics

cs.RO ·2026-11-04 ·Shu'ang Sun, Yibo Qiu, Haoliang Ye, Zan Huang, Ao Tian +3 more
similarity 0.70
Abstract · 8 pages, 6 figures

Online reinforcement learning offers an effective approach to precise laboratory manipulation, yet costly real-world demonstrations and limited access to dense process rewards hinder its practical application. We present TwinBridge, a digital twin-driven framework that supplies policy initialization and dense visual rewards for real-world online learning. We construct digital twins by combining 3D Gaussian splatting with physics simulation. Here, simulator states provide task events and geometric progress under physical prerequisites and ordering constraints, yielding rewards for policy pretraining with simulated demonstrations and reinforcement learning. We distill this supervision into a visual reward model. During real-world interaction, its predictions are combined with execution history to compute dense rewards for human-in-the-loop policy refinement. Across five real-world laboratory tasks, TwinBridge achieves a final mean autonomous success rate of 98% with at most 25 minutes of real-world interaction per task. Experiments show that dense rewards improve interaction efficiency over sparse rewards, and simulation-based policy initialization further accelerates learning.

0.20

ForceTwin: Physics-informed Digital Twins for Robotic Manipulation from Instrumented Human Interaction

cs.RO cs.AI ·2026-09-18 ·Tim Engelbracht, René Zurbrügg, Mayank Mittal, Marco Hutter, Marc Pollefeys +2 more
code/data similarity 0.70
Abstract

Manipulating objects requires understanding not only their motion, but also the physical properties that determine it. For articulated objects, these include inertia, friction, and mechanisms such as springs or door closers, whose effects can vary with configuration and velocity. Such properties are not directly observable from appearance: visually identical doors may require very different effort to manipulate. Existing digital-twin pipelines recover primarily kinematics or assign static physical parameters from visual and language priors, which can yield physically implausible estimates. As a result, state-dependent mechanism dynamics remain unidentified and are not represented in standard asset formats. We present ForceTwin, a system for identifying physics-informed digital twins of articulated objects from instrumented human interaction. A person probes an object using a handheld force-sensing gripper, providing synchronized poses and interaction forces from which we estimate the articulation, parametric dynamics including inertia, Coulomb friction, viscous damping, and a structured neural residual capturing state-dependent mechanism forces. ForceTwin nearly halves the inertial-parameter error of a VLM prior. As a feedforward dynamics model for impedance control on a Spot and a Franka FR3, ForceTwin achieves 87% goal completion across nine object-embodiment pairs, compared with 60% using VLM-prior and 57% using kinematics-only twins, with the largest gains on objects whose strong mechanisms cause both baselines to stall. We further use the identified twins to train whole-body door-traversal policies and deploy them in the real world. Project Page: https://timengelbracht.github.io/forcetwin-website/

0.58

AuRA: From Atoms to Bits through Autonomous Robot Interaction

cs.RO ·2026-10-25 ·Xinchen Yu, Yoonkyo Jung, Haoran Chen, Zikui Cai, Aayush Talreja +3 more
code/data similarity 0.69
Abstract · Project page: https://aura-robot.github.io/

Simulation enables robot-learning experiments at scale, but acquiring physics-informed digital twins often requires human manipulation and manual annotation of recorded interactions. To reduce this reliance on human effort, we present AuRA, a system that automates robot interaction and data preparation for digital-twin reconstruction. AuRA combines execution-aware grasp selection, gripper-motion measurements derived from robot states, and registered pre-interaction geometry and appearance priors. These choices enable autonomous collection, avoid reliance on visual gripper tracking, and recover object information obscured by the robot. Across 13 deformable objects, AuRA reduces active human labor and wall-clock collection time per successfully reconstructed asset by 88.5% and 55.4%, respectively. Relative to the human-collected PhysTwin baseline, AuRA improves seven of nine metrics for future prediction under known gripper motion, including a 28.1% reduction in one-way $L_1$ Chamfer distance. It also achieves similar mean silhouette IoU and Chamfer distance under the evaluated unseen interactions. These results highlight the potential of autonomous robot interaction to support more efficient digital-twin acquisition with comparable reconstruction quality.

0.24

Physical Twins: Accelerating and Enabling Robot Learning with Phantom Platforms

cs.RO ·2026-10-03 ·Elizabeth Peiros, Derek Chen, Lucas Yager, Edward Lee, Kevin Wang +2 more
similarity 0.67
Abstract · 8 pages, 7 figures

Improvements in human-robot physical interaction (pHRI) can have major implications for physical therapy, search and rescue, and telemedicine. However, a major challenge concerns human constraints and safety in human-robot physical experiments. Concerns about human studies also include repeatability, scalability, and participant diversity. To conduct such experiments, an IRB and willing human participants are required. In this work, we present an improved phantom device, a physical twin, that enables real-world RL-type testing for physically interactive algorithms. The new device not only replicates the ball-and-socket motion of the shoulder but also renders scapular and protraction/retraction motions. The experiments showcase the device's ability to render multiple 3D joint limits and demonstrate a Franka Panda arm sensing limits and interacting with the device as an arm for ADLs (activities of daily living) and pHRI tasks.

0.24

Operational digital twin clinics enable task-based evaluation of embodied AI

cs.RO cs.AI ·2026-08-12 ·Xinyuan Wu, Jingrao Zhang, Mengdi Xu, Henry K. Chu, Mingguang He +1 more
similarity 0.64
Abstract

Embodied artificial intelligence (AI) must be tested in the clinical environments where it will operate, but building realistic, robot-testable settings is costly and difficult to scale. Here we show that routine clinic images can be transformed into operational digital twins for task-based evaluation of embodied AI. Using 39 ophthalmic clinic scenes, we converted single photographs into editable, simulator-ready environments and assessed reconstruction quality, room-scale geometry, mesh grounding, multi-robot feasibility, perturbation sensitivity and closed-loop policy performance. The reconstructed scenes preserved workspace structure, while local editing enabled controlled device reconfiguration. Device meshes, collision proxies and semantic anchors converted visual reconstructions into contact-aware simulation scenes. Across three robot embodiments, shared task targets showed different patterns of reachability and contact feasibility. Small device translations and rotations produced task-specific changes in contact margins that were not captured by visual similarity alone. Digital-twin trajectories also supported local policy learning and closed-loop evaluation. These findings establish operational validity as a key principle for clinical digital twins and provide an intermediate layer between offline development and physical deployment of embodied AI in healthcare.

0.04

Physical AI: Concepts, Techniques, and Challenges

cs.AI cs.RO ·2026-10-19 ·Weinan Zhang, Chunhua Shen, Tianfan Xue, Bin Zhao, Bingqi Jiang +47 more
similarity 0.63
Abstract · Monograph, 12 chapters, 388 pages, 64 figures

In recent years, physical artificial intelligence (physical AI) has experienced rapid development, capturing significant attention in academia and industry. However, the field has yet to reach consensus on its definition, technical landscape, and core challenges. To address this gap, this monograph seeks to define physical AI, present its technical landscape, and examine each key technical subfield in depth. In doing so, it establishes a unified framework for Physical AI: artificial systems that perceive, model, and act upon the physical world through continuous feedback under real-world constraints. Further, this monograph distinguishes physical AI from conventional digital AI, embodied AI, cyber-physical systems, and digital twins. It develops the foundations of physical AI across physics, control, cognition, learning, and reasoning. It reviews key methodologies, including multimodal perception, physical data and simulation, world models, embodied foundation models, motor-control systems, multi-agent coordination, and hardware-software co-design. Furthermore, it examines applications spanning robotics, manufacturing, infrastructure, scientific experimentation, and autonomous services. Particular attention is given to sim-to-real transfer, reliability, real-time execution, evaluation, safety, governance, and long-term deployment. By connecting perception, prediction, decision-making, action, and adaptation in a closed loop, this monograph identifies the technical challenges and research directions required to advance from narrow automation toward more generalizable physical AI and, ultimately, physical artificial general intelligence (Physical AGI).

0.20

Decision-Making in Digital Twins: Optimization, Control, and Learning

eess.SY math.OC ·2026-10-12 ·Enlu Zhou
similarity 0.63
Abstract

A digital twin is a virtual representation of a physical system, continuously updated with data from the physical system, enabling dynamic information exchange and decision-making. This bidirectional interaction, which distinguishes digital twins from conventional simulation models, introduces new challenges to simulation-based optimization, control, and learning. This paper reviews recent developments in decision-making methodologies motivated by digital twins. We focus on three interconnected areas: simulation optimization, sequential decision making, and risk-aware decision making. For simulation optimization, we discuss data-driven simulation optimization under input uncertainty, streaming input data, decision-dependent observations, and limited computational and data-collection resources. For sequential decision making, we review the connections among digital twins, stochastic optimal control, and reinforcement learning, with emphasis on Bayesian approaches that update models and policies as new information becomes available. For risk-aware decision making, we discuss distributionally robust and Bayesian risk formulations as alternative approaches for addressing model uncertainty. Throughout the paper, we emphasize the common problem of making decisions while the underlying model is being updated. We conclude by identifying research challenges and opportunities.

0.27