CS Paper Radar

Mixed-Reality Feedback Control: Towards Blending of Control and Digital Twinning

Melisa Soydan, Mario Gleirscher

cs.RO eess.SY ·submitted 2026-10-14 08:13 UTC · v1

Abstract

Technologies, such as co-simulation, in particular, digital twins (DTs) are used for comprehensive and reliable prototyping, verification and validation (V&V) of control systems. There, the combination of feedback loops across multiple realities (e.g., simulations, physical setups) enables various forms of experimentation with robots and autonomous systems. However, the handling of multiple realities requires a more specific support by the modelling techniques used in control engineering. To streamline the latter in the system life cycle, we propose a framework for mixing (i.e., fusing, defusing) control input and feedback signals (i.e., stimuli, responses) in simultaneously operated plants for multiple realities.

Comment
19 pages, 7 figures

Closest papers

Nearest neighbours by meaning across the whole index.

A Human-Centric Adaptive Multi-Fidelity Digital Twin Framework for Cyber-Physical Systems

cs.HC eess.SY ·2026-11-18 ·Clarisa Grijalva-Lugo, Yu-Zheng Lin, Shalaka Satam, Sicong Shao, Jesus Pacheco +2 more
similarity 0.63
Abstract · 20 pages, 3 figures, 11 tables; submitted to Advanced Engineering Informatics

Digital twins of cyber-physical systems must operate with scarce data, on assets that change after deployment, and for humans whose needs differ, yet existing implementations fix fidelity at design time and do not match the representation to the human or the task. This paper proposes a human-centric adaptive multi-fidelity digital twin framework that models the evolving human, the physical twin, and the digital twin core, and adds a human inference and adaptation function that estimates the human state, predicts the response, and adapts the representation. The framework defines low-, medium-, and high-fidelity tiers distinguished by adaptivity and by coupling, quantified as the ratio of the digital twin latency to the duration of the process step it follows. Three proof-of-concept instantiations are evaluated: a virtual reality tutor, a computer numerical control milling model, and an industrial training station. The low tier estimated learner sentiment with an accuracy of 0.86. The forecasters of the milling model and the training station, both medium-tier, showed a normalized error decline of 0.39 to 0.89 and a human adaptation gain of 1.16 to 1.51, but their latency exceeded the physical process step by 25 to 87 times. Only the station's vision model met the physical step. The tiers are separated by which functions adapt and where the latency constraint lies, not by a monotone increase in gain or coupling.

0.29

Missing Dimensions: Integrating Human and Social Systems into Digital Twin Engineering

cs.SE ·2026-09-10 ·Francis Bordeleau, Mark van den Brand
similarity 0.59
Abstract

Digital twins (DTs) have emerged as a key technology at the core of digital transformation, yet their engineering practice remains too narrowly focused on engineered and natural systems. This paper argues that four system dimensions must be explicitly recognized in DT engineering: Engineered, Natural/Biological, Human, and Social. Each dimension brings distinct properties, modeling requirements, and ethical obligations that fundamentally shape what a DT must represent and how it must be built. We further argue that as DTs extend into the Human and Social dimensions, the emphasis should shift from automated control toward decision support and occupant empowerment. We illustrate our arguments using a smart building as a running example and identify four open research challenges for multi-dimensional DT engineering.

0.30

Physical AI: Concepts, Techniques, and Challenges

cs.AI cs.RO ·2026-10-19 ·Weinan Zhang, Chunhua Shen, Tianfan Xue, Bin Zhao, Bingqi Jiang +47 more
similarity 0.59
Abstract · Monograph, 12 chapters, 388 pages, 64 figures

In recent years, physical artificial intelligence (physical AI) has experienced rapid development, capturing significant attention in academia and industry. However, the field has yet to reach consensus on its definition, technical landscape, and core challenges. To address this gap, this monograph seeks to define physical AI, present its technical landscape, and examine each key technical subfield in depth. In doing so, it establishes a unified framework for Physical AI: artificial systems that perceive, model, and act upon the physical world through continuous feedback under real-world constraints. Further, this monograph distinguishes physical AI from conventional digital AI, embodied AI, cyber-physical systems, and digital twins. It develops the foundations of physical AI across physics, control, cognition, learning, and reasoning. It reviews key methodologies, including multimodal perception, physical data and simulation, world models, embodied foundation models, motor-control systems, multi-agent coordination, and hardware-software co-design. Furthermore, it examines applications spanning robotics, manufacturing, infrastructure, scientific experimentation, and autonomous services. Particular attention is given to sim-to-real transfer, reliability, real-time execution, evaluation, safety, governance, and long-term deployment. By connecting perception, prediction, decision-making, action, and adaptation in a closed loop, this monograph identifies the technical challenges and research directions required to advance from narrow automation toward more generalizable physical AI and, ultimately, physical artificial general intelligence (Physical AGI).

0.20

Ruleless Digital Twins: Toward Declarative Decision-Making Through Standardized Frameworks and Technologies

cs.SE ·2026-10-07 ·Ivan Spajić, Volker Stolz
similarity 0.58
Abstract · 47 pages (bibliography included), expanded version of the Spajić and Stolz (2026) DataMod paper

Digital twins (DTs) can be thought of as digital counterparts of physical objects, or more generally, twinning targets (TTs). To enact changes on and optimize for properties of their TTs, DTs use an element of decision-making. Traditionally, many domains, such as home automation, utilize rule-based decision-making models to imperatively define DT actions based on desired TT conditions. With evolving user specifications, rule-based models have become increasingly complex to develop and maintain, particularly in scenarios involving optimizations under dynamically changing systems, such as those affected by weather or dynamic energy pricing. We present an alternative in the form of ruleless digital twins (RDTs) that automatically produce optimal decisions with respect to purely declarative user specifications, similarly to the well-established ruleless approach of model predictive control. They do so through a combination of a semantic knowledge base, logical inference, simulation models, and an autonomic computing architecture, each exclusively based on a widely-used standard or technology. We evaluate our proof of concept through an incubator case study and experiments against a rule-based bang-bang controller in the context of a virtual office room environment. Results show core that RDT functionality performs better than the control in terms of maintaining desired room temperatures and minimizing energy costs under dynamically changing spot pricing. Additionally, due to combinatorial decision tree construction, RDTs require considerable computational power for longer prediction horizons, although their applicability to specific problem domains (and use of shorter horizons) ultimately falls on their users.

0.31

Reinforcement Twinning via Twin-Calibrated Bayesian Policy Search and Replay-Driven Identification

eess.SY ·2026-11-02 ·Miguel A. Mendez, Lorenzo Schena, Romain Poletti, Yannick Lecomte, Sebastiano Randino +1 more
similarity 0.57
Abstract

The feasibility of updating and synchronising digital twins of actuated systems from operational data depends strongly on how the system is controlled and, consequently, on the data generated during operation. This establishes a direct link between control actuation and model identifiability. This work presents a reinforcement-twinning framework that exploits this link through three computational components. A Gaussian-process surrogate of the physical return combines the twin prediction, used as a calibrated trend, with a discrepancy learned from measured returns, allowing Bayesian policy search to exploit the twin without being tied to its predictions. Candidate policies are proposed around the policy that is optimal for the twin, while measured returns can expand this region. The resulting physical episodes are replayed on the twin after observer-based synchronisation with sparse measurements, and its parameters are identified by combining Bayesian optimisation with gradient-based refinement. Selected information-oriented episodes deploy an excitation policy to improve parameter identifiability. The framework is demonstrated on a Burgers-flow analogue of dynamic-induction control in wind farms, with a deliberately misspecified model structure. Despite substantially biased return predictions, the twin provides a useful starting policy and search region, placing the guided search ahead of unguided Bayesian optimisation from its first physical evaluations. This advantage is maintained when the search region is allowed to expand in response to changes in measured returns, whereas searches over-constrained by the twin-defined region are eventually overtaken by the unguided alternatives. The results support reinforcement twinning as a framework for jointly adapting models and control policies when physical interaction is limited and model discrepancy cannot be neglected.

0.25

Probabilistic Physics Digital Twin for Human-Robot Interaction

cs.RO ·2026-10-14 ·Yushen Hu, Harshvardhan Chandirasekar, Apurv Kushwaha, Jonathan King, Chen Zhu-Tian
similarity 0.57
Abstract

Digital twins are increasingly used in robotics to simulate interactions, predict action outcomes, and support planning. However, constructing accurate digital twins requires precise estimates of physical properties such as mass, friction, and center of mass, which are often difficult to obtain in real-world environments. We propose probabilistic digital twins, which represent uncertain physical properties as probability distributions rather than fixed values. By simulating actions across these distributions, the twin predicts a range of possible outcomes and their likelihoods. These probabilistic simulations can guide both human and robot actions by identifying actions that are effective despite physical uncertainty. We demonstrate this framework through human-robot interaction scenarios, including interfaces that communicate simulated outcomes to users and robot planning strategies that explicitly account for uncertainty.

0.21

ForceTwin: Physics-informed Digital Twins for Robotic Manipulation from Instrumented Human Interaction

cs.RO cs.AI ·2026-09-18 ·Tim Engelbracht, René Zurbrügg, Mayank Mittal, Marco Hutter, Marc Pollefeys +2 more
code/data similarity 0.57
Abstract

Manipulating objects requires understanding not only their motion, but also the physical properties that determine it. For articulated objects, these include inertia, friction, and mechanisms such as springs or door closers, whose effects can vary with configuration and velocity. Such properties are not directly observable from appearance: visually identical doors may require very different effort to manipulate. Existing digital-twin pipelines recover primarily kinematics or assign static physical parameters from visual and language priors, which can yield physically implausible estimates. As a result, state-dependent mechanism dynamics remain unidentified and are not represented in standard asset formats. We present ForceTwin, a system for identifying physics-informed digital twins of articulated objects from instrumented human interaction. A person probes an object using a handheld force-sensing gripper, providing synchronized poses and interaction forces from which we estimate the articulation, parametric dynamics including inertia, Coulomb friction, viscous damping, and a structured neural residual capturing state-dependent mechanism forces. ForceTwin nearly halves the inertial-parameter error of a VLM prior. As a feedforward dynamics model for impedance control on a Spot and a Franka FR3, ForceTwin achieves 87% goal completion across nine object-embodiment pairs, compared with 60% using VLM-prior and 57% using kinematics-only twins, with the largest gains on objects whose strong mechanisms cause both baselines to stall. We further use the identified twins to train whole-body door-traversal policies and deploy them in the real world. Project Page: https://timengelbracht.github.io/forcetwin-website/

0.58

Operational digital twin clinics enable task-based evaluation of embodied AI

cs.RO cs.AI ·2026-08-12 ·Xinyuan Wu, Jingrao Zhang, Mengdi Xu, Henry K. Chu, Mingguang He +1 more
similarity 0.57
Abstract

Embodied artificial intelligence (AI) must be tested in the clinical environments where it will operate, but building realistic, robot-testable settings is costly and difficult to scale. Here we show that routine clinic images can be transformed into operational digital twins for task-based evaluation of embodied AI. Using 39 ophthalmic clinic scenes, we converted single photographs into editable, simulator-ready environments and assessed reconstruction quality, room-scale geometry, mesh grounding, multi-robot feasibility, perturbation sensitivity and closed-loop policy performance. The reconstructed scenes preserved workspace structure, while local editing enabled controlled device reconfiguration. Device meshes, collision proxies and semantic anchors converted visual reconstructions into contact-aware simulation scenes. Across three robot embodiments, shared task targets showed different patterns of reachability and contact feasibility. Small device translations and rotations produced task-specific changes in contact margins that were not captured by visual similarity alone. Digital-twin trajectories also supported local policy learning and closed-loop evaluation. These findings establish operational validity as a key principle for clinical digital twins and provide an intermediate layer between offline development and physical deployment of embodied AI in healthcare.

0.04