CS Paper Radar

Learning a Resolution-Consistent Jacobian Field for Bio-Inspired Rigid-Soft Finger

Tianyou Liang, Haisen Zeng, Shanjun Chen, YiMing Zhu, Zhongyue Lu, Zirong Luo

cs.RO ·submitted 2026-10-01 13:29 UTC · v1

Abstract

Bio-inspired tendon-driven rigid-soft coupled dexterous fingers exhibit strong nonlinearity and configuration-dependent sensitivity, making accurate modeling challenging. In discrete-time control, Jacobian-based kinematic algorithms typically rely on point-wise local linear approximations, which makes their performance sensitive to sensor sampling frequency and controller update frequency. To address this issue, we propose Jacobian Flow Matching (JFM), a structured learning framework based on Conditional Flow Matching (CFM), to learn a resolution-consistent Jacobian field that models actuation-to-motion transitions as a dynamical flow. The proposed framework supports both single-step prediction and continuous rollout via ODE integration, enabling consistent inference across temporal resolutions. Experiments on a tendon-driven rigid-soft finger show that the proposed method suppresses outlier errors and improves single-step prediction accuracy, reducing the global average RMSE by over 53% compared with a baseline discrete Jacobian learning approach. For long-horizon prediction, trajectories recovered via ODE integration achieve higher fidelity under sparse sampling (Stride = 8), reducing the RMSE median by 14.43% and the error variance by 24.87%. These results demonstrate that the learned flow-based Jacobian field provides an effective local model for offline multi-step trajectory optimization in rigid-soft coupled nonlinear systems.

Comment
27 pages, 8 figures, including supplementary material. Under review at Robotics and Autonomous Systems

Closest papers

Nearest neighbours by meaning across the whole index.

History-Conditioned Flow Matching for Probabilistic Dynamics of Tendon-Driven Continuum Robots

cs.RO ·2026-09-22 ·Hang Yang, Tingcong Liu, Junjie Xiong, Fangju Yang, Ke Wu
similarity 0.66
Abstract

Deterministic dynamics modeling of tendon-driven continuum robots remains challenging owing to uncertainties in material behavior, tendon transmission, friction, and contact. Measured joint configurations and nominal tendon commands do not fully characterize these internal mechanical factors, leaving uncertainty in the subsequent motion. We therefore develop a history-conditioned, physics-informed flow-matching framework for probabilistic dynamics prediction, using motion and actuation histories to predict the distribution of the next complete joint configuration. By recursively sampling next-step configurations under prescribed commands, the model predicts distributions of future whole-body motions. In simulation, scenario-specific models achieve five-second trajectory Energy Scores (lower is better) of 12.05 mm under internal friction variation and 9.29 mm under unobserved actuation disturbances. Relative to the conditional variational autoencoder and diffusion baselines, Flow attains lower Energy Scores and coverage closer to the nominal level in both scenarios. Ablations support history and structural conditioning in both scenarios. On the physical robot, predictions under two tendon-command profiles excluded from training capture the principal motion sequences, with five-second Energy Scores of 11.91 and 11.42 mm, lower than the compared baselines. The predicted-to-measured spread ratios are 1.65 and 1.22 (closer to 1 is better). These results support history-conditioned probabilistic dynamics prediction under incomplete mechanical observations.

0.23

Task Convergent Flow Matching for Generative Robotics

cs.RO cs.LG ·2026-11-17 ·Lorenzo Paiola, Maximilian Stölzle, Giorgio Grioli, Daniela Rus, Cosimo Della Santina
similarity 0.63
Abstract · Submitted to L4DC 2027

Successful manipulation requires the manipulated objects to progress toward a desired task outcome. Generative policies can produce plausible robot motions without ensuring this progress, allowing successive action chunks to stall or undo earlier changes to the scene. We introduce a task-centric inference-time refinement framework for flow matching policies that evaluates candidate action chunks through parallel physics-based rollouts and searches for motions satisfying sufficient conditions for task convergence. Task-level performance funnels bound temporary regressions while requiring net progress over each chunk. We establish sufficient conditions under which repeated funnel satisfaction yields convergence throughout execution, and use interval-wise Lipschitz bounds to verify continuous-time funnel satisfaction from sampled rollouts. To accommodate contact-rich hybrid dynamics, we refine the flow's source seeds using derivative-free Covariance Matrix Adaptation Evolution Strategy (CMA-ES), constrained to radial quantile bands of the source distribution to help retain the learned motion prior. This formulation supports both Euclidean and pose-valued Riemannian Flow Matching (RMFM) policies. On PushT and three-cube stacking, refinement raises success from 31.3% to 88.5% and from 77.0% to 86.0%. On PushT, whose coverage score responds continuously to the motion of the block, a learned funnel lowers the average maximum loss of achieved progress by 19% relative to constraining the endpoint of each chunk alone, at a similar success rate. With the radial band, 11.3% of the executed motion windows are out of distribution, a rate within the range of held-out demonstrations, compared with 34.1% for an unconstrained search.

0.23

DEAL-Grasp: Decoupled Alignment Representation for Geometry-Aware Dexterous Grasp Generation

cs.RO cs.GR ·2026-09-23 ·Fuqiang Zhao, Qian Liu
code/data similarity 0.62
Abstract

Synthesizing realistic articulated hand-object interactions is a fundamental problem in virtual reality, embodied intelligence, and digital human applications. Existing methods for dexterous grasp synthesis typically regress or denoise poses in a joint space that couples global rigid motion with local articulation, which often yields unstable samples and physically implausible contacts. We introduce DEAL-Grasp, built upon the Decoupled Alignment (DEAL) representation, which reformulates grasp synthesis as alignment-space generation: the interaction state comprises task-space geometric anchors and articulation parameters, from which the rigid transform is recovered via closed-form Procrustes alignment while preserving local articulation. On this mixed state, we model grasp generation using heterogeneous-state flow matching with component-wise vector fields, incorporating time-adaptive physical regularization during training. At inference, grasps are synthesized solely by integrating the learned vector field, without test-time optimization or auxiliary physical guidance. Across MultiDex and zero-shot RealDex benchmarks, DEAL-Grasp attains high force-perturbation success rates alongside minimal penetration and high diversity of generated grasps, while substantially reducing native inference latency compared to optimization-heavy baselines. The project page is available at https://wmtlab.github.io/DEAL-Grasp/.

0.26

DeepSoMo: Deformable Object Motion Estimation under Proprioceptive Partial Observability via Predictive Latent Dynamics

cs.RO ·2026-11-08 ·Hanwen Zhao, Weicheng Dai, Luoyao Chen, Chen Feng
position similarity 0.62
Abstract

Estimating the motion of deformable bodies in soft robotics from proprioceptive sensing is challenging due to partial observability, as internal observations provide only indirect access to global motion, contact, and shape evolution. In such settings, instantaneous observations are insufficient, and accurate estimation requires temporally consistent latent representations. This paper studies deformable motion estimation under proprioceptive partial observability from a latent dynamics perspective. We argue that a key requirement for long-horizon estimation is latent temporal consistency, where latent states remain predictive of future observations. To this end, we present DeepSoMo, a learning-based framework that enforces predictive structure in latent dynamics through a self-supervised latent prediction constraint. We evaluate the approach in a canonical deformable dynamics setting using high-fidelity finite-element simulation as a controlled source of ground truth. Results demonstrate consistent improvements in trajectory estimation, orientation recovery, shape reconstruction, and contact inference over architecture-matched recurrent baselines.

0.22

OATH: Object-Agnostic Tracking for Dexterous Hands, Enabling Rapid Physics-based Retargeting

cs.RO ·2026-10-09 ·Solvin Sigurdson, Prasanna Sriganesh, Juan Borbon, Joel Burdick, Dingqi Zhang +1 more
code/data similarity 0.61
Abstract · 8 pages, 7 figures, 3 tables. Project page: https://multiplylabs.github.io/oath/

Retargeting human demonstrations into dynamically feasible dexterous hand trajectories requires resolving complex contact physics, yet current methods either omit physical constraints or rely on computationally expensive sampling-based optimization in simulation. Reinforcement learning has previously been used to achieve planning and control through contact dynamics; however, standard policies require object-specific training and fail to generalize across arbitrary geometries. This work reframes physics-based retargeting as a residual learning problem, where we demonstrate that zero-shot generalization across objects relies on structured state representations. Specifically, formulating the observation space to include a localized contact-error signal guiding each robot link to its target coordinate on the object surface enables a single, unified residual policy to successfully grasp novel, unseen object geometries. Rolling out this policy in simulation on a kinematic reference trajectory yields a physics-based retargeter, OATH. Operating at 0.4 seconds of compute per second of reference, OATH achieves 2.4x faster-than-real-time performance and a 60-350x reduction in computational cost compared to sampling-based methods. The system recovers 58% of held-out and 54% of unseen-object DexYCB trajectories while maintaining a sub-2 cm tracking error.

0.29

TactileLab: Sim-to-Real Dexterous Robot Manipulation with Efficient Shear-Sensitive Tactile Simulation

cs.RO ·2026-11-05 ·Yijiong Lin, Wenhao He, Toby Jack, Max Yang, Haoran Li +1 more
similarity 0.61
Abstract

Robotic simulators that represent shear within tactile sensing are becoming increasingly important for contact-rich manipulation. However, simulating contact shear with high-fidelity deformable or soft-body simulation is computationally demanding, limiting scalability to large-scale reinforcement learning. Here we introduce \textit{TactileLab}, a scalable multimodal tactile simulation framework built on IsaacLab, with a particular focus on a new optical-flow-based tactile representation termed \textit{simulated tactile flow}. Instead of relying on soft-body physics to model shear deformation, TactileLab uses rigid-body simulation to efficiently estimate dense tangential motion at the contact interface, producing tactile flow fields that encode shear-relevant contact dynamics. This flow field complements geometric features from a contact depth field. We further introduce a novel policy architecture for jointly processing long-horizon proprioceptive signals and rich tactile observations. Across multiple contact-rich tasks, policies trained using simulated tactile flow learn efficiently at scale and exhibit emergent contact-aware behaviours. To facilitate sim-to-real policy deployment, we then propose a novel real-to-sim tactile translation method that maps real tactile image streams to simulated tactile flow and depth fields, together with a new tactile data collection method, and validate it across diverse tactile sensor designs. Using this pipeline, we demonstrate real-world deployment on tasks encompassing dual-arm and dexterous in-hand manipulation. Overall, TactileLab provides an end-to-end sim-to-real framework for scalable tactile policy learning, enabling diverse shear-sensitive manipulation across heterogeneous tactile robotic systems.

0.19

Primitive-Informed Sampling-Based MPC for Multi-Fingered Dexterous Manipulation

cs.RO ·2026-09-14 ·Emek Barış Küçüktabak, Karankumar Patel, Jinda Cui, Zhaodong Yang, Kazuhiro Sasabuchi +1 more
hot direction similarity 0.60
Abstract

We present a primitive-informed sampling-based model predictive control (MPC) framework for multi-fingered dexterous manipulation. Sampling-based MPC evaluates candidate control trajectories through forward simulation without requiring gradients through complex contact dynamics. However, directly sampling these trajectories in the high-dimensional joint space of a dexterous hand is inefficient and makes performance strongly dependent on the sampling distribution. Our framework biases sampling using low-dimensional manipulation primitives that encode coordinated finger motions, while simultaneously optimizing joint-level residuals to adapt these motions to the current hand-object configuration. Task-related rollout constraints reject infeasible trajectories during forward simulation, improving the effective use of the sampling budget. We evaluate the approach on a 16 DoF Allegro hand using a synchronized MuJoCo digital twin. Ablations show that both the primitive and residual are necessary for reliable continuous in-hand rotation, that increasing the sampling budget alone does not recover this coordination, and that rollout constraints substantially improve success rate. A primitive extracted for one object size transfers to other sizes and remains effective under model mismatch. The framework further supports grasping, object reorientation, and coordinated arm-hand manipulation, using primitives extracted from both a simulation-trained policy and human hand-motion data.

0.46

DexTouch-WM: Learning Action-Conditioned Tactile World Models from Human Touch for Dexterous Robot Manipulation

cs.RO cs.CV ·2026-09-17 ·Yan Qin, Yue Chen, Wenwei Lin, Shujia Liu, Chuqiao Lyu +7 more
publishedhighly cited similarity 0.60
Abstract · Accept to IROS 2026 Workshop RoBoWoMo (Lightning Talk)

Learning predictive models of contact-rich dexterous manipulation requires dense tactile interaction, but such data are costly to scale on real robots and remain tied to embodiment-specific sensors. We introduce DexTouch-WM, an action-conditioned world model that learns from scalable human touch to jointly predict future RGB observations and bilateral tactile dynamics. Our insight is that human and robot manipulation share transferable contact dynamics when their tactile observations and action spaces are made compatible. We deploy flexible piezoresistive arrays with a shared sensing layout on both human and dexterous robot hands, and retarget human motion into the robot action space so that human interaction can supervise the same dynamics model used for real-robot prediction. DexTouch-WM couples a pretrained video expert with a lightweight tactile expert using anatomy-aware tactile tokens and aligned action conditioning. In human-to-robot scaling experiments, we keep five hours of real-robot supervision fixed while increasing human interaction from 0 to 100 hours, and observe substantial improvements in held-out robot-domain visual, geometric, and contact prediction despite disjoint human and robot task sets. Beyond prediction, we evaluate the world models as surrogate environments for policy evaluation and as generators of synthetic trajectories for real-robot policy learning, showing that scalable human interaction provides a complementary data axis for learning dexterous robot world models.

0.52