CS Paper Radar

Pose accuracy is not grasp reliability: residual pose posteriors improve simulated robotic grasp planning

Md Selim Sarowar, Sungho Kim

cs.RO cs.CV ·submitted 2026-09-02 13:22 UTC · v2
benchmarkcode/data

Abstract

Model-based robotic grasping estimates an object's six-dimensional (6D) pose and plans on its mesh as if that pose were exact. We show that this assumption reduces simulated grasp success even when the pose passes the standard accuracy test. We built LM-Grasp, a benchmark of 15{,}800 real LINEMOD RGB-D images with grasp outcomes simulated on scanned meshes. FoundationPose meets the ADD(-S) criterion (mean model-point error below 10\% of the object diameter) on 0.997 of test scenes, yet planning on its estimate succeeds in 0.791 of scenes, compared with 0.856 using the annotated pose. Because the estimator reports no uncertainty, we build a pose posterior from its own errors on annotated training frames, resampled from frames with similar observable fit, and score each grasp by its mean success across sampled poses and four equally weighted physics scenarios. Without retraining the estimator, posterior planning yields 0.859 (difference from the annotated-pose result +0.003; 95\% scene-wise paired-bootstrap interval [$-$0.003, +0.009]). This interval is compatible with no difference and a small shortfall; we did not conduct an equivalence test. For pose errors of 5 to 10\% of the diameter, simulated success rises from 0.796 to 0.984. Among test scenes assigned singleton-success conformal prediction sets, observed held-out precision is 0.972, compared with 0.904 for the point-estimate planner; these are empirical precision estimates, not conditional guarantees. On occluded LM-O scenes never used for tuning, posterior planning closes 66\% of the gap to the annotated pose.

Comment
Project Page : https://physical-agi.github.io/MSP-FRAMEWORK/

Closest papers

Nearest neighbours by meaning across the whole index.

Continual Learning for 6-DoF Grasp Synthesis via Experience and Demonstrations

cs.RO cs.LG ·2026-10-01 ·Giulio Schiavi, Andrei Cramariuc, Michael Pantic, Roland Siegwart
published similarity 0.75
Abstract · Accepted to CoRL 2026

Most current grasp synthesis systems are trained offline and remain fixed during deployment. While this works well when deployment conditions resemble the training data, performance can degrade when robots encounter conditions they have not seen before, such as unfamiliar objects. In this work, we present a continual-learning framework for single-view 6-DoF grasp synthesis for a parallel-jaw gripper in cluttered scenes. Rather than finetuning a large parametric model, our method adapts through memory in a learned embedding space: grasp outcomes update future grasp scores, while optional user demonstrations are recalled and transferred to new scenes as additional candidate grasps. We evaluate our method in simulation and in extensive real-world experiments comprising over 1500 grasp trials. We show that our method matches the performance of existing 6-DoF grasping baselines even before adaptation, improves online on unseen objects from categories absent or underrepresented during training, and supports long-horizon continual learning with limited forgetting. In real-world experiments, our method reaches over 90\% success rates on several challenging object categories after only 50 online grasp attempts. Videos and code at https://giuschio.github.io/cl_grasping/.

0.36

LAGR: Language-Aware Grasp Re-Ranking for Task Oriented Grasping

cs.RO cs.AIcs.LG ·2026-11-18 ·Manav Kulshrestha, Shreyans Jain, Apoorva Vashisth, Ankush Gupta, Damon Conover +1 more
dataset similarity 0.73
Abstract · Manav Kulshrestha and Shreyans Jain contributed equally to this work

Grasping is a fundamental capability for robotic manipulation, with the appropriate grasp depending on the downstream task. For example, a hammer should be held by its handle for hammering but by its head when it is handed over. Selecting such a grasp requires consideration of two criteria: physical feasibility on the object and suitability for the task. Developing such an approach requires dense grasp labels with both feasibility and task scores on real observations, but existing task-oriented grasping datasets are small, synthetic, or costly to annotate. To address this gap, we construct a dataset of 2.68 million scored 5-DoF grasp annotations over 100,911 image-task pairs from $17$ object categories. Additionally, we introduce LAGR, a task-oriented grasp detection method that explicitly addresses both these challenges. Given vision-language inputs, LAGR first generates task-independent grasp proposals along with a physical feasibility score, and then scores the generated grasps with a language-conditioned re-ranker that reasons over all proposals jointly. Experiments show that LAGR outperforms the next best method by 9.3 percentage points. We further validate LAGR on real objects by executing task-oriented grasps with a Franka Research 3 robot.

0.25

T(R,O) Grasp++: Cross-Embodiment Dexterous Grasping in Cluttered Environments at Scale

cs.RO ·2026-10-18 ·Xin Fei, Zhixuan Xu, Huaicong Fang, Fangjun Chen, Zhaoyu Mu +3 more
similarity 0.73
Abstract

Dexterous grasping in cluttered scenes remains a central challenge in robotic manipulation, as it requires high-dimensional hand coordination under complex object-environment constraints. To address this problem, we introduce one of the largest simulation benchmarks to date, comprising around 0.5B grasp annotations across eight dexterous hands and approximately 50K objects. Furthermore, we propose T(R,O) Grasp++, a scalable diffusion-based framework for cross-embodiment dexterous grasp synthesis in cluttered environments. Extensive experiments demonstrate that our model significantly outperforms existing baselines, achieving an average success rate of 88% across eight dexterous embodiments. Meanwhile, our pipeline achieves only 0.3s inference latency while improving inference sample efficiency by over 10 times.

0.13

VerifLM: Physically Verified Image-Edit-Guided Goal Pose Generation for Robotic Manipulation

cs.RO ·2026-11-02 ·Xuanyou Chen, Jinyu Chen, Glen Chou
similarity 0.72
Abstract · Submitted to IEEE ICRA 2027

Many manipulation goals are easier to visualize than to specify numerically through explicit geometric constraints. We present VerifLM, a framework that converts image-edited goal configurations into physically verified 6-DoF object poses for planning. Using an edited image, VerifLM recovers an initial relative pose between an active object and a passive object, then prompts a vision-language model to generate axis-wise sampling constraints and a task-specific validation program. Candidate poses are evaluated in a physics simulator, and the VLM selects the most task-consistent retained candidate. To operate without predefined object meshes or physical parameters, VerifLM reconstructs metric object geometry from RGB-D observations and constructs multiple hypothetical simulations with varying physics and geometry. Parallel-world validation evaluates candidates across geometry--physics variations and ranks them by the number of worlds in which they satisfy the validation program. We evaluate VerifLM on nine manipulation tasks. VerifLM achieves the highest task success on eight of nine tasks, with the largest performance gain on goals requiring precise orientation and contact relationships.

0.19

OmniDex: Scaling Dexterous Hand Grasping to Diverse Cluttered Scenes

cs.RO cs.CV ·2026-10-08 ·Naiyu Fang, Zhongjin Luo, Yuxin Mo, Siyuan Huang, Jianbo Liu +5 more
benchmark similarity 0.71
Abstract

Dexterous grasping is the foundational primitive in embodied AI, demanding massive data to train robust models. As real-world data collection is expensive, simulation has become the mainstream paradigm. Yet, while cluttered scenes best reflect real-world applications, learning to grasp within them is bottlenecked by a critical scarcity of large-scale data. To resolve this, we curate high-quality 3D objects and supporting bases, proposing a scalable seed-and-filter strategy that bypasses sluggish scene-level optimization. This yields an unprecedented benchmark comprising over 2.6 million scenes and 0.4B scene-specific grasp ground truths, featuring diverse realistic layouts paired with rich semantic and geometric observations. Furthermore, we introduce the OmniDex model to overcome the grasp multimodality and last-millimeter precision errors plaguing current generative models. By coupling Soft Winner-Takes-All learning with human-inspired physical constraints during training, and utilizing physics-driven ranking, our approach achieves robust dexterous grasping without the latency of post-optimization. Experimental results show that OmniDex model achieves state-of-the-art performance and strong generalization across diverse scenes, views, and unseen objects.

0.15

GoalDex: Learning Image-Goal-Conditioned Policies for Dexterous In-Hand Manipulation

cs.RO ·2026-11-01 ·Arjun Bhardwaj, Mayank Mittal, Maximum Wilder-Smith, Vaishakh Patil, Marco Hutter
similarity 0.71
Abstract

Dexterous in-hand manipulation is commonly formulated around explicit object pose estimation, yet reliable pose tracking is challenging under severe hand-object occlusions and often requires object-specific supervision. This dependence creates a bottleneck between perception and control that can limit robustness and scalability. We present an end-to-end visuomotor framework for dexterous in-hand reorientation that operates on RGB observations and proprioception without intermediate pose estimation. Our approach jointly learns visual representations and control policies through behavior distillation from privileged state teachers and auxiliary reconstruction objectives. Furthermore, we introduce image-goal-conditioned reorientation, allowing desired object configurations to be specified directly through images rather than canonical pose representations. We evaluate our approach across a diverse set of object geometries and instances in simulation, where it consistently outperforms pose-based methods and policies built on pretrained visual encoders. Finally, we demonstrate direct sim-to-real transfer of a single image-goal-conditioned policy across five real-world objects, showing that accurate dexterous reorientation can be achieved without explicit pose estimation. For videos and code release, please refer to the project website: https://rffr.leggedrobotics.com/works/goaldex/

0.17

2nd Place Solution to the HANDS 2026 Workshop Challenge-Dexterous Grasp Motion Track: Single-Shot Trajectory Warping for Grasp Motion Generation

cs.RO cs.AIcs.CV ·2026-09-18 ·Muneeb A. Khan, Woojin Kim, Shinwoo Kim, Muhammad Munsif, Binod Bhattarai +1 more
similarity 0.70
Abstract

This report describes our 2nd place solution to the HANDS 2026 workshop challenge (Dexterous Grasp Motion track) in conjunction with ECCV 2026. In this challenge, we address grasp motion generation for the 12-DoF LinkerHand O6, aiming to produce physically plausible reach-and-lift trajectories for unseen objects from randomized initial hand poses in simulation. This task is particularly challenging because each grasp requires a per-step policy to make approximately $70$ twelve-dimensional decisions, with errors accumulating over time, while test objects and physical dynamics may differ from those encountered during training. To address these challenges, we propose editing a single successful GraspM3 demonstration instead of generating the motion step by step: a policy observes the object once and outputs a 12-D warp of the demonstration, which is then replayed open-loop. Moreover, we train the warp policy with one-step PPO over all $4{,}824$ training objects in parallel. As a result, our method achieved success rates of $94.61\%$ on the easy track, the highest of all submissions, and $57.18\%$ on the hard track of the private test set.

0.21

Predicted Futures Are Not Enough: Learning Executable Goals for Robot Manipulation

cs.RO ·2026-10-07 ·Tzu-Yu Chuang, Ching-Hsiang Chang, Yi-Hsiu Lee, Yi-Ting Chen, Min Sun +1 more
code/data similarity 0.70
Abstract · 9 pages, 8 figures, 4 tables. Project page: https://claire0730.github.io/executable-goals/ Code and models: https://github.com/Claire0730/ex…

Generative world models provide rich predictions of how manipulation scenes may evolve toward task objectives, yet those futures do not directly expose the compact task variables required by control. When training supervises future prediction alone, terminal goal accuracy is not an explicit learning objective, even when geometric recovery is available. We present Entity-Level Goal Readout, a learned prediction-to-execution interface that makes the executable terminal goal an explicit output of a 3D trace world model. It combines object-centric pose prediction with translation grounded in observed depth to produce a compact goal in SE(3). A shared Pose-Native Executor consumes this fixed goal with online object-pose feedback for closed-loop control without rerunning the world model. Across five manipulation tasks, the pipeline achieves a mean success rate of 79.69%. Goal diagnostics directly measure terminal goal accuracy, while controlled translation perturbations characterize how execution degrades under goal error. Zero-shot deployment on a Franka arm achieves 73.33% success on nominal StackCube, 66.67% with distractors, and 75.00% on PickPlate with a target unseen during policy training. These results support treating the prediction-to-execution interface as an explicit learned component of world-model planning rather than incidental post-processing in the control pipeline itself. Project page: https://claire0730.github.io/executable-goals/

0.37