CS Paper Radar

Touch Where It Matters: Learning Local Tactile Policies for Fine-Grained Gripper Control

Mingyuan Ba, Yanan Zhou, James Zhao, Weiming Zhi

cs.RO ·submitted 2026-10-17 06:10 UTC · v1

Abstract

Visual imitation policies can organise task motion, yet contact-rich object use demands gripper adjustments beyond coarse opening and closing. We study fine-grained gripper control as a local feedback problem within ongoing task execution. Our approach assigns tactile feedback a dedicated control role during contact-rich task stages. We introduce ContactPolicy, a learned local controller that takes over the gripper while a visual base policy continues to control the arm. ContactPolicy combines tactile and force histories with current robot state to jointly predict gripper increments and their modulation, updating the gripper target from each new observation. An Intent policy manages control handover between the visual base policy and ContactPolicy. Real-world experiments on container opening, object handling, and tool operation show improved task success over both the Visuomotor Policy and Tactile-Force Policy baselines. Execution analysis illustrates how the gripper adjusts contact within an ongoing task, including periods of target retention and fine-grained changes as interaction conditions evolve. Arm-state ablations and failure analysis further examine how execution context supports local regulation. These results support task-local tactile policies for fine-grained gripper control within visual manipulation.

Comment
9 pages, 11 figures, 3 tables

Closest papers

Nearest neighbours by meaning across the whole index.

PredTac: Learning Contact-Rich Manipulation with Predicted Touch

cs.RO ·2026-09-14 ·Weijia Fan, Daqiang Guo
similarity 0.80
Abstract · 8 pages, 4 figures

Contact-rich manipulation benefits from tactile feedback, yet physical tactile sensors introduce hardware, calibration, synchronization, and maintenance costs that complicate policy learning and deployment. We formulate predicted touch as an alternative to measured tactile input and present PredTac, a framework that learns to infer tactile states from causal visual observations and robot states and uses the predicted touch as an explicit interface for policy learning and execution. A tactile predictor is first trained with tactile supervision and then used to provide contact information without requiring measured tactile input during downstream policy training or execution. We evaluate PredTac across three contact-rich manipulation tasks in simulation and on a real robot, and further examine how policy performance depends on the predicted contact content. In simulation goal-offset evaluations, predicted-touch policies achieve 27.0%, 52.0%, and 44.7% success on USB, Barbed-spike, and Valve, respectively, improving over the visual baseline by 8.0-13.7 percentage points. On the real robot, predicted-touch ACT achieves 70.0%, 50.0%, and 90.0% success on USB insertion, Barbed extraction, and Valve rotation, respectively, with a three-task mean of 70.0%, approaching measured-touch ACT at 72.2% and substantially outperforming visual ACT at 21.1%. Fixed-policy interventions further show that performance is sensitive to the spatial structure of predicted contact, with spatial rearrangement at fixed value distributions reducing Valve success by 10.7 percentage points. These results demonstrate that predicted touch can provide useful contact information for contact-rich manipulation without requiring tactile sensing as a policy input.

0.25

Visuo-tactile Diffusion Policy for Robotic Manipulation with Auxiliary Tactile Dynamics Prediction

cs.RO ·2026-11-02 ·Kaihong Ouyang, Zihao Su, Chengxuan Zheng, Weiyi Lai, Dong Zhou
published similarity 0.79
Abstract · 7 pages, 2 figures. Presented at ICCSSE 2026

Long-horizon robotic assembly requires reliable perception and action across grasping, alignment, and insertion. Vision captures global object pose and scene context, where as fingertip touch provides local contact cues that are especially useful as contact states change. We present a visuotactile diffusion policy that combines four-step histories from two camera views and two fingertip tactile images to condition action-sequence generation. A recurrent temporal representation integrates recent observations from both modalities. To encourage it to capture how contact evolves under action, an auxiliary objective predicts fingertip-image changes two steps ahead from the temporal representation and demonstrated actions. This objective is used only during training and adds no inference-time computation. Across three independent training seeds in MuJoCo assembly experiments, the full model achieves an average success rate of $59.0\% \pm 2.6\%$, improving on the visual-only baseline ($42.7\% \pm 1.5\%$) by 16.3 percentage points.

0.27

Learning Reflexive Behavior for Contact-Rich Manipulation

cs.RO ·2026-10-02 ·Quan Nguyen, Yunho Kim, Joonho Lee
similarity 0.79
Abstract

During contact-rich manipulation, interactions between a robot and its environment carry information about local geometry: a surface prevents penetration, a bore guides a peg. A controller that exploits these interactions can comply with environmental constraints while preserving task intent. Robot-earning systems commonly use position, hybrid force-position, or Cartesian impedance control. Their prescribed tracking objectives, stiffness, or force-control directions may not match local constraints and may degrade performance. We learn a proprioceptive reflex policy in simulation on three simple interaction primitives: a spring, a plane, and a rail. The policy maps task-space commands to joint-position targets using state history, without direct force or geometrical measurements. Once trained, it serves as a frozen execution layer beneath higher-level controllers. We evaluate it in dual-arm box lifting, peg insertion, and surface following. In box lifting, the reflex kept the force below the threshold while the baseline failed. In rough-surface following it stayed below the 10 N reference, on par with tuned hybrid force-position control. In 0.02 mm peg insertion It reduced mean estimated contact force to less than half that of the baseline while increasing hardware success rates from at most 22 % to 36-58 %. By separating contact response from command generation, the reflex policy provides motion planners, learned policies, and teleoperators with robust contact-rich execution.

0.15

GPT-6-Astra Understands Touch and Outperform Vision--Tactile--Language--Action Models

cs.RO cs.CL ·2026-10-20 ·Ziyong Wang, Yupei Lin, Runxuan Huang, Liang Lin, Dongyu Zhang +1 more
similarity 0.79
Abstract · https://gpt-as-tactile-policy.github.io/

General-purpose multimodal models have shown promise as direct robot action policies, yet whether they grasp the physics underlying real-world interactions and can apply that understanding to guide action remains underexplored. We investigate this question using contact-rich manipulation as a representative testbed, examining tactile understanding as a concrete instance of this broader capability. Contact-rich manipulation requires interpreting tactile feedback and adjusting actions after obstruction. Existing learned tactile policies acquire these capabilities through specialized pretraining and per-task optimization, with substantial data and training costs. We present GPT-as-Tactile-Policy, which summarizes expert demonstrations into task guidance and tactile references for contact assessment and corrective action. GPT-6-Astra combines this context with multimodal observations and interaction history to generate control commands and adjust actions through contact feedback. Without additional robotics-specific pretraining or per-task policy fine-tuning, our method achieves 93.8\% mean success across five UniVTAC tasks. Overall performance is comparable to state-of-the-art policies, with higher success on selected tasks. On Insert Hole, tactile feedback improves success from 31\% to 99\%. Execution analysis further demonstrates contact-guided probing and correction, supporting manipulation progress through explicit expert experience and online feedback.

0.40

GraspTune: Tactile-Driven Execution Refinement for Robust Grasping

cs.RO ·2026-09-21 ·Juntao Li, Xingke Xia, Sichao Liu, Daqiang Guo
similarity 0.78
Abstract

Visual grasp proposal generation has advanced rapidly, yet converting a selected proposal into a stable physical grasp remains a central execution-stage challenge. This paper introduces GraspTune, a tactile-driven execution-stage refinement framework that starts from a nominal proposal and applies bounded residual TCP motions during approach, contact formation, and final grasp execution. GraspTune learns control-facing contact semantics from local depth, tactile signals, state, and history using state-conditioned expert contact queries and multi-task supervision for contact change, contact risk, and post-close readiness. The representation conditions a diffusion-pretrained residual policy and is aligned with PPO for closed-loop execution. Across more than 60,000 simulated executions over 20 object categories, GraspTune establishes an execution-layer benefit across four proposal generators, raising stable grasp success by +19.22, +9.55, +12.45, and +20.70 percentage points for GraspNet, Contact-GraspNet, AnyGrasp, and VGN. A four-fold held-out category study raises unseen-object execution from 54.58% to 70.33%, showing category-disjoint generalization of contact correction. Across more than 1,000 real-robot trials on a UR5e setup with Xense fingertip sensors, GraspTune raises GraspNet execution from 71.0% to 84.3%, validating direct transfer without realworld policy fine-tuning. Together, these results turn visually plausible proposals into stable physical grasps for downstream contact-rich manipulation. A supplementary video is available at https://youtu.be/kcq7fSLNtzU.

0.28

Robot Learning with Visual Predicted Force

cs.RO cs.AI ·2026-10-03 ·Haonan Chen, Feiyang Wu, Yuxiang Ma, Mustafa Mete, Pengfei Ye +9 more
similarity 0.78
Abstract · 9 pages, 8 figures. Project Website: https://visual-force.github.io/

Force-aware manipulation typically relies on specialized force or tactile sensors. We show that force-aware manipulation can instead be achieved through visual force prediction from the deformation of a compliant Fin Ray gripper. Our approach trains two models. First, we train a visual force estimator on calibration data and use it to annotate task demonstrations with force estimates. Second, we train an action--force proposal policy on these force-augmented demonstrations to jointly generate candidate robot actions and their associated forces. At test time, we sample candidate actions and the forces they are expected to produce, then execute the action whose predicted force is closest to a target from the demonstrations. We evaluate our approach on berry picking, empty-can grasping, in-hand reorientation, and plug insertion. Our results show that visual force prediction can guide inference-time action selection for contact-rich manipulation without requiring force or tactile sensors at deployment.

0.25

Embodied Task Agents Can Perform Fine Manipulation through Tactile In-Context Learning

cs.RO ·2026-10-20 ·Yubang Wang, Zezheng Huai, Yifei Zhang, Jinhao Wu, Yitong Chen +6 more
code/data similarity 0.78
Abstract · 24 pages. Project page: https://no-518.github.io/tactile-eta/

Can embodied task agents use tactile experience to perform fine manipulation without task-specific model training? We investigate this capability through Tactile-ETA, a system that incorporates tactile experience into an embodied task agent through in-context learning. Successful expert demonstrations show how contact changes during manipulation. We pair each recorded robot movement with its camera views and bilateral tactile observations. We keep tactile observations at the start and end of each movement and add two neighboring time points when the tactile images change markedly in between. The agent combines these demonstrations with current vision, tactile observations, and proprioception to select actions through general-purpose robot tools, without updating model parameters. Across four simulated manipulation tasks, adding tactile observations to the same two expert demonstrations increases average success from 15.75% to 24.00%, with current sensing and execution interfaces held fixed. Further experiments show that performance depends on the sensory content and representation of demonstrations, while comparing agent models on Lift Bottle reveals substantial differences under the same system configuration. On the real-robot tasks, demonstrations with aligned visual and tactile observations increase the task success rate from 0% to 60%. These findings demonstrate the feasibility of using tactile experience in context for fine manipulation and suggest a path toward extending embodied task agents to additional sensory inputs.

0.46

ContactBench 1.0: A Benchmark for Everyday Contact-Rich Robotic Manipulation

cs.RO ·2026-10-17 ·Jingyi Xiang, Yixuan Wang, Jeannette Bohg, Wenzhen Yuan
benchmark similarity 0.78
Abstract · Project website: https://jingyi-xiang.github.io/contactbench

Contact-rich manipulation underlies many everyday human activities, yet evaluating robots on such tasks remains difficult: the tasks involve diverse contact behaviors and are hard to reproduce across laboratories. Drawing on observations of everyday human manipulation, we identify eight essential skills that provide a practical framework for studying contact-rich manipulation. We introduce ContactBench 1.0, an extensible benchmark that instantiates these skills through 36 reference tasks and more than 40 object assets. The assets capture task-relevant geometry and contact behavior, can be fabricated reproducibly on consumer-grade 3D printers, and are each paired with a matching simulation model. Controlled task variations in both the real world and simulation enable evaluation of how well robot systems generalize. We evaluate recent representative vision-only and visual-tactile policy learning methods on six benchmark tasks. The results reveal that while vision-only policies struggle to perceive or react to contact events, visual-tactile policies do not meaningfully improve their performance, highlighting that contact-rich manipulation remains a significant challenge for current policy learning methods. By providing common task definitions alongside physical and simulated assets, ContactBench 1.0 offers a shared testbed for developing, evaluating, and comparing contact-rich manipulation systems. The benchmark will be open-sourced at https://jingyi-xiang.github.io/contactbench.

0.20