CS Paper Radar

Autonomous Droplet Navigation via Model-Based Reinforcement Learning: Zero-Shot Transfer and Emergent Dynamics

Rajneesh Anand, Mayuresh V. Kothare

cs.RO cs.LGeess.SY ·submitted 2026-10-03 04:06 UTC · v1
published

Abstract

Self-driving laboratories (SDLs) are transforming chemical and materials discovery through closed-loop automation, yet automated infrastructure for physical manipulation of soft, deformable matter remains beyond current robotic platforms. A critical instance is autonomous droplet transport on an open surface, where contact-angle hysteresis, capillary pinning, and surface heterogeneity produce partially observable dynamics that pose significant challenges for classical model-based controllers. We introduce the first robotic platform for closed-loop autonomous liquid droplet navigation on an open, unconfined surface using model-based reinforcement learning. A two-axis tilting board coated with a thin silicone oil film drives the droplet, while an overhead camera provides real-time feedback. A learned policy was trained on just 50 to 150 physical episodes depending on geometric complexity, without simulation or analytical models. Beyond performance alone, the platform demonstrates three capabilities of interest to the SDL community: it robustly transfers zero-shot to unseen geometries; it autonomously discovers an oscillatory depinning strategy to free the droplet when it sticks; and it completes its full training pipeline in under 90 minutes. These results extend reinforcement-learning manipulation from rigid microrobots to deformable soft-matter systems for next-generation SDLs.

Comment
Accepted for presentation at the Robotics & Automation in Self-Driving Laboratories 2026 Workshop, IROS 2026. https://sites.tufts.edu/selfdrivinglabs-iros2026/

Closest papers

Nearest neighbours by meaning across the whole index.

Autonomous Droplet Navigation via Model-Based Reinforcement Learning

cs.LG cs.ROeess.SYphysics.flu-dyn ·2026-09-14 ·Rajneesh Anand, Mayuresh V. Kothare
code/datahot direction similarity 0.89
Abstract · 43 pages, 15 figures, 3 tables including supplementary material. The source code is available via GitHub at https://github.com/rajneeshanand…

Precise manipulation of liquid droplets underpins lab-on-a-chip platforms for diagnostics, chemical synthesis, and biological assays. Yet autonomous droplet transport through confined geometries of varying complexity remains an open challenge. Droplets exhibit contact-angle hysteresis, deformability, and capillary pinning, which make their response to actuation nonlinear and history dependent, that classical controllers and pre-programmed trajectories cannot cope in multi-turn environments. Here we demonstrate autonomous navigation of a liquid droplet through geometries of increasing complexity on a gravity driven (Labyrinth) platform using model-based reinforcement learning. A thin silicone oil film reduces contact-line pinning while two-axis tilt supplies the gravitational driving force, and an overhead camera tracks the droplet in real time. An offline-trained policy discovers effective tilt strategies from limited physical interaction data, without simulation or analytical droplet models. The system operates under partial observability, as oil-film thickness, instantaneous contact angle, and droplet deformation state remain hidden from the controller. Despite these challenges, the learned policy achieves reliable navigation across straight, right-angle, and curved-arc paths, including outside-corner geometries. We further demonstrate that a policy trained on a simpler geometry transfers to complex ones, succeeding zero-shot on right-angle and staircase paths and reaching full success on a curved arc with a fifth of the training data. The findings suggest promising avenues for enabling droplet based microfluidic systems to serve as intelligent chemical laboratories.

0.72

Ferrofluid Shape Control with Deep Reinforcement Learning

cs.RO ·2026-11-18 ·Anthony Czubarow, Antonio Terpin, Raffaello D'Andrea
similarity 0.65
Abstract

Ferrofluids are liquids containing magnetic nanoparticles, enabling shape control through external magnetic fields. This deformability supports their use as soft robotic bodies that can navigate confined spaces and manipulate objects. Their nonlinear continuum dynamics, however, make reliable control challenging. Existing work shows how spatial magnetic fields from electromagnet arrays can establish and maintain desired ferrofluid configurations. A complementary objective is to exploit the fluid's dynamics under limited actuation to reach prescribed shapes that may not be sustainable at equilibrium. Here, we use deep model-based reinforcement learning to control these transient shapes. Learning directly through physical interaction, our policies select current commands for a single fixed electromagnet based on visual observations, guiding the moving ferrofluid toward prescribed geometries. The learned policies repeatedly attain a variety of shapes spanning polygons, asterisks and pentominoes. These results demonstrate that learning directly from physical interaction enables reliable fluid shaping, opening new possibilities for controlling ferrofluid-based soft robots.

0.28

Fetch My Beer: Synthetic-to-real Hierarchical Policy for Smooth Pick-and-place

cs.RO ·2026-09-16 ·Yingyue Li, Chenyangguang Zhang, Ruida Zhang, Bowen Fu, Guangyao Zhai +1 more
top venuecode/data similarity 0.63
Abstract · 8 pages, 4 figures. Accepted by IEEE Robotics and Automation Letters (RA-L)

Many real-world robotic applications require dynamically sensitive manipulation, where success depends not only on reaching a target state but on maintaining stable object dynamics throughout execution. We study the stable transport of liquid-filled containers, where a robot must move objects to target locations while suppressing sloshing and preventing spillage. Unlike conventional pick-and-place, this task imposes stringent requirements on motion smoothness and trajectory-level stability, exposing clear limitations in existing systems. Specifically, fluid simulation remains too costly for online reinforcement learning; human teleoperation introduces unintended accelerations that induce sloshing during imitation learning; and current policy pipelines optimize for task completion rather than dynamic stability. We propose a synthetic-to-real framework coupling physically validated data generation with a hierarchical, diffusion-based controller. The scalable data pipeline synthesizes grasps, filters unstable poses via a vision-language model, and validates transport trajectories through fluid simulation. The policy is organized with a high-level module that translates language and visual observations into SE(3) control targets, and a latent diffusion controller that first plans efficiently in a compact latent space and then decodes dense action chunks, enabling the high control frequency needed for smooth and stable motion. Extensive experiments show our system outperforms state-of-the-art manipulation policies in transport smoothness and dynamic stability. Our project page: https://fetch-my-beer.github.io/

0.55

Programmable Droplets as Active Matter in a Reconfigurable Environment

cs.RO cond-mat.mtrl-sci ·2026-10-31 ·Ivan Shmarov, Abhishek Sharma, Michael Jirasek, Kristine Laws, Leroy Cronin
similarity 0.63
Abstract · 28 pages, 8 figures, 26 references

Exploring systems that bridge the gap between non living and living matter requires a reconfigurable environment coupled to synthetic systems capable of autonomous motion, adaptive behaviour and persistent dynamics. Self propelled droplets provide a minimal platform for such life like behaviour, but quantitative measures capturing both persistence and adaptability to enable closed loop reconfiguration remain lacking. Here we introduce an autonomous closed loop robotic platform to characterise and dynamically program the autonomous behaviour of multicomponent oil in water droplets. We explored two chemical composition spaces, with and without external pH perturbations, analysing droplet behaviour in real time and iteratively selecting formulations. To quantify this behaviour, we introduce two descriptors, persistence and diversity. Persistence is quantified by mapping temporal dynamics in a speed shape complexity space, where we identify a persistence domain that distinguishes active from inactive droplets. Diversity captures spontaneous or environmentally induced variation in motion and the propensity of droplets to reorient or adapt under perturbation. We show that persistence and diversity are constrained by a behavioural trade off: high persistence limits diversity, whereas high diversity reduces persistent activity. Our results provide a quantitative framework for navigating and programming life like behaviours in active droplets and show how closed-loop experimentation can uncover organising principles in complex active matter.

0.30

Embracing Flow Unsteadiness: A High-Throughput Learning Platform Enables Vortex-Exploiting Bioinspired Propulsion

cs.RO physics.flu-dyn ·2026-09-13 ·Fei Han, Xinyu Cui, Zhipeng Wang, Ning Yang, Hang Xu +5 more
similarity 0.62
Abstract

Biological swimmers and flyers exploit unsteady vortices for propulsion, whereas engineered vehicles usually suppress them as disturbances. Learning such flow exploitation in machines is difficult because real-fluid interaction data are scarce and unstructured exploration is unstable in high-dimensional, history-dependent flows. Here we present REEF, a co-designed physical-learning framework that integrates SHOAL, an eight-channel high-throughput array for real fluid--structure interaction, with V-STAR, a staged algorithm that converts these interactions into policies through imitation, offline internalization, and online adaptation. Across lift-based, drag-based, and momentum-jet propulsors, REEF expands the attainable force envelope to more than twice that of parameterized search. Particle image velocimetry shows that these gains arise from coordinated vortex formation, growth, and force projection, rather than refinement of a fixed motion-to-force mapping. Force-trained policies transfer zero-shot to free-moving robots whose body motion changes the surrounding flow, suggesting that REEF learns transferable wake-coupling principles for embodied propulsion in unsteady fluids.

0.18

Glass-Box Deep Reinforcement Learning for High-Speed Flow Control

cs.LG ·2026-10-19 ·Ameya D. Jagtap
position similarity 0.61
Abstract · 22 pages

Deep reinforcement learning (DRL) is challenging long-held assumptions in active flow control. By learning directly from interaction with a flow, DRL agents can find actuation strategies for nonlinear, unsteady flows without explicit models, often exploiting mechanisms that classical control would not anticipate. High-speed flows are a demanding test of this promise: shocks, shock/boundary-layer interactions, transition, aerothermal loading and microsecond-scale dynamics strain both algorithms and hardware. Yet most demonstrations remain idealized, low-speed simulations with limited evidence of robustness, interpretability, transferability or experimental deployment. In this perspective, we examine what constrains DRL-based high-speed flow control, including compressible simulation cost and fidelity, data efficiency, sensing in extreme environments, reward design, explainability, and generalization across Mach and Reynolds number. We argue that these limits reflect a tension between data-driven optimization and the structured physics, constraints and uncertainties of real flows. We then highlight where DRL may go beyond conventional approaches: adaptive control of shock-induced separation and turbulent boundary layers, coordination of distributed actuators, and discovery of unconventional flow-manipulation mechanisms.The central challenge, however, is translating these capabilities into reliable technology, which will require validated compressible simulation environments, physics- and constraint-aware learning, robustness to off-design conditions and sensor degradation, real-time operation, hardware-in-the-loop and wind-tunnel validation, and credible pathways to safety and certification.

0.29

Physical Diffusion Policy via MPM-Guided 3D Gaussian Simulation

cs.RO ·2026-11-03 ·Can Wang, Qiyuan Qiao, Lei Liu, Ge Yuan, Xinhui Liu +1 more
similarity 0.61
Abstract · 10 pages

Learning robust manipulation policies for diverse everyday objects remains challenging because objects with different materials exhibit distinct deformation and contact dynamics. While diffusion policies provide a powerful framework for modeling multimodal manipulation behaviors, they typically lack explicit physical reasoning and may produce averaged actions when facing heterogeneous objects. We present a physics-aware diffusion policy that augments visual imitation learning with object physical understanding. Our framework recovers material properties from visual demonstrations by coupling 3D Gaussian Splatting with differentiable Material Point Method simulation, where video guidance and particle-level constraints enable accurate physical parameter estimation. The estimated physics conditions are further integrated into a material-aware Mixture-of-Experts diffusion policy, allowing adaptive behavior generation across different physical regimes. For practical deployment, we introduce a physics memory bank that provides efficient physical retrieval without requiring online simulation optimization. Experiments on a wide range of simulated and real-world objects demonstrate improved manipulation success and generalization over existing diffusion-based approaches. These results show that incorporating physical knowledge is critical for scalable robot manipulation in diverse environments.

0.13

Grasp2Twist: Learning Bimanual Dexterous Jar Opening by Reinforcement Learning

cs.RO ·2026-09-25 ·Mo Xu, Yunfu Deng, Jianuo Wang, Josiah Hanna, Bilge Mutlu
similarity 0.60
Abstract · 9 pages, 6 figures. Under review

This paper presents Grasp2Twist, a bimanual dexterous manipulation system that learns to grasp and twist open jar lids using reinforcement learning. Learning this task raises three challenges: learning a unified policy for a multi-stage task, sustaining lid twisting, and sim-to-real transfer. To address the first challenge, we introduce a continuous enclosure measure to guide grasp formation and a binary enclosure indicator to guide the grasp-to-twist transition for unified policy learning. We derive both from the geometric relationship between the object center and the convex hull formed by the hand's palm and fingertips. Kinematic constraints limit how far the hand can rotate the lid with fixed contacts, so sustained twisting requires finger contact reconfiguration. We use a three-stage curriculum to facilitate exploration of these contact changes and also improve robustness for sim-to-real transfer. With our approach, the learned policy demonstrates finger gaiting, reconfiguring hand-object contacts to sustain lid rotation. It transfers zero-shot to the physical system and achieves an 88% task success rate across six household containers, including peanut-butter, vitamin, and instant-coffee jars. Ablations further validate the roles of the geometric enclosure in grasp formation and the curriculum in contact-reconfiguration exploration.

0.27