CS Paper Radar

Glass-Box Deep Reinforcement Learning for High-Speed Flow Control

Ameya D. Jagtap

cs.LG ·submitted 2026-10-19 18:30 UTC · v1
position

Abstract

Deep reinforcement learning (DRL) is challenging long-held assumptions in active flow control. By learning directly from interaction with a flow, DRL agents can find actuation strategies for nonlinear, unsteady flows without explicit models, often exploiting mechanisms that classical control would not anticipate. High-speed flows are a demanding test of this promise: shocks, shock/boundary-layer interactions, transition, aerothermal loading and microsecond-scale dynamics strain both algorithms and hardware. Yet most demonstrations remain idealized, low-speed simulations with limited evidence of robustness, interpretability, transferability or experimental deployment. In this perspective, we examine what constrains DRL-based high-speed flow control, including compressible simulation cost and fidelity, data efficiency, sensing in extreme environments, reward design, explainability, and generalization across Mach and Reynolds number. We argue that these limits reflect a tension between data-driven optimization and the structured physics, constraints and uncertainties of real flows. We then highlight where DRL may go beyond conventional approaches: adaptive control of shock-induced separation and turbulent boundary layers, coordination of distributed actuators, and discovery of unconventional flow-manipulation mechanisms.The central challenge, however, is translating these capabilities into reliable technology, which will require validated compressible simulation environments, physics- and constraint-aware learning, robustness to off-design conditions and sensor degradation, real-time operation, hardware-in-the-loop and wind-tunnel validation, and credible pathways to safety and certification.

Comment
22 pages

Closest papers

Nearest neighbours by meaning across the whole index.

Embracing Flow Unsteadiness: A High-Throughput Learning Platform Enables Vortex-Exploiting Bioinspired Propulsion

cs.RO physics.flu-dyn ·2026-09-13 ·Fei Han, Xinyu Cui, Zhipeng Wang, Ning Yang, Hang Xu +5 more
similarity 0.66
Abstract

Biological swimmers and flyers exploit unsteady vortices for propulsion, whereas engineered vehicles usually suppress them as disturbances. Learning such flow exploitation in machines is difficult because real-fluid interaction data are scarce and unstructured exploration is unstable in high-dimensional, history-dependent flows. Here we present REEF, a co-designed physical-learning framework that integrates SHOAL, an eight-channel high-throughput array for real fluid--structure interaction, with V-STAR, a staged algorithm that converts these interactions into policies through imitation, offline internalization, and online adaptation. Across lift-based, drag-based, and momentum-jet propulsors, REEF expands the attainable force envelope to more than twice that of parameterized search. Particle image velocimetry shows that these gains arise from coordinated vortex formation, growth, and force projection, rather than refinement of a fixed motion-to-force mapping. Force-trained policies transfer zero-shot to free-moving robots whose body motion changes the surrounding flow, suggesting that REEF learns transferable wake-coupling principles for embodied propulsion in unsteady fluids.

0.18

Physics Foundation Models for Efficient Control of Fluid Flows

cs.LG physics.flu-dyn ·2026-10-10 ·Mahindra Rautela, Shaifalee Saxena, Alexander Scheinker
code/data similarity 0.64
Abstract

We investigate whether a physics foundation model (PFM) pretrained for PDE forecasting can be repurposed for closed-loop fluid-flow control. We construct a PFM-based controller by attaching lightweight Gaussian actor and critic heads to frozen pretrained representations and train the policy using proximal policy optimization (PPO). We evaluate the approach on HydroGym's two-dimensional Kolmogorov-flow environment using vorticity observations at \(64\times64\) and \(128\times128\) resolutions and Reynolds numbers \(\mathrm{Re}=100\) and \(200\). Across these settings, we compare the frozen PFM controller against a baseline MLP policy trained with the same PPO framework. The PFM controller achieves consistently higher reward while requiring smaller executed action magnitudes over training, indicating improved control performance with reduced control effort. These results provide initial evidence that representations learned through self-supervised next-frame prediction can be transferred directly to closed-loop fluid flow control. Project page: https://mahindrautela.github.io/morph.

0.61

Reinforcement learning for control of partial differential equations

cs.LG ·2026-10-29 ·Nicolò Botteghi, Matteo Tomasetto, Andrea Manzoni
similarity 0.62
Abstract

Controlling dynamical systems governed by partial differential equations (PDEs) remains a challenging task in scientific computing, owing to high-dimensional state spaces, distributed actuation, nonlinear dynamics, and stringent real-time constraints. Reinforcement learning (RL) has emerged as a promising paradigm for synthesizing feedback controllers that adapt to varying operating conditions and can be deployed efficiently in real-time. This review provides a unified perspective on RL and PDE-constrained optimization, and surveys model-free, model-based, and multi-agent RL approaches to PDE control, with particular emphasis on sample efficiency, generalization across parametric variations, and scalability to high-dimensional problems. Two demonstrative examples, the stabilization of a chaotic system and a flow control problem, illustrate the performance and pitfalls of RL agents in practice. Finally, we discuss evaluation and benchmarking practices, current limitations, and open research directions toward reliable, data-efficient, and scalable RL for scientific computing.

0.17

LLM-Enhanced Deep Reinforcement Learning with Human-in-the-Loop Calibration for Real-Time Optimal Power Flow

eess.SY cs.AI ·2026-11-11 ·Jiayang Tu, Bozhen Jiang, Bojun Du, Yunxiao Ling, Ershun Du +1 more
similarity 0.61
Abstract

Real-time optimal power flow (RT-OPF) requires fast and reliable decision-making under time-varying operating conditions. However, many practical operating preferences and safety related requirements are difficult to encode using conventional numerical rewards alone, which limits the effectiveness and controllability of deep reinforcement learning (DRL) for RT-OPF. To address this issue, this paper proposes an LLM-enhanced DRL framework with human-in-the-loop (HITL) for RT-OPF. The proposed method augments the physics-based reward with sparse language evaluative feedback during offline training, so that semantic operational requirements can be incorporated into policy learning. To improve reliability in RT-OPF, a selective HITL calibration mechanism is further introduced to inspect and correct questionable LLM judgments before they continuously shape the learning process. Case studies on the IEEE 30-bus and IEEE 118-bus systems show that LLM feedback improves policy safety over vanilla TD3, while further HITL calibration achieves the best overall performance and more stable training behavior. These results demonstrate the effectiveness of combining language guidance and expert calibration for RT-OPF.

0.14

Autonomous Droplet Navigation via Model-Based Reinforcement Learning: Zero-Shot Transfer and Emergent Dynamics

cs.RO cs.LGeess.SY ·2026-10-03 ·Rajneesh Anand, Mayuresh V. Kothare
published similarity 0.61
Abstract · Accepted for presentation at the Robotics & Automation in Self-Driving Laboratories 2026 Workshop, IROS 2026. https://sites.tufts.edu/selfdr…

Self-driving laboratories (SDLs) are transforming chemical and materials discovery through closed-loop automation, yet automated infrastructure for physical manipulation of soft, deformable matter remains beyond current robotic platforms. A critical instance is autonomous droplet transport on an open surface, where contact-angle hysteresis, capillary pinning, and surface heterogeneity produce partially observable dynamics that pose significant challenges for classical model-based controllers. We introduce the first robotic platform for closed-loop autonomous liquid droplet navigation on an open, unconfined surface using model-based reinforcement learning. A two-axis tilting board coated with a thin silicone oil film drives the droplet, while an overhead camera provides real-time feedback. A learned policy was trained on just 50 to 150 physical episodes depending on geometric complexity, without simulation or analytical models. Beyond performance alone, the platform demonstrates three capabilities of interest to the SDL community: it robustly transfers zero-shot to unseen geometries; it autonomously discovers an oscillatory depinning strategy to free the droplet when it sticks; and it completes its full training pipeline in under 90 minutes. These results extend reinforcement-learning manipulation from rigid microrobots to deformable soft-matter systems for next-generation SDLs.

0.21

Learned Warm Starts for Steady Transonic RANS Simulations of Wing and Aircraft Configurations

cs.CE physics.flu-dyn ·2026-10-31 ·Muhammad Irfan Zafar, Saleh Abuhanieh, Şahin Yiğit
similarity 0.61
Abstract

Neural surrogates can accelerate many-query aerodynamic analyses, but their standalone predictions may be insufficiently accurate or robust to support reliable design evaluations. We develop a learned warm-start formulation that predicts mean-flow and turbulence fields directly on native CFD meshes for subsequent correction by the RANS solver. A common architecture and objective structure are used across subcritical and transonic flows, including challenging conditions with shock-induced separation. A condition-modulated Transolver combines Fourier and wall features with target-gated regularization, a Sobolev-type loss, adaptive loss weighting and sampled training. Prediction requires no preliminary flow solution, and training requires no solver-residual evaluation. Extensive 3D CFD evaluations on SuperWing, the NASA Common Research Model (CRM) and DLR-F6 assess geometric and operating-condition generalization, including interpolation, extrapolation and cross-mesh deployment, using Spalart-Allmaras and $k$-$ω$ SST closures on million-cell meshes. Unseen nominal cases achieve geometric-mean iteration speed-ups of approximately $2$-$4\times$ under a common convergence criterion based on cold-run aerodynamic coefficients. On CRM, learned initialization provides greater aggregate acceleration than nearest-condition restarts using the same solution library. Resolution invariance enables direct deployment of coarse-mesh models on finer meshes without retraining, solution interpolation or target-condition coarse solves. Including data generation and training, estimated campaign speed-ups reach $2.1$-$2.3\times$. No warm run diverges; fewer than $2\%$ fail to satisfy the cold-reference convergence criterion. These results support learned initialization for repeated aerodynamic analyses, while highlighting initialization sensitivity near separation as a remaining limitation.

0.06

Autonomous Droplet Navigation via Model-Based Reinforcement Learning

cs.LG cs.ROeess.SYphysics.flu-dyn ·2026-09-14 ·Rajneesh Anand, Mayuresh V. Kothare
code/datahot direction similarity 0.61
Abstract · 43 pages, 15 figures, 3 tables including supplementary material. The source code is available via GitHub at https://github.com/rajneeshanand…

Precise manipulation of liquid droplets underpins lab-on-a-chip platforms for diagnostics, chemical synthesis, and biological assays. Yet autonomous droplet transport through confined geometries of varying complexity remains an open challenge. Droplets exhibit contact-angle hysteresis, deformability, and capillary pinning, which make their response to actuation nonlinear and history dependent, that classical controllers and pre-programmed trajectories cannot cope in multi-turn environments. Here we demonstrate autonomous navigation of a liquid droplet through geometries of increasing complexity on a gravity driven (Labyrinth) platform using model-based reinforcement learning. A thin silicone oil film reduces contact-line pinning while two-axis tilt supplies the gravitational driving force, and an overhead camera tracks the droplet in real time. An offline-trained policy discovers effective tilt strategies from limited physical interaction data, without simulation or analytical droplet models. The system operates under partial observability, as oil-film thickness, instantaneous contact angle, and droplet deformation state remain hidden from the controller. Despite these challenges, the learned policy achieves reliable navigation across straight, right-angle, and curved-arc paths, including outside-corner geometries. We further demonstrate that a policy trained on a simpler geometry transfers to complex ones, succeeding zero-shot on right-angle and staircase paths and reaching full success on a curved arc with a fifth of the training data. The findings suggest promising avenues for enabling droplet based microfluidic systems to serve as intelligent chemical laboratories.

0.72

TactileLab: Sim-to-Real Dexterous Robot Manipulation with Efficient Shear-Sensitive Tactile Simulation

cs.RO ·2026-11-05 ·Yijiong Lin, Wenhao He, Toby Jack, Max Yang, Haoran Li +1 more
similarity 0.61
Abstract

Robotic simulators that represent shear within tactile sensing are becoming increasingly important for contact-rich manipulation. However, simulating contact shear with high-fidelity deformable or soft-body simulation is computationally demanding, limiting scalability to large-scale reinforcement learning. Here we introduce \textit{TactileLab}, a scalable multimodal tactile simulation framework built on IsaacLab, with a particular focus on a new optical-flow-based tactile representation termed \textit{simulated tactile flow}. Instead of relying on soft-body physics to model shear deformation, TactileLab uses rigid-body simulation to efficiently estimate dense tangential motion at the contact interface, producing tactile flow fields that encode shear-relevant contact dynamics. This flow field complements geometric features from a contact depth field. We further introduce a novel policy architecture for jointly processing long-horizon proprioceptive signals and rich tactile observations. Across multiple contact-rich tasks, policies trained using simulated tactile flow learn efficiently at scale and exhibit emergent contact-aware behaviours. To facilitate sim-to-real policy deployment, we then propose a novel real-to-sim tactile translation method that maps real tactile image streams to simulated tactile flow and depth fields, together with a new tactile data collection method, and validate it across diverse tactile sensor designs. Using this pipeline, we demonstrate real-world deployment on tasks encompassing dual-arm and dexterous in-hand manipulation. Overall, TactileLab provides an end-to-end sim-to-real framework for scalable tactile policy learning, enabling diverse shear-sensitive manipulation across heterogeneous tactile robotic systems.

0.19