CS Paper Radar

Curriculum-Based Deep Reinforcement Learning for Robust Autonomous Rocket Landing under Atmospheric Uncertainty

Atin Chowdhury, Piyush Joshi

cs.AI ·submitted 2026-10-28 10:51 UTC · v1

Abstract

Autonomous far-range rocket landing is a challenging control problem because navigation, descent-rate regulation, attitude management, and fuel use must be coordinated in the presence of uncertain atmospheric and vehicle dynamics. This paper presents a curriculum-driven proximal policy optimization (PPO) framework for three-dimensional rocket landing in a stochastic environment incorporating altitude-dependent aerodynamic drag, wind shear, temporally correlated turbulence, discrete gusts, uncertain propulsion and aerodynamic parameters, thrust-vector misalignment, engine lag, actuator-gain variation, noisy observations, and command latency. Learning is organised into far-range approach, soft descent, and integrated landing stages. Policy transfer, critic warm-up after reward changes, rehearsal of earlier tasks, and progressive expansion of weather severity are used to support learning across these coupled phases. Across 100,000 evaluation episodes, the policy achieves 96,656 successful landings, corresponding to a success rate of 96.656\% and a 95\% Wilson confidence interval of 96.543--96.766\%. Success decreases from 98.84\% under zero-disturbance conditions to 88.07\% in the highest weather-severity band. Successful touchdowns exhibit small miss distances and low residual horizontal and vertical velocities. An exploratory logistic-regression analysis identifies mean wind speed as the strongest measured correlate of success, with initial distance, initial altitude, initial vertical velocity, and wind-shear exponent also associated with outcomes. The results quantify both the effectiveness of the learned policy over the evaluated disturbance distribution and the degradation that occurs under severe compound atmospheric conditions.

Comment
21 pages, 12 figures

Closest papers

Nearest neighbours by meaning across the whole index.

Curriculum-Driven Deep Reinforcement Learning for Autonomous Far-Range Rocket Landing

cs.LG cs.AI ·2026-10-24 ·Atin Chowdhury, Piyush Joshi
published similarity 0.84
Abstract · This is the extension of our previous paper that got accepted in ICAART 2026

Autonomous vertical landing of reusable launch vehicles is a critical yet formidable challenge in aerospace engineering, primarily due to highly nonlinear dynamics, stringent precision require- ments, and inherent environmental uncertainties. While reinforcement learning (RL) has emerged as a promising alternative to traditional model-based control, existing approaches frequently rely on flat training paradigms and highly simplified initial conditions, drastically limiting their real-world applicability. This paper proposes a comprehensive curriculum-driven deep reinforcement learning framework designed to master both far-range navigation and precision landing. By employing a three-stage curriculum augmented with a critic warm-up procedure and a rehearsal mechanism to prevent catastrophic forgetting, alongside a high-dimensional observation vector utilizing a full 3 X 3 rotation-matrix, the agent is trained to navigate from an initial horizontal distance of 1,000 to 1,500 units vastly farther than prior benchmarks before executing a soft touchdown. Crucially, we intro- duce a redesigned reward function that reduces unnecessary computational overhead and addresses limitations of earlier formulations, including poorly calibrated reward components and potentially conflicting optimization objectives. The redesigned function bounds the per-step reward magnitude by (O(1)), while its progress term accumulates the total distance closed over the trajectory. The available 100,000-episode results report a 99.98% success rate for the proposed policy and 68.31% for the baseline under the evaluation conditions. In the evaluation, the proposed policy retained 69.1% of its initial fuel capacity and achieved a mean final distance from the pad centre of 7.07 units, compared with 13.10 units for the baseline. Landing visualizations are available in Video 1 and Video 2.

0.23

DreamLander: A 6-DOF World Model based Reinforcement Learning Planetary Lander

cs.RO math.OC ·2026-11-09 ·George Nehma, Patrick Quinn, Madhur Tiwari, Camilo Riano-Rios
published similarity 0.70
Abstract

Soft powered descent landings are becoming evermore important in the field of space exploration and scientific advancement, because of their ability to deliver mission critical payloads, both highly accurately and safely. The high efficiency restriction on planetary landings due to the severely limited fuel supply results in the need for optimal guidance trajectories and control. Model based reinforcement learning is proving to be an important tool in simplifying the highly complex process of generating optimal guidance trajectories and control. In this work, we present DreamLander, a state of the art model based reinforcement learning algorithm based on DreamerV3, to generate optimal guidance and control for a realistic 6 degree of freedom entry descent landing problem. The model based approach creates a sample efficient and highly generalizable model that can be applied to a variety of scenarios whilst maintaining a high degree of accuracy, landing within 2 meters of the designated landing zone, and with velocity norm less than 0.3 meters per second. DreamLander makes use of state sensor observations, making progress toward End-to-End solutions of the Entry Descent Landing problem.

0.42

Adversarial Reinforcement Learning for Robust Planetary Landing

cs.LG eess.SY ·2026-10-14 ·Miguel Leiva-Vélez, Nicolas Gaston-Rozado, Andrea Scorsoglio, Hodei Urrutxua, Víctor Rodríguez-Fernández +1 more
similarity 0.69
Abstract

Deep reinforcement learning (DRL) policies for planetary landing and other aerospace guidance, navigation, and control tasks are commonly trained under randomized disturbances to improve robustness. However, typical stochastic perturbation sampling assumes independent variations, whereas real mission disturbances often exhibit spatial and temporal correlation and cannot be reproduced through independent random draws. This work investigates adversarial reinforcement learning to discover challenging off-nominal conditions. A secondary agent generates physically constrained perturbations that degrade the performance of a landing controller, forming an asymmetric sequential zero-sum game. Training proceeds in three stages: baseline controller learning, robustness enhancement with stochastic perturbations, and an iterative adversarial phase in which the controller and disturbance generator are alternately optimized. Preliminary experiments in a three-degree-of-freedom landing environment show that adversarial disturbances reveal vulnerabilities missed by conventional randomization and provide early indications that, with a more extensive training campaign, the approach can yield improved robustness and stronger generalization under previously unseen perturbation regimes.

0.23

CORAL: Curriculum-Optimized Reward Adaptation for LiDAR-Based Goal-Directed Urban Driving

cs.RO cs.LG ·2026-08-14 ·Anisa Saleem, Duksu Kim
similarity 0.67
Abstract · 13 pages, 6 figures

Reinforcement learning is promising for autonomous urban driving, but long-horizon goal-directed navigation asks a policy to acquire several competing behaviors at once--reaching a distant goal, tracking a route, avoiding obstacles, obeying signals--and a fixed objective gives no order in which to learn them. This paper presents CORAL, which advances two schedules together: a five-stage curriculum that progressively lengthens routes and tightens behavioral constraints, and a stage-aware reward whose component weights shift emphasis from mission progress toward route following, safety, smoothness, and rule compliance as the task hardens. The policy is a multi-stream actor-critic network trained with Proximal Policy Optimization (PPO) in CARLA on a compact 99-dimensional state pairing a polar LiDAR histogram with vehicle telemetry, ego-frame route geometry, and traffic-rule indicators--no point-cloud encoder, no bird's-eye-view rasterization. Against two PPO baselines under an identical protocol, CORAL reaches the goal in all twenty evaluation episodes on the longest routes under the full set of behavioral constraints, where the baselines reach 5% and 10%; a factorial ablation shows that neither schedule alone matches their combination: removing either lowers both success and route completion, and disabling both drops success to 55%. Trained in one town, the policy transfers zero-shot to seven unseen towns, succeeding in 68-98% of episodes on routes of the same 100-150 m length, with mean lateral deviation below 0.35 m.

0.10

Curriculum-Based Adversarial Heterogeneous Agent Reinforcement Learning for Autonomous Quad-Copter Landing in Maritime Settings

cs.LG ·2026-09-11 ·Allan Minh-Tam Nguyen, Sree Showrya Kotala, Stefan Banioi-Crijman, Kurt Driessens, Rico Möckel
benchmark similarity 0.66
Abstract · 20 pages, 5 figures, submitted to BNAIC 2026

Recovering unmanned aerial vehicles (UAVs) in maritime environments is challenging due to wind turbulence and ship-deck motion, making it a valuable test case for alternative control and learning approaches as conventional landing approaches often become unreliable. We study simulated mid-air capture of quadrotor UAVs by a ship-mounted robotic arm, learning robust cooperative control policies with Heterogeneous-Agent Proximal Policy Optimization (HAPPO) Reinforcement Learning. We train with HAPPO using a curriculum and an adversarial wind agent (HARL-AC) in NVIDIA Isaac Lab, and compare the obtained control policies against those generated through curriculum-based domain randomization and a benchmark trained on a single sea state. In-distribution evaluation on sea states $0/4/5$ shows comparable success for HARL-AC and domain randomization of up to $97.5\%$. On out-of-distribution sea states $7/8/10$, HARL-AC generalizes better, achieving up to $16\%$ higher median success rate at sea state 10, and substantially lower crash rates of up to $14\%$ compared to the domain randomization policy. Furthermore, we show that the adversarially trained policy shows more cautious behavior, slightly increasing timeouts by $<3\%$, but yields safer recovery behavior in severe, unseen conditions.

0.24

Deep Reinforcement Learning with Massively Parallel Simulation for Low-Thrust Multi-Revolution Rendezvous

eess.SY ·2026-11-07 ·Chenglong Wu, Yi Zhou, Bin Cheng, Nan Zhang, Hexi Baoyin
similarity 0.66
Abstract · 23 pages, 8 figures, 13 tables

The growing demand for on-orbit servicing and constellation management in low Earth orbit requires autonomous and reliable low-thrust rendezvous. Traditional numerical methods struggle with the long-horizon nature of multi-revolution maneuvers under perturbations, while standard deep reinforcement learning (DRL) methods are limited by low simulation throughput and poor sample efficiency. To address these challenges, this paper proposes a massively parallel DRL guidance framework. A fully tensorized, GPU-resident orbital propagator is developed to integrate $J_2$-perturbed orbital dynamics and policy optimization within a zero-copy pipeline, achieving up to a 67-fold increase in simulation throughput over conventional CPU baselines. In addition, the rendezvous problem is formulated in modified equinoctial elements and trained with a progressive curriculum learning strategy to improve convergence and generalization across increasingly challenging rendezvous scenarios. Simulations show that the proposed framework can learn robust rendezvous trajectories for complex multi-revolution rendezvous over a broad range of orbital configurations. Additional evaluations under a high-fidelity model and eclipse-induced thrust interruptions further demonstrate the robustness of the learned policy under practical mission constraints. Moreover, the trajectories generated by the proposed framework can be used as effective initial guesses for optimization-based trajectory solvers, improving the convergence of high-precision trajectory optimization. The method offers a scalable approach for real-time low-thrust guidance and trajectory optimization.

0.36

Reinforcement Learning Optimized Flight Control for Interception of High-Altitude Maneuvering Targets

eess.SY ·2026-11-17 ·Brian Gaudet
similarity 0.65
Abstract

We implement and compare two approaches to improving flight control system (FCS) performance using reinforcement meta-learning (meta-RL). Both approaches are evaluated in a terminal phase hypersonic intercept engagement requiring hit-to-kill accuracy against a highly agile maneuvering target. In the first approach (FCS-RL), meta-RL is used to optimize a full FCS that maps acceleration tracking error and other measurements directly to commanded channel deflection rates of change, combining pitch, yaw, and roll control into a single policy. In contrast, the second approach (FCS-3L-RL) starts with a standard FCS (FCS-3L) implementing a three-loop autopilot for pitch and yaw control, but adapts the scheduled autopilot gains in real time using a policy optimized with meta-RL. Both approaches are compared to the FCS-3L in a 6-DOF simulator that captures most factors relevant to guidance and flight control. Simulations show that under ideal conditions the three systems have comparable performance. However, under conditions of aerodynamic uncertainty, radome refraction, or target evasive maneuvers, the FCS-RL significantly outperforms the FCS-3L-RL with improved intercept probability and less control effort. In turn, the FCS-3L-RL outperforms the FCS-3L by a similar margin. Importantly, we demonstrate that both FCS-RL and FCS-3L-RL generalize well to novel engagement scenarios, flight conditions, and target maneuvers that are not experienced during the optimization process.

0.14

Information-Guided Safe Reinforcement Learning for Autonomous Gas Source Localization using sUAS

cs.RO ·2026-09-04 ·Sachin Giri, Thomas Zhao, Matthew Huynh, YangQuan Chen
code/data similarity 0.64
Abstract · 8 pages, 5 figures, 2 tables. Submitted to the 2026 IEEE Conference on Decision and Control (CDC). Code: https://github.com/sachingirime/Inf…

The autonomous localization of fugitive gas emissions using small Unmanned Aircraft Systems (sUAS) constitutes a fundamentally ill-posed inverse problem. In turbulent atmospheric boundary layers, highly intermittent scalar concentration fields violate the assumptions of classical gradient-based navigation, causing data-driven estimators to suffer from severe noise and spurious local minima. To address these challenges, we introduce an Information-Guided Safe Reinforcement Learning framework evaluated within a custom, GPU-accelerated 3D simulation environment coupling an Eulerian wind solver with a Lagrangian puff dispersion model. We identify a critical vulnerability in deterministic information-seeking planners - a Gramian bias where agents act greedily upon flawed early estimates, starving the estimator of spatial diversity. To systematically break this degeneracy, our architecture integrates a classical empirical observability Gramian (EMGR) planner with a learned Soft Actor-Critic (SAC) exploratory policy. A deterministic meta-supervisor actively monitors estimator reliability via Kullback-Leibler (KL) divergence, dynamically blending deterministic exploitation with learned exploration to steer the sUAS into high-information zones. Trained via a progressive curriculum and safeguarded by a strictly enforced Robust Control Barrier Function (RCBF), our RL framework achieves nearly 80% localization success on complex, mobile sources - drastically outperforming classical baselines (~30%) - while ensuring zero safety violations.

0.37