CS Paper Radar

Reinforcement Learning for Target Jerk Estimation in Interceptor Homing Phase

Brian Gaudet, Dalton Durant, Roberto Furfaro

eess.SY ·submitted 2026-11-11 19:17 UTC · v1

Abstract

We demonstrate the use of reinforcement meta-learning (meta-RL) to optimize a target jerk estimation policy, which is then used as the target acceleration rate of change in the propagation model of an unscented Kalman filter (UKF). The policy is learned through simulated engagements of a terminal phase high-altitude endoatmospheric interception of a highly maneuverable target, with a hit-to-kill accuracy requirement. As benchmarks, we tuned both a standard UKF and an interacting multiple model (IMM) filter, where the IMM uses a different constant jerk assumption for each model. The benchmarks and the proposed augmented UKF with the target jerk policy are tested in a 6-DOF simulation environment, which has sufficient fidelity to capture most factors relevant to guidance, navigation, and flight control. The augmented UKF demonstrates significant improvement in target interception rate, with more challenging maneuvers giving the largest performance differential. The meta-RL jerk policy is then tested on an expanded engagement scenario, with increased initial state estimate uncertainty and novel target maneuvers, demonstrating that the meta-RL policy generalizes well to conditions not seen during optimization.

Comment
Submitted to Journal of Spacecraft and Rockets

Closest papers

Nearest neighbours by meaning across the whole index.

Reinforcement Learning Optimized Flight Control for Interception of High-Altitude Maneuvering Targets

eess.SY ·2026-11-17 ·Brian Gaudet
similarity 0.75
Abstract

We implement and compare two approaches to improving flight control system (FCS) performance using reinforcement meta-learning (meta-RL). Both approaches are evaluated in a terminal phase hypersonic intercept engagement requiring hit-to-kill accuracy against a highly agile maneuvering target. In the first approach (FCS-RL), meta-RL is used to optimize a full FCS that maps acceleration tracking error and other measurements directly to commanded channel deflection rates of change, combining pitch, yaw, and roll control into a single policy. In contrast, the second approach (FCS-3L-RL) starts with a standard FCS (FCS-3L) implementing a three-loop autopilot for pitch and yaw control, but adapts the scheduled autopilot gains in real time using a policy optimized with meta-RL. Both approaches are compared to the FCS-3L in a 6-DOF simulator that captures most factors relevant to guidance and flight control. Simulations show that under ideal conditions the three systems have comparable performance. However, under conditions of aerodynamic uncertainty, radome refraction, or target evasive maneuvers, the FCS-RL significantly outperforms the FCS-3L-RL with improved intercept probability and less control effort. In turn, the FCS-3L-RL outperforms the FCS-3L by a similar margin. Importantly, we demonstrate that both FCS-RL and FCS-3L-RL generalize well to novel engagement scenarios, flight conditions, and target maneuvers that are not experienced during the optimization process.

0.13

Autonomous fixed-wing interception using reinforcement learning

cs.RO cs.AIcs.NEeess.SY ·2026-10-27 ·Daan van Hamersveld, Ewoud Smeur, Iurii Kriachko
similarity 0.68
Abstract · MSc thesis, Faculty of Aerospace Engineering, Delft University of Technology, 2026. 24 pages, 29 figures, 4 tables

Medium altitude, long endurance unmanned aerial systems such as Shahed-type strike drones are an unanswered threat in modern conflicts. Conventional air defense systems are too expensive to provide a feasible large-scale solution. Quadcopter interceptors are a cheap, battle-proven alternative, but their extremely limited flight time restricts them to low altitude targets at short range, whereas fixed-wing platforms have the endurance to reach higher altitude targets from longer distances. However, such interceptors are still operated manually. Recent developments in learning-based controllers show potential for automation. In this work, a fixed-wing interceptor is presented that is controlled by a neural network that maps filtered monocular vision observations directly to actuator commands. The policy was trained in a custom Python environment using reinforcement learning and proximal policy optimization. The policy was then deployed on a fixed-wing platform and tested in real interception scenarios. As a baseline, two classical guidance controllers were also developed and tested in simulation. The policy is able to intercept targets both in simulation and in real flight. In simulation, the policy outperforms the classical controllers under most conditions. The interceptor was ultimately successfully validated in real flight by intercepting maneuvering fixed-wing targets. This novel system shows the potential of artificial intelligence for autonomous control of fixed-wing interceptor drones.

0.31

Nonlinear Model Predictive Control for Guidance Law with Target Input Estimation

eess.SY ·2026-08-27 ·Minho Jang, Minjeong Kim, Sungsu Park
similarity 0.65
Abstract

This paper presents a look angle-based nonlinear model predictive control guidance (MPCG) method for missiles equipped with strapdown seekers. Conventional proportional navigation guidance (PNG) requires line-of-sight (LOS) rate measurements, which are not directly available in strapdown systems. MPCG instead employs look angles and their derivatives as state variables, eliminating body-rate coupling and associated parasitic feedback. The guidance problem is formulated as a continuous-time optimal control problem (OCP), discretized via the Legendre-Gauss-Radau pseudo-spectral method (LGRPM), and solved as a nonlinear program (NLP) incorporating explicit field-of-view (FOV) and acceleration constraints. Target acceleration at the first step of the prediction horizon is estimated using an adaptive extended Kalman filter (AEKF) integrated with an interacting multiple model (IMM) framework. Simulation results under single-maneuver scenarios, which include pitch and yaw plane weaving as well as barrel-roll maneuvers, demonstrate that MPCG achieves reliable interception while satisfying operational constraints, outperforming pure PNG (PPNG) in stability and resilience. This indicates that MPCG offers a practical and effective solution for modern missile guidance systems constrained by seeker measurement limitations.

0.19

ISB-NMPC: Interception-Oriented Spherical-Bearing Nonlinear Model Predictive Control for Quadrotors

cs.RO ·2026-10-26 ·Li Hong, Linkai Liu, Xiangyu Ren, Shuli Lv, Chen Min +4 more
code/data similarity 0.62
Abstract · 9 pages, 10 figures, and 1 table. Submitted to IEEE/ASME Transactions on Mechatronics

Quadrotor interceptors equipped with strapdown cameras offer a low-cost solution for physically countering unauthorized UAVs. However, this configuration inherently couples vehicle maneuvering and camera pointing, creating a conflict between rapid closing and maintaining target visibility. We present interception-oriented spherical-bearing nonlinear model predictive control (ISB-NMPC), which couples spherical-bearing prediction with nonlinear quadrotor dynamics to jointly optimize target-directed closing motion and visibility. An attitude-compensated extended Kalman filter on S^2 estimates target bearing motion, compensates for perception delay, and predicts target bearings to guide scale-adaptive detection. Across 30 paired simulation trials, ISB-NMPC achieves an interception success rate of 86.7%, the highest among the evaluated controllers, with a mean per-trial target-visible time of 99.11%. Outdoor flights demonstrate acquisition of a 30 cm target at approximately 150 m, with interceptor speed reaching 17 m/s in the long-range flight and approximately 14 m/s while pursuing a quadrotor-towed maneuvering target. The software and hardware are open source at https://github.com/HOLYNOTBAD/ISB-NMPC and https://github.com/HOLYNOTBAD/holydrone, respectively.

0.32

Curriculum-Based Deep Reinforcement Learning for Robust Autonomous Rocket Landing under Atmospheric Uncertainty

cs.AI ·2026-10-28 ·Atin Chowdhury, Piyush Joshi
similarity 0.61
Abstract · 21 pages, 12 figures

Autonomous far-range rocket landing is a challenging control problem because navigation, descent-rate regulation, attitude management, and fuel use must be coordinated in the presence of uncertain atmospheric and vehicle dynamics. This paper presents a curriculum-driven proximal policy optimization (PPO) framework for three-dimensional rocket landing in a stochastic environment incorporating altitude-dependent aerodynamic drag, wind shear, temporally correlated turbulence, discrete gusts, uncertain propulsion and aerodynamic parameters, thrust-vector misalignment, engine lag, actuator-gain variation, noisy observations, and command latency. Learning is organised into far-range approach, soft descent, and integrated landing stages. Policy transfer, critic warm-up after reward changes, rehearsal of earlier tasks, and progressive expansion of weather severity are used to support learning across these coupled phases. Across 100,000 evaluation episodes, the policy achieves 96,656 successful landings, corresponding to a success rate of 96.656\% and a 95\% Wilson confidence interval of 96.543--96.766\%. Success decreases from 98.84\% under zero-disturbance conditions to 88.07\% in the highest weather-severity band. Successful touchdowns exhibit small miss distances and low residual horizontal and vertical velocities. An exploratory logistic-regression analysis identifies mean wind speed as the strongest measured correlate of success, with initial distance, initial altitude, initial vertical velocity, and wind-shear exponent also associated with outcomes. The results quantify both the effectiveness of the learned policy over the evaluated disturbance distribution and the degradation that occurs under severe compound atmospheric conditions.

0.08

Curriculum-Driven Deep Reinforcement Learning for Autonomous Far-Range Rocket Landing

cs.LG cs.AI ·2026-10-24 ·Atin Chowdhury, Piyush Joshi
published similarity 0.61
Abstract · This is the extension of our previous paper that got accepted in ICAART 2026

Autonomous vertical landing of reusable launch vehicles is a critical yet formidable challenge in aerospace engineering, primarily due to highly nonlinear dynamics, stringent precision require- ments, and inherent environmental uncertainties. While reinforcement learning (RL) has emerged as a promising alternative to traditional model-based control, existing approaches frequently rely on flat training paradigms and highly simplified initial conditions, drastically limiting their real-world applicability. This paper proposes a comprehensive curriculum-driven deep reinforcement learning framework designed to master both far-range navigation and precision landing. By employing a three-stage curriculum augmented with a critic warm-up procedure and a rehearsal mechanism to prevent catastrophic forgetting, alongside a high-dimensional observation vector utilizing a full 3 X 3 rotation-matrix, the agent is trained to navigate from an initial horizontal distance of 1,000 to 1,500 units vastly farther than prior benchmarks before executing a soft touchdown. Crucially, we intro- duce a redesigned reward function that reduces unnecessary computational overhead and addresses limitations of earlier formulations, including poorly calibrated reward components and potentially conflicting optimization objectives. The redesigned function bounds the per-step reward magnitude by (O(1)), while its progress term accumulates the total distance closed over the trajectory. The available 100,000-episode results report a 99.98% success rate for the proposed policy and 68.31% for the baseline under the evaluation conditions. In the evaluation, the proposed policy retained 69.1% of its initial fuel capacity and achieved a mean final distance from the pad centre of 7.07 units, compared with 13.10 units for the baseline. Landing visualizations are available in Video 1 and Video 2.

0.23

State-Dependent Objective Coordination for End-to-End Balance and Navigation

cs.RO ·2026-10-21 ·Xuhao Huang, Xitao Xu, Yuxin Zheng, Yunfan Ren, Zheng Gong
code/data similarity 0.61
Abstract · 8 pages, 7 figures, 3 tables

Traditional robotic navigation systems separate mapping, planning, and low-level control through cascaded modules, introducing computational overhead and accumulated latency. End-to-end reinforcement learning offers a compact unified policy that maps onboard observations directly to control actions. However, on inherently unstable platforms such as two-wheeled inverted-pendulum robots, navigation progress and obstacle avoidance can conflict with attitude stabilization in a strongly state-dependent manner. We propose a lightweight state-dependent objective coordination method for integrated navigation and control. A pitch-conditioned scalarization coefficient $β(s)\in[0,1]$ adjusts the relative contributions of balance stabilization and navigation objectives during training, without extra control hierarchies or auxiliary attitude-stabilization controllers, such as proportional-integral-derivative (PID) or linear-quadratic regulator (LQR) controllers. To support zero-shot deployment, simulation training models execution delay, action hold, asynchronous and noisy observations, and dynamics uncertainty. Gradient analysis reveals persistent directional interference and state-dependent variation in the objectives' relative scales. Comparisons with alternative reward-weighting, constraint, training, and control-structure strategies show that PPO+$β(s)$, where PPO denotes proximal policy optimization, achieves 98% and 95% success rates with small attitude excursions on the fixed obstacle map and the randomly generated high-obstacle-density map, respectively. Zero-shot hardware experiments further demonstrate bounded attitude behavior and consistent obstacle avoidance using only onboard observations. Training and evaluation code will be made publicly available upon acceptance.

0.37

BRACE: Belief-guided Residual Adaptive Control for Safe Navigation under Unknown Disturbances

cs.RO eess.SY ·2026-11-07 ·Jaeyoun Choi, Niklas Schmid, Oswin So, Augustinos Saravanos, Chuchu Fan
similarity 0.61
Abstract

Disturbances encountered during autonomous navigation generally vary across space, often discontinuously, as obstacles redirect wind and changing terrain alters traction. Tracking controllers typically assume sufficient control authority to mitigate disturbances and maintain safety; the regime where this assumption fails remains underexplored. We present Belief-guided Residual Adaptive Control (BRACE), a hierarchical reinforcement learning (RL) framework for reach-avoid navigation under unknown, bounded, spatially varying disturbances. BRACE infers a disturbance belief from model prediction errors and uses that to condition a high-level policy that adapts the reference trajectory. The policy combines a nominal and a belief-guided residual term, trained jointly with a likelihood-ratio advantage reweighting heuristic. We evaluate BRACE in three simulated quadcopter environments with abruptly switching wind fields, and validate it on Crazyflie hardware with fan-generated disturbances. Across these settings, BRACE improves reach-avoid performance over model predictive control and RL baselines.

0.12