CS Paper Radar

Reachability-Informed Reinforcement Learning for Multi-Impulse Interplanetary Transfers

Yashdeep Chaudhary, Roberto Armellin, Harry Holt

cs.LG math.OCeess.SY ·submitted 2026-10-01 09:16 UTC · v1
benchmark

Abstract

Reinforcement learning offers the prospect of a reusable sequential decision-making mechanism for spacecraft trajectory design, motivating policy interfaces that connect learned decisions to the underlying maneuver geometry. This paper develops Reachability Analysis-Informed Reinforcement Learning (RARL) for deterministic multi-impulse interplanetary transfers, placing intermediate waypoint selection at the center of the learned decision process. Local first-order reachability maps bounded velocity perturbations into an ellipsoidal set of next-node positions, within which the policy selects its waypoint. Lambert reconstruction then determines the corresponding maneuver to reach this selected waypoint along a dynamically consistent ballistic arc, coupling learned transfer-geometry selection with classical astrodynamics. A terminal two-impulse reconstruction completes the rendezvous, supported by a linear maneuver-demand assessment used for reward shaping. Numerical studies characterize this interface on a two-body Earth-Mars benchmark. Across three independent training runs, RARL achieves a mean maneuver cost of 10.23 km/s, 1.72% above a validated local sequential convex programming reference. Training over dispersed initial states extends policy reuse across a departure family with fixed target state and transfer duration. Each of the three independently trained multi-state policies completes all 10,000 held-out Monte Carlo departures without impulse-cap violations, compared with a mean feasibility rate of 6.49% for single-state policies. This broader sampled feasibility is accompanied by a 0.61% increase in mean nominal maneuver cost, without further training across departures. These results demonstrate that a reachability-informed decision interface supports benchmark-quality trajectory construction and policy reuse across dispersed departure conditions.

Comment
Preprint. 23 pages, 10 figures

Closest papers

Nearest neighbours by meaning across the whole index.

From Orbit Matching to Docking: A Hierarchical Reinforcement-Learning Approach to Multiphase Rendezvous Trajectory Design

cs.RO cs.LGeess.SY ·2026-10-12 ·Mariko Storey-Matsutani, Anatoly Zlotnik, Lee Holguin, Zigfried Hampel-Arias, Richard Linares
similarity 0.69
Abstract

As the complexity of space missions increases, autonomous spacecraft guidance must accommodate transitions between mission phases with distinct dynamics, objectives, and constraints. Rendezvous and proximity operations (RPO) are particularly challenging because they span multiple operating regimes, from orbital-scale matching and far-range approach to close-range maneuvering and terminal docking. Existing reinforcement-learning approaches generally address only a subset of this sequence or rely on manually defined switching conditions between controllers. This study presents a hierarchical reinforcement-learning framework for autonomous, three-degree-of-freedom RPO that spans the complete trajectory from orbit matching to docking. Five specialized policies are trained for orbit matching, far-range approach, mid-range approach, close-range approach, and docking. A learned high-level manager policy then selects among these phase-specific controllers based on the evolving spacecraft state. The hierarchical approach is evaluated over the full end-to-end rendezvous sequence and compared with a fixed sequential-invocation heuristic to assess trajectory performance and transition behavior. Training and inference metrics are also analyzed for the individual phase policies and the complete hierarchical system.

0.12

Temporal Attention-Enhanced Mixture-of-Experts Guidance for Multi-Revolution Low-Thrust Rendezvous

eess.SY ·2026-11-14 ·Chenglong Wu, Yi Zhou, Hongxu Zhu, Jialong Song, Hexi Baoyin
similarity 0.65
Abstract · 27 pages, 11 figures. Submitted to IEEE Transactions on Aerospace and Electronic Systems

Multi-revolution low-thrust rendezvous requires both long-duration orbital transfer and precise terminal control. This paper proposes a temporal attention-enhanced mixture-of-experts guidance law trained by deep reinforcement learning. A far-range expert acts on modified equinoctial elements, a terminal expert acts on the relative state in the target's local frame, and a learned gate blends their thrust commands. The terminal expert encodes its observation history with a Transformer to address partial observability, and the gate uses a separate encoder to account for the resulting history-dependent command. The experts are pretrained separately, the gate is trained with both experts frozen, and the complete guidance law is then fine-tuned end to end. On 600 test missions, it achieves rendezvous in every case at tolerances of 10 m and 1 m/s, reduces mean flight time and delta-v by 50.6% and 44.5% relative to distance-based hard switching, and remains successful without retraining under navigation errors, unmodeled orbital perturbations, eclipse-induced thrust outages, and thrust loss.

0.29

Satellite Trajectory Optimization via Proximal Policy Optimization for Space Debris Avoidance

cs.LG cs.RO ·2026-08-10 ·Logan Luna, Juan Ortiz Couder, Raul Alejandro Vargas-Acosta
publishedcode/data similarity 0.63
Abstract · 18 pages, 16 figures. Published in IEEE Access, vol. 14, pp. 18138-18154, 2026

Collision avoidance systems are commonly used to avoid fragmentation events occurring in Low-Earth Orbit (LEO) and Geosynchronous Equatorial Orbit (GEO). However, these events have been growing in frequency as orbital congestion worsens with the launch of megaconstellations. Consequently, conjunction alerts and collision risks are becoming increasingly common. Current practices, which are commonly manual or rule-based, have difficulty scaling to these worsening dynamic environments. To address this intensifying situation, we propose a reinforcement-learning policy for autonomous collision avoidance, trained via Proximal Policy Optimization (PPO) along with an open-source, high-fidelity astrodynamics simulator for training and evaluation. In 1,000 deterministic GEO episodes, our agent achieves a 97.5% collision avoidance success rate, outperforming traditional controllers such as a rule-based baseline (20.7% success) and an impulsive delta-v planner baseline (27.5% success). To achieve these results, we designed a simulator to train and evaluate our agent, using real-world and simulated debris. We simulate Newtonian two-body dynamics using Sun/Moon third-body perturbations, fuel-dependent thrust, and configurable debris fields. The agent is trained with curriculum learning and shaped rewards oriented toward encouraging survival, adequate projected miss distance, and delta-v conservation. Finally, our evaluation consisted of a fully deterministic pipeline, including shared seeds, per-episode logs, and telemetry exports. Our work is a publicly available framework at https://purl.org/sat-trajectory-avoidance

0.78

Deep Reinforcement Learning with Massively Parallel Simulation for Low-Thrust Multi-Revolution Rendezvous

eess.SY ·2026-11-07 ·Chenglong Wu, Yi Zhou, Bin Cheng, Nan Zhang, Hexi Baoyin
similarity 0.63
Abstract · 23 pages, 8 figures, 13 tables

The growing demand for on-orbit servicing and constellation management in low Earth orbit requires autonomous and reliable low-thrust rendezvous. Traditional numerical methods struggle with the long-horizon nature of multi-revolution maneuvers under perturbations, while standard deep reinforcement learning (DRL) methods are limited by low simulation throughput and poor sample efficiency. To address these challenges, this paper proposes a massively parallel DRL guidance framework. A fully tensorized, GPU-resident orbital propagator is developed to integrate $J_2$-perturbed orbital dynamics and policy optimization within a zero-copy pipeline, achieving up to a 67-fold increase in simulation throughput over conventional CPU baselines. In addition, the rendezvous problem is formulated in modified equinoctial elements and trained with a progressive curriculum learning strategy to improve convergence and generalization across increasingly challenging rendezvous scenarios. Simulations show that the proposed framework can learn robust rendezvous trajectories for complex multi-revolution rendezvous over a broad range of orbital configurations. Additional evaluations under a high-fidelity model and eclipse-induced thrust interruptions further demonstrate the robustness of the learned policy under practical mission constraints. Moreover, the trajectories generated by the proposed framework can be used as effective initial guesses for optimization-based trajectory solvers, improving the convergence of high-precision trajectory optimization. The method offers a scalable approach for real-time low-thrust guidance and trajectory optimization.

0.36

Learning Waypoints and Passage Events for Decentralized Multi-Quadrotor Trajectory Planning

cs.RO eess.SY ·2026-10-18 ·Yunhan Shi, Jia Deng, Zehao Jia, Lin Cheng, Shengping Gong
similarity 0.63
Abstract · 9 pages, 7 figures, 3 tables

Multi-quadrotor passage planning requires coordinated spatial routes and passage times under vehicle dynamics and actuation constraints. Optimizing these decisions jointly couples temporal allocation, nonlinear motion, and inter-vehicle interactions, creating a computational challenge for local trajectory generation. We propose a learning-optimization framework that predicts spatial waypoints and ordered passage events for independently solved quadrotor planners. A common physical time grid aligns the vehicles, with intermediate events fixed by the networks and terminal progress optimized locally to permit different completion metrics. Differentiable sequential convex programming incorporates rigid-body dynamics and rotor-thrust bounds, allowing collision and completion gradients to train both networks through the generated trajectories. In a two-quadrotor evaluation across 24 layouts, the adapted policy passes physical feasibility and sampled separation checks in all five held-out layouts. Their mean completion metric decreases by 1.52% relative to the inherited policy. The median forward time is 21.01 ms on the held-out exchange layout with matching numerical trajectory references. These results demonstrate that learned spatial and event parameters can coordinate independent rigid-body planners while preserving local vehicle constraints in the evaluated scenarios.

0.13

Graph-Based RL for Sequential Rendezvous Planning with Multiple Patrolling USVs

cs.RO cs.AI ·2026-11-09 ·Seung Woo Han, Hyeon Bin Yeo, Seung Heon Oh
similarity 0.63
Abstract

In patrol missions involving multiple unmanned surface vehicles (USVs), a designated command USV can support fleet management by sequentially rendezvousing with each patrolling USV. These operations require motion-feasible route planning and priority-aware rendezvous sequencing, with the objective of minimizing mission completion time and total tardiness relative to priority-dependent due dates. We propose a framework combining grid-based path planning, surrogate travel-time prediction, and graph-based reinforcement learning. The path planner accounts for ocean currents and approximates turn-rate constraints through multi-stage maneuvers, with path feasibility empirically validated through trajectory-tracking simulations. For rendezvous sequencing, a Feasible Transition Graph (FTG) represents candidate rendezvous events and temporally feasible transitions, enabling a graph neural network to exploit spatiotemporal rendezvous relationships for policy learning. A surrogate predicts travel times from compact current and motion features to avoid repeated path searches during FTG construction and policy training. Evaluation using grid-based path planning shows statistically significant average objective reductions of 10.5%, 3.5%, and 1.5% compared with greedy, MLP-RL, and 2-step lookahead greedy, respectively. The policy achieves these improvements while requiring approximately 1/35 of the latter's computation time. It maintains its advantage over MLP-RL under unseen priority-tier settings, with larger gains as fleet size increases.

0.32

Beyond Policy Alignment: Closing the Planning-Learning Loop for Robot Control with Learned World Models

cs.RO ·2026-09-30 ·Kowndinya Boyalakuntla, Yuhan Liu, Abdeslam Boularias
benchmark similarity 0.62
Abstract · 8 pages

Planning with learned world models combines online trajectory optimization with learned value and policy functions for high-dimensional control. Because the planner determines the experience used for learning, while the learned critic and actor in turn score and propose future plans, planning and learning form a closed feedback loop. TD-MPC is a prominent instance of this design. Recent policy-constrained variants strengthen one part of the loop by aligning the learned policy with planner behavior. We introduce PL-MPC (Planning-Learning MPC), which additionally modifies critic supervision and planner terminal-value estimation. Hybrid multi-step TD targets expose critic updates to more realized rewards before bootstrapping; disagreement-aware terminal estimates reduce the influence of uncertain critic values during MPPI planning; and return-weighted actor distillation emphasizes planner-executed actions from high-return episodes. The world-model architecture and MPPI optimizer are otherwise unchanged. On HumanoidBench, the largest gains occur on \texttt{balance-hard}, where Total Average Return (TAR) increases from $98\pm18$ to $387\pm255$, and \texttt{hurdle}, from $199\pm13$ to $466\pm200$; performance across the broader benchmark remains task dependent, and PL-MPC remains competitive on DMControl. Controlled ablations show different component interactions across the two tasks. We further demonstrate zero-shot sim-to-real transfer on wrench-nut alignment with a 7-DoF KUKA IIWA14, obtaining higher observed success than TD-M(PC)^2 on the training object size and two unseen sizes. Code and data will be available at: https://pl-mpc-humanoid.github.io.

0.33

Diffusion-Based Multiple-Shooting Indirect Optimal Control for Fuel-Optimal Spacecraft Trajectory Generation

eess.SY cs.LG ·2026-09-12 ·Saeid Tafazzol, Ehsan Taheri, Ryne Beeson
benchmarkcode/data similarity 0.62
Abstract · Submitted to IEEE Control Systems Letters (L-CSS) for possible publication

Diffusion-based generative models (DMs) have found applications in control problems, and in particular robotics, where the DMs enable exploration of possible control solutions. A critical shortcoming of these applications is that they have lacked optimality guarantees. This is a problem for their potential use in fuel-optimal spacecraft trajectories that are characterized with long time-horizons and bang-bang profiles. Alternatively, indirect optimal control methods ensure explicit satisfaction of necessary conditions, but are highly sensitive to the initial costate estimation needed to solve the resulting Hamiltonian boundary-value problems (HBVPs). To alleviate this sensitivity and enlarge the convergence domain of HBVPs, advanced indirect methods have been developed that use smoothing approaches and continuation. We propose a diffusion-based multiple shooting indirect control method that combines the exploration capability of DMs with indirect method to generate fuel-optimal spacecraft trajectories. We benchmark our method against an advanced indirect method on a fuel-optimal Earth-Mars low-thrust transfer problem, showing higher convergence robustness than the advanced indirect method that is based on random costate initialization. Code and visualizations are available at https://saeidtafazzol.github.io/Diffusion_Indirect_Control/.

0.21