CS Paper Radar

From Orbit Matching to Docking: A Hierarchical Reinforcement-Learning Approach to Multiphase Rendezvous Trajectory Design

Mariko Storey-Matsutani, Anatoly Zlotnik, Lee Holguin, Zigfried Hampel-Arias, Richard Linares

cs.RO cs.LGeess.SY ·submitted 2026-10-12 22:30 UTC · v1

Abstract

As the complexity of space missions increases, autonomous spacecraft guidance must accommodate transitions between mission phases with distinct dynamics, objectives, and constraints. Rendezvous and proximity operations (RPO) are particularly challenging because they span multiple operating regimes, from orbital-scale matching and far-range approach to close-range maneuvering and terminal docking. Existing reinforcement-learning approaches generally address only a subset of this sequence or rely on manually defined switching conditions between controllers. This study presents a hierarchical reinforcement-learning framework for autonomous, three-degree-of-freedom RPO that spans the complete trajectory from orbit matching to docking. Five specialized policies are trained for orbit matching, far-range approach, mid-range approach, close-range approach, and docking. A learned high-level manager policy then selects among these phase-specific controllers based on the evolving spacecraft state. The hierarchical approach is evaluated over the full end-to-end rendezvous sequence and compared with a fixed sequential-invocation heuristic to assess trajectory performance and transition behavior. Training and inference metrics are also analyzed for the individual phase policies and the complete hierarchical system.

Closest papers

Nearest neighbours by meaning across the whole index.

Deep Reinforcement Learning with Massively Parallel Simulation for Low-Thrust Multi-Revolution Rendezvous

eess.SY ·2026-11-07 ·Chenglong Wu, Yi Zhou, Bin Cheng, Nan Zhang, Hexi Baoyin
similarity 0.73
Abstract · 23 pages, 8 figures, 13 tables

The growing demand for on-orbit servicing and constellation management in low Earth orbit requires autonomous and reliable low-thrust rendezvous. Traditional numerical methods struggle with the long-horizon nature of multi-revolution maneuvers under perturbations, while standard deep reinforcement learning (DRL) methods are limited by low simulation throughput and poor sample efficiency. To address these challenges, this paper proposes a massively parallel DRL guidance framework. A fully tensorized, GPU-resident orbital propagator is developed to integrate $J_2$-perturbed orbital dynamics and policy optimization within a zero-copy pipeline, achieving up to a 67-fold increase in simulation throughput over conventional CPU baselines. In addition, the rendezvous problem is formulated in modified equinoctial elements and trained with a progressive curriculum learning strategy to improve convergence and generalization across increasingly challenging rendezvous scenarios. Simulations show that the proposed framework can learn robust rendezvous trajectories for complex multi-revolution rendezvous over a broad range of orbital configurations. Additional evaluations under a high-fidelity model and eclipse-induced thrust interruptions further demonstrate the robustness of the learned policy under practical mission constraints. Moreover, the trajectories generated by the proposed framework can be used as effective initial guesses for optimization-based trajectory solvers, improving the convergence of high-precision trajectory optimization. The method offers a scalable approach for real-time low-thrust guidance and trajectory optimization.

0.36

Temporal Attention-Enhanced Mixture-of-Experts Guidance for Multi-Revolution Low-Thrust Rendezvous

eess.SY ·2026-11-14 ·Chenglong Wu, Yi Zhou, Hongxu Zhu, Jialong Song, Hexi Baoyin
similarity 0.70
Abstract · 27 pages, 11 figures. Submitted to IEEE Transactions on Aerospace and Electronic Systems

Multi-revolution low-thrust rendezvous requires both long-duration orbital transfer and precise terminal control. This paper proposes a temporal attention-enhanced mixture-of-experts guidance law trained by deep reinforcement learning. A far-range expert acts on modified equinoctial elements, a terminal expert acts on the relative state in the target's local frame, and a learned gate blends their thrust commands. The terminal expert encodes its observation history with a Transformer to address partial observability, and the gate uses a separate encoder to account for the resulting history-dependent command. The experts are pretrained separately, the gate is trained with both experts frozen, and the complete guidance law is then fine-tuned end to end. On 600 test missions, it achieves rendezvous in every case at tolerances of 10 m and 1 m/s, reduces mean flight time and delta-v by 50.6% and 44.5% relative to distance-based hard switching, and remains successful without retraining under navigation errors, unmodeled orbital perturbations, eclipse-induced thrust outages, and thrust loss.

0.29

Reachability-Informed Reinforcement Learning for Multi-Impulse Interplanetary Transfers

cs.LG math.OCeess.SY ·2026-10-01 ·Yashdeep Chaudhary, Roberto Armellin, Harry Holt
benchmark similarity 0.69
Abstract · Preprint. 23 pages, 10 figures

Reinforcement learning offers the prospect of a reusable sequential decision-making mechanism for spacecraft trajectory design, motivating policy interfaces that connect learned decisions to the underlying maneuver geometry. This paper develops Reachability Analysis-Informed Reinforcement Learning (RARL) for deterministic multi-impulse interplanetary transfers, placing intermediate waypoint selection at the center of the learned decision process. Local first-order reachability maps bounded velocity perturbations into an ellipsoidal set of next-node positions, within which the policy selects its waypoint. Lambert reconstruction then determines the corresponding maneuver to reach this selected waypoint along a dynamically consistent ballistic arc, coupling learned transfer-geometry selection with classical astrodynamics. A terminal two-impulse reconstruction completes the rendezvous, supported by a linear maneuver-demand assessment used for reward shaping. Numerical studies characterize this interface on a two-body Earth-Mars benchmark. Across three independent training runs, RARL achieves a mean maneuver cost of 10.23 km/s, 1.72% above a validated local sequential convex programming reference. Training over dispersed initial states extends policy reuse across a departure family with fixed target state and transfer duration. Each of the three independently trained multi-state policies completes all 10,000 held-out Monte Carlo departures without impulse-cap violations, compared with a mean feasibility rate of 6.49% for single-state policies. This broader sampled feasibility is accompanied by a 0.61% increase in mean nominal maneuver cost, without further training across departures. These results demonstrate that a reachability-informed decision interface supports benchmark-quality trajectory construction and policy reuse across dispersed departure conditions.

0.29

Graph-Based RL for Sequential Rendezvous Planning with Multiple Patrolling USVs

cs.RO cs.AI ·2026-11-09 ·Seung Woo Han, Hyeon Bin Yeo, Seung Heon Oh
similarity 0.68
Abstract

In patrol missions involving multiple unmanned surface vehicles (USVs), a designated command USV can support fleet management by sequentially rendezvousing with each patrolling USV. These operations require motion-feasible route planning and priority-aware rendezvous sequencing, with the objective of minimizing mission completion time and total tardiness relative to priority-dependent due dates. We propose a framework combining grid-based path planning, surrogate travel-time prediction, and graph-based reinforcement learning. The path planner accounts for ocean currents and approximates turn-rate constraints through multi-stage maneuvers, with path feasibility empirically validated through trajectory-tracking simulations. For rendezvous sequencing, a Feasible Transition Graph (FTG) represents candidate rendezvous events and temporally feasible transitions, enabling a graph neural network to exploit spatiotemporal rendezvous relationships for policy learning. A surrogate predicts travel times from compact current and motion features to avoid repeated path searches during FTG construction and policy training. Evaluation using grid-based path planning shows statistically significant average objective reductions of 10.5%, 3.5%, and 1.5% compared with greedy, MLP-RL, and 2-step lookahead greedy, respectively. The policy achieves these improvements while requiring approximately 1/35 of the latter's computation time. It maintains its advantage over MLP-RL under unseen priority-tier settings, with larger gains as fleet size increases.

0.32

SCoCaT: Success Conditioned Constrained Reinforcement Learning for Spacecraft Docking

cs.RO ·2026-09-05 ·Aman Arora, Ricard Marsal I Castan, Matteo El-Hariry, Miguel Olivares-Mendez
published similarity 0.67
Abstract · Accepted at CoRL 2026 Conference (https://www.corl.org/)

Termination-based constrained reinforcement learning is attractive for safety-critical robotic deployments: it avoids online optimization at inference, scales easily to many constraints via a single scalar per constraint, and is simpler to implement than commonly used Lagrangian methods. Instead of pricing violations through summed cost penalties, this approach makes violations structurally unprofitable by shortening the effective horizon for each violation. We identify a structural failure mode of this method class on terminal-navigation tasks: reaching a precise goal configuration while satisfying safety constraints that tighten along the final approach. When the goal sits inside the region close to where the constraints become active, the survival-weighted objective makes dwelling outside the goal region strictly preferable to entering, producing high constraint compliance with low task completion. We formalize this pathology and show that a minimal augmentation to off-the-shelf RL algorithms like PPO resolves this ``feasibility collapse''. We empirically demonstrate that adding a dense per-step success signal via an auxiliary value critic improves the task completion rate while maintaining safety-critical constraint compliance. Validation across two representative spacecraft platforms: a 6U-CubeSat spanning the mass and degree-of-freedom envelope of operational proximity operations, and a floating platform testbed for zero-shot sim-to-real transfer in our laboratory, supports the generality of these findings.

0.36

Spacecraft Rendezvous Trajectory Generation with Modular Constraints via Diffusion Model Composition

cs.RO cs.LG ·2026-10-05 ·Mariko A. Storey-Matsutani, Richard Linares
similarity 0.67
Abstract

Emerging mission classes such as on-orbit servicing, satellite inspection, and active debris removal require trajectory design methods that are adaptable to a variety of mission scenarios. We present a diffusion-based trajectory generation approach for rendezvous and proximity operations (RPO) that enables flexible configuration of mission constraints. First, individual energy-based diffusion models are trained to satisfy distinct constraints such as approach cone and sensor line-of-sight from a set of optimized trajectories. Then, at inference time, the learned energy models can be composed with one another, or with an analytically defined energy field, to enforce specific constraint combinations. We validate this framework with the composition of a learned approach cone model and a learned sensor line-of-sight model, as well as a learned approach cone model and synthetic obstacle avoidance model, both of which yield constraint satisfaction rates that are within 1 percentage point of the single-constraint models or higher. These results indicate that our compositional diffusion framework can provide a modular approach to RPO trajectory design and enable reconfiguration for new constraint combinations without requiring model retraining.

0.30

Implicit Q-learning-bootstrapped ant colony optimization for maritime moving-target observation scheduling with agile satellites

cs.AI ·2026-08-25 ·He Wang, Junyu Wu, Yeye Liu, Yifan Zhou, Jie Zhang +3 more
published similarity 0.64
Abstract · Revised manuscript incorporating changes made during peer review. Published in Aerospace Science and Technology

Maritime moving-target observation scheduling with agile Earth observation satellites is a dynamic, sequence-dependent combinatorial optimization problem. Sea-surface targets move continuously, causing feasible observation windows to vary with target motion and satellite orbital geometry. The scheduler must jointly determine task selection, satellite assignment, observation-window selection, and observation ordering under time-window, attitude-maneuvering, and onboard-resource constraints. This paper proposes an implicit Q-learning-bootstrapped ant colony optimization method, termed IQACO, for multi-satellite maritime moving-target observation scheduling. Rather than directly learning a task-selection policy, IQACO embeds an offline implicit Q-learning module into constructive ant colony optimization to adaptively adjust the pheromone factor, heuristic factor, and evaporation rate. A compact search-state representation captures pheromone distribution, current and historical-best solution quality, and iteration progress. During online scheduling, ant colony optimization constructs feasible observation sequences, while the learned policy adjusts the search behavior according to the current search state. Experiments on 14 scenarios with different scales and satellite configurations show that IQACO consistently outperforms the compared algorithms, improving the mean objective value over conventional ant colony optimization by 2.86%-9.41%. Further comparative and supplementary experiments demonstrate its effectiveness and robustness across different scheduling conditions and problem settings. These results indicate that offline value learning provides an effective adaptive search-control mechanism for constrained maritime moving-target observation scheduling.

0.40

Risk-Sensitive Mission Planning for Multi-Target On-Orbit Inspection via Distributional Reinforcement Learning

cs.AI math.OC ·2026-10-30 ·Adam Abdin
published similarity 0.64
Abstract

Multi-target on-orbit servicing missions require planning tools that can adapt to evolving operational conditions while accounting for the outcome variability inherent to safety-critical spaceflight. This paper presents an adaptive planning approach for multi-target servicing campaigns in low Earth orbit and applies it to inspection missions flown by a low-thrust servicer. The approach is based on Distributional Reinforcement Learning, which learns the full distribution of mission returns and thereby supports explicit risk-sensitive decision-making at each planning step. Specifically, a Quantile-Regression Deep Q-Network (QR-DQN) agent selects which target to inspect next by jointly considering the expectation and the variance of the predicted return through a mean-variance criterion. The mission environment models circular orbits with a spread in inclination and uses Edelbaum's combined-manoeuvre low-thrust transfer estimates. The observation comprises the orbital elements, the visit history, the observed risk state of each target and the remaining mission-time budget. The risk state of each target changes stochastically during the campaign, making the outcome of an inspection uncertain and creating operationally meaningful distinctions between risk-neutral and risk-averse planning strategies. Because the mission-time budget generally admits only a subset of the candidate targets, the planner must also select which targets to inspect. Numerical experiments on scenarios with five target satellites show that one risk-aversion weight moves the policy across a range of mean return and return variance, over an unchanged environment and reward. Risk-averse agents reduce the variance of the episodic return by approximately one half relative to risk-neutral agents and lower the probability of a mission loss, at a cost in mean return. To validate the learned policies, the efficient frontier, which gives the highest mean return attainable at each level of return variance, is computed exactly by enumerating the admissible visit orders. At equal variance, the learned policies lie close to this frontier in mean return, and the risk-averse policies have a lower probability of a mission loss than the frontier policies.

0.36