CS Paper Radar

Integrating Low-Thrust Transfer Dynamics into POMDP Planning for On-Orbit Servicing

Susmitha Patnala, Vincent Mousseau, Adam Abdin

cs.RO math.OC ·submitted 2026-10-30 14:55 UTC · v1
published

Abstract

Recent developments in both high-thrust (HT) and low-thrust (LT) propulsion technologies broaden design options for on-orbit servicing (OOS) missions. However, incorporating LT transfers into OOS mission planning requires accounting for the trade-off between prescribed transfer duration and propellant consumption. Multi-servicing, multi-transfer OOS mission planning is inherently sequential and requires adaptive planning tools that account for propulsion-dependent transfer durations and mass-dependent propellant consumption. Thus, transfer dynamics must be integrated into mission planning and scheduling to increase flexibility and adaptability. In addition, the planning is often done with imperfect information about the system's state and stochastic service demands. This paper focuses on modeling orbital transfers, particularly LT transfers, within a Partially Observable Markov Decision Process (POMDP) framework for OOS mission planning. To enable offline policy computation, the transfer model must provide explicit state transitions and rewards for all admissible actions. The state space is therefore augmented with the servicing vehicle's remaining propellant to enable consistent mass updates. HT transfers are modeled using a precomputed transfer matrix that maps each transfer to the time of flight and propellant consumed. LT transfers are represented using an analytical GEO phasing model with prescribed transfer-duration alternatives. For each alternative, the model calculates propellant consumption and maneuver feasibility as functions of the required phase change and the current servicing-vehicle mass. The proposed modeling approach is evaluated on simplified OOS planning scenarios using circular phasing with HT and LT propulsion. The resulting POMDPs are solved using the Successive Approximations of the Reachable Space under Optimal Policies (SARSOP) algorithm, and the policies obtained under HT and LT are compared. This comparison provides a quantitative assessment of the feasibility and operational impact of LT options, particularly regarding the potential role of advanced electric propulsion in future servicing architectures.

Journal ref
77th International Astronautical Congress (IAC), Oct 2026, Antalya (Turkey), Turkey

Closest papers

Nearest neighbours by meaning across the whole index.

Offline Approximate Value Iteration: A Reinforcement Learning Approach for Satellite Constellation Deployment

cs.AI math.OC ·2026-11-02 ·Tristan Leuridan, Yiping Fang, Adam Abdin
top venuehot direction similarity 0.58
Abstract

This paper introduces a reinforcement learning-based approximate dynamic programming approach to optimize the deployment of Walker-type satellite constellations under uncertainties, including launch failures and satellite malfunctions. We formulate the problem as a Markov Decision Process that incorporates low-injection orbit launches, continuous low-thrust orbit raising, and variable satellite loads. We adopt the offline approximate value iteration solution method, using linear function approximation with basis functions and action pruning strategies to substantially reduce the computational and memory requirements for finding optimal decision policies. Preliminary results demonstrate convergence to policies with low expected costs and deployment times, both of which are lower than those of a rolling-horizon baseline, showcasing a novel application of approximate dynamic programming in space logistics.

0.48

Mission-Centric Requirements Analysis of Model Predictive Control in Spacecraft Rendezvous and Proximity Operations

eess.SY ·2026-10-02 ·Nils Maier, Torbjørn Cunis, Tam W. Nguyen
similarity 0.58
Abstract

Rendezvous and proximity operations (RPO) missions are commonly designed as a sequence of distinct phases each governed by its own sample time, constraint set, and cost structure, with the translation from mission-level requirements to model predictive control (MPC) design parameters typically performed ad hoc. This report takes a first step toward a systematic, traceable mapping between mission requirements and MPC design choices, using the ADRIOS / ClearSpace-1 active debris removal mission as a case study across three sequential operations. Its central hypothesis is that Economic MPC (EMPC), in which the stage cost reflects propellant consumption directly, is structurally better suited to proximity operations than fixed-point regulation or reference tracking, because it can exploit the natural, fuel-free relative-orbital-motion trajectories admitted by the Clohessy-Wiltshire-Hill (CWH) dynamics instead of fighting them. We show that a pure-fuel stage cost is fundamentally degenerate for this class of problem as the closed-loop optimum is not strictly dissipative with respect to any single target state or orbit, and analyse, operation by operation, which terminal ingredient (a terminal region, a terminal cost, a terminal equality, or a stage-cost regularisation) is required to repair this degeneracy, at what fuel and time cost, and under what formal guarantee. Across all three operations the economic controllers cut fuel consumption by roughly 40-99% relative to standard quadratic-tracking MPC, at the cost of longer manoeuvre times, and we synthesise the results into a cross-scenario table of mission requirements, MPC ingredients, and their residual limitations.

0.14

A physics-constrained deep reinforcement learning method for autonomous mission planning of Titan deep space probes

cs.AI cs.RO ·2026-11-11 ·Hanmin Li, Tao Wang, Shuangliang Liu, Zhe Zhang, Maodeng Li
similarity 0.57
Abstract

Titan is a compelling target for future deep space exploration, while its extreme Earth-Titan distance and complex environment make continuous ground-in-the-loop planning impractical. Existing planning methods, primarily developed for Earth-observation scenarios, have limited generalizability to uncertain deep space conditions and adaptability to time-varying mission states. To address these challenges, this paper proposes a physics-constrained maskable proximal policy optimization (PCM-PPO) method for Titan autonomous mission planning. A Titan-specific discrete-time Markov decision process is established to characterize mission feasibility and resource constraints. A system dynamics and geometric model is incorporated to enhance physical representation, including central gravity, J2 perturbation, third-body attraction, and downlink occultation. To accommodate opportunity targets under a bounded action dimension, PCM-PPO organizes predefined and opportunity targets through a fixed-capacity target pool and represents the mission state with a target-wise state matrix and a global state vector. A physics-constrained action mask constructs an admissible decision domain from current mission states, guiding policy learning toward physically executable decisions and avoiding infeasible exploration. An integrated training strategy facilitates policy optimization under coupled constraints. Comparative, ablation, and uncertainty experiments validate the effectiveness of PCM-PPO. PCM-PPO provides a feasible and interpretable scheme for autonomous mission planning of Titan deep space probes.

0.26

Reachability-Informed Reinforcement Learning for Multi-Impulse Interplanetary Transfers

cs.LG math.OCeess.SY ·2026-10-01 ·Yashdeep Chaudhary, Roberto Armellin, Harry Holt
benchmark similarity 0.57
Abstract · Preprint. 23 pages, 10 figures

Reinforcement learning offers the prospect of a reusable sequential decision-making mechanism for spacecraft trajectory design, motivating policy interfaces that connect learned decisions to the underlying maneuver geometry. This paper develops Reachability Analysis-Informed Reinforcement Learning (RARL) for deterministic multi-impulse interplanetary transfers, placing intermediate waypoint selection at the center of the learned decision process. Local first-order reachability maps bounded velocity perturbations into an ellipsoidal set of next-node positions, within which the policy selects its waypoint. Lambert reconstruction then determines the corresponding maneuver to reach this selected waypoint along a dynamically consistent ballistic arc, coupling learned transfer-geometry selection with classical astrodynamics. A terminal two-impulse reconstruction completes the rendezvous, supported by a linear maneuver-demand assessment used for reward shaping. Numerical studies characterize this interface on a two-body Earth-Mars benchmark. Across three independent training runs, RARL achieves a mean maneuver cost of 10.23 km/s, 1.72% above a validated local sequential convex programming reference. Training over dispersed initial states extends policy reuse across a departure family with fixed target state and transfer duration. Each of the three independently trained multi-state policies completes all 10,000 held-out Monte Carlo departures without impulse-cap violations, compared with a mean feasibility rate of 6.49% for single-state policies. This broader sampled feasibility is accompanied by a 0.61% increase in mean nominal maneuver cost, without further training across departures. These results demonstrate that a reachability-informed decision interface supports benchmark-quality trajectory construction and policy reuse across dispersed departure conditions.

0.29

From Orbit Matching to Docking: A Hierarchical Reinforcement-Learning Approach to Multiphase Rendezvous Trajectory Design

cs.RO cs.LGeess.SY ·2026-10-12 ·Mariko Storey-Matsutani, Anatoly Zlotnik, Lee Holguin, Zigfried Hampel-Arias, Richard Linares
similarity 0.57
Abstract

As the complexity of space missions increases, autonomous spacecraft guidance must accommodate transitions between mission phases with distinct dynamics, objectives, and constraints. Rendezvous and proximity operations (RPO) are particularly challenging because they span multiple operating regimes, from orbital-scale matching and far-range approach to close-range maneuvering and terminal docking. Existing reinforcement-learning approaches generally address only a subset of this sequence or rely on manually defined switching conditions between controllers. This study presents a hierarchical reinforcement-learning framework for autonomous, three-degree-of-freedom RPO that spans the complete trajectory from orbit matching to docking. Five specialized policies are trained for orbit matching, far-range approach, mid-range approach, close-range approach, and docking. A learned high-level manager policy then selects among these phase-specific controllers based on the evolving spacecraft state. The hierarchical approach is evaluated over the full end-to-end rendezvous sequence and compared with a fixed sequential-invocation heuristic to assess trajectory performance and transition behavior. Training and inference metrics are also analyzed for the individual phase policies and the complete hierarchical system.

0.12

Diffusion-Based Multiple-Shooting Indirect Optimal Control for Fuel-Optimal Spacecraft Trajectory Generation

eess.SY cs.LG ·2026-09-12 ·Saeid Tafazzol, Ehsan Taheri, Ryne Beeson
benchmarkcode/data similarity 0.57
Abstract · Submitted to IEEE Control Systems Letters (L-CSS) for possible publication

Diffusion-based generative models (DMs) have found applications in control problems, and in particular robotics, where the DMs enable exploration of possible control solutions. A critical shortcoming of these applications is that they have lacked optimality guarantees. This is a problem for their potential use in fuel-optimal spacecraft trajectories that are characterized with long time-horizons and bang-bang profiles. Alternatively, indirect optimal control methods ensure explicit satisfaction of necessary conditions, but are highly sensitive to the initial costate estimation needed to solve the resulting Hamiltonian boundary-value problems (HBVPs). To alleviate this sensitivity and enlarge the convergence domain of HBVPs, advanced indirect methods have been developed that use smoothing approaches and continuation. We propose a diffusion-based multiple shooting indirect control method that combines the exploration capability of DMs with indirect method to generate fuel-optimal spacecraft trajectories. We benchmark our method against an advanced indirect method on a fuel-optimal Earth-Mars low-thrust transfer problem, showing higher convergence robustness than the advanced indirect method that is based on random costate initialization. Code and visualizations are available at https://saeidtafazzol.github.io/Diffusion_Indirect_Control/.

0.21

Chance-Constrained Inspection Viewpoint Sequencing for On-Orbit Servicing via Safe Reinforcement Learning

cs.AI math.OC ·2026-10-30 ·Chloé Carer, Adam Abdin
published similarity 0.56
Abstract

Autonomous inspection of a target spacecraft is typically the initial step in on-orbit servicing. Planning close-proximity observation viewpoints involves a complex combinatorial problem constrained by coverage, fuel, and safety considerations. This paper models the viewpoint selection as a Constrained Markov Decision Process (CMDP) and applies Proximal Policy Optimization with Lagrangian relaxation to solve it. The system combines a high-level planner, which sequences camera viewpoints on an observation sphere around the target, with a low-level model that assesses each transfer using the Clohessy--Wiltshire (CW) equations for relative motion. The deterministic constraints on single-impulse reachability and $Δv$ cap are enforced directly on the action set. Safety margins are affected by navigation and manoeuvre errors, making the actual clearance stochastic. To address this, a chance constraint on the realised trajectory is implemented, with a Lagrangian multiplier governed by PID control to adapt the penalty during training. The approach is tested on a Galileo-class navigation satellite in medium Earth orbit, modelled as a point cloud where visibility from each viewpoint considers field-of-view and line-of-sight occlusion. Across various risk budgets, the trained policy maintains the keep-out violation rate within the specified limit, achieving coverage close to its maximum and staying within the mission $Δv$ budget. Compared to the usual practice of applying a fixed margin to the keep-out radius and planning deterministically, the chance-constrained policy attains less than half the violation rate at comparable coverage. This makes the collision risk an explicit design parameter, eliminating the need for re-tuning per case. The learned policy selects a viewpoint without the repeated Monte Carlo risk estimates required by a greedy, chance-constrained planner at every decision point.

0.32

Risk-Sensitive Mission Planning for Multi-Target On-Orbit Inspection via Distributional Reinforcement Learning

cs.AI math.OC ·2026-10-30 ·Adam Abdin
published similarity 0.56
Abstract

Multi-target on-orbit servicing missions require planning tools that can adapt to evolving operational conditions while accounting for the outcome variability inherent to safety-critical spaceflight. This paper presents an adaptive planning approach for multi-target servicing campaigns in low Earth orbit and applies it to inspection missions flown by a low-thrust servicer. The approach is based on Distributional Reinforcement Learning, which learns the full distribution of mission returns and thereby supports explicit risk-sensitive decision-making at each planning step. Specifically, a Quantile-Regression Deep Q-Network (QR-DQN) agent selects which target to inspect next by jointly considering the expectation and the variance of the predicted return through a mean-variance criterion. The mission environment models circular orbits with a spread in inclination and uses Edelbaum's combined-manoeuvre low-thrust transfer estimates. The observation comprises the orbital elements, the visit history, the observed risk state of each target and the remaining mission-time budget. The risk state of each target changes stochastically during the campaign, making the outcome of an inspection uncertain and creating operationally meaningful distinctions between risk-neutral and risk-averse planning strategies. Because the mission-time budget generally admits only a subset of the candidate targets, the planner must also select which targets to inspect. Numerical experiments on scenarios with five target satellites show that one risk-aversion weight moves the policy across a range of mean return and return variance, over an unchanged environment and reward. Risk-averse agents reduce the variance of the episodic return by approximately one half relative to risk-neutral agents and lower the probability of a mission loss, at a cost in mean return. To validate the learned policies, the efficient frontier, which gives the highest mean return attainable at each level of return variance, is computed exactly by enumerating the admissible visit orders. At equal variance, the learned policies lie close to this frontier in mean return, and the risk-averse policies have a lower probability of a mission loss than the frontier policies.

0.36