CS Paper Radar

A physics-constrained deep reinforcement learning method for autonomous mission planning of Titan deep space probes

Hanmin Li, Tao Wang, Shuangliang Liu, Zhe Zhang, Maodeng Li

cs.AI cs.RO ·submitted 2026-11-11 12:12 UTC · v1

Abstract

Titan is a compelling target for future deep space exploration, while its extreme Earth-Titan distance and complex environment make continuous ground-in-the-loop planning impractical. Existing planning methods, primarily developed for Earth-observation scenarios, have limited generalizability to uncertain deep space conditions and adaptability to time-varying mission states. To address these challenges, this paper proposes a physics-constrained maskable proximal policy optimization (PCM-PPO) method for Titan autonomous mission planning. A Titan-specific discrete-time Markov decision process is established to characterize mission feasibility and resource constraints. A system dynamics and geometric model is incorporated to enhance physical representation, including central gravity, J2 perturbation, third-body attraction, and downlink occultation. To accommodate opportunity targets under a bounded action dimension, PCM-PPO organizes predefined and opportunity targets through a fixed-capacity target pool and represents the mission state with a target-wise state matrix and a global state vector. A physics-constrained action mask constructs an admissible decision domain from current mission states, guiding policy learning toward physically executable decisions and avoiding infeasible exploration. An integrated training strategy facilitates policy optimization under coupled constraints. Comparative, ablation, and uncertainty experiments validate the effectiveness of PCM-PPO. PCM-PPO provides a feasible and interpretable scheme for autonomous mission planning of Titan deep space probes.

Closest papers

Nearest neighbours by meaning across the whole index.

Multi-Horizon Future Discovery for Physics-Informed Exploration

cs.LG ·2026-10-26 ·Shuwei Wang, Qiang Guan, Brian Qiu, Haoxiang Huang, Haoxi Wang +3 more
similarity 0.65
Abstract

Exploration in physical control environments requires agents to identify actions whose consequences reveal previously unexplored aspects of the underlying dynamics. However, many intrinsic motivation methods rely on immediate prediction errors, state novelty, or local physical changes, which may overlook informative outcomes that emerge across extended temporal horizons. We investigate whether learning structured representations of reachable futures can provide a more effective basis for exploration. To this end, we introduce Hamiltonian Temporal Contrastive Curiosity (HTC$^2$), a framework that combines multi-horizon temporal representation learning with Hamiltonian phase-space structure. A horizon-conditioned contrastive critic learns temporal relationships from future-state pairs sampled within the same episode. The distance between positive-pair representations is then used as an intrinsic reward, encouraging the agent to explore futures that remain insufficiently organized in the learned representation. To incorporate information about physical dynamics, we additionally modulate this reward using bounded multi-step Hamiltonian variation, rather than directly rewarding changes in the Hamiltonian itself. We integrate HTC$^2$ with both Soft Actor-Critic (SAC) and Proximal Policy Optimization (PPO), and evaluate it across six DeepMind Control Suite tasks. Comparisons with ICM, RIDE, E3B, Plan2Explore, and a single-step Hamiltonian curiosity baseline are complemented by state-coverage evaluations, component ablations, and phase-space diagnostics to characterize the contributions of temporal learning and physical structure to exploratory behavior.

0.07

Deep Reinforcement Learning with Massively Parallel Simulation for Low-Thrust Multi-Revolution Rendezvous

eess.SY ·2026-11-07 ·Chenglong Wu, Yi Zhou, Bin Cheng, Nan Zhang, Hexi Baoyin
similarity 0.64
Abstract · 23 pages, 8 figures, 13 tables

The growing demand for on-orbit servicing and constellation management in low Earth orbit requires autonomous and reliable low-thrust rendezvous. Traditional numerical methods struggle with the long-horizon nature of multi-revolution maneuvers under perturbations, while standard deep reinforcement learning (DRL) methods are limited by low simulation throughput and poor sample efficiency. To address these challenges, this paper proposes a massively parallel DRL guidance framework. A fully tensorized, GPU-resident orbital propagator is developed to integrate $J_2$-perturbed orbital dynamics and policy optimization within a zero-copy pipeline, achieving up to a 67-fold increase in simulation throughput over conventional CPU baselines. In addition, the rendezvous problem is formulated in modified equinoctial elements and trained with a progressive curriculum learning strategy to improve convergence and generalization across increasingly challenging rendezvous scenarios. Simulations show that the proposed framework can learn robust rendezvous trajectories for complex multi-revolution rendezvous over a broad range of orbital configurations. Additional evaluations under a high-fidelity model and eclipse-induced thrust interruptions further demonstrate the robustness of the learned policy under practical mission constraints. Moreover, the trajectories generated by the proposed framework can be used as effective initial guesses for optimization-based trajectory solvers, improving the convergence of high-precision trajectory optimization. The method offers a scalable approach for real-time low-thrust guidance and trajectory optimization.

0.36

Spacecraft Rendezvous Trajectory Generation with Modular Constraints via Diffusion Model Composition

cs.RO cs.LG ·2026-10-05 ·Mariko A. Storey-Matsutani, Richard Linares
similarity 0.64
Abstract

Emerging mission classes such as on-orbit servicing, satellite inspection, and active debris removal require trajectory design methods that are adaptable to a variety of mission scenarios. We present a diffusion-based trajectory generation approach for rendezvous and proximity operations (RPO) that enables flexible configuration of mission constraints. First, individual energy-based diffusion models are trained to satisfy distinct constraints such as approach cone and sensor line-of-sight from a set of optimized trajectories. Then, at inference time, the learned energy models can be composed with one another, or with an analytically defined energy field, to enforce specific constraint combinations. We validate this framework with the composition of a learned approach cone model and a learned sensor line-of-sight model, as well as a learned approach cone model and synthetic obstacle avoidance model, both of which yield constraint satisfaction rates that are within 1 percentage point of the single-constraint models or higher. These results indicate that our compositional diffusion framework can provide a modular approach to RPO trajectory design and enable reconfiguration for new constraint combinations without requiring model retraining.

0.30

Chance-Constrained Inspection Viewpoint Sequencing for On-Orbit Servicing via Safe Reinforcement Learning

cs.AI math.OC ·2026-10-30 ·Chloé Carer, Adam Abdin
published similarity 0.64
Abstract

Autonomous inspection of a target spacecraft is typically the initial step in on-orbit servicing. Planning close-proximity observation viewpoints involves a complex combinatorial problem constrained by coverage, fuel, and safety considerations. This paper models the viewpoint selection as a Constrained Markov Decision Process (CMDP) and applies Proximal Policy Optimization with Lagrangian relaxation to solve it. The system combines a high-level planner, which sequences camera viewpoints on an observation sphere around the target, with a low-level model that assesses each transfer using the Clohessy--Wiltshire (CW) equations for relative motion. The deterministic constraints on single-impulse reachability and $Δv$ cap are enforced directly on the action set. Safety margins are affected by navigation and manoeuvre errors, making the actual clearance stochastic. To address this, a chance constraint on the realised trajectory is implemented, with a Lagrangian multiplier governed by PID control to adapt the penalty during training. The approach is tested on a Galileo-class navigation satellite in medium Earth orbit, modelled as a point cloud where visibility from each viewpoint considers field-of-view and line-of-sight occlusion. Across various risk budgets, the trained policy maintains the keep-out violation rate within the specified limit, achieving coverage close to its maximum and staying within the mission $Δv$ budget. Compared to the usual practice of applying a fixed margin to the keep-out radius and planning deterministically, the chance-constrained policy attains less than half the violation rate at comparable coverage. This makes the collision risk an explicit design parameter, eliminating the need for re-tuning per case. The learned policy selects a viewpoint without the repeated Monte Carlo risk estimates required by a greedy, chance-constrained planner at every decision point.

0.32

SCoCaT: Success Conditioned Constrained Reinforcement Learning for Spacecraft Docking

cs.RO ·2026-09-05 ·Aman Arora, Ricard Marsal I Castan, Matteo El-Hariry, Miguel Olivares-Mendez
published similarity 0.64
Abstract · Accepted at CoRL 2026 Conference (https://www.corl.org/)

Termination-based constrained reinforcement learning is attractive for safety-critical robotic deployments: it avoids online optimization at inference, scales easily to many constraints via a single scalar per constraint, and is simpler to implement than commonly used Lagrangian methods. Instead of pricing violations through summed cost penalties, this approach makes violations structurally unprofitable by shortening the effective horizon for each violation. We identify a structural failure mode of this method class on terminal-navigation tasks: reaching a precise goal configuration while satisfying safety constraints that tighten along the final approach. When the goal sits inside the region close to where the constraints become active, the survival-weighted objective makes dwelling outside the goal region strictly preferable to entering, producing high constraint compliance with low task completion. We formalize this pathology and show that a minimal augmentation to off-the-shelf RL algorithms like PPO resolves this ``feasibility collapse''. We empirically demonstrate that adding a dense per-step success signal via an auxiliary value critic improves the task completion rate while maintaining safety-critical constraint compliance. Validation across two representative spacecraft platforms: a 6U-CubeSat spanning the mass and degree-of-freedom envelope of operational proximity operations, and a floating platform testbed for zero-shot sim-to-real transfer in our laboratory, supports the generality of these findings.

0.36

Phase-Gated Constrained Reinforcement Learning for End-to-End Capture and Detumbling of Non-Cooperative Targets

cs.RO eess.SY ·2026-10-11 ·Álvaro Belmonte-Baeza, Celia Redondo-Verdú, José L. Ramón, Miguel Cazorla, Leonard Felicetti +1 more
similarity 0.63
Abstract · 20 pages. Submitted for review to an IEEE Scientific Journal

Active debris removal requires a servicer spacecraft to approach a non-cooperative tumbling object, capture it, and arrest its rotation. This sequence is conventionally divided among phase-specific controllers, each depending on knowledge of the target's inertia, a dedicated grappling fixture, or a hand-designed switch at the contact-rich seam where such pipelines are most fragile. We show that a single constrained reinforcement learning policy can execute the entire mission, driving a dual-arm free-flying servicer from free-space closing through friction-based fixture-free capture to full detumbling of a target whose mass and inertia are unknown and whose state is observed only through noisy estimates. The enabling mechanism is a smooth proximity gate applied at three loci of a constrained Markov decision process. In the reward it induces the phase structure rather than imposing it; in the action space it forecloses a degenerate grab-and-pull exploit as a hard guarantee; and in the constraint set it resolves a normalization failure of constraint-based learning, in which a constraint spanning phases that load it unequally is rendered nearly inert in the phase where precision matters most. Both the failure mode and the remedy apply to any multi-phase problem of this kind. Over 1000 randomized trials the policy succeeds in 99.5% while holding its contact-force, actuator, base-reaction and collision limits throughout, transfers zero-shot to unseen geometries and centre-of-mass offsets, and extrapolates across a tenfold range of target mass.

0.28

DreamLander: A 6-DOF World Model based Reinforcement Learning Planetary Lander

cs.RO math.OC ·2026-11-09 ·George Nehma, Patrick Quinn, Madhur Tiwari, Camilo Riano-Rios
published similarity 0.63
Abstract

Soft powered descent landings are becoming evermore important in the field of space exploration and scientific advancement, because of their ability to deliver mission critical payloads, both highly accurately and safely. The high efficiency restriction on planetary landings due to the severely limited fuel supply results in the need for optimal guidance trajectories and control. Model based reinforcement learning is proving to be an important tool in simplifying the highly complex process of generating optimal guidance trajectories and control. In this work, we present DreamLander, a state of the art model based reinforcement learning algorithm based on DreamerV3, to generate optimal guidance and control for a realistic 6 degree of freedom entry descent landing problem. The model based approach creates a sample efficient and highly generalizable model that can be applied to a variety of scenarios whilst maintaining a high degree of accuracy, landing within 2 meters of the designated landing zone, and with velocity norm less than 0.3 meters per second. DreamLander makes use of state sensor observations, making progress toward End-to-End solutions of the Entry Descent Landing problem.

0.33

Physics-informed neural networks for spacecraft trajectory optimization with exact boundary conditions

cs.LG ·2026-11-16 ·Samuel Schönherr, Günther Waxenegger-Wilfing, Sascha Ranftl
code/data similarity 0.63
Abstract · 35 pages, 8 figures, for reproducibility: https://github.com/s-schoenherr/SpacePINN

We use neural networks to optimize trajectories by modeling the position as a function of time and minimizing the thrust with respect to the network parameters. This corresponds to an unsupervised training procedure in which physical laws are incorporated directly into the optimization objective. In standard physics-informed neural networks, initial and terminal states are enforced through additional weighted loss terms, resulting in a multi-objective optimization problem that is highly sensitive to the choice of loss weights and can lead to poor convergence and reduced accuracy. To address this challenge, we construct a simple transformation, exploiting the mathematical structure of trajectory optimization problems, to enforce boundary conditions exactly a-priori. This transformation can be interpreted as a reparameterization of the solution space that embeds the boundary conditions directly into the function. This converts the original constrained optimization problem into an unconstrained one over admissible trajectories. I.e. the transformation eliminates the boundary condition penalty terms and reduces the problem to a single-objective optimization. Crucially, trajectory optimization comes with a low-dimensional boundary, which allows particularly simple and stable constructions of distance functions compared to general domains in partial differential equations. Numerical experiments demonstrate that the proposed approach meaningfully improves convergence behavior and solution accuracy.

0.43