CS Paper Radar

Implicit DeepReach Without Implicit Differentiation

Xingjian Li, Kelvin Kan, Deepanshu Verma, Krishna Kumar, Hayden Schaeffer, Samy Wu Fung

eess.SY math.OC ·submitted 2026-11-09 22:00 UTC · v1

Abstract

We consider the problem of approximating backward reachable and reach-avoid tubes for high-dimensional nonlinear systems whose Hamiltonians require numerical optimization over constrained controls and disturbances. Existing neural Hamilton-Jacobi (HJ) reachability methods alleviate the dimensionality bottleneck of grid-based solvers, but evaluating Hamiltonians without closed-form expressions introduces an additional computational challenge. We show that, under suitable assumptions, the gradients required for value-function training can be computed without differentiating through the inner optimization problem. Specifically, we establish an envelope-type gradient identity for the saddle point Hamiltonian, under which the optimal control and disturbance enter backpropagation as constants without gradient tracking. This enables end-to-end neural HJ reachability with numerically evaluated Hamiltonians while avoiding both solver unrolling and implicit differentiation. We demonstrate the accuracy and computational efficiency of the proposed approach on nonlinear reachability and reach-avoid problems of different scales.

Closest papers

Nearest neighbours by meaning across the whole index.

Gradient-Free Neural Hamilton-Jacobi Reachability for Scalable Safety-Critical Control

cs.RO eess.SY ·2026-09-12 ·Zeyuan Feng, Ali Fuat Sahin, Santiago Thorup, Somil Bansal
benchmarkhighly cited similarity 0.76
Abstract

Hamilton-Jacobi (HJ) reachability provides a principled framework for synthesizing safety certificates and robust controllers for safety-critical robotic systems. However, applying reachability analysis to high-dimensional nonlinear systems remains challenging: classical grid-based solvers suffer from the curse of dimensionality, continuous-time neural solvers require accurate spatial value gradients, and reinforcement-learning-based approaches often suffer from weak boundary anchoring and non-stationary adversarial policy optimization. We propose a discrete-time neural reachability framework for control-disturbance-affine systems that learns backward reachable tubes (BRTs) and backward reach-avoid tubes (BRATs) through Bellman-Isaacs value propagation. Our key idea is to combine equation-driven self-supervision with structured policy learning: rather than computing explicit PDE-gradients, we exploit the bang-bang structure of optimal safety interventions to construct approximate teacher actions from gradient-free value probes, converting adversarial actor learning into supervised policy learning. To stabilize long-horizon value propagation, we leverage the learned actor to train the value function backward from the terminal boundary using a windowed temporal curriculum, where each window is used as the boundary condition for the next window. Across benchmark problems up to 80 dimensions, our method learns accurate reachability value functions while improving stability over existing learning-based solvers. We further demonstrate observation-space scalability on F1-tenth racing with over 16,000-dimensional egocentric inputs. The learned safety filter generalizes zero-shot to unseen tracks and transfers to a physical RC car, achieving real-time robust collision avoidance.

0.47

Forward Trajectory Steering for Hamilton-Jacobi Reachability Analysis

eess.SY cs.LG ·2026-08-11 ·Sungje Park, Stephen Tu
similarity 0.73
Abstract · IEEE CDC 2026

Hamilton-Jacobi (HJ) reachability provides a mathematically rigorous framework for safe control of dynamical systems, but its practical application is bottlenecked by the computational complexity of solving Hamilton-Jacobi-Isaacs variational inequality PDEs in high dimensions. Physics-informed neural networks (PINNs) have recently emerged as a promising alternative to classical mesh-based solvers, yet their performance is highly sensitive to the choice of collocation sampling. In order to learn accurate safety value functions, existing PINNs-based HJ reachability solvers must rely on complex training pipelines and auxiliary supervision. In this work, we propose STEER2REACH (S2R), a PINNs-based HJ reachability solver that requires minimal modification on top of standard PINNs training. S2R's key contribution is a lightweight, low-overhead adaptive collocation sampling distribution constructed by steering forward trajectories using a combination of the optimal control and disturbance signals induced by the current value function, with injected stochastic exploration noise. We demonstrate that despite its simplicity, S2R achieves competitive--and in some cases improved--performance on safety metrics while reducing relative L2 error across a range of reachability benchmarks compared with SoTA MPC-guided HJ reachability solvers, all without requiring multi-stage training or MPC-based supervision.

0.12

VertexCBF: Improving Neural Control Barrier Functions via Vertex-Restricted Control Search

cs.RO cs.LGeess.SY ·2026-09-11 ·Bojan Derajić, Sebastian Bernhard, Wolfgang Hönig
similarity 0.66
Abstract

As the number of autonomous robots continues to grow, safety becomes increasingly important. Control barrier functions (CBFs) provide a theoretically grounded framework for ensuring safety, but existing design methods often face limitations in effectiveness, scalability, or interpretability, and may result in overly conservative safe sets. In this paper, we propose \emph{VertexCBF}, a framework for learning neural CBFs in a scalable, systematic, and explainable way. We approximate the stationary Hamilton--Jacobi value function using a neural network trained via a combination of physics-informed and sparsely supervised learning. By exploiting control-affine dynamics and a convex polytope control set, under which the Hamiltonian is maximized at the control vertices, we efficiently generate supervision points via GPU-parallel vertex-restricted tree search, while a residual architecture guarantees that the learned CBF is never larger than the specified constraint function. We evaluate the method on 15 systems and compare it against relevant baselines, showing that it reliably recovers large safe sets where the baselines are conservative or fail completely. In addition, we perform a hardware experiment in which a mobile robot safely avoids pedestrians using a neural CBF trained with our method.

0.26

ConDR: Conservative Reachability through Neural Hamilton-Jacobi Subsolutions

cs.RO cs.LGeess.SY ·2026-11-04 ·Kendra Givens, Taylor T. Johnson, Bryan C. Ward, Thomas Beckers
theoryhot direction similarity 0.65
Abstract · 8 pages, 4 figures

Hamilton-Jacobi (HJ) Reachability identifies the unsafe states of a dynamical system from which it can be forced into failure under worst-case disturbance. However, classical grid-based solvers scale exponentially with state dimension. DeepReach improves scalability by replacing the grid with a neural value function trained on the HJ-Isaacs (HJI) variational inequality (VI), but penalizes both sides of the residual equally. This allows the learned value to lie above the true one and unsafe states to be misclassified as safe. We propose Conservative DeepReach (ConDR) which aims to learn a subsolution of the HJI VI such that the learned unsafe set contains the true one. ConDR enforces the obstacle branch and terminal condition by construction, pushes the HJI residual to a safety margin $ε$ below zero with a one-sided loss, and prevents the degenerate solution of classifying everything as unsafe. We prove that the exact solution of the margin-shifted VI that ConDR trains toward is a subsolution of the original VI so its unsafe set contains the true one and its value lies within $ετ$ of the true value function. We compare against DeepReach and MADR in the established systems of Air3D, Air6D, a 9D multi-vehicle system, and a 13D quadrotor. ConDR reports the lowest rate of unsafe state misclassified and the lowest closed-loop collision rates with at most a small increase in intervention. On Air3D and Air6D, ConDR tolerates model mismatch that scales with $ε$. At matched false-negative rates, ConDR's unsafe sets are smaller than the baselines showing the safety is not obtained by trivial over-approximation.

0.34

Data-Driven Hamiltonian for Neural Optimal Control

eess.SY ·2026-10-26 ·Christopher A. Strong, Aswin Surya, Namhoon Cho, Jason J. Choi, Claire J. Tomlin
benchmark similarity 0.65
Abstract · 21 pages, 9 figures

Reinforcement learning (RL) has emerged as a prominent data-driven approach to optimal control, but often requires substantial amounts of data and provides limited guarantees when encountering out-of-distribution state-action pairs. In this paper, we propose an alternative approach grounded in the well-established Hamilton-Jacobi framework in optimal control theory. Rather than requiring an explicit analytical model of the system dynamics to evaluate the Hamiltonian, we replace this with a data-driven Hamiltonian (DDH), which we generalize to a broad class of optimal control problems. In contrast to standard RL approaches, the DDH provides a principled mechanism, requiring only minimal prior system information such as a Lipschitz constant, for quantifying and accounting for uncertainty in regions not covered by the data, enabling the derivation of controllers with performance guarantees. Additional prior system knowledge, when available, can be readily incorporated to improve performance. For high-dimensional systems, we further introduce a physics-informed neural approximation of the value function, trained using a novel procedure that generates worst-case fictitious rollouts evaluated through the DDH. On an LQR problem, the DDH reaches near-optimal cost on every dataset we test, including random data on which offline RL methods remain far from optimal, and needs far fewer samples than online RL. On a nonlinear lunar lander benchmark, the DDH achieves an optimality gap to a trajectory-optimization reference more than $3\times$ smaller than CQL and FQL, and matches the performance of SAC with approximately $37\times$ fewer samples. These results establish the DDH as a promising framework for data-driven optimal control, particularly in settings where available data are limited in quantity or quality.

0.33

Data-Driven Hamiltonian Under Parametric Uncertainty

eess.SY ·2026-11-19 ·Zach Liu, Namhoon Cho, Jason J. Choi
similarity 0.65
Abstract

Hamilton--Jacobi (HJ) reachability computes the states from which a system can be kept out of an unsafe region, but it requires knowledge of dynamics. The data-driven Hamiltonian (DDH), under a black-box dynamics setting, lower bounds the Hamiltonian directly from trajectory data, which produces a guaranteed conservative safe set. However, it needs a Lipschitz constant of the dynamics, and its bound loosens with the distance from the data. We consider dynamics that are linear in unknown parameters through a known feature vector, with a Gaussian prior on the parameters, and use this structure to construct a parametric DDH. From a few short trajectories with noisy velocity measurements, the parametric DDH lower-bounds the Hamiltonian at all states and costates with a probability of at least $1-δ$. The safe set obtained by replacing the Hamiltonian with this bound is then contained in the true safe set with the same probability. For control-affine systems, evaluating the bound amounts to a second-order cone program, so it can be solved efficiently within existing grid-based and neural PDE solvers. We demonstrate the approach on a two-state system and a six-state lunar lander.

0.23

Temporal Difference Learning of Probabilistic Reach-Avoid Value Functions

cs.RO eess.SY ·2026-10-20 ·Elias Fontanari, Pietro Noah Crestaz, Matteo Saveriano, Andrea Del Prete
similarity 0.65
Abstract · Submitted to ICRA 2027

Reach-avoid sets, defining the states from which a task can be completed while avoiding unsafe regions, are fundamental tools for deploying robots in environments subject to disturbances and uncertainty, as well as for runtime task-feasibility monitoring, allowing prediction of whether a state is safe and will lead to successful task completion. Classical Hamilton-Jacobi Reachability (HJR) and its recent neural extensions (e.g., Lipschitz Continuous Reachability Learning, DeepReach) treat disturbances adversarially, computing the reach-avoid set under worst-case disturbances from a bounded set. This guarantees safety but can yield overly conservative sets when disturbances are stochastic rather than adversarial, and additionally requires the disturbance set to be known or specified in advance. We propose Probabilistic reach-avoid Value Function (PVF), a self-supervised method, applicable to black-box systems, for computing reach-avoid sets as the states with a certain reach-avoid probability under stochastic, potentially unbounded disturbances. We evaluate our approach against HJR-based baselines on systems of increasing complexity, up to a Hopper locomotion task, and show that our probabilistic sets are less conservative and more accurate than adversarial HJR sets when disturbances are non-adversarial.

0.08

Spectral Koopman-Hopf Formula for Reachability with Adversary

eess.SY math.OC ·2026-09-06 ·Sarang Sutavani, Umesh Vaidya
similarity 0.64
Abstract

This paper develops a spectral Koopman-Hopf framework for adversarial reachability analysis of nonlinear systems. By lifting the nonlinear drift dynamics into Koopman eigenfunction coordinates, the proposed approach transforms the original state-dependent Hamilton-Jacobi-Isaacs (HJI) equation into an approximate state-independent optimization problem in spectral coordinates. The transformed control and disturbance directions are approximated using least-squares spectral projections, and upper and lower bounds on the transformed Hamiltonian are used to derive corresponding bounds on the value function and backward reachable sets. Koopman eigenfunctions are computed using a path-integral-based Galerkin framework that avoids spatial discretization of the associated eigenfunction PDEs. Numerical examples demonstrate tractable approximation of backward reachable sets for nonlinear adversarial systems.

0.20