CS Paper Radar

Data-Driven Hamiltonian Under Parametric Uncertainty

Zach Liu, Namhoon Cho, Jason J. Choi

eess.SY ·submitted 2026-11-19 01:50 UTC · v1

Abstract

Hamilton--Jacobi (HJ) reachability computes the states from which a system can be kept out of an unsafe region, but it requires knowledge of dynamics. The data-driven Hamiltonian (DDH), under a black-box dynamics setting, lower bounds the Hamiltonian directly from trajectory data, which produces a guaranteed conservative safe set. However, it needs a Lipschitz constant of the dynamics, and its bound loosens with the distance from the data. We consider dynamics that are linear in unknown parameters through a known feature vector, with a Gaussian prior on the parameters, and use this structure to construct a parametric DDH. From a few short trajectories with noisy velocity measurements, the parametric DDH lower-bounds the Hamiltonian at all states and costates with a probability of at least $1-δ$. The safe set obtained by replacing the Hamiltonian with this bound is then contained in the true safe set with the same probability. For control-affine systems, evaluating the bound amounts to a second-order cone program, so it can be solved efficiently within existing grid-based and neural PDE solvers. We demonstrate the approach on a two-state system and a six-state lunar lander.

Closest papers

Nearest neighbours by meaning across the whole index.

Data-Driven Hamiltonian for Neural Optimal Control

eess.SY ·2026-10-26 ·Christopher A. Strong, Aswin Surya, Namhoon Cho, Jason J. Choi, Claire J. Tomlin
benchmark similarity 0.73
Abstract · 21 pages, 9 figures

Reinforcement learning (RL) has emerged as a prominent data-driven approach to optimal control, but often requires substantial amounts of data and provides limited guarantees when encountering out-of-distribution state-action pairs. In this paper, we propose an alternative approach grounded in the well-established Hamilton-Jacobi framework in optimal control theory. Rather than requiring an explicit analytical model of the system dynamics to evaluate the Hamiltonian, we replace this with a data-driven Hamiltonian (DDH), which we generalize to a broad class of optimal control problems. In contrast to standard RL approaches, the DDH provides a principled mechanism, requiring only minimal prior system information such as a Lipschitz constant, for quantifying and accounting for uncertainty in regions not covered by the data, enabling the derivation of controllers with performance guarantees. Additional prior system knowledge, when available, can be readily incorporated to improve performance. For high-dimensional systems, we further introduce a physics-informed neural approximation of the value function, trained using a novel procedure that generates worst-case fictitious rollouts evaluated through the DDH. On an LQR problem, the DDH reaches near-optimal cost on every dataset we test, including random data on which offline RL methods remain far from optimal, and needs far fewer samples than online RL. On a nonlinear lunar lander benchmark, the DDH achieves an optimality gap to a trajectory-optimization reference more than $3\times$ smaller than CQL and FQL, and matches the performance of SAC with approximately $37\times$ fewer samples. These results establish the DDH as a promising framework for data-driven optimal control, particularly in settings where available data are limited in quantity or quality.

0.33

Forward Trajectory Steering for Hamilton-Jacobi Reachability Analysis

eess.SY cs.LG ·2026-08-11 ·Sungje Park, Stephen Tu
similarity 0.66
Abstract · IEEE CDC 2026

Hamilton-Jacobi (HJ) reachability provides a mathematically rigorous framework for safe control of dynamical systems, but its practical application is bottlenecked by the computational complexity of solving Hamilton-Jacobi-Isaacs variational inequality PDEs in high dimensions. Physics-informed neural networks (PINNs) have recently emerged as a promising alternative to classical mesh-based solvers, yet their performance is highly sensitive to the choice of collocation sampling. In order to learn accurate safety value functions, existing PINNs-based HJ reachability solvers must rely on complex training pipelines and auxiliary supervision. In this work, we propose STEER2REACH (S2R), a PINNs-based HJ reachability solver that requires minimal modification on top of standard PINNs training. S2R's key contribution is a lightweight, low-overhead adaptive collocation sampling distribution constructed by steering forward trajectories using a combination of the optimal control and disturbance signals induced by the current value function, with injected stochastic exploration noise. We demonstrate that despite its simplicity, S2R achieves competitive--and in some cases improved--performance on safety metrics while reducing relative L2 error across a range of reachability benchmarks compared with SoTA MPC-guided HJ reachability solvers, all without requiring multi-stage training or MPC-based supervision.

0.12

Neural Barriers: An Online Certifiable Learning-enhanced Adaptive High Order Safety Critical Control

cs.RO ·2026-10-04 ·Lishuo Pan, Mattia Catellani, An Cao, Lorenzo Sabattini, Nora Ayanian
similarity 0.66
Abstract · 14 pages, 11 figures, 3 tables

Control barrier functions are an effective model-based tool to formally certify the safety of a system. However, transferring their theoretical guarantees to real-world robotics systems requires high model fidelity. For example, payloads or wind disturbances can cause significant model perturbations to an aerial vehicle, leading to safety compromises. In this work, we propose a certifiable online learning-enhanced robust adaptive control barrier function, which adapts to disturbances using a Neural ODE and quantifies its adaptation uncertainty with conformal prediction. Our approach guarantees safety at all time under unknown time-varying model disturbances. It adopts a conservative strategy when the adaptation uncertainty is high; and efficiently adapts to reduce controller conservativeness as it receives more data. Our approach provides a provable safety guarantee with a probability bound under suitable Lipschitz smoothness assumptions on the underlying model and trajectory. These results demonstrate the potential of our method as a practical safety controller for robotics system operating under model perturbations.

0.23

Estimation-Aware Reachability

cs.RO eess.SY ·2026-11-18 ·Himanshu Paudel, Javier Borquez
code/data similarity 0.65
Abstract

Hamilton-Jacobi Reachability computes the states from which a system can be kept safe under worst-case disturbances, but it assumes the state is known exactly, whereas real robots act on noisy estimates whose uncertainty evolves with their motion. We extend Hamilton-Jacobi Reachability to the robot's belief, estimate and uncertainty, treating unexpected measurements as a bounded adversary and enlarging the unsafe set by the current uncertainty, so that certified states are safe with a prescribed probability. The formulation is compatible with existing solvers. In simulation, the resulting safety filters nearly eliminate the collisions of certaintyequivalent HJ and are less conservative than equally safe uncertainty-padding baselines. The implementation code is available at https://github.com/HimanshuPaudel/Estimation-Aware-Reachability

0.33

Implicit DeepReach Without Implicit Differentiation

eess.SY math.OC ·2026-11-09 ·Xingjian Li, Kelvin Kan, Deepanshu Verma, Krishna Kumar, Hayden Schaeffer +1 more
similarity 0.65
Abstract

We consider the problem of approximating backward reachable and reach-avoid tubes for high-dimensional nonlinear systems whose Hamiltonians require numerical optimization over constrained controls and disturbances. Existing neural Hamilton-Jacobi (HJ) reachability methods alleviate the dimensionality bottleneck of grid-based solvers, but evaluating Hamiltonians without closed-form expressions introduces an additional computational challenge. We show that, under suitable assumptions, the gradients required for value-function training can be computed without differentiating through the inner optimization problem. Specifically, we establish an envelope-type gradient identity for the saddle point Hamiltonian, under which the optimal control and disturbance enter backpropagation as constants without gradient tracking. This enables end-to-end neural HJ reachability with numerically evaluated Hamiltonians while avoiding both solver unrolling and implicit differentiation. We demonstrate the accuracy and computational efficiency of the proposed approach on nonlinear reachability and reach-avoid problems of different scales.

0.07

VertexCBF: Improving Neural Control Barrier Functions via Vertex-Restricted Control Search

cs.RO cs.LGeess.SY ·2026-09-11 ·Bojan Derajić, Sebastian Bernhard, Wolfgang Hönig
similarity 0.65
Abstract

As the number of autonomous robots continues to grow, safety becomes increasingly important. Control barrier functions (CBFs) provide a theoretically grounded framework for ensuring safety, but existing design methods often face limitations in effectiveness, scalability, or interpretability, and may result in overly conservative safe sets. In this paper, we propose \emph{VertexCBF}, a framework for learning neural CBFs in a scalable, systematic, and explainable way. We approximate the stationary Hamilton--Jacobi value function using a neural network trained via a combination of physics-informed and sparsely supervised learning. By exploiting control-affine dynamics and a convex polytope control set, under which the Hamiltonian is maximized at the control vertices, we efficiently generate supervision points via GPU-parallel vertex-restricted tree search, while a residual architecture guarantees that the learned CBF is never larger than the specified constraint function. We evaluate the method on 15 systems and compare it against relevant baselines, showing that it reliably recovers large safe sets where the baselines are conservative or fail completely. In addition, we perform a hardware experiment in which a mobile robot safely avoids pedestrians using a neural CBF trained with our method.

0.26

AegisFlow: Training-Free Non-Myopic Path-Safe Guided Flow Matching

cs.AI ·2026-10-29 ·Kunpeng Liu, Boshi Zhang, Keyou You
code/data similarity 0.65
Abstract · 18 pages, 6 figures

Pretrained generative models, such as diffusion and flow matching models, have demonstrated exceptional capabilities in synthesizing high-quality data. However, steering their generative trajectories to satisfy strict constraints--such as identity preservation in image editing or obstacle avoidance in robotics--remains fundamentally challenging. While recent training-free guidance methods enforce path-safety, they rely heavily on min-norm controllers that only account for immediate constraint satisfaction. These myopic approaches often yield suboptimal trajectories that hover dangerously close to the boundary, severely restricting the model's ability to optimize toward its target. In this paper, we propose AegisFlow, a training-free, non-myopic path-safe guidance framework that recasts guided generation as a safety-critical optimal control problem with global scope. By integrating a time-varying barrier function directly into the global cost functional, AegisFlow naturally enforces constraints along the entire generative path. To compute the global optimal control, we derive a tractable, mathematically grounded approximation of the Hamilton-Jacobi-Bellman (HJB) equation. Comprehensive experiments demonstrate that AegisFlow outperforms state-of-the-art methods across text-guided image manipulation and robotic planning, establishing a significantly better trade-off between target alignment and constraint satisfaction. Code is available at https://github.com/LiuKunpeng03/AegisFlow.

0.25

Temporal Difference Learning of Probabilistic Reach-Avoid Value Functions

cs.RO eess.SY ·2026-10-20 ·Elias Fontanari, Pietro Noah Crestaz, Matteo Saveriano, Andrea Del Prete
similarity 0.64
Abstract · Submitted to ICRA 2027

Reach-avoid sets, defining the states from which a task can be completed while avoiding unsafe regions, are fundamental tools for deploying robots in environments subject to disturbances and uncertainty, as well as for runtime task-feasibility monitoring, allowing prediction of whether a state is safe and will lead to successful task completion. Classical Hamilton-Jacobi Reachability (HJR) and its recent neural extensions (e.g., Lipschitz Continuous Reachability Learning, DeepReach) treat disturbances adversarially, computing the reach-avoid set under worst-case disturbances from a bounded set. This guarantees safety but can yield overly conservative sets when disturbances are stochastic rather than adversarial, and additionally requires the disturbance set to be known or specified in advance. We propose Probabilistic reach-avoid Value Function (PVF), a self-supervised method, applicable to black-box systems, for computing reach-avoid sets as the states with a certain reach-avoid probability under stochastic, potentially unbounded disturbances. We evaluate our approach against HJR-based baselines on systems of increasing complexity, up to a Hopper locomotion task, and show that our probabilistic sets are less conservative and more accurate than adversarial HJR sets when disturbances are non-adversarial.

0.08