CS Paper Radar

On the Value Function of Infinite-Horizon Optimal Control of Piecewise Affine Systems

Francesco Cordiano, Kanghui He, Bart De Schutter

eess.SY math.OC ·submitted 2026-08-20 01:04 UTC · v1

Abstract

In this paper, we study the structure of the value function in constrained infinite-time optimal control (CITOC) problems of piecewise affine (PWA) systems, with $\ell_1$ or $\ell_\infty$ stage cost. Existing works, such as [1], establish that the resulting value function is PWA in the state. However, existing results do not analyze whether the value function is a proper PWA function, i.e., with a finite number of affine pieces over compact sets, or whether the number of pieces can be infinite. We show that the latter case is indeed possible by means of an explicit example, which is also instrumental in establishing rigorous and easily verifiable sufficient conditions that ensure that the resulting value function is a proper PWA function. Our theoretical findings complement well-known results, e.g., the linear-quadratic case, and serve as support for recent learning-based control schemes for PWA systems. Throughout the paper, the proposed results are illustrated by means of a numerical example.

Comment
Submitted to IEEE Transactions on Automatic Control

Closest papers

Nearest neighbours by meaning across the whole index.

Risk-Sensitive Stochastic Optimal Control on Positive Cones

eess.SY math.OC ·2026-11-04 ·Mohamad Al Ahdab, John-Josef Leth
similarity 0.64
Abstract

We consider stochastic optimal control problems in which the state is constrained to a proper cone. Specifically, we consider risk-neutral, risk-averse, and risk-seeking formulations within a single entropic-risk framework. We investigate classes of problems whose Bellman operator preserves affine functions and characterize their optimal value functions and feedback policies for both finite-horizon and time-average infinite-horizon problems. Additionally, we establish convergence and state-bound results under finite-step cost observability. Moreover, we demonstrate an exact connection between entropic-risk optimal control on proper cones and deterministic infsup control for systems whose dynamics and stage costs are affine in the disturbances. In addition, we provide examples of classes of systems for which explicit solutions can be obtained.

0.31

Reach-Stabilize Control of Control-Affine Systems with Unknown Affine Parameters

eess.SY math.OC ·2026-10-08 ·Alexander Dorsey, Kunal Garg, Ankit Goel
similarity 0.63
Abstract

This paper considers the reach-stabilize prob- lem for a class of nonlinear control-affine systems with unknown parametric uncertainties, where the system states must remain in a safe set at all times, or enter a safe set in finite time and then remain in it, and converge to a goal point. An estimator is designed that generates a parameter estimate and a computable, nonincreasing bound on the estimation error from a known initial error bound, without requiring persistence of excitation. The bound defines margins that are added to a control barrier and a control Lyapunov function condition, which are then enforced in a quadratic program for efficient control design. It is shown that, for the closed-loop system, the safe set is forward invariant when the system starts within the safe set and is finite-time reachable from outside the safe set with a recovery-time bound that does not depend on the unknown parameter, and that the goal point is exponentially stable. In two numerical examples, with the initial state outside and inside the safe set, the proposed control policy, a baseline oracle control policy that uses the true parameter, both recover or remain in the safe set and converge to the goal point, while another baseline control policy that uses a fixed, incorrect, parameter does not remain

0.18

On the recursive feasibility and stability of MPC without terminal constraints

eess.SY math.OC ·2026-11-17 ·Mara Perez, Ignacio Sanchez, Filiberto Fele, Daniel Limon
similarity 0.62
Abstract · 6 pages, 1 figure. Comments are welcome!

In this paper, we analyze stability and recursive feasibility of a model predictive control (MPC) scheme that does not rely on terminal ingredients -- such as terminal constraints or a terminal cost based on a terminal controller. In the proposed formulation, we augment the cost function by a weighted finite tail which is optimized online, and we show it plays the role of a stabilizing terminal cost. Under a relaxed, local dynamic programming condition on the tail, we certify a family of regions -- parameterized by the tail's weight coefficient -- on which the problem is recursively feasible and the origin is asymptotically stable. These regions are larger as the weight increases: in particular, we show that any state that can be steered within the control horizon to a predefined neighborhood of the origin is contained in a certified region for a sufficiently large weight. Finally, we characterize the closed-loop performance over the certified region, bounding the suboptimality of the closed loop with respect to the infinite-horizon optimal cost. We conclude the discussion by illustrating these results on a simple numerical example.

0.29

Infinite Horizon Mean-Field Terminal Value Problem

eess.SY math.OC ·2026-09-11 ·Uğur Aydın, Tamer Başar, Naci Saldi
similarity 0.61
Abstract · 39 pages, 2 figures

We introduce a class of finite- and infinite-horizon discrete-time control problems for studying macroscale systems in discrete time, which we refer to as mean-field terminal value problems (MFTVPs). An MFTVP takes a terminal \(Q\)-function as input and, starting from this terminal \(Q\)-function, seeks an indefinite backward evolution of optimal policies and state measures, as required for an infinite-horizon non-stationary mean-field equilibrium (MFE) of a discrete-time mean-field game (MFG). We study the existence and uniqueness properties of this problem. As an application, using the uniqueness properties of MFTVPs, we show that, in the average-cost setting (and in the discounted setting for sufficiently small discount factors), MFTVPs can be used as an intermediate step to develop approximation schemes between finite-horizon MFEs and average-cost MFEs (respectively, stationary MFEs). In contrast to approaches that use infinite-horizon MFGs as intermediate problems, MFTVPs can be employed when the discount factor is sufficiently small (or, in the average-cost setting, when the drift factor is sufficiently small), together with a regularizer, to approximate a stationary MFE. Assuming uniqueness of the solution of a MFTVP, the restrictions we impose on the Lipschitz parameters of the system are weaker than the contraction conditions available for regularized stationary MFGs, and our approximation scheme remains applicable even in the presence of multiple stationary MFEs.

0.30

Infinite-Horizon Sparse Optimal Control: Solution through a Finite-Horizon Subproblem and Its Receding-Horizon Implementation

eess.SY math.OC ·2026-08-18 ·Yasuaki Oishi, Takumi Iwata, Masaaki Nagahara
similarity 0.61
Abstract · 17 pages, 4 figures

Sparse optimal control is considered in the infinite horizon. In the literature, sparse control has been considered mostly in a finite horizon for its formulation into a finite-dimensional optimization problem. It is shown in this paper that an optimal solution of the infinite-horizon sparse control problem can be obtained through a solution of some finite-horizon subproblem. This is due to sparsity of the optimal solution in the sense that the optimal control input is constantly equal to zero at its tail. An estimate is given on the horizon length required by this subproblem and its adaptive choice is also discussed. Implementation with a receding-horizon technique is considered and its optimality and sparsity are guaranteed.

0.58

Predicted-Flow Control Barrier Functions for Non-Control-Affine Systems

eess.SY ·2026-09-11 ·Amirsaeid Safari, Jesse B. Hoagg
similarity 0.61
Abstract

Control barrier functions (CBFs) enforce safety through conditions imposed pointwise in time without consideration of state evolution over a future horizon. Thus, CBF-based controls are typically myopic. Predicted-flow CBFs (P-CBFs) generalize CBFs from a function of the current state to a functional of the predicted flow under a parametrized control. P-CBFs can certify safety over the entire prediction horizon while simultaneously allowing for performance optimization over the horizon. However, prior work with P-CBFs only applies to control-affine systems, and suffers from a limitation where the prediction horizon can shrink or even vanish. This article addresses both of these shortcomings by introducing a planning control that smoothly transitions from a parametric plan to a backup control (if needed) and a time-shift parameter that determines if transition is needed. The evolution of the control-plan and time-shift parameters is determined from a single guaranteed-feasible convex optimization, which reduces to a quadratic program (QP) if the control limits are a convex polyhedron. The real-time executed control is determined from the instantaneous planning control and time shift. This method simultaneously addresses safety certification and performance optimization over the fixed prediction horizon. The approach is compared to nonlinear model predictive control in simulation of an autonomous car navigating a dense obstacle environment.

0.48

Reinforcement learning for constrained stochastic linear--quadratic control in continuous time

cs.LG math.OC ·2026-10-26 ·Weiping Wu, Qitong Pan, Xun Li, Peng Wu
similarity 0.59
Abstract

We study continuous-time stochastic linear--quadratic control with a scalar state and nonnegative control constraints. With quadratic running costs, randomized exploration can change the optimal mean feedback. We introduce an auxiliary problem without entropy in which the conditional policy mean is nonnegative and the covariance is prescribed in proportion to the squared state. Moment reduction and state separation yield two regional Riccati equations and characterize the finite-horizon value function and optimal conditional mean. We give a sufficient control--cost compatibility condition under which this mean coincides with the optimal feedback of the original constrained problem. For time-invariant coefficients, we characterize the undiscounted infinite-horizon optimum and establish mean square asymptotic stability under the stated assumptions. For unknown dynamics, we develop model-free actor--critic schemes for finite and infinite horizons. Numerical experiments compare the learned policies with their analytical targets and examine the effect of exploration on the optimal mean.

0.25

Conditions for Normality of Pontryagin's Maximum Principle in the Absence of State Constraints

eess.SY math.OC ·2026-11-03 ·André Tits, Pierre-Antoine Absil, Noé Tits
similarity 0.59
Abstract

Pontryagin's Maximum Principle is said to be "normal" for the problem at hand when it has a solution (maximizer u*) that explicitly involves the problem's cost function, making the maximum principle helpful (while in the opposite situation it is of no help). The aim of the present paper is, for Lagrange optimal control problems without state constraints, to identify classes of problems for which the Principle is normal. General conditions are introduced that guarantee normality, and classes of problems that satisfy these conditions are identified. Such classes include problems for which the constrained (by bounds on the control values) "augmented" reachable set, with cost along an upward vertical axis, has a convex lower-boundary function and the optimal terminal state lies in the relative interior of the reachable state set, with the terminal constraint within the affine hull of that set. Such classes include problems for which the constrained (by bounds on the values of the control signal) "augmented" (with the cost value along an upward "vertical axis") reachable set has a "convex" lower boundary for which the optimal terminal state lies in the relative interior of the set of reachable states. Special attention is given to Lagrange steering problems: sufficient conditions are obtained on a nested sequence of sub-classes of such problems, culminating with the class of steering problems with scalar control signal and linear time-invariant (LTI) state dynamics and objective integrand, for which normality is guaranteed whenever all eigenvalues of the state dynamics matrix A are real (e.g., if A is symmetric or triangular or the system is overdamped) or have small enough imaginary parts; simple 2 x 2 examples show that, in contrast with the situation in the finite-dimensional case, this property does not extend to arbitrary A matrices.

0.31