CS Paper Radar

Vector Lyapunov functions for nonconvex constrained optimization

Torbjørn Cunis, Ilya Kolmanovsky

eess.SY math.OC ·submitted 2026-11-05 21:23 UTC · v1

Abstract

Nonconvex constrained optimization problems are commonly solved by a Newton method for necessary conditions (sequential linear/quadratic programming) or a barrier reformulation (interior-point methods). We present a vector Lyapunov analysis for Newton-type algorithms based on constraint violation and optimality of the primal solutions to a sequence of approximate optimization problems. We thus prove local stability and convergence by constructing a strict Lyapunov function for the primal solution.

Comment
Extended version of submission to the 2027 European Control Conference

Closest papers

Nearest neighbours by meaning across the whole index.

Interior-point proximal methods for nonsmooth optimization in Hilbert spaces with cone-ordered constraints

eess.SY math.OC ·2026-09-12 ·Behzad Azmi, Alberto De Marchi
similarity 0.65
Abstract · 37 pages, 7 figures, 3 tables, 2 algorithms

We study an inexact interior-point method for nonsmooth, possibly nonconvex optimization in a Hilbert space with inequality constraints ordered by a cone in a Banach lattice, with particular emphasis on infinite-dimensional state-constrained optimal control. The objective function is given by the sum of a smooth, possibly nonconvex term and a convex, possibly nonsmooth term with a computable proximal mapping. The constraints are formulated by means of an order cone in a Banach lattice. This setting covers finite-dimensional nonsmooth nonlinear problems with componentwise constraints as well as infinite-dimensional PDE-constrained optimization problems with pointwise state constraints. The method is based on barrier-regularized subproblems, which are solved inexactly by a proximal-gradient method. We consider logarithmic and power-type barriers and derive the differentiability and curvature estimates needed for the convergence and complexity analysis. Under suitable constraint qualifications and compactness assumptions, we establish approximate KKT conditions for the original problem and convergence of the inexact interior-point sequence. For logarithmic and power-type barriers, we derive complementarity estimates and outer iteration bounds; the corresponding power-barrier rates and total inner-outer complexity bounds are stated under explicit barrier-path and uniform smoothness assumptions. For convex problems, we obtain stronger convergence results. We apply the framework to state-constrained semilinear elliptic optimal control and sparse dictionary learning with nonlinear side constraints. Numerical experiments illustrate the proposed method.

0.23

Accelerated Safe Gradient Flow

eess.SY math.OC ·2026-10-01 ·Xuting Gao, Sonia Martínez
similarity 0.60
Abstract · Submitted to IEEE Control Systems Letters

This letter introduces an accelerated safe gradient flow (A-SGF) for strongly convex constrained optimization problems. The A-SGF filters a nominal second-order accelerated flow through a second-order control barrier function quadratic program. We establish conditions for strict feasibility and local Lipschitz continuity of the resulting vector field, prove forward invariance of a lifted safe set, and show that its unique equilibrium is the constrained minimizer. A variational Lyapunov analysis isolates the effect of the safety correction and gives explicit conditions on the barrier gains under which A-SGF retains the same convergence rate as the unconstrained Nesterov accelerated method. A nonlinear 2D example demonstrate anytime feasibility and faster convergence of the algorithm.

0.24

Primal Acceleration of Newton's Method

cs.AI math.OCcs.LG ·2026-08-21 ·Nikita Doikov
similarity 0.58
Abstract

We develop a new direct accelerated Newton method for minimizing convex functions with Lipschitz continuous Hessian. The algorithm uses only primal variables and performs just one linear solve per iteration. With a simple predetermined choice of parameters, it achieves the global convergence rate of $O(1/k^3)$ in terms of the functional residual. To the best of our knowledge, this is the first second-order method for this problem class attaining this rate while relying solely on one linear system solve per iteration (without solving auxiliary nonlinear regularized subproblems, such as cubic regularization, performing nonlinear parameter searches, or using dual extragradient corrections). Our method can be implemented in a Hessian-free way, using an inexact linear system solver, while preserving the fast global rate. We further extend our construction to arbitrary geometry through Bregman divergence, and to composite optimization problems.

0.11

Randomized and Oracle-Based Feasible Directions Methods for Nonsmooth Nonconvex Constrained Optimization

cs.LG math.OC ·2026-11-05 ·Dan Greenstein, Nadav Hallak
similarity 0.57
Abstract

Minimizing nonsmooth nonconvex objectives over general convex sets using no derivatives of the objective, together with structural access to the constraint set, poses a fundamental geometric difficulty: at boundary points, the feasible direction cone may fail to be closed, preventing the construction of finite sets of search directions that capture all feasible descent directions. We develop a feasible-directions framework for problems structured as the difference between a continuously differentiable function and a convex function over a closed convex set, and access the objective without knowledge of its decomposition, through exact line searches along feasible segments. To address the geometric obstruction, we introduce $η$-feasible directions, which remain uniformly separated from the boundary of the feasible direction cone, enabling finite-direction search even in the presence of nonlinear constraints. We present two algorithmic realizations of this framework. The first assumes access to an $η$-direction oracle and extends feasible-directions methods from linear to general convex constraints. The second replaces the oracle by a noise-augmented randomization scheme that constructs spanning directions from the affine-hull inner normal and almost surely avoids nonsmooth boundary points when initialized outside that set. Under curvature bounds and Lipschitz continuity of the smooth component's gradient, we jointly control the spanning coefficients and feasible step lengths of the oracle construction. This yields finite-time approximate $d$-stationarity along every unit feasible direction admitting a prescribed positive step length. For both methods, we also establish a rate in the proximal directional minimization (PDM) measure, assuming uniformly bounded direction-set sizes. Randomized exploration ensures that, almost surely, every accumulation point is $d$-stationary.

0.11

First-order smoothed exact-penalty methods for equality-constrained regularized nonconvex optimization

cs.LG math.OC ·2026-11-15 ·Shiji Zuo, Qihang Lin, Wei Liu, Yangyang Xu
theory similarity 0.56
Abstract

We consider regularized nonconvex optimization over a closed convex set $\mathcal{X} \subseteq \mathbb{R}^d$, which involves a convex nonsmooth regularizer $r$ in the objective and $n$ smooth equality constraints. To find a near-KKT point of the problem, we propose a prox-linear first-order method, named as SEPi-Prox, based on a smoothed exact-penalty formulation. At each outer iteration of SEPi-Prox, a strongly convex prox-linear subproblem is formed and solved inexactly to a sufficient accuracy. Under a local non-singularity condition that is weaker than some other commonly used regularity conditions or constraint qualifications, SEPi-Prox finds an $ε$-KKT point within $O(ε^{-2})$ first-order oracle calls, matching the lower bound for unconstrained smooth nonconvex optimization. We employ Nesterov's accelerated proximal-gradient method (APG) and a dual cutting-plane method (DCP) for the prox-linear subproblems. Let $\mathcal{C}_{\mathrm{FO}}$ and $\mathcal{C}_r$ denote the numbers of arithmetic operations of evaluating one first-order oracle and one proximal mapping of $r$ over $\mathcal{X}$, respectively. Then SEPi-Prox requires $\widetilde{O}(\mathcal{C}_{\mathrm{FO}}ε^{-2} + (dn + \mathcal{C}_r)ε^{-5/2})$ and $\widetilde{O}((\mathcal{C}_{\mathrm{FO}} + dn^2 + n\mathcal{C}_r + n^{3.373})ε^{-2})$ total arithmetic operations to produce an $ε$-KKT point, respectively by using APG and DCP as the subsolver. The latter result is preferable when $n$ is small and nearly matches the optimal dependence on $ε$. Numerical experiments on instances of the multi-class Neyman-Pearson classification demonstrate the efficiency of SEPi-Prox with both subsolvers.

0.04

Safe Newton-Based Extremum Seeking for Static Maps with Delayed Output Measurements

eess.SY ·2026-09-14 ·Azad Ghaffari, Tiago Roux Oliveira
similarity 0.56
Abstract

This work presents a delayed safe Newton-based extremum seeking (SANES) framework for minimizing an unknown static map subject to an unknown safety constraint. The objective and safety measurements are assumed to be affected by the same constant time delay. To compensate for delayed measurements, a model-free predictor is developed to construct the quantities required for optimization, including the nominal Newton-based extremum-seeking control input and the gradient information used to formulate control Lyapunov function (CLF) and control barrier function (CBF) conditions. Robust CLF--CBF quadratic programs (QPs), subject to parameter-update constraints, are then formulated to account explicitly for derivative-estimation and prediction errors. For the delay-free case, robustness margins are derived from bounds on the extremum-seeking estimation errors, whereas for the delayed case, the margins incorporate both estimation and prediction errors. Practical stability of the nominal Newton-based extremum-seeking dynamics is established directly through a Lyapunov analysis, thereby providing convergence guarantees over the admissible parameter set without relying exclusively on local averaging arguments. Robust CLF and CBF conditions are subsequently derived to establish practical convergence and forward invariance of a robust subset of the prescribed safe set. A numerical case study demonstrates the effectiveness of the proposed SANES framework in achieving constrained optimization despite unknown objective and safety maps and delayed measurements.

0.44

A Local-Linearly Convergent Algorithm for Nonconvex Equality-Constrained Optimization

cs.LG math.OCstat.ML ·2026-08-12 ·Frank E. Curtis, Lingjun Guo, Daniel P. Robinson
similarity 0.56
Abstract

For solving nonconvex equality-constrained optimization problems, a recent Gradient-Eigenstep Algorithm by Goyens et al.~is an iteration-efficient approach, based on minimizing Fletcher's augmented Lagrangian function, for finding an approximate second-order stationary point from an arbitrary starting point. In this paper, the analysis of this algorithm is extended, offering a two-fold contribution. First, it is shown that a local-linear rate of convergence can be obtained by this method if it is initiated sufficiently close to a strong second-order stationary point and employs a sufficiently small step-size parameter and sufficiently large penalty parameter. In this case, the algorithm reduces to a gradient descent algorithm applied to minimize Fletcher's augmented Lagrangian. Second, as a particularly useful application of the first result, it is shown that the Gradient-Eigenstep algorithm can be used as an iteration-efficient subproblem solver in the context of a progressive sampling strategy for solving equality-constrained optimization problems when the objective and constraint functions are defined by large sample averages, ultimately offering an algorithm with an improved worst-case sample complexity when compared to an approach that solves a full-sample problem directly.

0.04

Distributed Optimization Involving Non-Lipschitzian, Linear Composite Functions: Unified Algorithm Design and Convergence Analysis

cs.GT ·2026-11-05 ·Liang Ran, Huaqing Li, Lifeng Zheng, Jun Li, Run Tang
similarity 0.56
Abstract · 15 pages; 3 figures

This paper formulates a generic distributed nonsmooth optimization problem subject to edge-based equality constraints, local inequality constraints characterized by general convex functions, as well as globally coupling constraints including affine equality and nonlinear inequality. The terminal goal is to minimize the sum of local non-differentiable convex functions and proximable nonsmooth convex composite terms attached to linear operators, over networked multi-agent systems. This formulated framework not only applies in complex practical problems but also reduces to existing distributed optimization structures. To cope with technical issues caused by local linear mappings, potential multiple nonsmooth functions and coupling nonlinear constraints, we first derive a new local first-order condition, and further present a novel fully distributed optimization algorithm based on variable decompositions and proximal operators. Specifically, the proposed algorithm allows each agent to update states via local information exchange and through {locally fixed step-sizes}, and to find inexact solutions of subproblems without loss of convergence speed. In contrast to existing tailor-made algorithms for the bounded gradients of objective functions or affine constraints, our algorithm is capable of solving more practical cases involving general convex functions and nonlinear constraints. Under mild assumptions, the convergence rate $o(1/k)$ in terms of the optimality and successive-difference metrics is achieved, by constructing the equivalent variational inequalities of the proposed algorithm. Finally, numerical experiments on practical application problems illustrate the theoretical results.

0.26