CS Paper Radar

Inequality-Constrained Optimization via Sliding Mode Control

Rawan Hoteit, Veronica Centorrino, Efe C. Balta, John Lygeros

eess.SY math.OC ·submitted 2026-11-15 00:27 UTC · v1
theory

Abstract

We present COSMIC (Constrained Optimization via Sliding Mode for Inequality Constraints), a continuous-time solver for inequality-constrained optimization. COSMIC couples primal gradient-flow dynamics with multiplier dynamics derived from sliding mode control laws on a manifold defined by the smoothed Fischer-Burmeister function. We prove that COSMIC's trajectories reach this manifold in finite time from any primal initial point, after which the dynamics reduces to the gradient flow of a logarithmic barrier function. This yields a control-theoretic interpretation of barrier methods while allowing infeasible initialization. For strongly convex problems, we further establish exponential convergence to the minimizer. To address the numerical stiffness that arises in discretization, we develop a modified discrete-time algorithm. We validate COSMIC across various machine learning applications, demonstrating its ability to solve optimization problems accurately while scaling effectively to high-dimensional settings.

Closest papers

Nearest neighbours by meaning across the whole index.

Reciprocal-Manifold Annealed KKT Flows for Constrained Optimization: Application to the Nonconvex AC Optimal Power Flow

eess.SY math.OC ·2026-08-30 ·M Parimi, Aditi Ramteke, Rachit Mehra, Arun Mahindrakar, Navdeep Singh
benchmark similarity 0.59
Abstract

Safety-critical optimization applications, such as real-time power system operation, maintain feasibility at every intermediate step, not merely at convergence. Existing approaches either violate constraints mid-solve (interior-point methods) or enforce feasibility through per-instant quadratic programming subproblems with cubic computational cost and unbounded worst-case execution time. We propose a continuous-time optimization framework for smooth constrained nonlinear problems that preserves feasibility throughout the optimization process without requiring projection operators, quadratic programming subproblems, or other per-iteration optimization routines. The method is built around a reciprocal multiplier manifold, which establishes an explicit relationship between inequality constraints and their associated Lagrange multipliers. By designing a continuous multiplier update law, the manifold is shown to remain forward invariant, while the resulting dynamics are equivalent to continuous-time logarithmic barrier gradient descent. The proposed framework naturally extends to multiple inequality constraints, equality constraints, nonconvex feasible sets, and infeasible initial conditions. The method is further enhanced through an augmented Uzawa flow that eliminates oscillatory transients commonly observed in classical primal-dual saddle-point dynamics. The effectiveness of the proposed approach is applied to the AC Optimal Power Flow problem of IEEE 9-bus and IEEE 57-bus systems. Numerical results show convergence to solutions within 0.4\% of the benchmark optimum while maintaining strict feasibility of all constraints. A computational complexity analysis shows that the proposed dynamics reduce the per-step computational cost from cubic to linear complexity. Finally, dynamic tracking studies under time-varying operating conditions demonstrate reliable feasibility preservation.

0.24

Fischer-Burmeister-based Multiplier Dynamics in Inequality-Constrained Optimization

eess.SY math.OC ·2026-11-02 ·Kang Liu, Jianchen Hu
similarity 0.57
Abstract

We study proportional-integral multiplier feedback for inequality-constrained optimization using the Fischer-Burmeister complementarity residual. We construct explicit dynamics and characterize gain-dependent stability and convergence rates. For a single active constraint, we derive exact local stability and decay-rate conditions, including a tradeoff between continuous-time convergence and discretization costs. For strongly convex objectives with affine constraints, we establish global convergence and characterize geometric conditions for uniform exponential stability. Numerical experiments compare feedback structures and computational costs.

0.33

Controlled-Minimax: Stabilizing Adversarial Optimization via Control Theory

cs.LG ·2026-10-09 ·Tianjiao Luo, Jianfei Chen, Jun Zhu
similarity 0.55
Abstract

Minimax optimization is fundamental to adversarial learning yet suffers from severe stability challenges, often manifesting as persistent limit cycles or divergence. This paper investigates these instabilities through the lens of continuous-time dynamical systems. By modeling training dynamics as a second-order Ordinary Differential Equation (ODE), we rigorously identify the root cause of non-convergence: the problematic eigenstructure of the loss function's Hessian near equilibrium points. To address this, we introduce Controlled-Minimax, a novel stabilization framework that reformulates the optimization trajectory as a controllable physical system. Unlike heuristic regularization techniques, our approach derives a specific feedback controller that provides theoretical guarantees of local exponential stability, regardless of the underlying Hessian spectrum. Consequently, our method stabilizes general minimax problems where traditional gradient descent-ascent methods fail. Extensive experiments on pathological benchmarks including the Dirac-GAN, 2D Gaussian mixtures, and Swiss Roll datasets, as well as high-resolution image synthesis with StyleGAN2-ada, demonstrate that Controlled-Minimax effectively suppresses oscillatory behavior while ensuring fast and robust convergence.

0.26

Safe Output-Regulation Flows for Constrained Optimization via Control Barrier Functions

eess.SY ·2026-11-05 ·Zihao Ren, Bingqi Liu, Lei Wang, Hongye Su, Guodong Shi
similarity 0.55
Abstract

Control-theoretic approaches to constrained optimization have been widely studied by interpreting optimization algorithms as dynamical systems. This paper formulates constrained optimization as an output-regulation problem with state constraints, where equality-constraint residuals are treated as regulated outputs and inequality constraints as state constraints. Proportional--integral feedback is employed to drive the equality-constraint residuals asymptotically to zero, while a control barrier function (CBF) safety filter is used to ensure inequality safety throughout the optimization process. We establish feasibility of the CBF filtering program throughout the inequality-feasible set and local exponential convergence of the resulting flow. Under stronger global conditions, we further prove convergence from an entire connected component of the inequality-feasible set. Since exact implementation of the flow requires solving a CBF quadratic program at each state, we further develop a flow that approximates the CBF solution through multiplier updates. Moreover, we provide a unified formulation encompassing the two proposed flows and projected primal--dual dynamics. Numerical examples demonstrate convergence, safety, and computational tradeoffs of the proposed flows.

0.13

Generalization as a robust performance property of learning-enabled dynamical systems

eess.SY cs.LGmath.OC ·2026-08-31 ·Filippo Fabiani
similarity 0.55
Abstract

By focusing on algorithmic stability as a means of establishing out-of-sample bounds, we provide a system-theoretic interpretation of generalization in learning-enabled dynamical systems arising in data-driven optimization and feedback control approximation. Given two neighboring datasets, we specifically model sample replacement as an exogenous disturbance acting on a sensitivity system, while the incremental behavior of the data-dependent operator is encoded through an integral quadratic constraint. By relying on dissipativity arguments, we establish a matrix inequality-based certificate and a uniform stability bound that separates the one-sample sensitivity of the learned operator, and an algorithm-dependent dynamical gain. The latter can then be optimized, offering a tractable tool for certifying and comparing generalization capabilities of learning dynamics. We show that our results recover classical ones for gradient descent, apply naturally to momentum-based methods such as heavy-ball and Nesterov acceleration, and extend to data-driven control.

0.20

Follow the Regularized Leader Does Not Converge in Constrained Optimization

cs.GT math.OC ·2026-10-26 ·Ioannis Anagnostides, Ioannis Panageas, Nikolas Patris, Tuomas Sandholm
similarity 0.55
Abstract

Follow the regularized leader (FTRL) is a foundational algorithm in online learning whose regret properties have been extensively studied for decades. However, in stark contrast to mirror descent---and gradient descent in particular---the convergence of FTRL to first-order stationary points in constrained nonconvex optimization was not well understood. In this paper, we show that continuous-time FTRL can fail to converge even asymptotically in infinitely smooth constrained optimization problems: its Karush-Kuhn-Tucker (KKT) gap can remain bounded away from zero indefinitely. This is especially surprising in light of the fact that continuous-time FTRL guarantees a monotonic improvement of the objective value. As a result, we establish that non-convergent behavior of no-regret dynamics---which besets general variational inequality problems---can occur even in potential systems. From a technical standpoint, our construction relies on an infinitely differentiable function where the gradient flow dynamics exhibit decelerating oscillations without ever converging pointwise, which we in turn embed into higher-dimensional FTRL dynamics that sustain a perpetually large KKT gap. We complement our negative result by showing that FTRL with Euclidean regularization does converge when the objective function is analytic. This shows that the analytic property plays a more fundamental role for FTRL than for gradient flow: it governs not only pointwise convergence, but convergence to first-order stationarity itself.

0.19

AegisFlow: Training-Free Non-Myopic Path-Safe Guided Flow Matching

cs.AI ·2026-10-29 ·Kunpeng Liu, Boshi Zhang, Keyou You
code/data similarity 0.55
Abstract · 18 pages, 6 figures

Pretrained generative models, such as diffusion and flow matching models, have demonstrated exceptional capabilities in synthesizing high-quality data. However, steering their generative trajectories to satisfy strict constraints--such as identity preservation in image editing or obstacle avoidance in robotics--remains fundamentally challenging. While recent training-free guidance methods enforce path-safety, they rely heavily on min-norm controllers that only account for immediate constraint satisfaction. These myopic approaches often yield suboptimal trajectories that hover dangerously close to the boundary, severely restricting the model's ability to optimize toward its target. In this paper, we propose AegisFlow, a training-free, non-myopic path-safe guidance framework that recasts guided generation as a safety-critical optimal control problem with global scope. By integrating a time-varying barrier function directly into the global cost functional, AegisFlow naturally enforces constraints along the entire generative path. To compute the global optimal control, we derive a tractable, mathematically grounded approximation of the Hamilton-Jacobi-Bellman (HJB) equation. Comprehensive experiments demonstrate that AegisFlow outperforms state-of-the-art methods across text-guided image manipulation and robotic planning, establishing a significantly better trade-off between target alignment and constraint satisfaction. Code is available at https://github.com/LiuKunpeng03/AegisFlow.

0.25

Newton-Based Methods for Bilevel Optimization Beyond Lower-Level Strong Convexity

cs.LG math.OC ·2026-10-19 ·Youran Dong, Junfeng Yang, Wei Yao, Jin Zhang
similarity 0.55
Abstract

Bilevel optimization provides a framework for machine learning tasks such as hyperparameter optimization, data reweighting, and meta-learning. Existing Newton-based bilevel methods typically rely on the classical hypergradient formula, which is generally unavailable when lower-level solutions are nonunique and the lower-level Hessian is singular. We develop two Newton-based methods beyond lower-level strong convexity by exploiting curvature shared by the lower-level and multiplier updates. Newton-based GALET handles potentially singular lower-level curvature under a positive spectral-gap condition, exploiting its range structure to avoid the squared-Hessian multiplier system used in GALET. Newton-based BAMM uses a strongly convex aggregation model when the lower-level objective is convex and the upper-level objective is strongly convex in the lower-level variable. We establish convergence guarantees and improved complexity bounds relative to GALET and sl-BAMM under the corresponding assumptions. We further introduce GNBO, a generalized Newton-based framework that captures the shared-curvature structure of the proposed methods and several existing Newton and quasi-Newton bilevel methods. Experiments on synthetic problems, data hyper-cleaning, and GPT-2 data reweighting evaluate the proposed methods across different bilevel learning tasks.

0.07