CS Paper Radar

Extremum Seeking Control: Three Revolutions and the Road Ahead

Tiago Roux Oliveira

eess.SY math.OC ·submitted 2026-08-31 23:31 UTC · v1
position

Abstract

The history of extremum seeking is not merely the history of an algorithm; it is the history of an idea. Few ideas in control engineering have demonstrated the remarkable longevity of extremum seeking control (ESC). Invented more than one hundred years ago, ESC has continually reinvented itself while remaining faithful to its original objective: enabling systems to optimize their performance without relying on accurate mathematical models. This perspective article proposes that the evolution of ESC is best understood through three scientific revolutions. The first Engineering Revolution (1922-1999) established the engineering principles of model-free optimization; the second Mathematical Revolution (2000-2010) provided the rigorous mathematical foundations that transformed ESC into a mature discipline of nonlinear control; and the third ongoing Infinite-Dimensional and Cyber-Physical Revolution (2010-present) continues to expand its scope toward delays, partial differential equations, distributed optimization, event-triggered implementations, and increasingly complex cyber-physical systems. Beyond recounting this historical evolution, we offer a personal perspective on why ESC has remained relevant across successive technological eras. We argue that its enduring influence arises because the fundamental engineering challenge has never changed: "how can a dynamical system learn to improve its own performance when the optimum is unknown?" As optimization, learning, and feedback control become increasingly intertwined, ESC appears uniquely positioned to contribute to a new generation of intelligent autonomous systems, pointing toward what may become the field's fourth scientific revolution.

Comment
Preprint submitted for consideration in The Journal of The Franklin Institute Bicentennial Invited Papers (36 pages, 16 figures)

Closest papers

Nearest neighbours by meaning across the whole index.

Towards Co-Designed Event-Triggered Extremum Seeking

eess.SY math.OC ·2026-08-10 ·Roberto Luo, Pedro Henrique Silva Coutinho, Victor Hugo Pereira Rodrigues, Tiago Roux Oliveira, Miroslav Krstic
similarity 0.63
Abstract · 7 pages, 3 figures

This paper studies event-triggered gradient-based multivariable extremum seeking for nonlinear maps with polytopic Hessian uncertainty. Unlike existing event-triggered extremum-seeking methods, which first fix the controller (typically diagonal) and then design the triggering mechanism, the proposed approach jointly synthesizes the controller and the triggering mechanism through a co-design framework that admits both diagonal and full controller gain matrices. The co-design problem is formulated as a convex optimization problem with linear matrix inequality constraints. Its solution guarantees exponential convergence of the average closed-loop system with a prescribed decay rate while maximizing the admissible triggering threshold to reduce communication. Lyapunov and averaging analyses establish exponential stability of the event-triggered system, and Zeno-freeness is proved to guarantee implementability. Numerical results illustrate that diagonal gains cannot achieve the same triggering thresholds and decay rates as the full controller gain matrices, highlighting the benefits of exploiting Hessian coupling information.

0.07

Safe Newton-Based Extremum Seeking for Static Maps with Delayed Output Measurements

eess.SY ·2026-09-14 ·Azad Ghaffari, Tiago Roux Oliveira
similarity 0.59
Abstract

This work presents a delayed safe Newton-based extremum seeking (SANES) framework for minimizing an unknown static map subject to an unknown safety constraint. The objective and safety measurements are assumed to be affected by the same constant time delay. To compensate for delayed measurements, a model-free predictor is developed to construct the quantities required for optimization, including the nominal Newton-based extremum-seeking control input and the gradient information used to formulate control Lyapunov function (CLF) and control barrier function (CBF) conditions. Robust CLF--CBF quadratic programs (QPs), subject to parameter-update constraints, are then formulated to account explicitly for derivative-estimation and prediction errors. For the delay-free case, robustness margins are derived from bounds on the extremum-seeking estimation errors, whereas for the delayed case, the margins incorporate both estimation and prediction errors. Practical stability of the nominal Newton-based extremum-seeking dynamics is established directly through a Lyapunov analysis, thereby providing convergence guarantees over the admissible parameter set without relying exclusively on local averaging arguments. Robust CLF and CBF conditions are subsequently derived to establish practical convergence and forward invariance of a robust subset of the prescribed safe set. A numerical case study demonstrates the effectiveness of the proposed SANES framework in achieving constrained optimization despite unknown objective and safety maps and delayed measurements.

0.45

MOON-RL: Multi-Objective Optimization via Objective Space Navigation with Reinforcement Learning

cs.LG ·2026-11-11 ·Andrii Artomov, Soheyl Massoudi, Jürg Schiffmann
similarity 0.58
Abstract · 66 pages, including appendices. Code, data and supplementary video links are provided in the manuscript

Many engineering design problems involve multiple conflicting objectives and strict feasibility constraints. They are typically solved with population-based optimizers that require many evaluations and offer limited online control over where the search is focused. We present MOON-RL, a reinforcement-learning-based optimization framework that formulates constrained multi-objective search as preference-directed navigation in the objective space. In MOON-RL, a lightweight PPO policy receives objective-space and constraint-related observations and acts as an instance-specific adaptive search process. It learns how to update the design so that the current solution advances along a user-specified trade-off direction, while a simple meta-algorithm periodically resets the policy, resamples starting designs from a continuously updated archive of non-dominated solutions, and adjusts step-size and penalty schedules. We evaluate MOON-RL on real-world actuator-design problems and additional convergence-hard constrained benchmarks, comparing against established evolutionary algorithms under matched evaluation budgets. In focused search scenarios, MOON-RL achieves competitive performance in the region of interest, and ablation studies show that both the RL-driven adaptive sampling process and the meta-algorithm contribute to this behaviour, suggesting that repurposing RL training as objective-space navigation is a viable building block for interactive, preference-guided design tools. Additional experiments show that the framework also accommodates adaptive non-RL samplers, while PPO remains a strong choice for fixed-target, preference-directed objective-space navigation.

0.26

First-Order Optimization as Minimum-Time Control

eess.SY math.OC ·2026-08-14 ·Liraz Mudrik, Isaac Kaminer, Pramod P. Khargonekar
benchmark similarity 0.58
Abstract · 13 pages, 4 figures

We formulate first-order optimization as a minimum-time control problem. The iterate is the state, the update, a combination of the gradients observed so far, is the control, and the points with gradient norm at most a prescribed tolerance form the target set. For a fixed objective and start, the minimum number of oracle queries needed to reach the target is a value function: it measures the complexity of the instance, not the worst case over a class. On a strongly convex quadratic, the conjugate gradient iterates emerge from the discrete Pontryagin conditions, and the value is a controllability index. Beyond the quadratic, a Hessian-generated reachable span replaces the controllability matrix, and feasibility becomes a reachability question: whether a critical point can be reached at all, and in how many steps, is read from the span. Curvature is a resource: a checkable condition certifies that an instance needs fewer steps than the controllability index of its quadratic model at the minimizer, and the gap can grow without bound with the dimension. The value is thus a benchmark for the intrinsic difficulty of each instance, against which any first-order method can be measured.

0.10

A Decade of Bayesian Optimization for Controller Tuning and Robot Learning: Tutorial, Review, and Future Prospects

cs.RO eess.SY ·2026-09-08 ·David Stenger, Paul Brunzema, Johanna Menn, Alexander von Rohr, Angela P. Schoellig +1 more
surveybenchmark similarity 0.58
Abstract · Currently under review

In the past decade, Bayesian optimization (BO) has emerged as a powerful and adaptable framework for automatic controller tuning and robot learning. This article offers a comprehensive overview of the state-of-the-art in BO, designed to support both researchers and practitioners in understanding recent advancements, practical applications, and future research directions. We begin by adopting a practitioner's perspective, illustrating how to effectively set up BO through a representative controller tuning example. We position BO within the broader context of learning paradigms, ranging from deep reinforcement learning to data-driven control, and highlight scenarios where BO is most advantageous. Next, we discuss the diverse range of BO methods that have been developed to tackle complex problems and specific applications. This article provides a unified perspective on the current landscape of BO, emphasizing its relevance to control systems and robotics, and it highlights future prospects by identifying key research challenges and promising avenues for advancing BO in the field. This includes addressing a significant gap in the BO landscape: the lack of standardized benchmark problems specifically for control-related applications. To foster future research and ensure rigorous evaluation, we start an effort towards a lightweight benchmark suite for control engineering and robotics. We also present metrics and best practices to facilitate direct comparisons between new BO algorithms and established state-of-the-art methods.

0.43

Resilient Extremum Seeking Control for Cyber-Physical Systems Under Denial-of-Service Attacks

eess.SY math.OC ·2026-09-12 ·Filipe da Silva Bastos Teixeira, Pedro Henrique Silva Coutinho, Tiago Roux Oliveira, Miroslav Krstic
similarity 0.58
Abstract · Preprint submitted to IEEE Transactions on Control of Network Systems (12 pages, 4 figures)

Extremum seeking control (ESC) relies on deliberately injected excitation to extract optimization information from measured outputs, making its networked implementation particularly vulnerable to denial-of-service (DoS) attacks. This paper develops a resilient discrete-time ESC architecture for cyber-physical systems subject to DoS attacks. By holding the most recently successfully transmitted signals during DoS intervals, the proposed mechanism yields averaged error dynamics with an exponentially contracting mode under successful communication and a neutral mode under attack. For deterministic DoS attacks, practical exponential convergence to the extremum is established under an average bound on the attack duration, with an explicit convergence rate depending on the fraction of time under attack. For probabilistic DoS attacks, almost-sure and in-probability convergence properties are established via stochastic averaging, with the attack success probability explicitly entering the convergence rate. A key finding is that a seemingly natural zero-input strategy can fundamentally compromise ESC: attacks synchronized with the dither generate an $\mathcal{O}(1/a)$ bias in the averaged dynamics, where $a$ is the dither amplitude, and can displace the equilibrium from the true optimizer. Thus, under the proposed hold-based architecture, increasingly severe DoS attacks primarily slow the optimization process rather than destroy its stability, provided that communication is not permanently blocked. Numerical simulations illustrate the theoretical guarantees and the failure mechanism induced by dither-synchronized attacks.

0.54

Dual Control: On Exploration-Exploitation in Linear Systems

eess.SY math.OC ·2026-08-20 ·Tomas J. Meijer, Anders Rantzer
published similarity 0.57
Abstract · To be published in Annual Review of Control, Robotics, and Autonomous Systems Vol. 10 (2027)

The term "dual control" refers to the dual objective of simultaneously balancing exploration and exploitation. Problems of this kind have been studied for nearly a century. This paper is devoted to theory and methodology relevant for optimal control of linear time-invariant systems whose parameters are initially unknown and must be learned by active probing. We review the main ideas underlying four major research directions: Multi-armed bandits, self-tuning regulators, regret rate minimizing controllers, and minimax optimal dual controllers. The first three have a long history and rich literature, whereas the fourth provides a promising framework for robust dual control.

0.34

Model-Free Adaptive Parameter Tuning for Efficient Multi-Robot Warehouse Operations

cs.RO eess.SY ·2026-08-21 ·Pratap Tokekar, Mouhacine Benosman, Rahul Chandan, Alexandre Ormiga Galvao Barbosa, Michael Caldara +1 more
similarity 0.56
Abstract

Robotic Fulfillment Centers (FCs) store inventory on shelves (pods) arranged in dense blocks. Retrieving a target pod that is buried deep in a block requires moving obstructing pods out of the way (i.e., digout). Multi-robot planners use parameterized cost functions to control digout behavior, producing a spectrum of strategies: at one extreme, obstructing pods are sent to other blocks (using more robots in travel lanes); at the other, pods are shuffled within the block (avoiding lane congestion but increasing extraction time). Each point on this spectrum has different downstream consequences for floor congestion and throughput. The optimal operating point depends on the specific facility configuration and shifts with operational conditions such as varying station demand and congestion patterns, making offline tuning impractical. We present an adaptive parameter tuning framework based on Extremum Seeking Control (ESC) that continuously adjusts planner parameters in response to measured throughput. ESC performs model-free optimization by perturbing parameters with sinusoidal dither signals and correlating perturbations with performance changes to estimate gradients, making it robust to the multi-minute delayed effects and credit assignment challenges inherent in large FC operations. Simulation studies demonstrate that the adaptive policy improves upon fixed policies across several conditions. We observe an improvement in throughput by an average of 5.0% across map and robot fleet size variations, and by 8.4% under dynamic operating conditions. This work eliminates manual parameter provisioning and enables real-time adaptation, providing a self-tuning paradigm for FC storage operations.

0.30