CS Paper Radar

Resilient Extremum Seeking Control for Cyber-Physical Systems Under Denial-of-Service Attacks

Filipe da Silva Bastos Teixeira, Pedro Henrique Silva Coutinho, Tiago Roux Oliveira, Miroslav Krstic

eess.SY math.OC ·submitted 2026-09-12 23:45 UTC · v1

Abstract

Extremum seeking control (ESC) relies on deliberately injected excitation to extract optimization information from measured outputs, making its networked implementation particularly vulnerable to denial-of-service (DoS) attacks. This paper develops a resilient discrete-time ESC architecture for cyber-physical systems subject to DoS attacks. By holding the most recently successfully transmitted signals during DoS intervals, the proposed mechanism yields averaged error dynamics with an exponentially contracting mode under successful communication and a neutral mode under attack. For deterministic DoS attacks, practical exponential convergence to the extremum is established under an average bound on the attack duration, with an explicit convergence rate depending on the fraction of time under attack. For probabilistic DoS attacks, almost-sure and in-probability convergence properties are established via stochastic averaging, with the attack success probability explicitly entering the convergence rate. A key finding is that a seemingly natural zero-input strategy can fundamentally compromise ESC: attacks synchronized with the dither generate an $\mathcal{O}(1/a)$ bias in the averaged dynamics, where $a$ is the dither amplitude, and can displace the equilibrium from the true optimizer. Thus, under the proposed hold-based architecture, increasingly severe DoS attacks primarily slow the optimization process rather than destroy its stability, provided that communication is not permanently blocked. Numerical simulations illustrate the theoretical guarantees and the failure mechanism induced by dither-synchronized attacks.

Comment
Preprint submitted to IEEE Transactions on Control of Network Systems (12 pages, 4 figures)

Closest papers

Nearest neighbours by meaning across the whole index.

Inverting Self-Triggered Control: Adversarial Reinforcement Learning for Sparse Denial-of-Service Attacks

cs.LG eess.SY ·2026-09-10 ·Adam Haroon, Erick J. Rodríguez-Seda, Tristan Schuler, Cody Fleming
theory similarity 0.63
Abstract · 8 pages, 3 figures

Self-triggered reinforcement learning control (RL-STC) learns the sparsest control schedule that preserves Lyapunov-decreasing stability under a Run-Time Assurance (RTA) override. We invert this: an adversarial RL agent learns the sparsest jamming or Denial-of-Service (DoS) schedule that destabilizes the closed loop, with a Lyapunov-increase admissibility predicate mirroring the defender's safety certificate. We prove a plant-property lower bound on the minimum jam count required for an immediate hold-last medium-access-control adversary to force a crash against a self-triggered controller (STC) satisfying a Lyapunov contract, and recover a certificate-level analog of the consecutive-grouping optimality of prior count-budget DoS scheduling as a corollary. This extends the DoS-scheduling count-budget analysis from periodic and linear-time-invariant to STC controllers. Empirically, we train against four fixed defenders per plant (one Linear Quadratic Regulator (LQR) and three RL-STC) on Pendulum, CartPole, and Quadrotor2D. The learned adversary is the only adversary that crashes every defender on every plant at $100\%$: greedy misses Quadrotor2D LQR on $42\%$ of episodes and periodic misses Pendulum LQR on $97\%$. On jam-time-per-failure it beats baselines by up to $2.8\times$, and shows its widest absolute margin on Quadrotor2D LQR. Robustness ablations show that Gaussian observation noise exceeding the initial-state magnitude and position-only observation both preserve $100\%$ failure rate and keep the learned adversary strictly ahead of both baselines on jam-time-per-failure.

0.36

Approximate Dynamic Programming for Link Reconfiguration to Reduce Malicious Input Controllability in Consensus Networks

eess.SY ·2026-10-29 ·Nathaniel Mailhot, Mohammad Pirani, Davide Spinello
similarity 0.61
Abstract

Communication links influence how strongly a malicious input at one sensor affects a consensus network. We study how a defender can reduce this influence by selecting one additional link after observing the currently attacked sensor. The attacked sensor changes over time, and vulnerability is measured by the trace of a finite-horizon controllability Gramian. A prescribed local epsilon-greedy attacker response rule defines a discounted Markov decision process for the defender. Exact dynamic programming (DP) provides an optimal policy using the full Gramian payoff matrix and attacker transition model. We then introduce approximate dynamic programming (ADP) that learns from sampled attacker transitions and empirical Gramian costs estimated from measured impulse responses. Under conditionally unbiased costs and standard sampling and learning-rate conditions, the learned action-value table converges to the exact DP solution. The analysis uses Bellman contraction and bounds the effect of persistent Gramian estimation bias, establishing when measured responses recover optimal defender decisions without constructing the full payoff matrix. In simulations on 20- and 100-node graphs, empirical ADP rollout costs are within 0.23% and 1.76%, respectively, of exact DP. In a two-dimensional intruder tracking example, both policies reduce network-wide root-mean-square tracking error spread by approximately 25% relative to an unmitigated attack.

0.18

Cross-Domain Joint DDoS Detection in Multi-Controller SDN via Confidence-Based Entropy Fusion

cs.CR ·2026-08-18 ·Zhaoyang Zhang, Shen Wang, Xiaofeng Tao
similarity 0.61
Abstract

In multi-controller Software-Defined Networking (SDN), Distributed Denial-of-Service (DDoS) attacks exhibit a "dispersed source, concentrated target" pattern across domains, i.e., attack traffic originates from multiple edge-controller domains but converges on a victim in a single aggregation controller domain. While entropy-based DDoS detectors are effective in single-controller settings, their direct application in multi-controller SDN reveals a previously overlooked anomaly. Through systematic experiments, we identify an aggregation bias: during the post-attack transition phase, the aggregation controller continues to generate excessive false positives, while edge controllers have already returned to normal. We attribute this phenomenon to the coupled effects of OpenFlow statistics lag and unconstrained dynamic-threshold drift. To address this issue, we propose a cross-domain confidence-fusion framework that leverages lightweight edge-side messages to calibrate aggregation-controller decisions without sharing raw traffic data. The framework is non-intrusive, communication-efficient, and incrementally deployable. Experiments on a three-controller linear Mininet testbed with 24 hosts over 10 runs show that the method preserves edge-controller performance while reducing the aggregation false positive rate from 8.87% to 1.96% and increasing the F1 score from 89.04% to 96.89%.

0.24

Extremum Seeking Control: Three Revolutions and the Road Ahead

eess.SY math.OC ·2026-08-31 ·Tiago Roux Oliveira
position similarity 0.58
Abstract · Preprint submitted for consideration in The Journal of The Franklin Institute Bicentennial Invited Papers (36 pages, 16 figures)

The history of extremum seeking is not merely the history of an algorithm; it is the history of an idea. Few ideas in control engineering have demonstrated the remarkable longevity of extremum seeking control (ESC). Invented more than one hundred years ago, ESC has continually reinvented itself while remaining faithful to its original objective: enabling systems to optimize their performance without relying on accurate mathematical models. This perspective article proposes that the evolution of ESC is best understood through three scientific revolutions. The first Engineering Revolution (1922-1999) established the engineering principles of model-free optimization; the second Mathematical Revolution (2000-2010) provided the rigorous mathematical foundations that transformed ESC into a mature discipline of nonlinear control; and the third ongoing Infinite-Dimensional and Cyber-Physical Revolution (2010-present) continues to expand its scope toward delays, partial differential equations, distributed optimization, event-triggered implementations, and increasingly complex cyber-physical systems. Beyond recounting this historical evolution, we offer a personal perspective on why ESC has remained relevant across successive technological eras. We argue that its enduring influence arises because the fundamental engineering challenge has never changed: "how can a dynamical system learn to improve its own performance when the optimum is unknown?" As optimization, learning, and feedback control become increasingly intertwined, ESC appears uniquely positioned to contribute to a new generation of intelligent autonomous systems, pointing toward what may become the field's fourth scientific revolution.

0.18

FoSeRL: Formal Sequential Robustness Certification for Reinforcement Learning Policies

cs.LG ·2026-10-03 ·Sara Taheri, Deep Kumar Ganguly, Jan Křetínský, Majid Zamani
similarity 0.57
Abstract

Even a few action perturbations can substantially degrade the performance of a deployed decision policy. Certifying the resulting return loss is challenging in stochastic environments, where returns vary even without an attack. We introduce FoSeRL, a framework for certifying deployed RL policies against precommitted, temporally sparse action attacks. The deployed policy is unchanged, with no smoothing or retraining. Certification requires a resettable simulator supporting shared randomness and independent one-step successor queries, but no analytical dynamics model. FoSeRL certifies that an attacked episode loses no more than a prescribed amount of return relative to the same episode unattacked, with at least a target probability and at a user-specified confidence level. Both runs share the initial state and randomness, so the measured loss reflects the attack, not the episode; carrying the running return gap as a state coordinate makes it the terminal value, reducing trajectory-level certification to terminal safety. Time-dependent barrier conditions on the augmented state bound the terminal failure probability: satisfied exactly, they certify every admissible precommitted attack; learned from sampled trajectories and verified on held-out data, they certify the same guarantee under a specified attack-episode setting. Across six stochastic continuous-control environments and three RL policy families (TD3, SAC, and PPO), FoSeRL certifies non-trivial cardinality--magnitude robustness frontiers, achieves substantially larger certified budgets than policy smoothing, and reveals marked robustness differences among policies with comparable nominal performance.

0.20

Tractable Defense against Advanced Persistent Threats in Networked Settings

cs.CR ·2026-09-14 ·Brandon Collins, Keith Paarporn, Shouhuai Xu, Philip N. Brown
publishedtheory similarity 0.57
Abstract · Includes proofs, to appear in CDC 2026

Recently, the theory of Boolean Dynamical Systems was proposed to study the decision theory surrounding the defense of computer networks against Advanced Persistent Threats (APTs). Boolean Dynamical Systems naturally capture four first principle primitives of APTs: the stealthy nature of attacks, limited and noisy information from automated systems like intrusion detection systems, lateral movement after the attacker penetrates into the network, and the defender's ability to secure a subset of computers at any time at the loss of resources such as system uptime. Currently, doing optimal/heuristic control in a computationally tractable manner is not possible because the emergent value function is computationally intractable (with respect to the network size). To resolve this, we propose a mean-field analysis inspired heuristic value function. We prove that our proposed heuristic is based on an exact computation of the value function under the assumption that the underlying state estimate distribution maximizes entropy. We numerically evaluate the quality of our heuristic as parameterized by the degree to which the entropy assumptions are violated.

0.36

Dynamical low-rank equilibrium computation for stochastic games between advanced persistent threats and moving target defense

cs.GT cs.AIcs.CReess.SY ·2026-09-20 ·Tian Zijian, Zhang He, Chen Xinjie, Wang Wenhai, Liu Xinggao
theorycode/data similarity 0.57
Abstract · Regular Paper, under review at Automatica (submission 26-2265). v2: corrected the utility-loss figure in Experiment 5 (2.3%). This preprint …

Moving target defense (MTD) against advanced persistent threats (APTs) in industrial control systems (ICS) has well-established game-theoretic formulations, but their practical value hinges on equilibrium computation, which faces two gaps: full-rank value iteration is prohibitively expensive at industrial scale, and the resulting defense strategies admit no certified robustness against adversarial perturbations. We first reveal that the attack and defense influence matrices of ICS dynamics are intrinsically low-rank: APTs infiltrate through a handful of entry points and MTD reconfigures only a few components per cycle. Our theory makes four contributions. First, an augmented gradient matrix certifies that the low-rank structure propagates through the non-smooth Bellman operator of the zero-sum stochastic game, so that every Bellman target lies near a low-dimensional subspace and low-rank truncation incurs an explicit error bound (Lemma 1, Theorem 1). Second, we propose the Dynamical Low-Rank Nash Equilibrium algorithm, named DLR-NE, which augments the rank-r search space each iteration, regularizes the core matrix spectrum, and retracts via truncated SVD, and prove that it converges geometrically to a neighborhood whose error decomposes into five physically interpretable sources (Theorem 2). Third, its per-step cost is O(nr^2), a Theta(n/r^2) speedup over full-rank value iteration (Theorem 3). Fourth, a single weight trades accuracy against a certified sensitivity bound of the induced defense strategy under core-matrix perturbations (Corollary 1). Six experiments on a nonlinear power-system testbed confirm each prediction, with 94% parameter compression at 2.3% utility loss. All experimental data and code are publicly available.

0.29

Security Games on Series-Parallel Attack Graphs with Adaptive Attackers

cs.GT ·2026-08-21 ·Russell Kai Min Tan, Hui Han Chin, Chun Kai Ling
similarity 0.57
Abstract · 68 pages, 10 figures. Added examples and updated acknowledgements

We study security games on attack graphs, where an adaptive attacker seeks to reach a target by sequentially attempting stochastic controls along the current attack frontier, while a defender allocates limited resources across controls to delay compromise. The attacker may choose among exponentially many attack routes and freely pivot between them as successes and failures are observed, yielding an exponentially large space of contingent attack policies. For any fixed defender allocation, we show that an optimal attacker policy on a two-terminal series-parallel attack graph is an index policy: at each step, the attacker selects an available control with the largest value of an extension of the classical Gittins index. The indices and the resulting attacker best response can be computed in polynomial time, without explicitly enumerating attack paths or contingent policies. To the best of our knowledge, this is the first optimal index characterization for adaptive attackers in security games on general series-parallel attack graphs. We further develop efficient algorithms for computing the attacker's exact utility and an exact defender subgradient, enabling deterministic first-order optimization of defensive resource allocations without sampling attack trajectories. Our framework strictly generalizes prior approaches restricted to parallel chains and out-trees, while exploiting the compositional structure of series-parallel graphs to support interpretable attacker policies and parallel computation across independent subgraphs. Experiments demonstrate that the resulting methods scale substantially better than naive explicit-state approaches while producing effective defensive allocations.

0.47