CS Paper Radar

Inverting Self-Triggered Control: Adversarial Reinforcement Learning for Sparse Denial-of-Service Attacks

Adam Haroon, Erick J. Rodríguez-Seda, Tristan Schuler, Cody Fleming

cs.LG eess.SY ·submitted 2026-09-10 07:21 UTC · v1
theory

Abstract

Self-triggered reinforcement learning control (RL-STC) learns the sparsest control schedule that preserves Lyapunov-decreasing stability under a Run-Time Assurance (RTA) override. We invert this: an adversarial RL agent learns the sparsest jamming or Denial-of-Service (DoS) schedule that destabilizes the closed loop, with a Lyapunov-increase admissibility predicate mirroring the defender's safety certificate. We prove a plant-property lower bound on the minimum jam count required for an immediate hold-last medium-access-control adversary to force a crash against a self-triggered controller (STC) satisfying a Lyapunov contract, and recover a certificate-level analog of the consecutive-grouping optimality of prior count-budget DoS scheduling as a corollary. This extends the DoS-scheduling count-budget analysis from periodic and linear-time-invariant to STC controllers. Empirically, we train against four fixed defenders per plant (one Linear Quadratic Regulator (LQR) and three RL-STC) on Pendulum, CartPole, and Quadrotor2D. The learned adversary is the only adversary that crashes every defender on every plant at $100\%$: greedy misses Quadrotor2D LQR on $42\%$ of episodes and periodic misses Pendulum LQR on $97\%$. On jam-time-per-failure it beats baselines by up to $2.8\times$, and shows its widest absolute margin on Quadrotor2D LQR. Robustness ablations show that Gaussian observation noise exceeding the initial-state magnitude and position-only observation both preserve $100\%$ failure rate and keep the learned adversary strictly ahead of both baselines on jam-time-per-failure.

Comment
8 pages, 3 figures

Closest papers

Nearest neighbours by meaning across the whole index.

FoSeRL: Formal Sequential Robustness Certification for Reinforcement Learning Policies

cs.LG ·2026-10-03 ·Sara Taheri, Deep Kumar Ganguly, Jan Křetínský, Majid Zamani
similarity 0.66
Abstract

Even a few action perturbations can substantially degrade the performance of a deployed decision policy. Certifying the resulting return loss is challenging in stochastic environments, where returns vary even without an attack. We introduce FoSeRL, a framework for certifying deployed RL policies against precommitted, temporally sparse action attacks. The deployed policy is unchanged, with no smoothing or retraining. Certification requires a resettable simulator supporting shared randomness and independent one-step successor queries, but no analytical dynamics model. FoSeRL certifies that an attacked episode loses no more than a prescribed amount of return relative to the same episode unattacked, with at least a target probability and at a user-specified confidence level. Both runs share the initial state and randomness, so the measured loss reflects the attack, not the episode; carrying the running return gap as a state coordinate makes it the terminal value, reducing trajectory-level certification to terminal safety. Time-dependent barrier conditions on the augmented state bound the terminal failure probability: satisfied exactly, they certify every admissible precommitted attack; learned from sampled trajectories and verified on held-out data, they certify the same guarantee under a specified attack-episode setting. Across six stochastic continuous-control environments and three RL policy families (TD3, SAC, and PPO), FoSeRL certifies non-trivial cardinality--magnitude robustness frontiers, achieves substantially larger certified budgets than policy smoothing, and reveals marked robustness differences among policies with comparable nominal performance.

0.20

When to Intervene? State-Aware Sparse Manipulation in Federated Reinforcement Learning

cs.LG ·2026-10-08 ·Shutong Zheng, Sijia Chen
code/data similarity 0.64
Abstract · 27 pages, 12 figures

Federated reinforcement learning (FRL) enables distributed agents to collaboratively train decision-making policies, but its decentralized training process also exposes global policy learning to Byzantine manipulation. Existing poisoning attacks primarily focus on how to construct malicious updates, while trajectory-level intervention timing remains largely implicit. In sequential decision making, however, where an intervention is applied can alter subsequent trajectories and learning signals. Through controlled experiments, we find that changing the selected trajectory states materially alters attack efficacy even when the malicious-update construction is fixed. We therefore identify when as a distinct attack dimension and introduce the Viability-constrained Behavioral Steering Attack (V-BSA), which uses local policy uncertainty to select sparse intervention states and applies envelope-constrained behavioral steering. Across discrete-action benchmarks, V-BSA achieves substantial degradation against robust aggregators and ensemble defenses with only a fraction of the interventions used by dense poisoning, while revealing task- and aggregation-dependent boundaries. Overall, our results highlight intervention timing as a distinct dimension of sequential robustness in FRL. The code is available at https://github.com/Yodeesy/V-BSA

0.33

A Minimally Informed Black-Box Fake-Report Attack on DRL Traffic-Signal Control

cs.CR ·2026-10-28 ·Amro Saleh, Loukas Lazos, Mohammad J. Abdel-Rahman
similarity 0.63
Abstract · 7 pages, 3 figures, 5 tables. Submitted to IEEE VTC2027-Spring (under review)

Deep reinforcement learning (DRL) traffic-signal controllers may use vehicle-to-infrastructure (V2I) reports, exposing their perception layer to false-data injection. Existing demonstrations typically assume either white-box access to the controller's weights or full observability of the traffic state. We study the impact of a weaker and more realistic adversary who has no access to the RL model and whose traffic observations are limited to her own approach. This black-box, minimally informed attacker trains an imitation surrogate of the controller. She then crafts bounded fake reports, restricted to inflating vehicle counts on her own approach, that transfer to the real controller. We demonstrate that despite the loss of accuracy relative to an attacker who can observe the entire state and action space, this limited attacker can significantly increase the average waiting time for other approaches while barely changing the waiting time on her own approach, i.e. externalizing the added delay onto the other approaches. Because the only realizable move is to inflate one's own reported queue, we find that a zero-knowledge attacker that simply reports the maximum feasible queue is as damaging as the surrogate-based one, while an unstructured random inflation is not: the surrogate is not required to reproduce the degradation, and the exposed vulnerability is the V2I sensing channel itself. The surrogate's role is directional control, selecting a disruptive or a selfish objective at the same budget. The attack persists under +/-20% uniform demand variation. Finally, on a common per-lane budget we place the attacker on a knowledge ladder against white-box adversaries, and find the black-box surrogate matches a realizable white-box attacker that uses the victim's own gradients; only abandoning realizability raises the damage further.

0.16

Resilient Extremum Seeking Control for Cyber-Physical Systems Under Denial-of-Service Attacks

eess.SY math.OC ·2026-09-12 ·Filipe da Silva Bastos Teixeira, Pedro Henrique Silva Coutinho, Tiago Roux Oliveira, Miroslav Krstic
similarity 0.63
Abstract · Preprint submitted to IEEE Transactions on Control of Network Systems (12 pages, 4 figures)

Extremum seeking control (ESC) relies on deliberately injected excitation to extract optimization information from measured outputs, making its networked implementation particularly vulnerable to denial-of-service (DoS) attacks. This paper develops a resilient discrete-time ESC architecture for cyber-physical systems subject to DoS attacks. By holding the most recently successfully transmitted signals during DoS intervals, the proposed mechanism yields averaged error dynamics with an exponentially contracting mode under successful communication and a neutral mode under attack. For deterministic DoS attacks, practical exponential convergence to the extremum is established under an average bound on the attack duration, with an explicit convergence rate depending on the fraction of time under attack. For probabilistic DoS attacks, almost-sure and in-probability convergence properties are established via stochastic averaging, with the attack success probability explicitly entering the convergence rate. A key finding is that a seemingly natural zero-input strategy can fundamentally compromise ESC: attacks synchronized with the dither generate an $\mathcal{O}(1/a)$ bias in the averaged dynamics, where $a$ is the dither amplitude, and can displace the equilibrium from the true optimizer. Thus, under the proposed hold-based architecture, increasingly severe DoS attacks primarily slow the optimization process rather than destroy its stability, provided that communication is not permanently blocked. Numerical simulations illustrate the theoretical guarantees and the failure mechanism induced by dither-synchronized attacks.

0.54

Energy as a Concealable State in Adversarial UAV Patrolling: Formulation, an Energy-Security Threshold, and the Limits of Self-Play

eess.SY ·2026-08-27 ·Sai Krishna Reddy Mareddy
similarity 0.63
Abstract

We study energy-constrained adversarial patrolling on a graph, in which a battery-limited UAV defends a cluster of high-value targets against a strategic attacker who chooses when and where to strike. Unlike prior adversarial patrolling, the patroller must periodically return to a base to recharge; unlike prior energy-aware patrolling, it faces a self-interested adversary. Our central observation is that the remaining energy is a hidden state: the attacker never observes the battery directly, but observes the patroller's trajectory and can infer when a recharge excursion, and thus a vulnerability window, is imminent. We formalize the interaction as a zero-sum partially observable stochastic game and report a negative result on the solver side: neither independent deep Q-learning nor Neural Fictitious Self-Play reaches a stable equilibrium at this scale; each improves transiently and then collapses, with the co-trained thwart rate falling from about 0.25 to about 0.09 over training. Using a structural analysis independent of the learning dynamics, we show that achievable security rises monotonically with the energy budget, from zero below a threshold to about 0.7 when the budget is ample, establishing the energy budget as the primary determinant of defensibility. We set out the program the model is built to answer: whether an inference-capable attacker concentrates its successful strikes in the recharge window, and whether the defender can learn deceptive recharge timing to keep that window closed.

0.19

Runtime Assurance Under Measurement Attack: Necessary and Sufficient Observability Conditions for Learned Control in Radio Access Networks

eess.SP cs.CR ·2026-09-25 ·Yasser Al Eryani
similarity 0.61
Abstract · 17 pages, 6 figures, 9 tables

Runtime assurance pairs a verified fallback with an untrusted controller and a switching monitor, and is the leading route to admitting learned policies into safety-relevant network control. Its guarantee rests on a condition it assumes rather than requires: that the monitor's estimation error is zero, or at worst stochastic with a characterisable rate. In mobile networks many measurements originate at untrusted endpoints, where the error is neither. We state this assurance precondition and prove it: at zero measurement noise the guarantee survives an adversary controlling $q$ channels if and only if the plant is $2q$-sparse observable with respect to the safety-relevant output. This is functional observability over attack supports, strictly weaker than full-state observability: on our topology a per-cell outage condition doubles the budget and one stated over total offered load triples it. The converse is a constructive attack making a safe and an unsafe trajectory observationally identical; it defeats every monitor on those channels, not one detector, at any noise level. As the trust split is static and known before deployment, confining the adversary to one family yields a confinement threshold, above which trusted channels alone resolve the state and all untrusted channels may be corrupt at once. Three uninfluenceable counters cross it here, lifting the budget from two to six and making placement the dominant lever. Below the precondition the monitor faces a frontier, not a dichotomy; a set-valued monitor is optimal and carries sufficiency into non-zero noise at the residual test's false-rejection rate. An adversary reaching the trigger radius owns the switch regardless. The same argument bounds any monitor reading agent messages rather than measurements. A rank test decides the budget and identification does not preserve it, so a published budget must name its margin floor.

0.20

Forward-Invariant Policy Classes for Safe Reinforcement Learning in Multicopter Control

eess.SY ·2026-09-29 ·Chieh Tsai, Muhammad Junayed Hasan Zahed, Jinzhi Shen, Yi Xie, Ruoshan Lan +3 more
similarity 0.61
Abstract

This paper proposes a reinforcement learning (RL) framework for safe gain scheduling based on forwardinvariance-induced action-space design. Under stated nominalmodel and inversion-domain assumptions, rather than enforcing safety through runtime shielding or penalty-based constraints, safety is embedded directly into the policy class. Specifically, we construct a finite library of feedback controllers sharing a common Lyapunov certificate that establishes forward invariance of a prescribed admissible set under arbitrary switching. Consequently, any policy whose actions are restricted to this library, including policies encountered during RL exploration, inherits the same certificate. Policy optimization can therefore focus on closed-loop performance without runtime safety filtering or action projection. The framework is instantiated for quadcopter hover regulation, where a DQN schedules among certified feedback controllers. Nonlinear MuJoCo simulations demonstrate state-dependent gain scheduling and empirically evaluate robustness to wind, model mismatch, sensor noise, and sensing delay. The results illustrate how safety certification can be separated from policy optimization by learning over a forward-invariant policy class.

0.12

Distributed Secure Learning Control for Large-scale Multirobots under Stealthy Actuator Attacks

cs.RO ·2026-09-07 ·Xinglong Zhang, Qingwen Ma, Cong Li, Hui Yin, Changxin Zhang +3 more
top venuehot direction similarity 0.60
Abstract · 23 pages, 23 figures. A revised version of this manuscript has been accepted to IEEE Transactions on Robotics

Distributed learning control for multirobot systems (MRS) offers significant flexibility in presence of uncertainties but lacks provable performance guarantees. A promising direction involves integrating reinforcement learning (RL) into distributed model predictive control (DMPC), leveraging the strengths of RL in nonlinear policy design and the receding-horizon replanning capabilities of DMPC. However, ensuring secure control within such a learning framework under malicious cyber attacks, particularly stealthy ones, remains a critical challenge, because the distributed policies generation depends on information exchange among neighbors, where compromised agents can rapidly influence the behavior of others through the communication network. This article proposes a distributed secure learning control (DSLC) framework for large-scale MRS under malicious, stealthy actuator attacks. Our framework offers two key features: (i) a unified approach that enables secure learning control across various coordination scenarios and (ii) a game-theoretic distributed learning-based predictive control strategy that learns how to balance the attacker and defender through a differential-game based DMPC framework. Specifically, DSLC employs a distributed attacker-actor-critic architecture to learn the optimal defense and attack policies online within each prediction interval. Unlike numerical optimization-based controllers that calculate open-loop control sequences, our method simultaneously generates adversarial attack policies and corresponding defense policies in analytical closed-loop form. The defense policies could be directly generalized to MRS with varying scales and diverse actuator attack probabilities. The effectiveness and scalability of DSLC are validated through comprehensive simulations and real-world experiments in multiple wheeled robots via various control tasks.

0.53