CS Paper Radar

Approximate Dynamic Programming for Link Reconfiguration to Reduce Malicious Input Controllability in Consensus Networks

Nathaniel Mailhot, Mohammad Pirani, Davide Spinello

eess.SY ·submitted 2026-10-29 19:45 UTC · v1

Abstract

Communication links influence how strongly a malicious input at one sensor affects a consensus network. We study how a defender can reduce this influence by selecting one additional link after observing the currently attacked sensor. The attacked sensor changes over time, and vulnerability is measured by the trace of a finite-horizon controllability Gramian. A prescribed local epsilon-greedy attacker response rule defines a discounted Markov decision process for the defender. Exact dynamic programming (DP) provides an optimal policy using the full Gramian payoff matrix and attacker transition model. We then introduce approximate dynamic programming (ADP) that learns from sampled attacker transitions and empirical Gramian costs estimated from measured impulse responses. Under conditionally unbiased costs and standard sampling and learning-rate conditions, the learned action-value table converges to the exact DP solution. The analysis uses Bellman contraction and bounds the effect of persistent Gramian estimation bias, establishing when measured responses recover optimal defender decisions without constructing the full payoff matrix. In simulations on 20- and 100-node graphs, empirical ADP rollout costs are within 0.23% and 1.76%, respectively, of exact DP. In a two-dimensional intruder tracking example, both policies reduce network-wide root-mean-square tracking error spread by approximately 25% relative to an unmitigated attack.

Closest papers

Nearest neighbours by meaning across the whole index.

Defense Filters for Robust Learning of Partially Observed Dynamical Systems

eess.SY math.OC ·2026-10-15 ·Jihun Kim, Javad Lavaei
similarity 0.65
Abstract · 8 pages, 1 figure

Robust estimation of unknown system dynamics from partial observations under attack is a core challenge in learning for control, increasingly critical in multi-agent networks subject to adversarial data poisoning. Existing methods either require highly restrictive assumptions on the adversary or yield weak error bounds. In this paper, we consider identifying partially observed linear dynamical systems subject to adversarial attacks with an attack probability $p < 0.5$ at each time. We propose a defense-filtering framework leveraging knowledge of the observation matrix to construct a defense filter satisfying full column rank and subspace separability conditions, which enables the evaluation of a single-attack scenario against observed data. Using this framework, we analyze the constrained $\ell_2$-norm estimator and establish finite-sample error bounds for recovering the Markov parameter matrix up to a linear transformation. This is the first approach to provide provable system identification guarantees via a defense filter for partially observed systems subject to harsh environments where the attack probability approaches $0.5$.

0.37

Preventing Model Extraction in Contextual Linear Optimization

cs.LG math.OC ·2026-10-21 ·Adam N. Elmachtoub, Tianyi Lin, Wenxuan Liu
theory similarity 0.64
Abstract · 25 pages, 11 figures

Consider a contextual linear optimization problem, where a predict-then-optimize (PTO) system returns optimal decisions without revealing the cost vectors used in its downstream linear program. Our focus is on how to prevent attackers, who can submit context queries, from extracting the underlying prediction model. Although the cost-to-decision map is many-to-one, observations across contexts can reveal the underlying prediction model. For linear models, we provide sufficient conditions where the attacker can learn the model up to positive scaling by only observing the decision. These conditions motivate an adaptive attack that alternates inverse optimization with boundary sampling, i.e., using context queries leading to decision boundaries of the current attacker model. To prevent extraction, we propose a mechanism that perturbs decisions only when the predicted cost vector is near a decision boundary. We establish a regret bound in terms of the perturbation radius. For the attackers that fit the defended responses as optimal decisions under the evaluation distribution, we also prove a lower bound on expected extraction error. Experiments on synthetic instances and real data quantify the security-performance trade-off of the defense mechanism and illustrate that our defense can increase the extraction error.

0.18

A Hidden Markov Preference Model for Online Prediction of Adversary Actions

cs.LG ·2026-11-05 ·Angel Daruna, Michael Cogswell, Tixiao Shan, Yi Yao, Han-Pang Chiu +1 more
benchmark similarity 0.62
Abstract

A defender anticipating an attacker needs a probability distribution over the attacker's next action from information available before that action occurs. Two structures complicate this: the attacker's behavioral regime persists and can switch within an episode, and each action combines an operation with state-dependent arguments whose feasible values change over time. We capture both with a hidden Markov model whose latent state is the active regime and whose regime-conditioned action law is defined on a feasibility-masked tree of executable actions. The same likelihood forms the next-action distribution and updates the regime belief once the action is seen, and a partially observed prefix contributes its resolved local factors. Estimation separates into convex logistic fits that Baum-Welch reuses when regimes are unlabeled. We derive a reach-weighted nodewise KL expression for regime information, a sequence-wise log-loss bound for switching regime paths, and a uniform fallback guarantee. On the CAGE 2 cyber defense benchmark the filtered latent state is worth 0.8 nats per step, and after a controller switch the loss begins to fall in three steps against ten for a recurrent baseline.

0.27

Resilient Extremum Seeking Control for Cyber-Physical Systems Under Denial-of-Service Attacks

eess.SY math.OC ·2026-09-12 ·Filipe da Silva Bastos Teixeira, Pedro Henrique Silva Coutinho, Tiago Roux Oliveira, Miroslav Krstic
similarity 0.61
Abstract · Preprint submitted to IEEE Transactions on Control of Network Systems (12 pages, 4 figures)

Extremum seeking control (ESC) relies on deliberately injected excitation to extract optimization information from measured outputs, making its networked implementation particularly vulnerable to denial-of-service (DoS) attacks. This paper develops a resilient discrete-time ESC architecture for cyber-physical systems subject to DoS attacks. By holding the most recently successfully transmitted signals during DoS intervals, the proposed mechanism yields averaged error dynamics with an exponentially contracting mode under successful communication and a neutral mode under attack. For deterministic DoS attacks, practical exponential convergence to the extremum is established under an average bound on the attack duration, with an explicit convergence rate depending on the fraction of time under attack. For probabilistic DoS attacks, almost-sure and in-probability convergence properties are established via stochastic averaging, with the attack success probability explicitly entering the convergence rate. A key finding is that a seemingly natural zero-input strategy can fundamentally compromise ESC: attacks synchronized with the dither generate an $\mathcal{O}(1/a)$ bias in the averaged dynamics, where $a$ is the dither amplitude, and can displace the equilibrium from the true optimizer. Thus, under the proposed hold-based architecture, increasingly severe DoS attacks primarily slow the optimization process rather than destroy its stability, provided that communication is not permanently blocked. Numerical simulations illustrate the theoretical guarantees and the failure mechanism induced by dither-synchronized attacks.

0.54

When to Intervene? State-Aware Sparse Manipulation in Federated Reinforcement Learning

cs.LG ·2026-10-08 ·Shutong Zheng, Sijia Chen
code/data similarity 0.61
Abstract · 27 pages, 12 figures

Federated reinforcement learning (FRL) enables distributed agents to collaboratively train decision-making policies, but its decentralized training process also exposes global policy learning to Byzantine manipulation. Existing poisoning attacks primarily focus on how to construct malicious updates, while trajectory-level intervention timing remains largely implicit. In sequential decision making, however, where an intervention is applied can alter subsequent trajectories and learning signals. Through controlled experiments, we find that changing the selected trajectory states materially alters attack efficacy even when the malicious-update construction is fixed. We therefore identify when as a distinct attack dimension and introduce the Viability-constrained Behavioral Steering Attack (V-BSA), which uses local policy uncertainty to select sparse intervention states and applies envelope-constrained behavioral steering. Across discrete-action benchmarks, V-BSA achieves substantial degradation against robust aggregators and ensemble defenses with only a fraction of the interventions used by dense poisoning, while revealing task- and aggregation-dependent boundaries. Overall, our results highlight intervention timing as a distinct dimension of sequential robustness in FRL. The code is available at https://github.com/Yodeesy/V-BSA

0.33

Dynamical low-rank equilibrium computation for stochastic games between advanced persistent threats and moving target defense

cs.GT cs.AIcs.CReess.SY ·2026-09-20 ·Tian Zijian, Zhang He, Chen Xinjie, Wang Wenhai, Liu Xinggao
theorycode/data similarity 0.60
Abstract · Regular Paper, under review at Automatica (submission 26-2265). v2: corrected the utility-loss figure in Experiment 5 (2.3%). This preprint …

Moving target defense (MTD) against advanced persistent threats (APTs) in industrial control systems (ICS) has well-established game-theoretic formulations, but their practical value hinges on equilibrium computation, which faces two gaps: full-rank value iteration is prohibitively expensive at industrial scale, and the resulting defense strategies admit no certified robustness against adversarial perturbations. We first reveal that the attack and defense influence matrices of ICS dynamics are intrinsically low-rank: APTs infiltrate through a handful of entry points and MTD reconfigures only a few components per cycle. Our theory makes four contributions. First, an augmented gradient matrix certifies that the low-rank structure propagates through the non-smooth Bellman operator of the zero-sum stochastic game, so that every Bellman target lies near a low-dimensional subspace and low-rank truncation incurs an explicit error bound (Lemma 1, Theorem 1). Second, we propose the Dynamical Low-Rank Nash Equilibrium algorithm, named DLR-NE, which augments the rank-r search space each iteration, regularizes the core matrix spectrum, and retracts via truncated SVD, and prove that it converges geometrically to a neighborhood whose error decomposes into five physically interpretable sources (Theorem 2). Third, its per-step cost is O(nr^2), a Theta(n/r^2) speedup over full-rank value iteration (Theorem 3). Fourth, a single weight trades accuracy against a certified sensitivity bound of the induced defense strategy under core-matrix perturbations (Corollary 1). Six experiments on a nonlinear power-system testbed confirm each prediction, with 94% parameter compression at 2.3% utility loss. All experimental data and code are publicly available.

0.29

Vulnerability Analysis for Safe Multi-Agent Reinforcement Learning

cs.AI ·2026-10-19 ·Taozheng Zhu, Shixiong Jiang, Fanxin Kong
similarity 0.60
Abstract · 21 pages, including references and appendices

Safe Multi-Agent Reinforcement Learning (SMARL) aims to learn cooperative control policies under prescribed safety constraints. Al- though existing SMARL methods focus on enforcing safety during policy optimization, their vulnerability remains largely unexplored. This paper investigates inference-time observation attacks against SMARL, where an adversary perturbs the observations of a subset of agents, thereby altering the actions produced by their policies. We propose MinMax Attack, an attack framework that jointly selects adversarial target actions for compromised agents while accounting for the Cooperative Resistance of benign agents. We further derive a bound on the induced safety cost and characterize its relation- ship with the Compromised-Agent Budget and the Perturbation Budget. Experiments across multiple safety-critical multi-agent benchmarks and representative state-of-the-art SMARL algorithms show that MinMax Attack induces substantially higher safety costs than existing attack baselines while preserving task reward, thereby reducing its detectability and achieving both attack effectiveness and stealthiness.

0.15

FoSeRL: Formal Sequential Robustness Certification for Reinforcement Learning Policies

cs.LG ·2026-10-03 ·Sara Taheri, Deep Kumar Ganguly, Jan Křetínský, Majid Zamani
similarity 0.59
Abstract

Even a few action perturbations can substantially degrade the performance of a deployed decision policy. Certifying the resulting return loss is challenging in stochastic environments, where returns vary even without an attack. We introduce FoSeRL, a framework for certifying deployed RL policies against precommitted, temporally sparse action attacks. The deployed policy is unchanged, with no smoothing or retraining. Certification requires a resettable simulator supporting shared randomness and independent one-step successor queries, but no analytical dynamics model. FoSeRL certifies that an attacked episode loses no more than a prescribed amount of return relative to the same episode unattacked, with at least a target probability and at a user-specified confidence level. Both runs share the initial state and randomness, so the measured loss reflects the attack, not the episode; carrying the running return gap as a state coordinate makes it the terminal value, reducing trajectory-level certification to terminal safety. Time-dependent barrier conditions on the augmented state bound the terminal failure probability: satisfied exactly, they certify every admissible precommitted attack; learned from sampled trajectories and verified on held-out data, they certify the same guarantee under a specified attack-episode setting. Across six stochastic continuous-control environments and three RL policy families (TD3, SAC, and PPO), FoSeRL certifies non-trivial cardinality--magnitude robustness frontiers, achieves substantially larger certified budgets than policy smoothing, and reveals marked robustness differences among policies with comparable nominal performance.

0.20