CS Paper Radar

Keeping Shielded Data-Driven Controllers Safe Under System Evolution via Model Consistency

Roland Stolz, Vitus Lüntzel, Romain Pascual, Eric Sax, Matthias Althoff

eess.SY cs.AI ·submitted 2026-11-18 13:45 UTC · v1

Abstract

Cyber-physical systems (CPS) increasingly contain data-driven controllers, which raise safety concerns given their black-box nature. A common mitigation is to enforce system safety via a shield around the controller, which restricts the inputs based on assumptions about the physical system. However, these assumptions are usually fixed and may become invalid as the system evolves (during development or due to component degradation), possibly leading to unsafe behavior. We address this limitation in a model-driven approach that treats both the data-driven controller and safety shield as first-class models that are integrated into a virtual single underlying model (V-SUM) alongside system models. We describe system evolution as changes in the associated models, such that consistency checking yields a continuous assessment of safety guarantees. In case of inconsistencies, consistency-preservation rules automatically update the models and restore a valid safety shield. We evaluate our approach on a reach-avoid task for a unicycle system. The results show that consistency-driven evolution effectively maintains safety guarantees under system changes and provides a principled basis for evaluating and restoring performance.

Closest papers

Nearest neighbours by meaning across the whole index.

Context-dependent Runtime Construction of Safe Regions of Attraction from Design-Time Identification for Multi-Candidate Simplex Architectures

cs.RO eess.SY ·2026-10-26 ·Georg Jäger, Nils-Jonathan Friedrich, Norman Seyffer, José Cecílio, António Casimiro
similarity 0.66
Abstract · 13 pages, 8 figures, 5 tables

Deploying safety-critical cyber-physical systems requires control strategies that reconcile high performance and adaptability with rigorous safety guarantees. Classical control strategies offer stability proofs but are conservative, while advanced optimization- and learning-based controllers achieve high performance but lack formal verification. Runtime Assurance (RTA) architectures bridge this gap by introducing their own (dis-)advantages. Safety filters use online optimization and introduce chattering, while Simplex permissiveness is bounded by the tightness of the certified Region of Attraction (RoA) of its High-Assurance (HA) controller. Targeting this conservatism, we present a two-stage RoA certification procedure leveraging contraction-based stability to establish a target set for Taylor-model-based reachability analysis. We then treat the closed-loop system under HA control as a switched system, enabling: (a) runtime construction of context-dependent safe RoA and (b) safe switching between HA candidates to optimize recovery and maximize the operating envelope of the uncertified High-Performance (HP) controller. We validate this framework on a robotic path-tracking task using a three-stage evaluation: Ordinary Differential Equation (ODE) simulation for theoretical verification, high-fidelity Gazebo simulation under unmodeled dynamics, and experiments on physical hardware. Results show that the Simplex architecture maintains safety under fault injection and successfully shields a behavioral-cloning-trained HP controller. Crucially, the decision module introduces negligible overhead (0.028 ms average, 4.13 ms worst-case), consuming under 10 % of the control period.

0.26

A Pragmatic Guide to Building Conservative Discrete Abstractions of Cyber-Physical Systems

eess.SY cs.LO ·2026-08-10 ·Jordan Peper, Krish Kapadia, James Gast, Ethan Howes, Ivan Ruchkin
similarity 0.64
Abstract

Symbolic model checking is an effective approach for verifying semantically rich temporal-logic properties of cyber-physical systems, but it hinges on discretizing continuous-state dynamics into a finite-state abstraction. To transfer verification guarantees from the abstract model to the concrete CPS, the abstraction must conservatively approximate the concrete state space and behaviors. Hence, model-builders must maintain this soundness while balancing pessimism with tractability. However, they face several common pitfalls such as under-approximating the state space, under-approximating transitions, unsound pruning of "degenerate" behaviors, and improper specification lifting. This tutorial presents a pragmatic, conservative-by-construction workflow for building discrete abstractions of closed-loop dynamical systems. The workflow consists of four modular steps with interchangeable subroutines: (i) state-space partition and abstraction-function design, (ii) conservative transition construction via axis-aligned bounding boxes, polytopes, or sampling with PAC coverage certificates, (iii) mitigation of spurious transitions and self-loops using certified erasure and counterexample-guided abstraction refinement, and (iv) sound lifting of LTL specifications using may-must semantics. We demonstrate the end-to-end pipeline on three case studies and report how these design choices affect abstraction structure, runtime, and verification outcomes.

0.04

Verification-Guided Safety-Repair of Learned Controllers

cs.LO eess.SY ·2026-11-16 ·Fabian Kresse, Sobhan Gharanfoli, Nikolaus Vertovec, Alessandro Abate, Thomas A. Henzinger +1 more
theory similarity 0.64
Abstract

Formal verification can certify that a learned controller is safe. But what should be done when verification fails? Simply suppressing actions that lead to unsafe next states is insufficient because the dangerous action that ultimately caused the safety violation might have occurred many steps earlier. Retraining modifies the policy globally, thereby potentially introducing new failures. Instead, we introduce ReachECO, a verification-guided repair procedure that exploits the information of failed multi-step reachability analysis for synthesizing explicit constraints on controller behavior. For Differentiable Weightless Controllers, the repairs compile into exact circuit patches that leave the controller unchanged everywhere outside the targeted policy cells. We prove soundness and termination of the repair procedure for ReachECO, as well as, assuming exhaustive continuation search, completeness relative to the reachability abstraction. Empirically, we demonstrate \method's ability to successfully repair otherwise unsafe controllers in three continuous-control benchmarks: ACC, Docking2D and Car2D.

0.26

Assuring Learning-Enabled Cyber-Physical Systems with Foundation Models: A Neuro-Symbolic Agenda for Testing, Verification, and Runtime Assurance

cs.RO eess.SY ·2026-10-29 ·Xi Zheng, Ivan Ruchkin, Hong Jin Kang, Tevfik Bultan, Vijay Ganesh +9 more
position similarity 0.63
Abstract

Learning-enabled cyber-physical systems (CPS), including autonomous vehicles and UAVs, increasingly incorporate foundation models, yet fragmented assurance practices provide insufficient support for system-level guarantees. We argue that restoring engineering control across stochastic neural, symbolic, and physical boundaries requires a shift from model-centric evaluation to system-level assurance. We introduce the Foundation Model Assurance Stack, connecting three directions: (i) neuro-symbolic architectures that constrain foundation models through explicit symbolic world models, reasoning layers, and interfaces; (ii) semantically grounded, conformance-based testing that combines formal specifications and symbolic world models with foundation-model-assisted specification mining, scenario generation, and test synthesis; and (iii) compositional verification and runtime assurance that target formally tractable components and interfaces through modular contracts, equivalence checking, verified synthesis, uncertainty-aware reasoning, and runtime monitoring and enforcement. Symbolic abstractions establish assurance boundaries between stochastic learning and formal reasoning, connecting testing, verification, and runtime evidence across the system lifecycle. Foundation models serve as constrained assistants whose outputs are independently checked wherever correctness can be formally specified. We identify research challenges in specification engineering, abstraction fidelity, evidence composition, and lifecycle certification, and outline a staged agenda toward trustworthy, deployable, and certifiable learning-enabled CPS.

0.24

Controller-Agnostic Runtime Safety Certificates for Tool-Using AI Agents via Data-Driven Robust Reachability

eess.SY ·2026-10-12 ·Ali Baheri, Peng Wei
published similarity 0.63
Abstract

Software agents built on large language models act by issuing tool calls, some of which cannot be undone. To control theory the model is an opaque feedback controller, replaced every few months, whose closed loop cannot be analyzed directly. We propose a runtime filter, or shield, between any such controller and its environment that certifies a bound on the probability of reaching an unsafe condition during a bounded task. Given a predicate abstraction and candidate successor sets, everything else comes from logs: the loop becomes a finite controlled Markov chain, the estimated kernel is surrounded by a distribution-free confidence region, and robust reach-avoid value iteration runs over it. The certificate is probably approximately correct and uniform over controllers respecting the filter, so one model may replace another without recomputation. In all tested datasets the true risk stayed within certificate and budget; in a file-system sandbox the filter stays sound against a hazard the logs barely covered, where a plug-in baseline certifies safety and fails; and it holds seven controllers, three of them open-weights models, to zero unsafe outcomes in 1050 episodes.

0.28

Environment Steering: Using Data Flow Control to Improve Agent Utility and Safety

cs.CL cs.AIcs.DB ·2026-09-19 ·Charlie Summers, Prajwal Raghunath, Aaditya Pai, Mayur Kulkarni, Zhuo Zhang +2 more
position similarity 0.62
Abstract · 9 pages, 11 figures, REALM Workshop, EMNLP 2026

LLM agents can make unsafe tool calls even when instructed to behave safely. Existing defenses constrain agents before execution, modify tool inputs/outputs, or rely on LLM judges; these approaches may depend on model behavior or block unsafe actions without helping the agent recover. We argue that the execution environment should instead enforce safety as the agent runs and steer it toward safe alternatives when violations occur---we call this Environment Steering. We implement this by modeling the agent and harness execution state as database tables, track the record-level data flows, and check these data flows against declarative policies during runtime. When violations are detected, policy- and context-specific feedback steers the agent toward safe trajectories. On AgentDyn, this enables the agent to improve task success rate over no-defense while achieving 0% attack success rate.

0.29

A Recursive CBF Framework for Safety under State Uncertainty

eess.SY ·2026-09-16 ·Rahal Nanayakkara, Aaron D. Ames, Paulo Tabuada
similarity 0.62
Abstract · 8 pages, 4 figures, IEEE CDC 2026

The practical implementation of Control Barrier Functions (CBFs) for safety-critical control is often hindered by uncertainty in the knowledge of the state. While existing robust CBF methods address state uncertainty, they often lack recursive feasibility guarantees or fail when uncertainty levels are high, allowing the system to enter regions where no safe control input exists. To resolve this, we propose a novel framework of enforcing recursive CBFs. Rather than merely ensuring the invariance of the original safe set, this approach enforces the forward invariance of a subset of the safe region where a robustly safe control input is guaranteed to exist. This holistic framework ensures that the system never strays into ambiguous regions, providing continued feasibility and safety guarantees, regardless of the level of state uncertainty.

0.27

Model-Free Generative Stealthy Attacks on Safety-Critical Vehicle Cyber-Physical Systems

eess.SY ·2026-10-22 ·Qazi Mairaj ud din, Sidra Ghayour Bhatti, Qadeer Ahmed
similarity 0.60
Abstract

Existing stealthy attacks presuppose white- or grey-box access to the target system, unrealistic in practice. This paper develops a model-free stealthy attack framework that learns an attack policy directly from nominal measurement data, without knowledge of the system dynamics, estimator, or detector. We establish a certified residual-impact bound and an observable stealth budget around a stealth-consistent anchor, and show that recurrence and context conditioning are structural necessities for closed-loop, multivariate cyber-physical systems (CPS). The resulting attack generator, trained against a Wasserstein critic under a stealth-constrained impact objective, induces a hazardous 2.51 m lane departure while evading five intrusion detection systems, and, in real-time hardware-in-the-loop tests, a speed deviation of approximately 8 km/h while staying below the detection threshold. Results show that safety-critical CPS remain exposed without attacker access to privileged information, motivating offense-informed defenses.

0.28