CS Paper Radar

Causal Analysis of Failures of Cyber-Physical Systems with Continuous Inputs

Tom Yaacov, David A. Kelly, Khen Elimelech, Hana Chockler

cs.RO ·submitted 2026-10-22 05:08 UTC · v1

Abstract

Modern autonomous Cyber-Physical Systems (CPSs), such as self-driving cars, operate in increasingly complex environments while expected to maintain high reliability. The black-box nature of many of these systems makes it impossible to fully verify their behavior prior to deployment. When a failure occurs, it is important to analyze it to guide improvements to system robustness. A recent work introduced the novel concept of leveraging the framework of actual causality for CPS failure explanation. However, that work considered only a discrete, binary causal model that cannot capture continuous changes in system inputs. In this paper, we address these limitations by extending the actual causality framework with the notion of distance to account for the continuous characteristics of the system. Specifically, we introduce the novel concepts of distance-weighted actual cause and distance-weighted degree of responsibility, enabling causal influence to account for the magnitude of changes required to avoid a failure, rather than only its cardinality. While motivated by the continuous nature of CPSs, this constitutes a general extension of the theory of actual causality and enables more fine-grained analysis of continuous inputs in any causal model. Building on these notions, we present an efficient, practical, system-agnostic explanation-derivation algorithm that uses distance-weighted responsibility to guide the explanation search. We experimentally evaluate our approach across a number of environments from two autonomous-driving simulators with black-box neural controllers. The results show that the proposed extension provides more effective guidance than the current discrete approach.

Closest papers

Nearest neighbours by meaning across the whole index.

Sufficient and Necessary Causal Explanations for the Analysis of Autonomous Systems

cs.RO ·2026-10-21 ·Tom Yaacov, Khen Elimelech, Hana Chockler, Moshe Y. Vardi
hot direction similarity 0.74
Abstract

Understanding failures of autonomous systems is challenging, particularly when their behavior depends on black-box components such as neural networks. In this work, we make a distinction between two objectives of failure analysis: reproduction, which seeks to identify conditions sufficient to reproduce an observed failure, and avoidance, which seeks to identify the changes required to prevent it. Previous work on causal explanations of system failures is based on the concept of sufficient explanations and naturally supports reproduction. Nevertheless, we observe that it does not address the problem of avoiding the failure. In this paper, we introduce and formalize the concept of necessary explanations for system analysis as the correct formalism for failure avoidance. We place necessary explanations in the framework of actual causality and show that, together with sufficient explanations, they provide a unified causal framework for addressing both objectives of failure analysis. We present an efficient approximation algorithm for generating necessary explanations based on the notion of causal responsibility. We evaluate the efficiency of our approach across multiple environments. We demonstrate the practical utility of necessary explanations for failure avoidance through counterfactual generation and show that it outperforms existing methods.

0.25

From Concepts to Actual Causes: Toward Closing the Concept-Malfunction Gap in Robot Foundation Models

cs.RO cs.AI ·2026-10-30 ·Gihyuk Ko
position similarity 0.60
Abstract · 4 pages, 1 figure, 1 table. Accepted at the Science of Physical AI Safety (SPAIS) Workshop at CoRL 2026

Post-incident analysis of a robot foundation model poses a singular causal question: which internal components caused this failure, and to what degree? Mechanistic interpretability (MI), however, establishes general causal claims by decoding the concepts a policy has learned, yet knowing which concepts are encoded is insufficient for identifying the actual cause of a specific incident. In this position paper, we analyze this concept-malfunction gap by posing fault isolation in the framework of actual causality. Specifically, we formalize a fault-isolation query as a tuple of intervened internal components, counterfactual values, contingencies, and read-outs, and lay out conditions for it to be well-specified. Through this lens, we identify shortcomings of current MI approaches: they assume fixed contingencies and read out only immediate effects, letting backups obscure actual causes. We demonstrate on a toy example that these shortcomings can lead to conflicting verdicts. Lastly, based on the well-specification conditions, we outline four open research tasks toward closing the gap.

0.39

From Constraints to Causality: Petri net Encodings for Explaining Binary Neural Network Inference

cs.LO cs.LG ·2026-11-17 ·Alex Chan, Mohamed Tarraf, Alex Yakovlev, Rishad Shafik
publishedtheory similarity 0.58
Abstract · In Proceedings FMAS 2026, arXiv:2611.21375

Binary Neural Networks (BNNs) are attractive for resource-constrained hardware and on-device components in autonomous systems. However, their discrete, non-linear behaviour makes them difficult to analyse and trust in safety-relevant settings. Existing approaches based on SAT and Satisfiability Modulo Theories can verify a BNN's input-output properties, but treat it as a monolithic constraint system, which reveal only whether a property holds and not how the computation reaches its result. In this paper, we present an early work that provides a complementary, discrete-event view. We give a compositional translation from the inference semantics of a fixed, trained BNN to a 1-safe Petri net (PN), where each computation step is an explicit transition, so that a firing sequence is a causal trace of the BNN's decision. On a worked example, we establish faithfulness via exhaustive input-output equivalence against a reference implementation, and show how contrasting firing sequences localises where a single-feature change alters the prediction. Finally, we outline two directions this opens: a correctness-of-translation theorem, and a causal-trace analysis of where two trained BNNs diverge in their decisions, a structural question that constraint-based verifiers cannot express.

0.65

SecODD: Adapting the Operational Design Domain from Driving Autonomy to Autonomous Cyber Defense

cs.CR ·2026-10-17 ·Jina Kang, Jeong-Han Yun, Hyesong Choi
similarity 0.58
Abstract · 13 pages, 5 tables

Security operations centers are beginning to delegate detection, triage, and containment to Autonomous Cyber Defense (ACD) agents at execution time. Erroneous high-impact playbooks can disrupt critical services and create accountability gaps among deployers, SOC supervisors, vendors, managed service providers, and infrastructure providers. We propose SecODD, an execution-time governance framework for ACD that adapts the Operational Design Domain concept from autonomous-vehicle regulation to cyber autonomy. It defines the authorized scope of automated defensive actions, models when meaningful human intervention must remain possible, and specifies auditable due-care conditions through a Mitigation Safe Harbor. Four engineering requirements---tamper-evident logging, human-in-the-loop orchestration, rollback playbooks, and safe-state transition controls---are evaluated through a per-scenario audit matrix that illustrates how two ACD failure modes can be translated into pass/fail audit outcomes. The result is an evidence-oriented framework that authorizes ACD only when autonomous actions are bounded, interruptible, recoverable, and reconstructable from evidence.

0.16

Assessing the Quality of Concrete Scenarios Generated from Scenario Classes for Automated Driving

cs.SE cs.RO ·2026-10-13 ·Lina Putze, Thies de Graaff, Jan Steffen Becker, Christian Neurohr
similarity 0.56
Abstract

A scenario-based development approach has become state of the art for automated driving systems. Once a set of scenario classes covering the operational design domain is available, concrete scenarios are generated from these scenario classes, e.g., to be used for hazard analysis and risk assessment or for the system's verification and validation. For such applications, a high diversity among the generated concrete scenarios is paramount. The question arises how the quality of collections of concrete scenarios can be assessed in such a setting. In this work, we aggregate scenario distance measures into so-called quality functions that quantify the diversity of concrete scenario collections in the context of automated driving. In particular, we consider the case where scenario classes are formalized as constraint systems and the associated concrete scenario space is not explicitly known. Based on a set of desired properties we construct candidates for such quality functions and give proofs for their property fulfillment. To show the practical relevance of quality functions, we perform a variety of experiments in the context of an abstract scenario-based virtual verification toolchain for automated driving systems. More specifically, we use quality functions to compare the effectiveness of variation methods for sampling, examine the choice of different scenario distance measures, and investigate the effects of temporal aspects on concrete scenario collection quality.

0.15

AWARE-RG: Abstraction Aware Explanation of Resiliency Graphs for Cyber Physical Systems

cs.CR cs.AI ·2026-10-14 ·Shadaab Kawnain Bashir, Rakesh Podder, Philip D. Hopkins, Sarath Sreedharan, Indrakshi Ray +1 more
similarity 0.56
Abstract

Understanding the security posture of cyber physical systems (CPS) requires reasoning across both information technology (IT) and operational technology (OT). However, a security analyst often holds an incomplete, role-specific view of the underlying system which has most likely resulted from their knowledge limited by a need-to-know framework. Such an incomplete view can potentially lead them to incorrectly assess if a cyber attack produces an untoward physical consequence resulting in making uninformed decisions about system resilience. In this work, we present AWARE-RG (Abstraction Aware Explanation of Resiliency Graphs), a semi-automated, human-in-the-loop framework that provides an analyst with a role-bounded, need-to-know explanations of resiliency paths in a CPS. The AWARE-RG abstraction structure is derived using Formal Concept Analysis over MITRE ATT&CK for ICS tactics, ordered by the ICS Cyber Kill Chain. For valid attack-fault queries, plan repair and backward means-end analysis identify the missing dependencies, explain why the physical hazard is reachable, and suggest defensive strategies. For invalid attack-fault queries, AWARE-RG localizes the first abstraction level at which the path fails, and reports confirmed blocking conditions as hardening pointers.

0.12

Towards Realistic Models of Change: A Causal Perspective on Decision-Making and Dynamical Systems

cs.LG cs.AI ·2026-10-20 ·Isacco Beretta
position similarity 0.55
Abstract · PhD thesis, University of Pisa (National PhD Program in Artificial Intelligence), 2026. 175 pages

While statistical learning has made remarkable progress in recent decades, driven by large datasets and data-driven protocols that have transformed scientific practice, it still faces a key limitation: it struggles to generalize beyond the data it has seen, often failing under distribution shift. Causality offers a promising solution: causal models focus on the underlying mechanisms of change rather than surface-level associations, enabling robust reasoning under external interventions. This allows systems not only to learn from data, but also to reason about hypothetical scenarios, adapt to new environments, and inform better decisions. This thesis explores how these approaches can inform each other, in two parts. The first part examines algorithmic recourse, that is, identifying which changes an individual can make to overturn an unfavorable decision. We propose an alternative perspective to the established framework, arguing for a more realistic cost function, a reformulation of the problem in causal terms, and the inclusion of a temporal dimension in policy evaluation. The second part focuses on dynamical systems, extending the notions of causal interventions and effects to discrete and continuous stochastic processes. We show that the equilibria of these processes, and how they change following an intervention, can be linked to Structural Causal Models (SCMs), viewed as compressed representations of the underlying temporal dynamics. We argue for the relevance of such temporal causal models, since in real-world scenarios actions take time to produce their effects. The common goal of both parts is practical: the utility, robustness, and trustworthiness of these models depend critically on their ability to realistically represent the underlying phenomena.

0.24

Assuring Learning-Enabled Cyber-Physical Systems with Foundation Models: A Neuro-Symbolic Agenda for Testing, Verification, and Runtime Assurance

cs.RO eess.SY ·2026-10-29 ·Xi Zheng, Ivan Ruchkin, Hong Jin Kang, Tevfik Bultan, Vijay Ganesh +9 more
position similarity 0.55
Abstract

Learning-enabled cyber-physical systems (CPS), including autonomous vehicles and UAVs, increasingly incorporate foundation models, yet fragmented assurance practices provide insufficient support for system-level guarantees. We argue that restoring engineering control across stochastic neural, symbolic, and physical boundaries requires a shift from model-centric evaluation to system-level assurance. We introduce the Foundation Model Assurance Stack, connecting three directions: (i) neuro-symbolic architectures that constrain foundation models through explicit symbolic world models, reasoning layers, and interfaces; (ii) semantically grounded, conformance-based testing that combines formal specifications and symbolic world models with foundation-model-assisted specification mining, scenario generation, and test synthesis; and (iii) compositional verification and runtime assurance that target formally tractable components and interfaces through modular contracts, equivalence checking, verified synthesis, uncertainty-aware reasoning, and runtime monitoring and enforcement. Symbolic abstractions establish assurance boundaries between stochastic learning and formal reasoning, connecting testing, verification, and runtime evidence across the system lifecycle. Foundation models serve as constrained assistants whose outputs are independently checked wherever correctness can be formally specified. We identify research challenges in specification engineering, abstraction fidelity, evidence composition, and lifecycle certification, and outline a staged agenda toward trustworthy, deployable, and certifiable learning-enabled CPS.

0.24