CS Paper Radar

Representative Dataset Generation Framework for AI-based Failure Analysis during real-time Validation of Automotive Software Systems

Mohammad Abboush, Christoph Knieke, Andreas Rausch

cs.SE ·submitted 2026-08-10 14:28 UTC · v1
published

Abstract

Recently, thanks to its ability to extract knowledge from historical datasets, the data-driven approach has been widely used in various phases of the system development life cycle. During real-time system validation, remarkable achievements have been accomplished in developing an intelligent failure analysis based on historical data. However, despite its superiority over other conventional approaches, e.g., model-based and signal-based, the availability of representative datasets persists as a major challenge. Thus, for different engineering applications, new solutions to generate representative faulty data in different forms should be explored. Therefore, in this study, a novel approach based on Hardware-in-the-Loop (HIL) simulation and real-time Fault Injection (FI) method is proposed to generate and collect data samples under single and simultaneous faults for Machine Learning (ML) applications during system validation phases. The developed framework can generate not only sequential data, but also textual data including fault logs. The results show the applicability of the proposed framework in simulating and capturing the system behaviour under faults within the system components.

DOI
10.24251/HICSS.2024.877

Closest papers

Nearest neighbours by meaning across the whole index.

Knowledge-Graph-Guided Retrieval-Augmented LLMs for Explainable Root Cause Analysis in Automotive HiL Validation

cs.CR cs.SE ·2026-08-11 ·Hamza Ouarrad, Mohammad Abboush, Andreas Rausch
published similarity 0.66
Abstract · 10 pages, 3 figures, 5 tables. Accepted for publication and oral presentation at the 10th International Conference on System Reliability and…

Hardware-in-the-Loop validation of automotive software systems generates large multivariate time-series recordings whose manual analysis is time-consuming and often limited to anomaly detection and fault classification rather than root-cause analysis. Although deep learning methods have shown strong performance in fault detection and classification, they usually require task-specific training or retraining when new fault locations, systems, or operating conditions are introduced. They also tend to treat localization as a classification task, without explicitly representing the spatial and functional relationships between fault locations, sensors, and downstream subsystem effects. This limits their generalizability and their usefulness for engineering root cause analysis and diagnosis. This paper proposes a knowledge-graph-guided retrieval-augmented large language model framework for RCA (root cause analysis) and fault localization in automotive HiL data. The method converts raw time-series recordings into compact diagnostic evidence, enriches this evidence with sensor-to-location and propagation knowledge, and retrieves similar historical cases to support the final reasoning step. The LLM is then used as a decision and explanation layer rather than as a direct time-series classifier, producing a ranked fault-location prediction together with an interpretable RCA explanation. The framework is evaluated on two automotive HiL case studies: an ASM gasoline engine and an electric vehicle system. The best-performing model achieves Top-1 accuracies of 90\% and 94\%, respectively, while recording-level aggregation reaches perfect file-level fault localization in the evaluated subset. These results demonstrate the potential of KG-guided RAG-LLM reasoning for explainable and generalizable HiL RCA.

0.25

FaultGene: LLM-Assisted Fault-Augmented Test Generation for Hardware-in-the-Loop Validation of Autonomous Driving Systems

cs.SE cs.AI ·2026-10-21 ·Chaoran Yuan, Fadi Kurdahi
similarity 0.62
Abstract

Reliable validation of autonomous driving cyber-physical systems requires testing not only nominal driving behavior but also failures arising from faults in sensing and embedded computation. Physical road testing provides realistic system behavior but is costly and difficult to reproduce, while simulation enables scalable evaluation but cannot fully capture hardware-dependent behavior and fault propagation. Hardware-in-the-loop (HIL) validation bridges this gap by executing the driving stack on real embedded hardware within a simulated environment. However, preparing fault-augmented scenarios for HIL testing remains largely a manual process, limiting test coverage, scalability, and efficiency. We present FaultGene, an LLM-assisted framework for automated generation of executable fault-augmented tests for HIL fault-resilience validation. Given a baseline Scenic/CARLA scenario and a structured fault specification, FaultGene generates a fault-augmented test case that captures the effects of HIL-level fault injection. During offline refinement, semantic-consistency checks and execution-level feedback are used to correct candidates that fail to preserve the intended scenario context or produce the desired fault-induced degradation, and to construct high-quality supervision for fine-tuning. The resulting generator then produces fault-augmented tests directly at deployment time without requiring manually engineered reference executions or repeated refinement loops.

0.12

A Hybrid Rule-Based and AI-Augmented Framework for Automatic Failure Recovery in DevOps Deployments

cs.SE cs.DC ·2026-09-21 ·Raju Chowdhury, Aniruddha Singh Gautam, Apurv Ghai
similarity 0.60
Abstract · This is the preprint version of a paper accepted at the ETNCC 2026 Conference. The final published version will appear in IEEE Xplore. This …

Automatic failure recovery is another difficult area of DevOps deployments due to the complexity of the system, high workload and the limitations of the traditional rules-based or manual approach. This study proposes a hybrid model integrating deterministic, rule-based recovery with the assistance of machine learning by fault prediction to automatically monitor failures, identify, classify and recover failures in real time. To test the architecture, a distributed log dataset of 100,000 records was used for models such as Decision Tree, Random Forest, Logistic Regression, LightGBM, Autoencoder with BiLSTM. The Decision Tree had a strong performance in defect identification with an F1 score of 80.8%, recall of 80.0%, precision of 81.6%, and accuracy of 89.4%. An impressive 83.3% success rate was achieved by the automated recovery measures, resulting in a 94.6% decrease in mean time to recovery (MTTR), a 95.4% reduction in downtime, and annual savings of 7,462. The combination of explainable rule-based logic and adaptive AI prediction in the framework means it is more resilient, operationally efficient, and scalable, and an effective, autonomous approach to DevOps environments evaluated under controlled experimental conditions.

0.35

Learning from Past Mistakes: AI-Assisted Analytics of Software Defects and Their Fixes for Future Defect Prevention - An Industrial Case Study

cs.SE ·2026-10-14 ·Vehid Goruslu
similarity 0.59
Abstract

Context. Despite advances in software testing and quality assurance, defects continue to recur in industrial software systems. Defect data are routinely collected in issue-tracking systems, but their use is largely confined to detection, reporting, and short-term resolution, leaving the knowledge embedded in historical defects and their fixes under-exploited for prevention and long-term improvement. Objective. This paper investigates how historical defect and defect-fix data can be analyzed, with AI assistance under human oversight, to identify the human-error origins of recurring defects and to feed that understanding back into prevention and QA improvement. Method. We developed an AI-assisted, human-in-the-loop analytics approach that classifies defects by failure type and by human-error origin across the SDLC, determines how each could have been prevented, and synthesizes engineer-focused capability-building insights. It was deployed and evaluated in a large software company through an industrial case study, using practitioner surveys and interviews, classification-reliability checks, and an effort-normalized analysis of defect density. Results. Practitioners and independent review judged the classifications accurate; the outputs were rated useful and actionable; and defect density decreased after the introduction of the feedback, a tentative association we interpret as consistent with, but not proof of, a positive effect, given the study's uncontrolled pre/post design. Conclusion. AI-assisted, human-governed analysis of defect-fix data can help shift organizations from defect detection toward prevention, continuous learning and improvement.

0.25

LLM Ensemble Fault Classification for Automotive HiL Validation

cs.SE ·2026-08-11 ·Hamza Ouarrad, Mohammad Abboush, Andreas Rausch
top venue similarity 0.58
Abstract · 8 pages, 3 figures. Accepted at the 23rd Workshop on Model Driven Engineering, Verification and Validation (MoDeVVa 2026), co-located with M…

Automotive HiL validation generates large multivariate test recordings whose analysis remains challenging due to manual review effort, rule-based limitations, and the need for explainable diagnostic decisions. Recent machine-learning and deep-learning approaches have improved fault diagnosis, but they often require large labelled datasets, generalise poorly across operating conditions, and provide limited insight into their predictions. This paper proposes an explainable multi-LLM ensemble framework for sensor-level fault classification in automotive validation. The framework uses compact evidence representations of fault-injection recordings and combines the outputs of heterogeneous large language models to improve diagnostic robustness, ranking quality, confidence reliability, and interpretability. The approach is evaluated on gasoline-engine and electric-vehicle HiL systems across three driving settings and ten single-fault classes. Among the individual models, Mistral Small~24B provides the strongest overall single-model trade-off, achieving 0.903 Top-1 accuracy, 0.887 MCC, and the lowest Brier score of 0.102. The final Top-3 ensemble combines Mistral Small~24B, Qwen2.5~32B, and Phi-4~14B using confidence-weighted voting, improving the scenario-averaged results to 0.917 Top-1 accuracy, 0.913 macro F1, and 0.902 MCC, while also providing the best calibration among the tested ensemble strategies. A Top-5 ensemble does not improve over the Top-3 configuration, indicating that model complementarity is more important than ensemble size. The results show that coordinated multi-LLM reasoning can support robust, calibrated, and engineer-interpretable fault classification for automotive HiL validation.

0.35

AI-Based End-to-End Automotive Design and Software Development for SDVs

cs.SE cs.AI ·2026-10-21 ·Nenad Petrovic, Alois Knoll
published similarity 0.56
Abstract

This paper explores the potential of Artificial Intelligence (AI) to enable end-to-end automotive design and software development for Software-Defined Vehicles (SDVs). Based on our contributions within the Central Car Server (CeCaS) and follow-up EU-funded projects, we present an AI-driven toolchain integrating Large Language Models (LLMs) with Model-Driven Engineering (MDE) to automate the transformation of natural-language requirements into system architectures, executable software, and validation artifacts. The approach emphasizes traceability, consistency, and compliance with safety-critical automotive development requirements. Validation is performed on a centralized vehicle computing platform supporting adaptive cruise control and emergency braking, tested in Hardware-in-the-Loop (HiL) and Software-in-the-Loop (SiL) environments. We discuss current capabilities, practical limitations, and future directions toward increasingly automated, reliable, and quality-assured automotive software development with minimal human intervention.

0.41

Reliability Engineering for AI Systems: Challenges, Methods, and Directions

cs.AI stat.AP ·2026-09-28 ·Rong Pan, Yili Hong, Min Xie
benchmark similarity 0.56
Abstract

AI reliability concerns whether an AI system performs its intended function dependably over a stated period and under stated operating conditions, with clear evidence. As these systems become more autonomous, this function should include more than only a correct output. Retrieval, memory, tool use, permissions, human oversight, and interactions among systems must operate consistently and safely, and, for a generative AI, so must the reasoning process that produces the output. Average benchmark accuracy only measures system capability; it does not quantify this broader reliability claim. This paper adapts established reliability engineering methods, from failure definitions and operational envelopes to FMEA, accelerated testing, field monitoring, and reliability growth, to AI systems. A four-level diagnostic framework classifies failures as component, operational-loop, agentic-conduct, or network and governance failures. Test, evaluation, verification, and validation (TEVV), sequential monitoring, and FRACAS create and refresh evidence. The SMART framework provides statistical guidance for measurement, analysis, assessment, and test planning, and the NIST's AI risk management framework provides organizational guidance for governance, evaluation, monitoring, and mitigation. Three cases are discussed to illustrate these programs. It is shown that while the established reliability engineering principles can still provide a usable foundation, new measurements and safety guardrails are urgently needed as these systems are self-evolving at a fast pace.

0.14

Formal Analysis of a Fault-Tolerant Extension of the ROSACE Flight Controller

cs.SE cs.SC ·2026-11-17 ·Marco Bozzano, Alessandro Cimatti, Moussa Fall, Stefano Tonetta
publishedbenchmarkcode/data similarity 0.55
Abstract · In Proceedings FMAS 2026, arXiv:2611.21375

Autonomous avionics controllers must satisfy stringent real-time functional and performance require- ments even in the presence of run-time faults. In this paper, we analyze ROSACE, an open-source control engineering benchmark for flight control systems, focusing on its run-time resilience against faults such as sensor noise and sensor failures. We extend the benchmark with an FDIR (Fault Detection, Identification and Recovery) component based on Triple Modular Redundancy. Our approach builds upon COMPASTA, a tool for Model-Based Design developed on top of the TASTE tool by ESA. COMPASTA enables the modeling and injection of faults into a system's design. Moreover, it implements a simulation-based verification and validation workflow, automating the generation and execution of a set of test cases, based on a user-defined notion of coverage. Using COMPASTA, we construct a set of formal models, with faults and with FDIR, and we formally analyze their impact on the system properties. The analysis is based on the coupling of the formal models with the Simulink models used for the simulations.

0.54