CS Paper Radar

Structuring occupational accident narratives for cross-sector safety analysis: Transferability of accident-process role classification

Aho Yapi, Pierre Latouche, Arnaud Guillin, Yan Bailly

cs.CL ·submitted 2026-09-18 17:59 UTC · v2

Abstract

Introduction: Occupational accident narratives describe work situations, unfavourable conditions, accident events, and consequences, but differences in terminology and reporting practices hinder systematic analysis across sectors and organisations. This study examined whether a model developed in one occupational sector could identify the same accident-process information in unseen sectors and reporting environments. Method: French accident narratives were segmented into factual units and expert-annotated as work situation, explicitly reported unfavourable condition, accident event or deviation, or reported consequence. Models were developed on 42,244 factual units from 6,040 construction-sector narratives and evaluated without retraining on metallurgy, chemistry-plastics, and an independently collected company corpus. We compared a TF-IDF-based lexical model, frozen pretrained text representations, and task-adapted pretrained models. Results: Average balanced accuracy was 75.0% for TF-IDF, 76.9% for frozen pretrained representations, and about 85.7% after task adaptation. Across repeated runs, leading adapted approaches showed similar overall performance, with no method consistently outperforming the others. Performance was lower and more variable on the company corpus, where transfer also involved a different reporting environment and data source. Conclusions: Accident-process roles learned from construction narratives remained identifiable in other sectors and an independent organisational setting. Practical Applications: The framework can support assisted coding, expert review, cross-sector analysis, and prevention-oriented analysis of large accident-report collections.

Closest papers

Nearest neighbours by meaning across the whole index.

AISA: AI Safety Assistant Framework for Continuous Improvement of Highway Construction

cs.CL ·2026-08-17 ·Mason Smetana, Trevor Neece, Lev Khazanovich
similarity 0.71
Abstract · 17 pages, 5 figures

Job Safety Analysis (JSA) and pre-task planning can benefit from prior incident records, yet historical accident data is often stored as unstructured narratives that are difficult to consult at the point of planning. A novel framework centered on large language models (LLMs) for highway construction safety reporting and planning is proposed as a foundation for future agentic applications, prioritizing deterministic, local inferencing. The first aim is to enable classification and quality scoring of incident narratives for existing and future reporting purposes. The second is to evaluate retrieval of relevant historical accidents, related imagery, and trusted industry documents for incorporation into daily safety plans. Neural probes were trained to classify incidents along four multiclass and two binary Occupational Injury and Illness Classification System (OIICS) fields and to derive an overall quality score, evaluated on a test set of over 15,000 narratives and a held-out set of 100 author-labeled records, benchmarked against a majority-vote LLM ensemble. The retrieval of historical accidents, reference imagery, and industry documents was benchmarked across embedding models using standard information retrieval metrics. OIICS classification reached 75% held-out accuracy, though the two binary flags were degenerate. The quality score, while meaningful on one database, was distorted on out-of-distribution fatalities in the held-out dataset. Accident retrieval recovered relevant incidents far above chance, performing best on lexically distinct construction activities. On document question answering, an open-weight decoder embedding model surpassed proprietary models. Overall, this work provides a new framework rooted in local inferencing and text embedding models for future agentic applications, with emphasis on bridging external data to JSA reports.

0.28

Semantic Causal-Factor Inference from Aviation Incident Narratives Using A Variational Autoencoder with Cosine-Similarity-Based Reconstruction

cs.AI ·2026-10-03 ·AZIIDA NANYONGA1, HASSAN WASSWA, UGUR TURHAN, KEITH FRANCIS JOINER, GRAHAM WILD
similarity 0.64
Abstract

Aviation accident and incident investigations generate extensive unstructured textual information containing evidence relevant to the causes and contributing factors of safety occurrences. Automatically extracting such information is challenging because causal evidence may be distributed across long and complex investigation narratives. This study proposes a semantic causal-factor inference framework combining natural language processing with a variational autoencoder (VAE) to learn the relationship between aviation investigation narratives and expert-reported probable causes. Investigation narratives and their corresponding probable causes are transformed into numerical representations, after which the encoder maps narrative representations to a probabilistic latent space. The decoder estimates representations of the corresponding probable causes and is trained using an objective that combines Kullback-Leibler divergence with cosine-similarity-based semantic reconstruction. The framework was evaluated using 20,919 finalized U.S. National Transportation Safety Board investigation reports from 2005 to 2020. On the held-out test set, the predicted and expert-reported probable-cause representations achieved a mean cosine similarity of 0.786 (SD = 0.120). The predicted representations also yielded interpretable terms associated with causal information in the reports. The results demonstrate the potential of probabilistic latent representation learning for AI-assisted extraction of causal information from aviation safety narratives while retaining expert investigation as the basis for formal causal determination.

0.14

Decomposed Relation Extraction from Aviation Accident Narratives: Evaluating Transformer-Based Safety Knowledge Discovery under Relational Sparsity

cs.CL ·2026-10-19 ·Aziida Nanyonga, Mustafa Tahsin Yilmaz, Saad Ezzini, Ugur Turhan, Graham Wild
benchmark similarity 0.63
Abstract

Relation extraction from aviation accident narratives is challenging because genuine safety relations are sparse among possible concept pairs. This study evaluates a decomposed framework separating relation classification from concept identification. A benchmark of 300 National Transportation Safety Board (NTSB) narratives containing 2,524 unique concept-text nodes was expanded into 21,038 ordered candidate pairs, with only 248 (1.18%) representing RESULTS_IN or CONTRIBUTES_TO relations. A TF-IDF Linear Support Vector Machine (SVM) and weighted RoBERTa classifier were developed using an independent graph-annotated resource with hard-negative sampling and frozen before sealed evaluation. Development Macro-F1 scores of 0.4738 (SVM) and 0.4809 (RoBERTa) declined to 0.2741 and 0.2420 under exhaustive evaluation, with Matthews correlation coefficients of 0.0128 and 0.0197. False positives dominated: the SVM recovered 96 reference relations but generated 7,018 false positives, while RoBERTa recovered 125 but generated 9,824. Post-test diagnostics revealed overlapping RoBERTa confidence distributions and false positives even within valid relation-category directions. Complementary end-to-end SmolLM2 evaluation identified missed and unsupported relations alongside malformed outputs. These findings demonstrate that known concept endpoints do not guarantee reliable relation extraction and that sampled-negative development performance may poorly reflect exhaustive evaluation. Controlling false positives and managing candidate-space sparsity remain essential for aviation safety knowledge extraction.

0.20

The record is part of the task: matched-record evaluation of text classifiers across maintenance, safety and recall reporting

cs.LG cs.CL ·2026-09-14 ·Hisham Ihshaish, Peter Mayhew, Tasnim M. A. Zayet, Ana Del Amo
similarity 0.61
Abstract · 37 pages (18-page article, 3 tables, 7 figures, plus a 19-page supplement with tables and figures numbered S1 onward)

Many operational cases are documented more than once, at different workflow stages and for different purposes, yet model evaluations normally select one of these records before model comparison begins. We treat that selection as part of the evaluation and compare matched records of the same cases under fixed labels and splits in three systems: GE Aerospace repair events, NASA ASRS safety reports and NHTSA vehicle recalls. The design fixes, for any decision a classifier would support, the records that existed and could be read when it was taken, and compares them retrospectively, as written. Across the three GE fields, for events whose label comes from parts transactions independently of the narratives, held-out macro-F1 ranged from 0.33 to 0.91. A difference of 0.46 separated the customer report, written before shop work, from the technician report, written after diagnosis but before the transaction that generates the label. That difference is substantially larger than the representation and architecture differences tested on the same events. The public systems showed different patterns: the NHTSA defect summary remained strongest under every model family tested, whereas the ASRS analyst synopsis outperformed the reporter narrative under learned sequence models but not under lexical baselines. Secondary analyses showed that some model comparisons were also record-dependent. Evaluations should be run on the information available at the intended decision point and should report how both the record and the label were produced.

0.35

Adapting a Large Language Model Crash-Severity Pipeline to Tennessee: Performance Across Sampling Strategies

cs.CE ·2026-09-24 ·Abhilasha Saroj, Pranav Govindu, Bharat Sharma, Usman Ahmed
similarity 0.60
Abstract

State crash databases differ in structure, coding, and injury-severity distributions, limiting direct reuse of predictive workflows across jurisdictions. This study adapts the SafeTraffic Copilot large language model (LLM) crash-severity workflow to a three-year Tennessee inventory of 624,392 crashes. Tennessee crash, roadway, vehicle, and person attributes were harmonized and converted into textual prompts while unavailable values were preserved rather than inferred. Llama 3.1 8B was fine-tuned using low-rank adaptation to classify five injury-severity categories. Random, county-based, and severity-balanced sampling strategies were evaluated using separate in-sample and unseen-test experiments; unseen-test experiments used 70/15/15 training, validation, and test splits. On their respective unseen test sets, random and county-based models achieved weighted F1-scores near 80%, but macro F1 remained below 47% and fatal-crash F1 below 27%, showing that strong aggregate performance can mask weak recognition of rare outcomes. Within its balanced evaluation population, the severity-balanced model achieved weighted and macro F1-scores of 57.7% and a fatalcrash F1-score of 65.9%, yielding more even class-level performance. Because each sampling strategy used a different test subset, crossstrategy differences are descriptive rather than controlled rankings. The results highlight the importance of interpreting LLM crashseverity performance together with sampling design, class balance, class-level metrics, and evaluation-population composition.

0.31

Advanced modelling and data analytics in aviation

cs.AI ·2026-08-14 ·Aziida Nanyonga
similarity 0.59
Abstract

The aviation industry characterized by its stringent safety standards has seen a growing need for innovative approaches to enhance safety measures. Despite the vast accumulation of aviation safety data over time, its full potential in predicting and preventing incidents has not been fully realized. This research addresses this gap by applying machine learning (ML) and natural language processing (NLP) techniques to analyze aviation safety data from Socrata, the Australian Transport Safety Bureau (ATSB), the National Transportation Safety Board (NTSB), and the Aviation Safety Network (ASN). By leveraging existing ML models, including deep learning and transformer-based architectures alongside NLP methods for mining aviation incident narratives, this study uncovers patterns contributing to safety related incidents such as accidents and near-misses. Additionally, it employs various topic modelling techniques to extract meaningful themes from unstructured safety reports, enhancing the interpretability of incident analysis. Causal inference techniques and interpretable AI frameworks are further explored to improve model transparency and trustworthiness. A key contribution of this work is the deployment of advanced ML methodologies in a structured aviation safety context, assessing their effectiveness and providing insights into their practical implementation. The findings offer valuable insights for aviation stakeholders, including regulators, airlines, and policymakers, by providing data-driven solutions that enhance incident analysis and decision making. Ultimately, this research supports the industry s ongoing efforts to minimize risks, improve passenger and crew security, and integrate AI driven methodologies into aviation safety management.

0.04

Toward Auditable and Calibrated AI for Dementia-Related Crash Severity Prediction: A Selective Deferral Framework to Support Human Review

cs.AI cs.HC ·2026-09-19 ·Gaurab Chhetri, Anika Baitullah, Subasish Das
similarity 0.59
Abstract · This is the author's preprint version of a paper accepted for presentation at HICSS 60 (Hawaii International Conference on System Sciences),…

Public crash databases increasingly support automated safety analysis, but crash severity prediction remains difficult to translate into public-sector decision workflows when models are evaluated primarily as ordinary classifiers. This study reframes dementia-related crash severity modeling as a decision-aware triage problem in which a system must classify crashes into no-injury/property-damage-only (O), minor or moderate injury (BC), and fatal or severe injury (KA), while also controlling outcome leakage, reporting severe under-triage, calibrating confidence, and preserving every raw prediction for audit. Using 4,781 Texas crash records with structured fields and police narratives, we evaluate structured, narrative, fusion, calibrated fusion, BERT-family, and local large-language-model baselines under a stratified 70/15/15 split. In the reported split, leakage-controlled Gemma obtains the highest observed macro-F1 (0.545; 95% bootstrap CI [0.507, 0.583]). The best calibrated fusion model obtains macro-F1 of 0.522 and expected calibration error of 0.033. Selective deferral improves performance among cases retained for automatic classification. At 70% coverage, macro-F1 rises to 0.573 and severity cost falls to 0.577, while deferred cases are treated as candidates for a proposed human-review process and are not further evaluated in the present experiment. The study contributes a reproducible, leakage-controlled, and uncertainty-aware evaluation framework for crash AI systems, emphasizing auditability and selective deferral rather than accuracy alone.

0.27

A Hybrid Framework for Explainable Aviation Risk: Causal Chain Extraction from Safety Reports with Physics-Based Failure Quantification and Interventional Analysis

cs.LG eess.SYstat.AP ·2026-10-26 ·Yutian Pang, Yifang Tian, Xinyu Zhao, Jueming Hu, John-Paul Clarke +1 more
similarity 0.59
Abstract · This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may…

Aviation authorities hold decades of accident reports whose narratives record how investigators reconstructed each accident. Computational work on these archives has mostly sorted reports into fixed taxonomies. In this work, we propose a hybrid framework in which a text-based corpus identifies the leading factors of a failure mode. Physics-based models then quantify how likely that failure is at a stated operating point. Ordered causal chains extracted from 56,202 American accident reports with a local large language model aggregate into a causation knowledge graph. On an evaluation sample of 2,000 reports, the chains recover 90.8% of investigator-flagged causes. Each physical factor whose drivers the record carries receives an occurrence model calibrated to the accident population. A leaky noisy-OR fitted to the corpus combines the factors into a failure mode probability that supports intervention. For engine power loss, the carburetor icing model separates 1,205 icing accidents from other accidents using temperature and dewpoint alone, reaching 0.61 in area under the receiver operating characteristic curve against a density ratio ceiling of 0.64. Routine weather hours give no valid control, because flying concentrates in fair weather. On later years the physics adds 0.03 to that area for occurrence and nothing for severity, so its value lies in interpretability. On four named accidents, abduction gives a probability of necessity of 0.29 to 0.50 for the heat omission, under monotonicity. The Fast Causal Inference algorithm does not license the largest backdoor adjustment on the learned networks.

0.22