CS Paper Radar

Characterizing Cross-Relay Generalization Failure in Machine Learning-Based Power System Protection

Nithin Pradeep Nadayil Kizhakkethil, Julian Oelhaf, Christian Bergler, Andreas Maier, Siming Bayer

cs.LG ·submitted 2026-10-20 15:13 UTC · v1
published

Abstract

Machine learning-based fault detectors are often evaluated at the relay locations used for training, although local voltage and current measurements vary across power networks. This work investigates cross-relay generalization for binary detection of faults anywhere in the network using local measurements, rather than line-selective tripping. Using 50 ms windows from a simulated 110 kV transmission network, we evaluate a multilayer perceptron across all 64 source target combinations of eight relays and five random seeds. A shared simulation-level split and source-only threshold selection provide a controlled held-out evaluation without target-specific calibration. Despite nearly unchanged mean F1-scores (0.730 within-relay versus 0.734 cross-relay), the mean false-positive rate (FPR) increases from 0.088 to 0.341, while the mean false-negative rate decreases from 0.399 to 0.318. The change is strongly directional: models trained at Bus 2 relays yield a mean FPR of 0.670 at Bus 1 and Bus 3 relays, compared with 0.096 for cross-relay transfers within the same bus group. Linear probing indicates that learned representations retain coarse bus-group information, while exploratory class-conditional latent distribution analysis shows no consistent monotonic association with transfer errors. These findings demonstrate that aggregate metrics can conceal location-structured changes in false-alarm behavior, highlighting the importance of evaluating error types separately when assessing cross-relay generalization.

Comment
6 pages, 3 figures. Accepted for presentation at the IEEE Power & Energy Student Summit (PESS 2026), Karlsruhe, Germany

Closest papers

Nearest neighbours by meaning across the whole index.

A Systematic Evaluation of Machine Learning Methods for Fault Detection and Line Identification in Electrical Power Grids

cs.LG eess.SP ·2026-09-15 ·Julian Oelhaf, Georg Kordowich, Paula Andrea Pérez-Toro, Tomás Arias-Vergara, Andreas Maier +2 more
publishedhighly cited similarity 0.73
Abstract · Accepted at ICASSP 2025. 5 pages, 4 figures. Published version: DOI 10.1109/ICASSP49660.2025.10890544

The integration of renewable energy sources into the electrical grid introduces complex challenges in fault detection and coordination of grid recovery mechanisms. Traditional relay protection systems, which operate based on static rules and predefined thresholds, are inadequate for addressing these challenges, particularly in detecting and isolating faults such as short circuits. Consequently, the conventional methodologies applied to electrical network protection frequently fail to achieve optimal performance in fault detection, especially in terms of adherence to safety standards and the selective limitation of damage. Recent research indicates that machine learning (ML)-based approaches can effectively tackle these issues; however, variations in grid configurations and analysis windows have impeded consistent comparative assessments. In this study, we assess the efficacy of various ML models in detecting electrical faults and pinpointing defective transmission lines within a 10 ms measurement interval - a critical time-frame for real-time operational viability, for the first time. The most effective model attained an F1 score of 0.991 +/- 0.018 and demonstrated a processing time of 0.342ms +/- 0.509ms.

0.52

Protection-Oriented Assessment of Machine Learning-Based Fault Detection in Power Systems

cs.LG eess.SP ·2026-11-13 ·Julian Oelhaf, Georg Kordowich, Changhun Kim, Christian Bergler, Andreas Maier +2 more
publishedhot direction similarity 0.73
Abstract · 9 pages, 3 figures, 5 tables. Accepted for oral presentation at the 21st International Conference on Developments in Power System Protection…

Machine learning (ML)-based fault detectors are increasingly proposed for power system protection, yet aggregate metrics can obscure the consequences of false and missed trips. We present a protection-oriented assessment of three ML detectors and a sequence-current reference on PROTECT-90, comprising 9,022 simulated fault episodes on a 90 kV double-circuit line. The assessment combines five window conditions, causal window-end labelling, episode-grouped cross-validation, and operating-point selection under a fixed security budget. It reports security, dependability, detection timing, decision margins, and episode-level nuisance assertions. Gradient-boosted trees, a convolutional neural network, and logistic regression achieve AUC-ROC values above 0.999, yet at a security budget of $10^{-3}$ their developed-fault missed-trip rates range from 0.007% to 1.99%. Post-clearing false-trip rates of approximately 0.12-0.16% per window coexist with nuisance assertions in 11.3-19.1% of episodes. Among the analysed physical descriptors, missed trips are most consistently associated with higher fault resistance within the sampled range. An analytical current proxy suggests a fault-current gradient only partly reproduced by waveform-derived current (rank correlation 0.62). No single topology state or faulted phase consistently dominates within the sampled domain. Because PROTECT-90 contains fault episodes only, security is assessed on pre-fault and post-clearing windows rather than independent non-fault disturbances.

0.34

EvEMTBench: An Open Benchmark for Machine Learning in Power System Protection

eess.SY cs.LG ·2026-09-23 ·Julian Oelhaf, Georg Kordowich, Christian Bergler, Andreas Maier, Johann Jäger +1 more
benchmarkcode/data similarity 0.70
Abstract · 15 pages, 3 figures. Supplementary information included. Code: https://github.com/EvEMTBench/evemtbench-benchmark

Studies of machine-learning-based power system protection are difficult to compare because task definitions, measurement access, data partitions, metrics, and generalization conditions often differ. EvEMTBench addresses this gap with an open, executable, and versioned benchmark that fixes these evaluation choices while leaving model design open. Across four grids spanning 20-345 kV, it defines 12 protection and event-analysis functions instantiated as 24 scored tasks and supports structured evaluation across observability conditions, predefined distribution shifts, and zero-shot and fine-tuned cross-grid transfer. Committed partitions, leakage controls, and reproducible reporting provide a common basis for comparing future methods. A reference evaluation spanning trivial, conventional, feature-based, and deep-learning baselines shows that wider observability is not uniformly beneficial, shifted conditions can reveal failures not apparent in-distribution, and cross-grid transfer is substantially stronger for fault detection than for fault localization. Protection-relevant diagnostics identify failure modes not apparent from primary metrics alone. EvEMTBench therefore makes generalization in machine-learning-based protection an explicit and reproducible evaluation problem.

0.22

A Standardized Framework for Machine Learning in Power System Protection

cs.LG cs.AIeess.SP ·2026-08-20 ·Julian Oelhaf, Georg Kordowich, Paula Andrea Pérez-Toro, Christian Bergler, Johann Jäger +2 more
publishedbenchmarkcode/data similarity 0.67
Abstract · 32 pages, 4 figures, 26 tables. Code: https://github.com/julianoelhaf/protection-eval-framework. Dataset: PROTECT-90, doi:10.5281/zenodo.211…

Studies of machine-learning-based power-system protection increasingly report near-perfect scores, yet the meaning of those scores depends strongly on the evaluation setting. Protection task, physical scope, measurements, timing, targets, preprocessing, and validation often vary jointly and remain incompletely specified. This paper proposes a standardization-oriented framework that treats evaluation design as part of the scientific contribution. It defines seven required study dimensions: protection objective, physical scope, observability, timing and decision windows, targets and sample validity, validation protocol, and evaluation outputs. The framework is instantiated in a bounded case study on the public PROTECT-90 electromagnetic-transient benchmark, comprising 9022 simulated episodes from a 90 kV double-line topology, for onset-conditioned fault classification and localization. Under centralized sensing, simulation-metadata-aligned 20 ms windows, and episode-grouped validation, a multi-layer perceptron (MLP) achieved a five-fold mean macro-averaged F1 score of 0.991 +/- 0.001 for classification and a localization mean absolute error of 10.20 +/- 0.25% of line length (mean +/- std across episode-grouped folds). Extending the decision horizon to 50 ms preserved this task-dependent performance asymmetry, while reduced observability approximately doubled the MLP localization error but had little effect on classification. A synchronized two-ended conventional locator outperformed the learning locators under its richer clean information set, and measurement degradation showed that clean predictive performance did not determine robustness. The framework turns evaluation assumptions into explicit, reproducible evidence and provides a basis for more comparable, auditable evaluation and future certification-oriented assessment of machine-learning protection functions.

0.77

Multi-Library Feature Engineering and Statistically Validated Classification for Power Swing Detection and Stability Assessment

eess.SP cs.LG ·2026-11-16 ·Pallav Kumar Bera, Samita Rani Pani, Ranjeeta Patel
published similarity 0.65
Abstract · Accepted at Annual Conference of the IEEE Industrial Electronics Society (IECON) 2026; 52nd Edition; Doha, Qatar

Faults occurring during power swings, and the evolution of a swing into an unstable condition, cause distance-relay maloperation and cascaded outages. This paper identifies faults, faults during power swings, and power swings, and distinguishes stable from unstable swings in a 9-bus WSCC system modeled in PSCAD/EMTDC. Three time-series feature libraries (tsfresh, TSFEL, and tsfeatures) are applied to the 3-phase relay voltages and currents, giving 3286 statistical, temporal, spectral, and fractal descriptors per event window. A two-step feature selection (Boruta all-relevant filtering followed by RFECV- and SHAP-guided minimal-optimal pruning) reduces the feature set to 18 compact, non-redundant features. 14 classifiers spanning linear, kernel/instance-based, probabilistic, neural, and tree-ensemble/boosting families are then benchmarked under repeated stratified cross-validation, with model differences assessed by a Friedman/Nemenyi omnibus-and-post-hoc test and a Nadeau--Bengio corrected paired t-test. Rotation Forest (ROF) identifies the three event types with 99.98% and SVM-RBF distinguishes stable from unstable swings with 97.97% balanced accuracy. The results show that automated, library-driven feature extraction combined with rigorous statistical model comparison achieves near-perfect discrimination while retaining feature-level interpretability through SHAP importance.

0.35

Directionally Supervised Neuromorphic Protection for Distribution Systems Using Local Measurements

eess.SP ·2026-11-12 ·Saurabh Prabhakar, Yubo Song, Pascal Schirmer, Bijaya Ketan Panigrahi, Subham Sahoo
similarity 0.65
Abstract · Submitted to IEEE Transactions on Smart Grid

Reliable protection under varying power system operating conditions requires rapid fault detection while preserving selectivity against external faults and non-fault disturbances. This requirement is particularly challenging in communication-free schemes, where relays must determine trip conditions solely from local electrical measurements. To address this challenge, this paper proposes a directionally supervised neuromorphic protection scheme that integrates disturbance-responsive leaky integrate-and-fire dynamics with a direction-dependent firing threshold to distinguish internal faults from reverse-direction events. The unified relay architecture combines local disturbance detection, directional discrimination, and spike-based trip generation without requiring inter-relay communication. Its performance is evaluated under grid-connected and islanded conditions using a ring microgrid and the IEEE 13-node distribution feeder. Across 240 evaluated fault scenarios, the framework achieves median operating times of 5.4--7.6 ms across the two networks, with 78% of cases operating within 10 ms and 95% within 20 ms. It also maintains protection security during load disturbances, while fault-resistance analysis confirms its response under high-resistance faults. Controller-hardware-in-the-loop (CHIL) experiments using an RTDS simulator and an external dSPACE controller demonstrate trip-command assertion in approximately 10 ms and 5 ms for the tested single-line-to-ground and three-phase-to-ground faults, respectively. The results establish real-time feasibility within the evaluated conditions and characterize operating-time variability across fault types and resistance levels for different network configurations.

0.38

A Hybrid Two-Stage Machine Learning Pipeline for Fault Detection and Classification in Power Transmission Systems

eess.SY cs.LG ·2026-08-24 ·Sahil Manikshete, Atharva Gujarathi, Thanh Long Vu, Akhtar Hussain, Van-Hai Bui
benchmarktheory similarity 0.61
Abstract · 21 pages

Rapid and accurate fault detection in high-voltage transmission networks is essential for grid reliability and equipment protection. Transmission fault datasets are frequently imbalanced, and certain fault types produce electrical signatures that fall within the normal operating envelope, causing single-model classifiers to fail on safety-critical cases. This paper proposes a hybrid two-stage machine learning pipeline that decouples detection from classification. Stage 1 combines an Isolation Forest anomaly detector with an optional supervised binary detector through an OR-fusion rule; the supervised branch is allocated automatically during training for any fault class the anomaly detector cannot resolve, and is omitted when no such class exists. Stage 2 applies a Random Forest multiclass classifier only to samples flagged by Stage 1. Feature engineering is expressed as a per-measurement-point operator mapping six raw channels to eighteen features, including zero-sequence symmetrical components derived from Fortescue's theorem, yielding 18L features for L measurement points. On the TLFaultDataset, the pipeline raises Line-fault end-to-end accuracy from 31.3% to 95.8%. On an independent single-point dataset, the same framework attains 97.25% end-to-end accuracy across all classes including normal operation, exceeding the TLFed federated benchmark of 94.84% without GPU or federated infrastructure, at 0.05 ms per sample on CPU. Ablation on both datasets shows zero-sequence features resolving the three-phase versus three-phase-to-ground ambiguity, raising the F1-score of that class pair from 0.39 to 0.997. The direction of the zero-sequence signature is found to be system-dependent, motivating a learned decision boundary in place of a fixed relay threshold.

0.28

One-Cycle Fault Classification and Faulted-Line Identification on the PROTECT-90 Dataset: An Initial Application Benchmark

eess.SY cs.LG ·2026-10-02 ·Emad Abukhousa, Abdulaziz Qwbaiban, Saman Zonouz, A. P. Sakis Meliopoulos
benchmarkcode/data similarity 0.60
Abstract

Open electromagnetic-transient datasets are beginning to make reproducible learning-based protection studies possible, but the practical use of these datasets still requires application-level benchmarks that define timing, sensing, and validation assumptions. This paper presents an initial application benchmark on the recently released PROTECT-90 dataset for two protection-oriented tasks: fault-type classification and discrete faulted-line identification. A compact one-dimensional convolutional neural network (CNN) is evaluated using post-inception windows of 0.25, 0.5, 1, and 2 cycles under strict episode-wise splitting. A non-convolutional multilayer perceptron (MLP) is also trained as an architecture-control baseline. The results show that both tasks are nearly saturated under full observability, with one-cycle test accuracies of 99.84% for fault type and 100.00% for line identification. The main performance variation appears under reduced observability: current-only inputs preserve line identification accuracy at 100.00%, whereas voltage-only inputs reduce line identification accuracy to 53.09% with the CNN and 50.57% with the MLP. This indicates that the limiting factor is measurement information rather than neural architecture. Additional stratified checks show stable performance across topology states and fault-resistance bins, while CPU inference contributes only 0.528 ms to the one-cycle total decision time of 20.53 ms.

0.29