Influence of nuisance variables on the PMU-based disturbance classification in power transmission systems

1.1 Disturbance classification in power transmission systems

The reliable operation of power transmission systems faces ongoing challenges by the integration of renewable energy sources, the utilization of active equipment (e.g. flexible AC transmission systems or high-voltage direct current links), and additional loads by active distribution systems. This reduces the available dynamic reserves of the system and increases the risk for critical grid situations. Disturbances like generator outages or line trips can lead to severe power supply disruptions and can cause system instabilities if not handled properly by suitable countermeasures (e.g. remedial action schemes like set point changes or load shedding [1]). Phasor measurement units (PMUs) provide high-resolution (typically between 10 and 50 frames per second) and time synchronized frequency, voltage, and current measurements from multiple sensors in the grid [2–5]. Some exemplary PMU frequency and voltage magnitude signals for a generator outage from a simulated power system (see Section 4.1) are given in Figure 1.

Figure 1:

PMU frequency and voltage magnitude signals for a generator outage (station 1D1) over 10 s.

The analysis of disturbance events from PMU signals have been investigated since many years and can be categorized into the tasks:

1. disturbance detection as the recognition of deviations from the normal operation,

2. disturbance identification as the recognition of disturbance types, and

3. disturbance localization as the recognition of disturbance locations.

Whereas the disturbance detection can be seen as a pre-analysis step to detect outliers or anomalies for subsequent classification tasks. This study focuses on the PMU based disturbance classification, which uses machine learning techniques to simultaneously identify and locate disturbance events in power systems. Most of the disturbance classification approaches derive features from principle component analysis [6–8], time-frequency transformations (e.g. S-Transform or wavelet transform) [8–10], or from the analysis of minimum volume enclosing ellipsoids [11]. The subsequent classification is implemented by decision trees [7, 12, 13], support-vector-machines [9, 14, 15], or k-nearest-neighbors [10, 16]. In addition, deep learning models have been investigated including convolutional [17, 18], recurrent [19], and spiking neural networks [20]. Most of the studies simulate PMU signals from a dynamic power system model (e.g. the IEEE 39 bus reference system [21]), whereas measured PMU signals are only used in some investigations.

Precise information about the location and type of a disturbance allows an assignment of the current grid situation to one of the precalculated contingencies, which enables the activation of appropriate countermeasures to restore a stable system state and power supply [2, 3]. For this purpose, the disturbances should be detected as early as possible and with current methods this can be achieved below 2 s after the start of the event. Any misclassifications (e.g. the detection of a short circuit on another line) and associated, incorrectly triggered countermeasures can have severe consequences for the system operation and may lead to a deterioration in the power supply or the stability of the grid. The investigation of the model robustness as well as the handling of the model uncertainties are therefore necessary to enable a practical use of disturbance classification methods in the power system operation.

1.2 Limitations of current approaches and nuisance variables

Disturbance classification algorithms rely on a sufficient set of examples for the disturbance events. As also mentioned in [22], measured PMU signals may not contain enough examples for all event types or locations and need additional effort to extract and label the relevant time ranges. Dynamic simulations provide a useful way to create PMU signals for specific disturbance events and operational points. As a downside, they rely on an accurate dynamic power system model and can only simulate power system dynamics ignoring effects like measurement noise or naturally changing operational conditions. Furthermore, only a specific subset of disturbance events (called critical contingencies) are relevant for the power system operation and require an activation of countermeasures. Especially for large power systems, a simulation of all possible disturbance events may therefore become infeasible.

In this work, we assume that the disturbance classification model is trained on simulated PMU signals X ^(S) and is applied on measured PMU field data X ^(P). To achieve a robust model design, the following nuisance variables can be taken into consideration:

1. measurement noise z ̲ R that results from errors of the data acquisition in the PMU sensors (e.g. errors in phasor estimation or filter effects),

2. model errors z ̲ M that result from inaccuracies of the dynamic power system model (e.g. wrong machine parameters, substitute models),

3. unknown operational points z ̲ O as operational points that are not included in the training data set, and

4. unknown disturbance events z ̲ U as disturbance events that are not included in the training data set.

Current classification approaches do not address these nuisance variables properly. Consequently, misclassifications are to be expected for these models, which will lead to falsely activated countermeasures and induce a severe threat to the power system operation. A corresponding overview of the nuisance variables is given in Table 1.

1.3 Main contributions and paper organization

This paper investigates the influence of multiple nuisance variables on the performance of disturbance classification models, which are trained on simulated PMU signals from a modeled power transmission system. For that, several test scenarios are considered to evaluate the robustness of different classification approaches in the presence of measurement noise, unknown operational points, and unknown disturbance events in the test data. With that, the disturbance classification approaches are evaluated under more realistic test conditions, which gives a better estimate of their generalization performance on PMU field measurements. As already introduced in [23–26], this study uses Siamese Sigmoid Networks (SSN) as a special type of a recurrent neural network based classification model with integrated rejection capability to simultaneously identify and locate disturbance events from PMU frequency and voltage signals. These SSNs are compared with different closed-set and open-set classification models.

The paper is organized as follows. Section 2 describes the disturbance classification problem and the nuisance variables from a systems theory point of view. Section 3 presents the classification approaches including Siamese Sigmoid Networks and several benchmark models as well as the used evaluation metrics. Section 4.1 describes the dynamic power system model and the simulated database for the experiments. Section 4.2 introduces the test scenarios to evaluate the different nuisance variables. Section 4.3 presents and discusses the classification performance results. Section 5 summarizes the results and gives an outlook on possible future work.

3.1 Siamese Sigmoid Networks

As introduced in [23, 25], SSN are a special type of recurrent neural networks to simultaneously identify and locate disturbance events from PMU signals. This means the class label y contains information about the disturbance type y _Type and the disturbance location y _Loc, such that y = y Type ; y Loc . The input data X consists of PMU frequency and voltage signals over T timesteps from N PMU sensors in the grid (no full PMU observability required), such that X ∈ R T × N ⋅ 2 . The basic architecture is given in Figure 3.

Figure 3:

Siamese Sigmoid Network architecture.

SSNs estimate the class affiliation of a query instance X _Q by the comparison with multiple support instances X _S of known disturbance events C ₁ … C _K. Therefore, a fixed number of support instances is selected for each class to create the support set S. Gated recurrent units (GRU) with additional attention mechanisms are used in the encoder network to compute the low-dimensional feature vectors l ̲ Q and l ̲ S . The Manhattan distance d between the support and query feature vectors is converted into a matching probability p _Pos and into a non-matching probability p _Neg with a double-sigmoid function based classifier, such that:

The weights w Pos / Neg C and biases b Pos / Neg C of the sigmoid functions are learned class-wise and are used to compute the positive threshold distance d Thr , pos C and the negative threshold distance d Thr , neg C for each class C:

The difference between the positive and negative threshold distance corresponds to the margin m C = d Thr , neg C − d Thr , pos C , which controls the separation between the positive and negative samples per class. To achieve low positive threshold distances and high negative threshold distances, the following additional margin loss is proposed:

Depending on the classifier results, the two cases can arise:

1. the support and query instance belong to the same class if d → 0, p _Pos → 1, and p _Neg → 0, or

2. the support and query instance belong to different classes if d → ∞, p _Pos → 0, and p _Neg → 1.

With that, the class affiliation of the query instance y ̂ is estimated from the maximum of the matching probabilities p _Pos of all support instances in the support set S following a simple one-against-all strategy. The query instance is rejected if the maximum matching probability is below the rejection probability p _R = 0.5, such that:

3.2 Benchmark models

The SSN model is compared with different benchmark models to evaluate the classification performance. The first benchmark model SSFD-SVM is a conventional disturbance classification approach from [27], which combines a strongest signal and fault detection (SSFD) algorithm for the localization task with a support-vector-machine (SVM) for the identification task. This approach requires a full PMU observability of the grid and is only applicable for closed-set classification problems. The second benchmark model Triplet-1NN is oriented at [28] and applies a triplet network architecture with hard negative online mining. This is a special training strategy for triplet networks and requires a sampling of negative examples beyond the margin. The feature vectors are generated with the same GRU encoder network as in Section 3.1 and the classification is performed with a 1-nearest-neighbor method (1NN). A full PMU observability is not required but again this approach is only applicable for closed-set classification problems. To solve open-set classification tasks, the third benchmark model GRU-DOC combines the GRU encoder network from Section 3.1 with a deep-open classifier (DOC) from [29]. Like the second benchmark model, a full PMU observability is not required.

3.3 Evaluation metrics

The classification models are evaluated for different classification tasks considering known and unknown disturbance events. For that, the class configuration of the K known classes will be extended with an additional container class C _U, which includes the rejections of the samples of all unknown classes, such that C ̄ = C 1 , … , C K , C U . Based on this, the following performance metrics are chosen according to [30]:

1. the classification accuracy of the known classes η _ACC,K,

2. the classification accuracy of the known and unknown classes η _ACC,

3. the F1-score with macro averaging of the known classes η _F1,K, and

4. the F1-score with macro averaging of the known and unknown classes η _F1.

Thus, the true positives of the unknown classes are excluded from the calculations of η _ACC and η _F1. The macro averaged F1-score corresponds to the arithmetic mean of the class-specific F1-scores.

4.1 Simulation model and database

As already described in [26], a generic power transmission system is simulated to extract the PMU frequency and voltage magnitude signals at all 400 kV busbars. The power system model includes 21 substations and 68 transmission lines with additional voltage regulators, power system stabilizers and protection systems. In total, the available database consists of 440 simulated contingencies at 7 operational points (OP1 to OP7) with different generation and load conditions. These operational points result from optimal power flow calculations and are given in Table 2. The grid topology is shown in Figure 4. The following disturbance types are considered within this study:

1. base load power plant outages (BLPP),

2. peak load plant outages (PLPP),

3. low (<50 % of the maximum capacity) and high (>50 % of the maximum capacity) load losses (LL ^Low and LL ^High ),

4. low (<50 % of the maximum capacity) and high (>50 % of the maximum capacity) PV losses (PV ^Low and PV ^High ),

5. line trips (L), and

6. short circuits at 10 % (SC ¹⁰), 50 % (SC ⁵⁰), and 90 % (SC ⁹⁰) line length.

Figure 4:

Topology of the simulated power system.

For the experiments, we considered 54 known and 60 unknown disturbance events with randomly chosen disturbance types and locations in the grid. We sampled the PMU frequency and voltage signals at the substations with a fixed window size T of 50 timesteps and a reporting rate of 25 f.p.s. from the 10 s post-disturbance records.

4.2 Test scenarios

To evaluate the influence of the nuisance variables on the disturbance classification results, we define the test scenarios listed in Table 3. For each scenario, we distinguish between a limited (“a”) and a full (“b”) observability of the grid with PMUs. In the “a”-scenarios the input signals are taken from 13 randomly selected PMUs of the grid. In the “b”-scenarios the input signals are taken from all 21 PMUs in the grid. Scenarios S.1a and S.1b investigate the influence of additional measurement noise with 20 dB and an AR-coefficient of AR = 0.2 in the test data. The measurement noise is generated by an autoregressive error model from [31]. Scenarios S.2a and S.2b investigate the influence of unknown operational points and scenarios S.3a and S.3b investigate the influence of unknown disturbance events in the test data. These unknown disturbance events correspond to new combinations of known disturbance types at known or unknown disturbance locations. Scenarios S.4a and S.4b include all nuisance variables.

4.3 Results and discussion

The experimental results for the different test scenarios correspond to the average values of the 10-fold cross-validation runs after hyper-parameter tuning. We used early stopping as regularization technique. Table 4 shows the accuracies and F1-scores of the classification models SSN, Triplet-1NN, and GRU-DOC for the scenarios S.1a and S.1b. Due to the limited PMU observability in the “a”-scenarios, the SSFD-SVM benchmark model is only applied for the “b”-scenarios (see also Section 3.2). As it can be seen, the accuracy values and F1-scores decrease between roughly between −4 % and −5 % due to the measurement noise in the test data. In case of a limited PMU observability (scenario S.1a), the SSN model achieves the highest test accuracies (93.19 %) and test F1-scores (93.05 %) compared to the Triplet-1NN and GRU-DOC benchmark models. In case of a full PMU observability (scenario S.2b), the accuracy and F1-score results increase slightly among all models due to the higher number of input signals. Here, the SSN model again achieves the highest test accuracies (95.15 %) and F1-scores (94.98 %). In contrast to that, the SSFD-SVM benchmark model performs very poorly with a test accuracy of 80.77 % and a F1-score of 81.76 %. Also, the loss of accuracy and F1-score is significantly higher with roughly −7.5 % compared to the other models, which indicates a bad generalization capability of the model.

Table 5 shows the accuracies and F1-scores of the classification models SSN, Triplet-1NN, GRU-DOC, and SSFD-SVM for the scenarios S.2a and S.2b. In this case, the accuracy and F1-score decrease significantly of about −20 % to −50 % compared to the training results due to the new operational points. For the scenario S.2a, the Triplet-1NN benchmark model achieves the highest test accuracy (77.01 %) and test F1-score (75.38 %). For all models, the accuracy value decrease on average of about −20.66 % and the F1-score values of about −23.43 % compared to the training results. The accuracy and F1-score losses far exceed the results of the scenarios S.1a and S.1b (see Table 4), which shows a greater impact of unknown operational points on the classification performance compared to measurement noise. In case of scenario S.2b, the SSN model achieves the highest test accuracy (81.48 %) and the lowest accuracy and F1-score losses compared to the training results. The Triplet-1NN benchmark model still achieves the highest test F1-score (78.28 %). Again, the accuracies and F1-scores of the SSFD-SVM benchmark model are significantly lower compared to all other models, with a remarkably large difference between the training and test results.

In case of the scenarios S.3a and S.3b, we also evaluate the accuracies and F1-scores for the known and unknown classes. The corresponding training and test results are given in Table 6. Due to the missing rejection capability, the Triplet-1NN and the SSFD-SVM benchmark models cannot be applied for these scenarios (see also Section 3.2). As it can be seen, the additional integration of unknown classes in the test data significantly reduces the accuracy results η _ACC and F1-score results η _F1. A direct comparison between the η _ACC,K and η _ACC values or the η _F1,K and η _F1 values is difficult due to the different class configurations. Still, the high number of misclassifications in the test data demonstrates the high impact of unknown disturbance events on the overall classification performance. The SSN model outperforms the GRU-DOC benchmark model in both scenarios and achieves the highest accuracies (81.09 % for known classes and 58.32 % for all classes) and F1-scores (78.99 % for known classes and 57.27 % for all classes).

The results regarding the influence of all nuisance variables is given in Table 7. Similar to the scenarios S.3a and S.4b, the SSN model outperforms the GRU-DOC benchmark model and achieves the highest accuracies (77.65 % for known classes and 56.93 % for all classes) and F1-scores (74.91 % for known classes and 51.82 % for all classes). The large differences between the training and test results indicate the impact of the nuisance variables on the performance of the classification models.