OPGW fault localization method based on transformer and federated learning

3.1 Cloud edge collaboration framework

Based on the cloud edge collaborative architecture, this article designs a model framework for OPGW fault localization as follows.

In Figure 1, at the edge, operational data of OPGW is collected through different sensors and fault feature extraction is performed. These data will be used for training local models at the edge, while transformers will be used to distribute input data to various data sources. Each data source conducts model training locally, updating only the parameters of the model, rather than the entire model. This can utilize the characteristics of transformer for efficient model training and updates.

Figure 1

A model framework for OPGW fault location. Source: Created by the authors.

In the cloud, the central server will receive the parameters of each trained edge model and form a joint model. Utilize the powerful computing power of cloud computing to perform big data analysis and processing on extracted features, thereby providing reliable data support for fault localization. Cloud computing can process large-scale data, improving the accuracy and reliability of fault localization. In addition, cloud computing can perform large-scale learning on training data to train more accurate fault localization models. By continuously optimizing the model, the accuracy and response speed of fault localization can be improved. The trained joint model can be distributed to the edge for use on the edge side. In this way, even if the cloud cannot respond in real time, fault location can also be carried out through edge computing to improve the real-time and reliability of the system.

The optimization of resources, data, and services can be achieved through cloud edge collaboration. In terms of resources, edge nodes can provide infrastructure resources such as computing, storage, and network, and collaborate with the cloud for scheduling and management. In terms of data, edge nodes are responsible for data collection and preprocessing, uploading results and related data to the cloud to form a complete data flow path. In terms of services, the cloud manages the lifecycle of edge side applications, including application deployment, start, stop, delete, and version update, while generating application orchestration strategies. The edge side executes applications according to cloud policies.

3.2 FedL

FedL is an emerging machine – learning algorithm framework. Initially, it was developed to address the issue of Android mobile terminal users updating language – production models at the edge. Its design objective is to safeguard privacy and data security during big – data exchanges, all while adhering to legal and compliance requirements. Consequently, it is structured in a way that obviates the need for direct data exchange or collection. By doing so, it not only protects user privacy but also resolves the data – silo dilemma that the artificial – intelligence industry encounters.

FedL aims to establish a federated model leveraging distributed devices. In the FedL framework, there should be at least two client – side entities. These clients collaborate with a central server to train a global model, and the training process mandates that the original data remains at the edge. The working principle of FedL is as follows:

As depicted in Figure 2, the shared global model is trained by a consortium of participating devices hosted by a central server. This approach enables edge nodes to collaboratively train models at the edge without sharing the original training data. The central server aggregates these parameter updates and generates a new iteration of the training model. This process is repeated until the global model converges.

Figure 2

Operational principle of FedL. Source: Created by the authors.

The effective combination of cloud edge collaboration theory and FedL greatly improves the performance of OPGW fault localization. First, the theory of cloud edge collaboration provides practical application scenarios and frameworks for FedL. In OPGW fault location, cloud edge collaboration tightly connects the cloud and edge ends, achieving remote control, processing, analysis, and intelligent decision-making of data. The cloud is responsible for global model training and optimization, while the edge is responsible for local data collection and processing, as well as collaborative work with the cloud. This architecture enables FedL to train and optimize models in a distributed environment, fully utilizing computing resources in the cloud and edge. Second, FedL provides the ability to protect data privacy for cloud edge collaboration. In OPGW fault location, data privacy and security are very important. Through FedL, various participants can jointly train a global model without exchanging raw data. This can not only avoid data leakage and abuse, but also improve the efficiency and accuracy of model training. Meanwhile, FedL can also address issues of uneven data distribution and inconsistent labels, making the model more robust and reliable. Finally, the combination of cloud edge collaboration and FedL can also improve the accuracy and real-time performance of OPGW fault localization. Through collaborative work between the cloud and edge, real-time fault data can be collected and processed, and the model can be updated and optimized online through FedL. This can make the model more adaptable to changes in the actual environment, improve the accuracy and real-time performance of fault localization. Meanwhile, cloud edge collaboration can also achieve functions such as fault warning and prediction, further improving the reliability and stability of the system.

The FedL system consists of one server and K users. Let d k represent the database of edge side user U k , k = 1 , 2 , 3 , … , K . The goal of the server-side is to learn the models constructed by each user based on their respective datasets, while the client-side participating in edge side training needs to find a model parameter α to minimize the center loss function:

where α k represents the parameters trained by the kth user, and α represents the parameters aggregated by the server. p k is defined as follows:

where ∑ k = 1 K ∣ d k ∣ represents the total number of data samples. Therefore, the optimization problem of FedL can be represented as follows:

where L k ( α , d k ) represents the edge side loss function of the kth user. The training process of a FedL system is shown in Figure 3.

Figure 3

The training process of FedL system. Source: Created by the authors.

The application of FedL in OPGW fault location has the following benefits:

1. The data for OPGW fault location may contain sensitive information, such as geographic location, equipment status, etc. FedL allows various data sources to conduct model training locally without uploading data to a central server, thus protecting data privacy and security.

2. Through FedL, distributed model training can be conducted using computing resources distributed across different regions, improving the speed and efficiency of data processing. Meanwhile, each data source can independently process local data, reducing data transmission latency and costs.

3. The global model built by FedL based on a wider dataset helps to improve the model’s generalization ability, enabling it to more accurately adapt to the OPGW fault location requirements under different geographical locations and environmental conditions.

4. FedL can support real-time model updates, meaning that when new fault data appear, various data sources can update the model locally and upload the updated model parameters to the central server for aggregation. This helps to maintain the real-time and dynamic nature of the model to adapt to constantly changing fault situations.

3.3 Feature fusion based on transformer

In many traditional methods, the process of fusing fault features extracted from the edge is too simple and crude, which cannot effectively capture the mapping relationship between multi-dimensional features.

In OPGW fault localization, the transformer model may learn and understand signal patterns in OPGW optical cables through its powerful feature extraction and pattern recognition capabilities. When a fault occurs, the model can recognize abnormal signals that are different from normal patterns, thereby achieving precise localization of the fault point. In the proposed cloud edge collaboration framework, the model’s role is as follows:

1. The transformer model, with its self-attention mechanism as its core, can effectively extract key features from fault signals. Under the cloud edge collaborative framework, these features can be extracted and analyzed through collaborative processing between the cloud and edge, which helps to more accurately identify fault modes in OPGW optical cables.

2. The cloud edge collaboration framework allows for the sharing of data and processing results between the cloud and edge. The transformer model can achieve the fusion of cloud and edge data by learning data features from different sources, enabling the system to respond to fault events more quickly.

3. In a cloud edge collaborative environment, the transformer model can be trained and optimized in a distributed manner through FedL and other methods. This means that the model can collect data on different edge nodes and conduct local training, and then summarize the training results to the cloud for global updates. This approach not only protects the privacy of data, but also improves the model’s generalization ability and adaptability.

Therefore, the transformer encoder is used to fuse features from different dimensions, and its structure is shown in Figure 4.

Figure 4

Multidimensional feature fusion based on transformer. Source: Created by the authors.

In Figure 4, local and global features are first extracted from the fault data information, and then the multi head self-attention mechanism is used to automatically ignore the noise present in the encoded information, thereby improving the model’s ability to capture important fault features.

By utilizing the multi head self-attention mechanism to fuse the multi-dimensional fault feature information matrix, the isolated row vectors in the matrix can transmit information to each other, establish connections, and adaptively focus on key features on this basis, improving the model’s understanding ability of OPGW fault features.

First, concatenate the fault features obtained from various dimensions together to synthesize the fault feature vector Z:

where S i ( i = 1 , 2 , … , n ) represents the OPGW operation data collected by the ith sensor, and n represents the total number of sensors.

In the process of feature fusion, the obtained fault features from various dimensions are first concatenated together, and then the multi-dimensional features are passed into the transformer encoder to obtain its output result. This process can be represented as follows:

where ω q , ω k , ω v represent linear transformation matrices. Q n , K n , V n , respectively, represent the Q, K, and V matrices used by the nth transformer encoder for self-attention operations. F n represents the output of the nth transformer encoder. F 0 = ME , n ∈ [ 1 , N ] , N is hyperparameters, which refer to the number of encoders. d t represents dimension. The fused multi-dimensional feature sequence is the output F N ∈ R ( n t + n i ) × d t of the last layer transformer encoder.

Due to the self-attention calculation of each token in multidimensional embedding with other tokens, it can simultaneously capture the internal effects of single dimensional features and the mapping relationships between multi-dimensional features, dynamically adjust the weights of multi-dimensional features, and allocate attention weights more reasonably. Due to the advantages of noise suppression, information redundancy reduction, model computation, and overfitting prevention in pooling operations, we choose to use combination pooling to obtain richer feature layers, choose maximum pooling to capture local features at each time step, and choose average pooling to make the model focus more on global features. Then, combine the results of max pooling and average pooling as the output of the attention module. The calculation process is as follows:

The fused feature vectors cannot be directly used for OPGW fault localization. Therefore, a simple attention layer is used to obtain the final feature representation for classification, as shown below:

where Re LU is the activation function, ω 1 and ω 2 are the weight matrices of the linear transformation, and b 1 and b 2 are the bias matrices.

Applying transformer to OPGW fault location may bring some potential benefits:

1. OPGW fault location may require processing a series of time series data, and through self-attention mechanism, transformer can capture long-term dependencies in the data, which is very useful for analyzing the historical trends and patterns of fiber optic cable faults.

2. In OPGW fault location, multiple data from different sensors and systems may be used. transformers can integrate data from different sources into a unified representation space through their powerful feature extraction and representation learning capabilities, thereby gaining a more comprehensive understanding and analysis of fault situations.

3. Through transformer’s deep learning and analysis of fault data, the model may be able to discover some subtle fault features or patterns that are difficult to capture by traditional methods. This helps to improve the accuracy and precision of fault localization, thus enabling faster localization and repair of faults.

4. The transformer model can process data streams from sensors in real-time and update the model as new data arrive. This means being able to respond in real-time to fiber optic cable failures and take necessary measures in a timely manner.

3.4 Fault localization process

The fault localization process of OPGW based on transformer and FedL in the proposed cloud edge collaborative environment is shown in Figure 5.

Figure 5

Fault localization process of OPGW. Source: Created by the authors.

4.1 Dataset and evaluation indicators

To improve the evaluation accuracy of the model, a large amount of data training samples are needed as support. The data collected from OPGW online monitoring devices deployed in Zhejiang, Hebei, Shanxi, and Yunnan provinces of China are used as the experimental simulation and analysis dataset. Due to the significant geographical differences among the four OPGW, their data are representative. These four datasets contain 850,008,000,085,000 and 90,000 monitoring data of OPGW operating status characterization under on-site time series, and each dataset is divided into three datasets. First, the data of each dataset is fused to obtain data fusion labels. Then, the monitoring data and data fusion labels are divided into training, testing, and testing sets in a 6:2:2 ratio. The detailed information of the dataset is shown in Table 1.

In the dataset used for health assessment, different feature quantities have different dimensions. For example, the temperature and environmental temperature and humidity of OPGW vary greatly in order of magnitude. Direct calculation increases complexity and also leads to a decrease in the accuracy of the assessment results. Normalization is used here to preprocess the data.

where x max is the maximum value in the sample data, and x min is the minimum value in the sample data.

The experiment uses the absolute error, relative error, and fault location time of OPGW fault location to determine the effectiveness of the proposed model.

The hardware environment of the experiment is given in Table 2.

The parameters of the experiment are shown in Table 3.

4.2 Comparative analysis

4.2.1 Experimental analysis

The following is a comparison between the proposed OPGW fault localization model based on transformer and FedL in the cloud edge collaborative environment and various other models under the same experimental conditions, including DBSCAN [26], BFS [28], multisource breadth-first search (M-BFS) [30], and WFBGA [31]. Comparative analysis is done using four different datasets.

The relative error of using different models for fault localization on different datasets is shown in Figure 6 and Table 4.

Figure 6

Comparison of relative errors in fault localization of different models. Source: Created by the authors.

Table 4

Comparison of relative errors in fault localization of different models

Model	Dataset 1 (%)	Dataset 2 (%)	Dataset 3 (%)	Dataset 4 (%)
Proposed model	0.85	0.78	0.82	0.93
DBSCAN	1.79	1.65	1.74	1.96
BFS	2.86	2.62	2.76	3.12
M-BFS	2.16	1.98	2.09	2.35
WFBGA	1.39	1.28	1.35	1.52

The absolute results of using different models for fault localization on different datasets are given in Figure 7 and Tables 5 and 6.

Figure 7

Comparison of fault localization results of different models. Source: Created by the authors.

Table 5

Comparison of fault localization results of different models

Fault distance (km)	Dataset 1	Dataset 2	Dataset 3	Dataset 4
Actual	3.800	3.800	3.800	3.800
Proposed model	3.7676	3.7703	3.7687	3.7646
DBSCAN	3.7318	3.7375	3.7341	3.7256
BFS	3.9085	3.8995	3.9050	3.9185
M-BFS	3.7180	3.7248	3.7207	3.7105
WFBGA	3.8529	3.8485	3.8511	3.8577

Table 6

Comparison of absolute errors in fault localization of different models

Absolute error (km)	Dataset 1	Dataset 2	Dataset 3	Dataset 4
Proposed model	0.0324	0.0297	0.0313	0.0354
DBSCAN	0.0682	0.0625	0.0659	0.0744
BFS	0.1085	0.0995	0.1050	0.1185
M-BFS	0.0820	0.0752	0.0793	0.0895
WFBGA	0.0529	0.0485	0.0511	0.0577

The time required for fault localization using different models on different datasets is shown in Figure 8 and Table 7.

Figure 8

Comparison of time used for fault localization of different models. Source: Created by the authors.

Table 7

Comparison of time used for fault localization of different models

Time usage (µs)	Dataset 1	Dataset 2	Dataset 3	Dataset 4
Proposed model	5.42	5.33	5.47	5.51
DBSCAN	8.38	8.25	8.46	8.52
BFS	10.95	10.77	11.05	11.14
M-BFS	9.95	9.78	10.04	10.11
WFBGA	7.34	7.22	7.41	7.47

4.3 Hyperparameter analysis

4.3.1 Learning rate analysis

The change in learning rate is a key factor affecting the faster convergence of model accuracy to maximum accuracy. Next we will analyze the impact of learning rate on model performance. The experiment set the learning rates to 0.01, 0.001, 0.0001, and 0.00001, respectively. The model was trained using dataset 1 and the test results were recorded.

Figure 9 shows the experimental content recorded from two aspects: updating the number of rounds during the model training process and the relative error of fault localization. It can be seen that when the learning rate is set to 0.001, the relative error of fault localization can reach the minimum with the increase in epoch, so setting the learning rate to 0.001 is the optimal value.

Figure 9

Relative error of fault localization obtained under different learning rates. Source: Created by the authors.

4.3.2 Transformer layer analysis

The encoder and decoder in transformer are both stacked with multiple layers. The number of layers determines the depth of the model. Increasing the number of layers can improve the representation ability of the model, but it also increases the number of parameters and computational complexity of the model. Therefore, different levels of transformer will have a significant impact on the fault localization results. The experimental results under different transformer layers are given in Figure 10.

Figure 10

The influence of different transformer layers on fault location results. Source: Created by the authors.

In Figure 10, the performance of the model varies when the number of layers in the transformer encoder is set to different values. When its value is 4, the model exhibits the best performance.