Just-in-time updated DBN BOF steel-making soft sensor model based on dense connectivity of key features

2.1 Maximum mutual information factor

In contrast to the correlation coefficient, the MIC (Maximum mutual information factor) takes into account information about the higher order relationship between two random variables, allowing for the exploration of not only linear correlations but also nonlinear relationships between two variables [25].

The MIC is calculated as follows: For two variables X and Y, the formula for calculating the mutual information between them is given in equation (2.1).

where p ( x , y ) is the joint probability distribution representing the two variables. p ( x ) and p ( y ) are the marginal probability distributions of x and y, respectively.

The MIC is obtained by selecting the maximum value of MI under different division methods, and the calculation is shown in equation (2.2).

where n x and n y denote the number of columns and rows, respectively, of the grid into which the partition is made. n is the number of scatter points, α is the threshold value, which takes the value of (0,1], and the larger the value of α, the more accurate the maximum mutual information coefficient is, but the computational complexity also rises dramatically, hence it is necessary to limit α to a suitable interval to achieve the optimal balance of computational accuracy and computational complexity.

The larger the calculated MIC value, the greater the higher order correlation between the two variables. In this study, the correlation between two vectors is assessed by introducing MIC.

2.2 UMAP streaming learning dimensionality reduction

UMAP is a nonlinear dimensionality reduction and visualization algorithm. It is a method based on graph theory and manifold learning for mapping high-dimensional data into low-dimensional spaces for visualization and analysis [26].

The specific steps of the UMAP algorithm are as follows:

1. High-dimensional similarity calculations. X = {x ₁, x ₂, …, x _N }, for any X _i, define ρ i and σ i according to equations (2.3) and (2.4)

where σ i is a length scale parameter and ρ i is a point connected to X _i with at least one edge weight of 1.

Define a directed weighted graph G ¯ = ( V , E , w ) , and using the symmetry of G ¯ to define the undirected weighted graph G, a symmetric matrix can be obtained by assuming that A is the weighted adjacency matrix of G

where A is the weighted adjacency matrix consisting of all W_ij; ⊗ is the Hadamard product of the matrices; and B is the weighted adjacency matrix of the final graph G.

UMAP evolves an equivalently weighted graph H containing the set of points {y _i }, i = 1, …, N by applying gravitational and repulsive forces along the boundary and vertices, respectively.

2. The gravitational forces of vertices i and j at coordinates y _i and y _j are obtained by the following equation:

where a and b are hyperparameters. The repulsive force is

where ε is a very small value that prevents the denominator from being 0 and a and b are hyperparameters.

The main goal of UMAP is to maintain the local and global structure between data points. It does this by constructing a graph of proximity relationships between data points and using the topology of the graph for flow shape approximation and optimization.

3.1 Key feature selection

For the soft sensor model, it is very important to be able to extract more information related to the target variable, while the traditional DBN deep neural network will lose some important information during the forward propagation process, in the process of BOF steel modeling, in order to make the model retain the information that has high relevance to the label information during the extraction process. This section uses the structure of dense connection through the innovative key feature selection method to supplement the information that the network may miss during the forward propagation process, so that the network can strengthen the ability to extract the information related to the target variable of BOF steelmaking during the forward propagation process. Information that may be missed by the network in the forward propagation process is supplemented, so that the network can be strengthened in its ability to extract information related to the target variable of BOF steelmaking in the forward propagation process. Cosine similarity is an excellent algorithm for evaluating the relative importance of features, so features with high relevance to the target variable can be selected by cosine similarity, and MIC is a statistical method used to measure the correlation between two variables. Redundant features can be removed by the MIC method measuring the higher order relationships between variables and implicit layer features. Therefore, in this work, the importance of features to the target variable and the redundancy between features are assessed by COS and MIC, respectively.

3.2 DBN instant update model for UMAP downscaling

Although the improvement of the residual structure mentioned above enables the model to improve the ability to extract information from the BOF steelmaking data, this model is still a static model, and in the process of BOF steelmaking, there are fluctuations between different furnaces, and a single static model cannot be well adapted to the effects caused by the changes in furnaces in the process of BOF steelmaking production. So, in order to maintain the performance of the model and adapt to the changes in the working conditions in a timely manner, a Just-In-Time Regression Network (JITRN) strategy is constructed to achieve rapid adaptive updating of the prediction model.

In the offline process, the historical data are constructed into a low-dimensional data pool through the UMAP feature dimensionality reduction method, and in the online update stage, the data to be measured are also queried with the low-dimensional data pool constructed in the offline process after dimensionality reduction through the UMAP features, and the few samples that are most similar to the query samples are selected. These samples have data structures and operating states that are closer to the current working conditions of the samples to be measured, and they are used to adaptively update the parameters of the offline network.

1. UMAP offline feature dimensionality reduction

The BOF steelmaking data not only exhibit high-dimensional characteristics but also demonstrate highly complex nonlinear relationships. The use of traditional instant learning metrics often face the problem of metric failure, in order to make the model in the instant learning online adaptive fine-tuning process more accurately select the historical data similar to the current furnace conditions, we through the UMAP stream learning method, the high dimensional complexity of the BOF steelmaking data in the reduction of dimensionality at the same time, but also to retain the structural characteristics of the data, this way of dimensionality reduction makes the complexity of the This way of dimensionality reduction makes the complex high-dimensional data dimensionality reduction which is conducive to the accuracy of the metrics in the immediate learning process, and because of the use of stream learning dimensionality reduction makes the preservation of the data structure is conducive to the metrics to select a higher similarity between the historical working condition data and the current working condition data.

The specific algorithmic process has the following steps:

Step 1. Offline process low-dimensional feature pool construction

Put the historical dataset X data = { x 1 , x 2 , . . . , x i } through UMAP dimensionality reduction algorithm.

where UMAP is a dimensionality reduction algorithm for flow learning, and X l is the low-dimensional feature pool after passing the historical dataset through dimensionality reduction.

Step 2. Online instant update of soft sensor model

After the dimensionality of the data to be measured is reduced by UMAP, the data to be measured can be measured against the low-dimensional database constructed by the offline process while maintaining the data structure, and then the historical data that are most similar to the current working conditions can be selected and the FDBN model can be fine-tuned online using these historical data.

The JITL of the model has the following specific steps

1. When the data to be measured Xtest comes, the dimension of the data to be measured is reduced by the UMAP dimensionality reduction algorithm

   (3.10) X t = UMAP ( x test ) ,

   where X _t is the feature in lower dimensions after dimensionality reduction by UMAP.

2. The dimensionality reduced data to be measured and the historical dataset N are measured by the cosine correlation metric

   (3.11) cos ( x , y ) = x ⋅ y | x | ⋅ | y | = ∑ i = 1 n x i y i ∑ i = 1 n x i 2 ∑ i = 1 n y i 2 .

3. Based on the values of the metrics with those in the history database, the small sample history subset in the history database that has the highest similarity to the current sample to be tested is selected according to the index they correspond to, and the online sample subset D = (x, y) is shown below:

   (3.12) D = { ( x u 1 , y u 1 ) , ( x u 2 , y u 2 ) , . . . , ( x un , y un ) } ,

   where the data selected by the corresponding index is the input feature, and the labeled value is the target.

4. A small subset of samples D = (x, y) is input to the offline trained model for online fine-tuning.

In particular, because of the highly nonlinear and complex structure of the BOF steelmaking process data, the selection of similar samples for on-the-fly learning used in this work does not use the original feature variables for direct metrics, but rather the low-dimensional vectors of the data structure preserved by the UMAP model, so as to improve the accuracy of the metrics and make the set of similar samples more in line with the characteristics of the furnace samples to be tested. The fine-tuned model is used to predict the current data to be measured.

The online adaptive fine-tuning in the above steps can update the connection weights W1 and b1 in the offline static model to W2 and b2. Moreover, since the selected small sample subset has a high similarity with the sample conditions of the furnace to be measured, the network only needs a few iterations to adaptively update the network online to improve the model performance to adapt to the current conditions of the BOF steelmaking forecasting.

3.3 FSDBN-UJTIL soft sensor model construction

When the implicit layer of the FSRBM in the first layer is passed backwards, the computational process of the FSRBM in the second layer is consistent with that of the FSRBM in the first layer. Typically, the FSDBN is stacked with multiple FSRBMs. Pre-training of the FSDBN refers to training the FSRBMs layer-by-layer on the labeled dataset, and then using these pre-trained FSRBMs to initialize the whole FSDBN in order to provide good initial parameters for the whole network. The training of the FSDBN consists of two phases, namely, off-line training and on-line fine-tuning.

Offline training

1. The historical process data of BOF steelmaking is used as an input variable, and the model is trained according to the method proposed in Section 3 to obtain the connection weights of each level and the dense connection block parameters.

2. Reduce the dimensionality of the historical production process data according to the UMAP stream learning dimensionality reduction algorithm and store it in a low-dimensional feature pool.

3. Save the global offline model and low-dimensional feature pool.

Online fine-tuning

1. When the samples to be tested come, first, the current samples to be tested are dimensionality reduced to obtain the structural low-dimensional features by UMAP algorithm.

2. The structural low-dimensional features of the to-be-tested sample are compared with the features in the low-dimensional feature pool computed in the offline process by similarity metric, and a subset of N small historical samples D that are similar to the current working conditions are selected.

3. The offline model is updated online using D to obtain a model suitable for the current operating conditions, and the updated model is used to predict the endpoint carbon content and temperature for the current sample to be measured.

The above two steps of modeling allow the model to be more compatible with the prediction requirements under different working conditions.

5.1 Experimental evaluation indicators and experimental setting

In order to evaluate the prediction performance of different soft-sensor models for the carbon content and temperature at the end point of BOF, regression accuracy (RA), root mean square error (RMSE), and mean absolute error (MAE) are used as the indexes in this work. Among them, RA is an important evaluation criterion, which represents the accuracy of the model in predicting the carbon content and temperature results at the end point of BOF steelmaking within the allowable error.

RMSE and MAE were used to evaluate the performance of the soft sensor model. The smaller the value, the better the predictive performance of the model.

All experiments were done on python 3.9, Cpu: 12th Gen Intel(R) Core (TM) i7-12700 2.10 GHz, RAM: 16.0 GB device.

5.2 Data description and parameterization

The data sources in this work are related to the actual BOF steel production process data. In order to enable the model to extract effective features for modeling forecasts, different features are selected as input variables for the model in conjunction with the results of the feature selection of the BOF steel production process data.

Table 2 describes the details of the two datasets, end point carbon content prediction and temperature prediction, in terms of the number of input features, the number of training samples, and the range of output results. Also, during the experiment, all data were normalized by zero mean normalization in order to eliminate the effect between the quantiles. During the experiment, a total of 4,150 samples under normal operating conditions were collected, of which 4,000 process samples were used as training set and 150 samples were used as test set for online evaluation of the model performance. And we preprocessed the data, specifically, first we performed a max-min normalization operation on these data to ensure that these data can be modeled on the same magnitude [0,1] as shown in equation (5.3). Normalization is a crucial and effective preprocessing step for distance-based algorithms that equalizes the impact and importance of feature scales, thus preventing wide ranges or higher magnitudes from having a greater impact on learning. The network parameters are shown in Table 3.

Because of the online adaptive updating process, the subset of samples selected by instant learning through metrics is important for fine-tuning the training, so the judgment of how many dimensions the input features should be reduced to after dimensionality reduction through UMAP in order to guarantee the best results is very important when metrics are being used; therefore, this section starts with tuning this important parameter in the algorithm, and the dimension of the features after dimensionality reduction through UMAP is set to W, which is compared and analyzed by varying the value of the parameter W. In order to find the optimal balance between information retention and metric accuracy after dimensionality reduction, we perform optimization by setting different values for W. Specifically, by setting the value of W to several different values of 2, 5, 8, 12, 15, and using the data for experiments, the appropriate dimension is selected based on the experimental results.

Table 4 summarizes the predictive assessment metrics for the parameter tuning process performed on the carbon content and temperature datasets. It is clearly visible that the predictive performance of the model shows differences under the variation in parameter W. When parameter W is set to 5, the model achieves the best Te regression accuracy and the corresponding RMSE of the predicted results reaches the minimum value, which indicates that the model achieves a better performance in this configuration. Therefore, the optimal value of parameter W can be determined as 5. The prediction accuracy is shown in Figure 3. The RMSE error is shown in Figure 4.

Figure 3

Radar plot of parameter W tuning accuracy.

Figure 4

RMSE metrics for tuning reference carbon and temperature.

5.3 Evaluation of the methodology

5.3.1 Ablation experiment

In order to verify the effectiveness of the proposed method, this section carries out step-by-step ablation experiments for the proposed method in chunks, specifically, the performance enhancement of the proposed method in this work mainly relies on the improvement of the two parts of the innovative key feature dense connection and the online adaptive updating strategy after the dimensionality reduction of the flow shape learning, so that the model is divided into the following parts for the experiments:

a. Soft sensor model of carbon content and temperature at the end of BOF steelmaking based on the original DBN.

b. DBN with the addition of key features densely connected, denoted as FSDBN.

c. DBN with online adaptive updating of flow shape downscaling, denoted as DBN-UJTIL.

d. Soft sensor model of carbon content and temperature at the end of BOF steelmaking based on the FSDBN-UJTIL model (Figure 5).

Figure 5

Accuracy results of ablation experiments.

According to the experimental results in Table 5, it can be seen that the prediction results of the FSDBN model are better than those of the original DBN because the features selected through the two-step key feature selection are complementary to the implicit layer features, and the ability of the network to extract the features is enhanced through the densely connected structure. The experimental prediction accuracy of the DBN-UJITL is higher than that of the original DBN because in the online prediction part, the adaptive updating strategy of streamline learning dimensionality reduction can make the model more suitable for the current working conditions of the samples to be tested, so the prediction accuracy can be improved. In contrast, the FSDBN-UJTIL model shows good performance through the collaboration of the enhanced feature extraction capability in the offline process and the adaptive updating strategy in the online process. The prediction accuracy is shown in Figure 6.

Table 5

Summary of ablation experiment results

Indicators	Te	DBN	FSDBN	DBN-UJTIL	FSDBN-UJTIL
Carbon content	0.01%	34.6666	45.333	46.666	65.333
Prediction	0.02%	69.3333	71.333	72.000	82.000
Accuracy	0.03%	83.3333	82.666	86.666	91.333
RMSE	—	0.02636	0.0263	0.0256	0.0178
MAE	—	0.01837	0.0174	0.0170	0.0112
Temperature	5°C	40.666	46.000	43.333	59.333
Prediction	10°C	66.666	72.000	74.000	80.000
Accuracy	15°C	82.666	84.666	90.000	91.333
RMSE	—	11.4464	9.9043	9.6137	8.1748
MAE	—	8.6079	7.4260	7.3079	5.9905

Figure 6

Predicted trend curve of carbon temperature at the end point of the ablation experiment.

5.4 Comparative experiments with other soft sensor methods

This section describes the comparison of the proposed method with other soft sensor models in the BOF steel production process. Specifically, this section divides the comparison into shallow model comparison, deep learning model comparison alone, and deep learning model plus traditional instant learning method comparison according to the content of the model. This section compares the prediction performance of deep neural network (DNN), stacked autoencoder (SAE), deep neural network-just in-time learning (DNN-JITIL), and stacked autoencoder just in-time learning (SAE-JTIL) the four soft sensor models, for carbon content and temperature at the end point of BOF steelmaking, respectively. After the model training is completed, the results of comparing the Te accuracy, RMSE, and MAE metrics of the SD-DBN model with the other eight soft sensors on the BOF steelmaking test dataset are presented in Table 6 (Figure 7).

Figure 7

Radar plot of prediction accuracy of end-point carbon temperature.

Based on the experimental results in Table 6, it can be clearly observed that with the other six soft sensor models, the method proposed in this study has the highest accuracy in forecasting the end point of BOF steelmaking, with better performance than the single model and the traditional deep learning methods, as well as shows a higher accuracy of the forecasting model than that of the traditional instantaneous learning methods. In the experiments, compared with the DNN and SAE models, the model prediction performance after the addition of traditional instantaneous learning shows a little decrease. This is because in the instantaneous learning stage, the dimension of the original input features is too high, resulting in the metrics cannot be similar to the current working conditions of the subset of the samples, so fine-tuning the model not only can not improve the training to adapt to the current working conditions by using instantaneous learning but also causes the model to fall into inaccurate working conditions due to inaccuracy of the small sample subset, resulting in performance degradation. The FSDBN-UJTIL model further achieves a significant improvement in prediction accuracy and successfully reduces the error level. One of the reasons for this significant improvement lies in the fact that the present research method enhances the feature extraction capability of the model by multiplexing the selected key features during the feature extraction process by means of thick linking. In addition, the accuracy of this study’s method is also higher compared to DNN-JITL and SAE-JITL, which is due to the fact that the model not only improves the performance of the model’s feature extraction in the offline process, but also improves the accuracy of selecting a small subset of samples in the instantaneous learning process in the online process through the dimensionality reduction of the UMAP manifold learning and then the metrics, and improves the model’s adaptability to the current conditions after the adaptive online updating. Adaptability – Through the experimental results, it can be found that the method in this study has higher accuracy than other methods for different carbon contents and temperature forecasts at the end point of the production process, which indicates that the FSDBN-UJTIL model is more robust to the complex data of the converter steel production process, and is able to extract the information related to the target variables from the high-dimensional and nonlinear data for training the model. The prediction accuracy is shown in Figures 8 and 9.

Figure 8

Predicted trend curves for end-point carbon content.

Figure 9

Predicted trend curves for end point temperatures.