A comparative study of different neural networks in predicting gross domestic product

2.1 GA-optimized back-propagation neural network (BPNN) model for GDP prediction

BPNN [12] has good performance in the prediction of time series, including short-term and long-term prediction, and has successful applications in the prediction of industrial [13] and medical [14] data. To enhance the performance of BPNN, this article uses a GA to improve it.

Taking a simple three-layer BPNN as an example, let the node in the input layer as

the node in the output layer as

the node in the hidden layer as

the input of the output layer is h _k and the output is y _k . Then,

where w _ij and w _ji refer to the weight between the hidden layer and the input layer and the weight between the hidden layer and the output layer and f(·) is the activation function, i.e., the sigmoid function

in this article.

Then, the error is back-propagated to optimize the weights and thresholds, and the objective function of the model can be written as:

where y _nk and y n k ^ are the expected and actual outputs of the kth neuron in the output layer.

A GA is a biological evolution-based method applicable to the optimization of complex systems [15], enabling the search for optimal solutions through the simulation of natural evolutionary processes. It can be well integrated with other algorithms [16]. The specific steps of the GA-BPNN model are as follows.

1. The population is initialized, and the parameters to be optimized are encoded. For a BPNN network with an N–M–L structure, the length S of chromosomes in GA is as follows:

   (10) S = L × ( N + 1 ) + M × ( L + 1 ) .

2. The adaptation function used is the error of the BPNN:

   (11) E n = 1 2 ∑ k ( y n k − y n k ^ ) 2 .

3. The selection, crossover, and mutation operations are performed. The selection is done by random traversal sampling. The crossover is done by a single-point crossover operator. The mutation is performed using a random method: the gene subject to mutation is selected, e.g., if the selected gene has a code of 1, it is mutated as 0; otherwise, it is mutated as 1.

4. Steps (2) and (3) are repeated until the error meets the requirements, and the optimized parameters are output.

5. The obtained parameters are input into BPNN to build the GDP prediction model.

2.2 Particle swarm algorithm-optimized Elman NN model for GDP prediction

Elman NN is a local regression network [17], which is characterized by an additional undertaking layer compared to BPNN. Let the input layer be U, the output layer be Y, the hidden layer be X, and the undertaking layer be X _c . The expressions of each layer can be written as:

where w ₁ is the weight from the input layer to the hidden layer, w ₂ refers to the weight from the undertaking layer to the hidden layer, b ₁ refers to the threshold value of the hidden layer, b ₂ refers to the threshold value of the output layer, g(·) is the activation function of the output layer, and f(·) is the activation function of the hidden layer.

Similar to BPNN, the weights and thresholds of Elman NN also have an impact on the performance of the network. Elman NN was optimized using the PSO algorithm in this article. The PSO algorithm is a global stochastic optimization algorithm [18]. Based on the predation process of birds [19], every bird is considered a particle. Suppose there are m particles in a D-dimensional space, where the position of the ith particle is

the velocity is

the current optimal position is

and the optimal global position is

Then, the update formula for the velocity and position of the particle can be written as:

where c ₁ and c ₂ are non-negative constants and r ₁ and r ₂ are random numbers in [0, 1].

The specific steps of the PSO-Elman NN model are as follows.

1. An Elman NN model is developed, and its network structure is determined.

2. The parameters of PSO are initialized.

3. The initial fitness value of the particle is calculated, and then the particle is updated according to the formula above.

4. The updated particle fitness value is calculated. The current optimal position and the global optimal position are updated until the maximum number of iterations is reached.

5. The obtained parameters are input into Elman NN to build the GDP prediction model.

2.3 Bat algorithm (BA) – LSTM neural network model for GDP prediction

LSTM is an improvement of recurrent neural network (RNN) [20], which has very wide applications in language models [20], artificial intelligence, etc. [21], and also performs well in the prediction of various data [22]. It has higher prediction accuracy when it is combined with other algorithms. In this article, the BA is combined with LSTM for GDP prediction.

The difference between LSTM and RNN is that the LSTM includes a memory unit to keep the historical information and includes three gates in the hidden layer: the input gate i _t , the forgetting gate f _t , and the output gate o _t . The update formula of the LSTM at layer t is written as:

where h _t−1 refers to the last output, x _t refers to the current input, σ refers to the activation function, w _i and b _i are the weight and bias of the input gate, w _f and b _i are the weight and bias of the forgetting gate, w _o and b _o are the weight and bias of the output gate, c _t−1 is the status of the previous memory unit, and c _t is the status of the current memory unit.

The loss function used in this article is the mean square error. The Adam function is used as the optimizer, the network is trained using the small-batch gradient descent algorithm, and the network parameters are optimized using the BA.

The BA is a heuristic algorithm [23], which finds the optimal global solution by simulating the bat localization process [24]. Let the number of bats in a D-dimensional space be N, the upper and lower limits of the pulse frequency are Q _max and Q _min, the position of the ith bat is

and the velocity is

then the update equations of its frequency, velocity, and position are as follows:

where β ∈ [0,1] and g _best is the optimal global position. A random number is generated to compare with the frequency; if the random number is larger, a random perturbation is performed:

and if the random number is smaller, the cross-border processing is performed:

A _t is the average loudness of the bat, and δ ∈ [−1,1]. The judgebound() function is used to implement cross-border processing. When the bat finds the target, the update formulas for frequency and loudness are as follows:

where α and ε are constants. If the new fitness value is smaller than the global optimal fitness value, then

The BA-optimized parameters are input into the LSTM model. The specific steps of the BA-LSTM model are as follows.

1. The LSTM model is established, and the network structure is set up.

2. The parameters of the BA are initialized, and all bats are iterated.

3. The frequency, speed, and position of the bat are updated according to the formula.

4. The fitness value is calculated. The frequency and loudness are updated until the termination condition is satisfied. The optimal parameters are output.

5. The obtained parameters are input into the LSTM to build the GDP prediction model.

3.1 Experimental data

This article focuses on the GDP forecast of Sichuan Province. In 2020, the GDP of Sichuan Province reached 485.98 billion yuan, showing an increase of 3.8% over the previous year, the economic growth was fast, the industrial system was complete, and industries, such as nuclear power equipment and heavy combustion engines, ranked among the top in the country. The GDP data of Sichuan province between 1992 and 2020 provided by the National Bureau of Statistics are shown in Table 1. The GDP of the first 4 years was used as input, the GDP of the fifth year was used as output, and so on. The model was validated using the data between 2010 and 2020, as shown in Table 2.

In the evaluation of the prediction model, to have a better understanding of the prediction effect, it is generally expressed by the prediction error, i.e., the distance between the actual value and the predicted value. Commonly used indicators included mean absolute error, median absolute error, etc. The performance of the different models was analyzed using the following two indicators:

Mean absolute percentage error (MAPE): it normalizes the error of every point, reflecting prediction precision, and its calculation formula is as follows:

Root-mean-square error (RMSE): it reflects the degree of deviation between the predicted value and the actual value, and its calculation formula is as follows:

where y _i and y i ′ are the actual and predicted values of GDP, respectively.

3.2 Experimental setup

As GDP belongs to normal time series, selecting a general three-layer structure was enough for the neural network model; therefore, the GDP of the first 4 years were used as the input, i.e., the number of input nodes was 4, and the GDP of the fifth year was used as the output, i.e., the number of output nodes was 1. The other settings are as follows.

1. GA-BPNN model: according to the experimental data, the node N of the input layer in the BPNN was 4 and the node L of the output layer was 1, +1. A network with a 4–9–1 structure was obtained. The parameters that GA needed to optimize were determined as 23 according to S = L × ( N + 1 ) + M × ( L + 1 ) . The population size was 60. The maximum number of iterations was 70. The mutation probability was 0.1. The crossover probability was 0.7. The training number of the model was 100. The learning rate was 0.1.

2. PSO-Elma and the node M of the hidden layer was determined as 9 using the empirical formula: M = 2Nn NN model: referring to the BPNN model, the structure of Elman NN was also set as 4–9–1. Let the population size of PSO be 20, r ₁ = 1, r ₂ = 0.5, c ₁ = c ₂ = 2, and the maximum number of iterations be 1,000.

3. BA-LSTM model: the node of the input layer was 4, the node of the output layer was 1, the node of the hidden layer was 10, the time step was 1, the maximum number of iterations was 1,000, the population size of BA was 30, and the frequency and loudness were 0.5.

3.3 Predicted results

The GDP prediction results of different models are shown in Figure 1.

Figure 1

GDP prediction results of different models.

It is seen from Figure 1 that there were some differences in the prediction results of different models for the prediction of GDP. The results of GA-BPN and BA-LSTM models had large errors with the actual values and large data volatility, showing unstable prediction performance, while the prediction results of the PSO-Elman NN model had a good agreement with the actual values. To further understand the model performance, the MAPE and RMSE values were calculated and compared, and the results are shown in Figures 2 and 3.

Figure 2

Comparison of MAPE values between different models.

Figure 3

Comparison of RMSE values between different models.

It is seen from Figure 2 that the MAPEs of the three models were 0.0578, 0.0236, and 0.0654, i.e., the MAPE of the BA-LSTM model was the smallest, followed by GA-BPNN and PSO-Elman NN models. The MAPEs of the GA-BPNN and BA-LSTM models were above 0.05. The MAPE of the PSO-Elman NN model was 0.0236, the smallest, 0.0342 smaller than the GA-BPNN model and 0.0418 smaller than the BA-LSTM model. The above results indicated that the PSO-Elman NN model had a smaller prediction error, showing better performance in predicting GDP.

It is seen from Figure 3 that the RMSE values of all the models were above 0.01; the RMSE value of the PSO-LSTM model was the smallest, followed by GA-BPNN and BA-LSTM models. The RMSE of the BA-LSTM model was 0.0465, the largest, and the RMSE of the PSO-Elman NN model was 0.0166, which was 0.0121 smaller than the GA-BPNN model and 0.0299 smaller than the BA-LSTM model. The smaller the RMSE value was, the smaller the difference between the prediction result and actual value was, i.e., the higher the degree of fitting was. The results indicated that the PSO-Elman NN model had better performance in GDP prediction and its prediction results were closer to the actual values; thus, it could achieve better applications in practice.