Statistical, machine learning, and deep learning models for COVID-19 forecasting in Kenya

2.1.1 ARIMA

ARIMA is a statistical model for forecasting univariate time series with autocorrelation and non-stationarity, integrating autoregressive (AR), differencing (I), and moving average (MA) components [13,19]. The AR component links current observations to past values, while the MA component models dependencies on past forecast errors. Differencing achieves stationarity. The ARIMA( p , d , q ) model, where p is the AR order, d is the differencing order, and q is the MA order, is expressed as follows:

where μ is a constant, ϕ i are the AR coefficients, θ j are the MA coefficients, and ε t is the white noise. The autocorrelation function (ACF) and partial autocorrelation function (PACF) guide p and q selection, with PACF cutoffs indicating AR processes and ACF cutoffs suggesting MA processes.

2.1.2 SVR

SVR is an extension of the SVM framework for regression tasks. It aims to find a function f ( x ) that deviates from the true target values y i by no more than a small margin ε , while keeping the model as flat as possible [3,11]. The function is defined as follows:

where w is the weight vector, x is the input vector, and b is the bias term. To enforce flatness, the norm ‖ w ‖ 2 is minimized. This leads to the following primal optimization problem:

subject to the constraint:

To allow for infeasible constraints due to noise or outliers, slack variables ξ i , ξ i * ≥ 0 are introduced. The problem becomes

subject to

Here, C > 0 is a regularization parameter that balances the model complexity and the amount up to which deviations larger than ε are tolerated.

Several commonly used kernels in SVR include the following:

Linear kernel:

Polynomial kernel:

Radial basis function (RBF) kernel:

where c is a constant, d is the degree of the polynomial, and γ controls the width of the RBF kernel.

Regularization through the parameter C controls the trade-off between model complexity and tolerance to errors. A large C assigns a higher penalty to errors, leading to less margin violation, while a small C allows for a more generalized model by tolerating larger deviations. The choice of kernel and regularization parameters significantly affects the SVR model’s ability to capture nonlinear relationships and its generalization performance.

2.1.3 RF

RF is an ensemble learning algorithm that builds multiple decision trees and aggregates their outputs to perform regression. It was introduced by Breiman [5] and is particularly effective in handling high-dimensional data, nonlinear relationships, and reducing overfitting.

In regression settings, the goal is to predict a continuous target variable y ∈ R based on an input vector x ∈ R d . RF achieves this by constructing a collection of T decision trees, each trained independently on a different subset of the training data. The final prediction for a new input x is obtained by averaging the predictions from all individual trees:

where h t ( x ) denotes the prediction made by the t th decision tree.

Each tree is trained on a bootstrap sample, which is created by randomly sampling (with replacement) from the original training dataset. This introduces variability among the trees. Additionally, during the tree-building process, RF introduces further randomness by selecting a random subset of features at each split rather than considering all features. This decorrelates the trees and enhances generalization.

At each node of a decision tree, the algorithm selects the best split point among the randomly chosen features based on a splitting criterion that minimizes prediction error. For regression, the most commonly used criterion is the mean-squared error between predicted and actual values in the resulting child nodes. This local optimization helps guide the recursive partitioning of the input space.

Unlike a single decision tree, which may overfit the training data, RF reduces the variance of predictions by averaging across many trees. The ensemble effect ensures that individual overfitting errors are averaged out, leading to more robust and accurate predictions.

2.1.4 RNN

RNNs are a class of neural architectures tailored for sequential data modeling. They maintain a hidden state h t that evolves over time by incorporating information from both the current input x t and the previous hidden state h t − 1 , thereby capturing temporal dependencies [26]. The hidden state is updated using the recurrence relation:

where W h and W x are the weight matrices corresponding to the hidden state and the input, respectively, b h is the bias term, and σ is a nonlinear activation function such as tanh or ReLU. The output at each time step y t is computed as follows:

where W y is the weight matrix for the output, and b y is the output bias term. This recurrent structure, combined with the output transformation, allows the network to retain and propagate information through time steps, making it suitable for tasks involving time series or sequences.

2.1.5 LSTM

LSTM is a specialized type of RNN designed to capture long-range dependencies in sequential data by using gates to control the flow of information. Each LSTM cell contains three primary gates: the forget gate, input gate, and output gate, which regulate how information is retained, updated, and outputted at each time step [6].

The forget gate f t determines what proportion of the previous cell state C t − 1 should be forgotten. It outputs values between 0 and 1, where 0 means “completely forget” and 1 means “completely retain.” The forget gate is calculated as follows:

where f t is the forget gate at time t , W f , U f , and b f are the weights and bias for the forget gate, and σ is the sigmoid activation function.

The input gate i t determines how much new information should be added to the cell state C t . It also uses a sigmoid function to output values between 0 and 1. The input gate is computed as follows:

where i t is the input gate at time t , W i , U i , and b i are the weights and bias for the input gate.

The cell candidate C t ˜ represents the new information that is to be added to the cell state. This candidate value is then filtered by the input gate i t . The cell state C t is updated as follows:

where C t − 1 is the previous cell state, f t is the forget gate, i t is the input gate, and C t ˜ is the cell candidate.

The output gate o t determines what part of the cell state should be output as the hidden state h t . It acts as a filter to decide how much of the cell state should contribute to the next hidden state. The output gate is calculated as follows:

Finally, the hidden state h t is updated as

where o t is the output gate, C t is the cell state, and tanh is the hyperbolic tangent activation function, applied to the cell state.

Figure 1 presents the LSTM network, which incorporates memory cells to handle long-term dependencies effectively.

Figure 1

Architecture of the LSTM network. Source: From ref. [39].

2.1.6 GRU

The GRU is a simpler version of LSTM with fewer parameters. It combines the forget and input gates of LSTM into a single update gate. The GRU has two main gates: the update gate and the reset gate [39].

The update gate z t controls the extent to which the previous hidden state h t − 1 should be retained. A higher value means more of the previous state is retained, while a lower value means more of the current candidate hidden state h t ˜ is used. The update gate is computed as

where z t is the update gate, and W z , U z , and b z are the weights and bias for the update gate.

The reset gate r t determines how much of the previous hidden state should be used when computing the candidate hidden state. It is calculated as follows:

where r t is the reset gate, and W r , U r , and b r are the weights and bias for the reset gate.

The candidate hidden state h t ˜ is computed using the reset gate to decide how much of the previous hidden state should be incorporated. It is calculated as follows:

where h t ˜ is the candidate hidden state and r t ⋅ h t − 1 is the element-wise multiplication of the reset gate and the previous hidden state.

Finally, the hidden state h t is updated as a linear interpolation between the previous hidden state h t − 1 and the candidate hidden state h t ˜ , controlled by the update gate z t :

where h t − 1 is the previous hidden state, h t ˜ is the candidate hidden state, and z t is the update gate.

The structure of the GRU is illustrated in Figure 2, where gating mechanisms help regulate information flow and improve efficiency.

Figure 2

Architecture of the GRU. Source: From ref. [39].

2.1.7 Evaluation metrics

The predictive performance of the models is evaluated using four standard metrics: RMSE, MAE, MAPE, and R 2 . These metrics assess the accuracy of point forecasts in both absolute and relative terms. Additionally, to quantify the improvement of the best-performing model (RF) compared to other models, the percentage improvement for RMSE, MAE, and MAPE is calculated, providing a relative measure of performance enhancement:

where M other represents the RMSE, MAE, or MAPE of a competing model (ARIMA, SVR, RNN, LSTM, or GRU), and M RF is the corresponding metric for RF, the best-performing model.

To statistically evaluate whether the forecasting performance of two competing models differs significantly, the DM test is applied [8]. The test is based on the loss differential series d t , defined as the difference in forecast losses from two models:

where L ( ⋅ ) is a loss function such as MSE, MAE, and MAPE, and e 1 t , e 2 t are the forecast errors at time t from the two models being compared.

The DM test evaluates the following hypotheses:

The test statistic is given by

where d ¯ = 1 T ∑ t = 1 T d t is the sample mean of the loss differential, T is the forecast horizon, and σ ˆ d 2 is a consistent estimate of the variance of d t .

Under the null hypothesis, the DM statistic asymptotically follows a standard normal distribution:

If the absolute value of the DM statistic exceeds the critical value from the standard normal distribution, the null hypothesis is rejected, indicating a statistically significant difference in predictive performance between the two models. In this study, the DM test is applied using MSE, MAE, and MAPE as loss functions to ensure comprehensive evaluation.

2.1.8 Experimental design

The experimental setup follows an 80:20 data split, where 80% of the time series is used for training and 20% for testing. We evaluate the forecasting performance of six models: ARIMA, SVR, RF, RNN, GRU, and LSTM. These models are applied to four key variables, with all learning carried out under a consistent framework.

To prepare the time series data for supervised learning for SVR, RF, and the RNNs, we apply a sliding window approach. 30-day sliding window is used to prepare the data. Given a univariate sequence y 1 , y 2 , … , y t , each input-output pair is constructed as:

where w is the window size. This transformation allows the models to capture and learn temporal dependencies by treating past observations as features for predicting the next value.

For the DL models (RNN, GRU, LSTM), feature scaling is performed using Min-Max normalization to stabilize and accelerate the training process. The normalization formula is

After prediction, the inverse transformation is applied to recover the original scale:

Rescaling ensures that input features are within a uniform range, preventing dominance by features with larger numerical values. This enhances the convergence behavior of DL algorithms and contributes to better generalization performance.

2.1.9 Parameter settings

Hyperparameter tuning was carried out for RF, ARIMA, and SVR using grid search or automated routines, while DL models (RNN, LSTM, GRU) employed fixed configurations.

For ARIMA, the model parameters were automatically selected using the auto_arima package in Python, resulting in an optimal configuration of ARIMA(4,1,0) for all datasets. The differencing order d = 1 was used to make the time series stationary.

For SVR, hyperparameter tuning was performed using grid search, which explored the following parameter space:

• C ∈ [ 1 , 10 , 100 ] ,

• γ ∈ [ 1 × 1 0 − 6 , 1 × 1 0 − 5 , 1 × 1 0 − 4 ] ,

• Kernel: RBF, linear and polynomial.

For the RF model, hyperparameter tuning was performed using a grid search over a comprehensive parameter space to optimize predictive performance. The following hyperparameters were explored:

• Number of trees ( n estimators ): { 50 , 100 , 200 } ,

• Maximum tree depth ( max_depth ): { None , 10 , 20 , 30 } ,

• Minimum samples required to split a node ( min_samples_split ): { 2 , 5 , 10 } ,

• Minimum samples required at a leaf node ( min_samples_leaf ): { 1 , 2 , 4 } ,

• Maximum number of features considered for splits ( max_features ): { auto , max_features } ,

• Splitting criterion ( criterion ): { mse , mae } .

For the DL models , we used a fixed architecture with three layers and ReLU activation. The optimizer for all DL models was Adam with a learning rate of 0.001. Dropout was applied with a rate of 0.2 to prevent overfitting, and early stopping was used to halt training if the validation loss did not improve. The models were trained for 100 epochs with a batch size of 16.

The parameter settings for the DL models are summarized in Table 1. These settings ensure consistency across all models, enabling a fair comparison of their performance.

Explanation of parameters for RNN, LSTM, and GRU models:

• Number of layers: All models have three layers of recurrent units (RNN, LSTM, or GRU). More layers allow the models to capture more complex patterns in sequential data.

• Activation function: The activation function used in all models is ReLU (rectified linear unit), which helps mitigate the vanishing gradient problem and accelerates convergence.

• Loss function: The MSE is used as the loss function for all models. It measures the average of the squares of the errors, making it sensitive to larger errors and ensuring that the model minimizes them.

• Optimizer: Adam (Adaptive Moment Estimation) is used for optimization in all models. It adjusts the learning rate during training to improve convergence speed and accuracy.

• Learning rate: Set to 0.001 for all models; this determines the step size during model training. A small learning rate helps with more stable training.

• Dropout rate: Dropout of 0.2 is used to prevent overfitting. It randomly disables 20% of neurons in the network during training, forcing the model to learn more robust representations.

• Epochs: All models are trained for 100 epochs to ensure enough iterations for convergence.

• Batch size: A batch size of 16 is used, meaning 16 samples are processed at once during each training iteration.

• Units per layer: The number of units in each layer is set to 100 for the first layer, 50 for the second, and 25 for the third layer, with the goal of progressively reducing the complexity of the model.

• Early stopping: This mechanism halts training when the validation loss stops improving, preventing overfitting and unnecessary training.

The Python packages used in this study include Numpy for numerical operations and array handling, Pandas for data manipulation and analysis, Seaborn for data visualization, Matplotlib for creating static, animated, and interactive visualizations, Scikit-learn for ML models including SVR, RF, and performance metrics, Statsmodels for statistical models and time series analysis, Pmdarima for automatic ARIMA model selection, and Keras for building and training DL models.