Comparative evaluation of machine learning models for thrust force estimation in aluminum 5083 H111

2.1 Drilling process

In this study, cutting speed, feed per tooth, and cooling method were selected as the input parameters, each evaluated at three distinct levels, as shown in Table 2. The output parameter was the maximum cutting force (F _zmax) generated during the drilling process.

Drilling experiments were performed on a CNC vertical machining center (Taksan TMC-700 V) equipped with a FANUC O-M Series control panel. Cutting force data were acquired using an ESIT AX3 load cell integrated with a National Instruments (NI) cDAQ-9188 data acquisition system and an NI 9237 strain gauge input module. The data were recorded and visualized using NI FlexLogger software.

A high-speed steel cobalt tool (HSSE-Co5) coated with titanium aluminum nitride (TiAlN) was used in the drilling experiments. The cutting tool had a nominal diameter of 8 mm and was selected based on standard industrial practices for aluminum alloy machining. The detailed specifications of the cutting tool are presented in Table 3.

During drilling operations, cutting forces along all three axes F _x , F _y , and F _z were measured using a three-axis load cell. Among these, the thrust force component in the Z-axis F _z was primarily considered, as it directly reflects the resistance encountered by the tool in the feed direction and is most relevant to drilling process performance. A full factorial design was employed, resulting in a total of 27 experimental runs. Each experiment was repeated three times under the same conditions, and the average value of maximum thrust force (F _zmax) was used in model training.

2.2 Model development

In this study, two single supervised machine learning algorithms (ANN, DT) and two ensemble supervised machine learning algorithms (RF and XGBoost) were used to predict F _zmax during drilling of aluminum 5083 H111 alloy.

Artificial neural networks are formed by connecting artificial nerve cells with a specified weight value. This model makes decisions based on existing examples and the relationships between them [22]. However, single-layer artificial neural networks cannot learn nonlinear relationships. Therefore, a multi-layer perceptron (MLP) is used in the study. In addition, the backpropagation neural network algorithm is used because it is the most widely used learning technique [23].

Decision trees are an heuristical and interpretable machine learning method that generates decision rules by iteratively subdividing the dataset [24]. The tree structure consists of a hierarchical structure that starts from the root node and applies a feature test at each internal node, reaching class labels or predicted values at the leaf nodes. Each data sample follows a unique path to only one leaf node, and this path reflects the model’s decision process in the form of explicit rules. Decision trees can work effectively with both categorical and numerical features and can be converted into a set of rules in the form of “IF-THEN”, which increases the transparency of the model and its understandability for users [22].

Random Forest is a statistically powerful ensemble learning algorithm that uses a community of decision trees and is widely preferred in classification and regression problems [25]. This algorithm creates many decision trees with randomly sampled subsets and feature subsets from the training data; thus, the generalization ability of the model increases and the tendency for overfitting decreases [26]. Each tree is trained independently and the model output is calculated by majority voting in classification and arithmetic mean in regression [27]. Random Forest produces stable results by reducing the variance of the model and stands out with its high accuracy in multidimensional datasets. While the randomness within the trees helps the algorithm maintain the bias–variance balance, the computation of feature importance levels also increases interpretability [28]. XGBoost is a machine learning algorithm based on decision tree-based ensemble learning approach, providing high accuracy, speed and scalability. XGBoost, an extended version of the gradient boosting method, stands out with its strong performance in both classification and regression problems. It can work efficiently on large data sets with parallel computing support, while reducing the risk of over-learning thanks to regularization techniques [29]. In addition, the cross-validation method was used in the training phase of all models. This method, which is based on resampling the data, divides the data set into subgroups and uses each of these subgroups as test data in turn. In addition, the remaining subgroups are used as training data for the model. This process is repeated as many times (k) as the number of subgroups into which the data set is divided. In this method, called K-fold cross validation, a general performance value is obtained by evaluating each measurement result [30]. In this way, more reliable estimation results are achieved. It is frequently preferred by researchers, especially when working with small data sets, as it ensures that the results are more balanced and unbiased [31].

2.3 Performance criteria

In regression analysis, evaluating model performance is critical to understanding the accuracy and reliability of predictions. Model performances were evaluated using mean absolute error (MAE) and root mean squared error (RMSE), squared correlation (R ²).

MAE: This criterion expresses the average of the absolute values of the differences between the values predicted by a regression model and the true values. MAE allows direct interpretation of the mean deviation of the model, i.e., how far the predictions are from the true values on average. Despite the high interpretability of MAE, comparison becomes difficult when the variables are on different scales [32].

RMSE: This criterion is a performance measure calculated by taking the square root of the mean square of the forecast errors and is particularly sensitive to large errors. RMSE, which is frequently used in the literature, provides a stronger indicator of the consistency of the forecasts, as it also reflects the variance of the errors [32].

R ² : Coefficient of determination is a common statistical metric that measures how much of the variance in the dependent variable a regression model explains. R ² can automatically increase when unnecessary variables are added to the model, which can increase model complexity and provide misleading signals about overall performance. While a higher R ² value generally indicates a better model fit, it is important to interpret this value in the context of data and model complexity [32].

3.1 Experimental results

In this study, drilling tests were conducted using a full factorial design with three levels of cutting speed, feed per tooth, and cooling method. As a result, a total of 27 experimental runs were performed. For each drilling operation, the thrust force component (F _z ) was measured as the primary cutting force response.

The maximum thrust force (F _zmax) obtained from each experiment is presented in Table 4, along with the corresponding input parameters. In the table, v _c denotes the cutting speed (m min⁻¹), f _z denotes the feed per tooth (mm tooth⁻¹), and F _zmax represents the maximum cutting force in the Z-axis direction.

A general trend observed in the results (Table 4) indicates that the thrust force increases with both cutting speed and feed rate across all cooling strategies. The lowest cutting force was measured as 313 N in experiment 7, conducted at 80 m min⁻¹ cutting speed, 0.06 mm tooth⁻¹ feed rate, and under liquid cooling conditions. Conversely, the highest force was recorded as 1,026 N in experiment 3, where drilling was performed under dry conditions with 80 m min⁻¹ cutting speed and 0.12 mm tooth⁻¹ feed.

A force–time graph was generated for each drilling trial to observe the evolution of thrust force throughout the cutting process. As shown in Figure 4, the raw force signal exhibited noticeable fluctuations and noise due to high-frequency vibrations and data acquisition sensitivity.

Figure 4:

Sample force-time graph.

To address this, the exponential smoothing method was applied to the raw data in order to preserve the overall trend while reducing short-term fluctuations. This filtering process enabled a clearer interpretation of the maximum force values and drilling phase characteristics.

The force–time graph of experiment 1, shown in Figure 4, illustrates how the maximum thrust force was extracted from the raw measurement data. This approach was systematically applied to all 27 drilling trials. For each trial, the maximum force in the Z-axis (machining direction) was identified and recorded.

3.2 Machine learning algorithms results

Following the experimental evaluation of cutting forces, machine learning algorithms were used to develop a prediction model for F _zmax based on the process parameters. The data were analyzed using four supervised learning algorithms as ANN, DT, RF, and XGBoost. The machine learning models were implemented in the Spyder 3.1 development environment using the Python programming language, Keras, scikit-learn and XGBoost libraries. The models were trained using cross validation. Hyperparameter values and performance values of the developed models are given in Table 5.

The predicted cutting force values obtained from the models are presented alongside the experimental results in Table 6. For each of the 27 drilling trials, F _zmax predicted by models was compared to the experimentally measured value as shown in Figure 5.

Figure 5:

Graphical comparison between observed and model-predicted Fmax.

3.3 Model comparison

To assess the predictive performance of all four machine learning models used in this study ANN, RF, DT and XGBoost through performance metrics as MAE, RMSE and R ^2. The performance metrics results are summarized in Table 7 and visualized in Figure 6.

Figure 6:

The performance metrics results of all models.

In regression problems, MAE and RMSE performance criteria can take values between 0 and infinity and this metric is negative. In other words, lower values are better. However, it is desired that the R ² value, which expresses the explanatory power of the model, is high [33]. According to the performance comparison of four different regression models, XGBoost achieved the most successful results in all metrics. It was the model with the least prediction error with the lowest MAE (56.742) and RMSE (67.179) values, while it explained 93 % of the data variance with an R ² value of 0.929. The ANN model showed a moderate success, while the RF and especially the DT models were weaker due to their high error rates and low explanatory power. As a result of the general evaluation, the XGBoost model stands out as the most suitable regression algorithm in terms of both accuracy and generalization power.

3.4 Effect of the parameters

Three-dimensional (3D) surface plots were generated to visualize the combined effects of the input parameters cutting speed v _c , feed per tooth f _z , and cooling strategy on the maximum thrust force. These plots, presented in Figure 7, provide a comprehensive view of how changes in machining conditions influence cutting force behavior.

Figure 7:

Surface plots of parameters, a) F _zmax versus v _c ; f _z , b) F _zmax versus v _c ; cooling type, c) F _zmax versus f _z ; cooling type.

In Figure 7a, it is observed that increasing the feed per tooth f _z consistently leads to a rise in cutting force, while variations in cutting speed v _c have a relatively limited influence on the output. This suggests that f _z is a more dominant factor in force generation under these conditions.

Figure 7b highlights the impact of cutting speed and cooling strategy. It can be seen that liquid cooling significantly reduces cutting forces compared to dry and air-cooling methods. Moreover, lower cutting speed levels are associated with more favorable (lower) force values, indicating that minimal thermal and mechanical stress is exerted on the tool and material under these conditions.

Finally, Figure 7c examines the combined effect of feed per tooth and cooling strategy. The cutting force increases markedly with higher f _z values, and the highest force values are recorded during dry machining operations, confirming the critical importance of adequate cooling in high-load scenarios.

Figure 8 presents the probability plot of the maximum thrust force (F _zmax) obtained from the drilling experiments. This plot is used to assess the normality of the data distribution, which is an important consideration when applying certain statistical and machine learning models. The plotted data points align closely with the reference line and lie within the confidence bounds, suggesting a good fit.

Figure 8:

Probability plot of experimental maximum thrust force (F _zmax).

The Anderson–Darling (AD) statistic is 1.067 and the corresponding p-value is 0.007, indicating a slight deviation from perfect normality. However, given the small sample size (n = 27), this level of deviation is considered acceptable in engineering applications. Thus, the dataset is deemed suitable for use in subsequent regression and machine learning analyses.

The machine learning models developed in this study are based on data obtained from drilling experiments conducted on aluminum 5083 H111 alloy. The modeling process was carried out with the maximum thrust force (F _zmax) values measured under various process conditions where the cutting speed, feed rate and cooling strategies were changed.

This application provided a solid basis for evaluating the adaptability of the developed models to the production environment. When the comparative performance of four different machine learning algorithms (ANN, DT, RF, XGBoost) was examined, it was seen that the XGBoost algorithm gave the most successful results in predicting the maximum thrust force with the values of MAE: 56.742 N, RMSE: 67.179 N, R ²: 0.929. Although the ANN model also reached a relatively high explanatory value as R ²: 0.792, it fell behind XGBoost in terms of error rates. The DT and RF models, on the other hand, exhibited a limited prediction ability with higher error and lower accuracy rates.

These findings reveal that ensemble methods provide more reliable results in complex manufacturing processes. These results show that XGBoost has a strong predictive capacity in application areas where multivariable and nonlinear structures are involved, such as manufacturing engineering. This situation is also consistent with the literature. For instance, Akdulum and Kayir reported that XGBoost was more successful than ANN and RF in their study on the AA6061-T651 alloy [34]. Alajmi and Almeshal achieved R ² ≈ 0.997 in tool wear prediction using another hybrid XGBoost approach, substantially outperforming SVM and MLP-ANN [35]. These examples underscore XGBoost’s capability in capturing nonlinearities and complex parameter interactions inherent in aluminum machining operations.