Cross-dataset evaluation of deep learning models for crack classification in structural surfaces

4.3.1 CNN

A special CNN structure was created for this work, where the issues of structural surfaces were matched to the two categories set for “Cracked” and “Non-Cracked.” The three-part structure consists of convolutional layers where each one is succeeded by ReLU activation functions that bring forward nonlinearity as well as max-pooling layers for lowering the spatial dimensions of the feature maps. Convolutional layers have a step-by-step change that the filter size becomes 32, then 64, and finally 128; this configuration involves a kernel size of 3 and the use of padding for keeping the dimensions of the feature maps at every stage of the process. This design allows for the step-by-step capture of complicated, hierarchical features from the input images. Apart from the convolutional layers, there is also a fully connected layer with 256 neurons arranged that the output from the convolutional layers operates as an input that is already in a dense form. This layer also utilizes ReLU activation and is then followed by a dropout layer with a rate that is set at 50%, which additionally lowers the risk of overfitting. The output of the model is the 1-neuron last layer that has sigmoid activation, and its primary function is to provide a regular value of the probability that a picture is damaged with binary classification being the task whereas computer vision is the field. Such a model is not only simple and straightforward but also powerful.

4.3.2 ResNet50

The adapted ResNet50 model, which is well known for its powerful feature extraction capabilities and the use of residual connections to prevent deep networks from becoming less powerful due to the vanishing gradient problem, has been subjected to transfer learning for the task of binary classification of cracks in structures [62]. In order to not lose the ImageNet acquired patterns, all layers of the pre-trained ResNet50 model are frozen. Instead of the usual fully connected layers, we developed a new and special classification head for our architecture. This head is composed of a dense layer which includes 256 neurons and ReLU activation to add nonlinearity to the cost of enabling the model to learn complex patterns. Furthermore, a 50% dropout rate is incorporated to prevent overfitting in this layer. The end result of this architecture is a happening of the output layer. The output layer consists of a single neuron that employs a sigmoid activation function which in turn is responsible to output the images’ probability scores and classify them as “Cracked” or “Non-Cracked.” This change exploits the pre-trained capabilities of ResNet50 for feature extraction while at the same time re-factoring the model for the purpose of efficacious and reliable crack detection in binary classification settings.

4.3.3 VGG16

The VGG16 model was initially pre-trained on ImageNet [63] and was then modified for the purpose of binary classification tasks in the field of crack detection. The freezing of the convolutional layers of the VGG16 model, in order to still utilize the features of ImageNet, by the way, is recognized as a process that not only saves computer resources but does it in such a way that it prevents overfitting, taking as an example those feature vectors, which are well generalized and coming from different types of images. The VGG16 model’s existing classifier is replaced by the newly designed dense layer structure that is dedicated to binary classification. Shaped from one layer with 256 fully connected neurons to the next, the activation function ReLU is used in order to add a nonlinear capacity. This in turn will increase the model’s power to decode complicated patterns in the data. This layer contains a 50% data dropout rate which ensures data are not overfit. A sigmoid output layer concludes the structure of architecture. For a given input, the outcome is a probability score which indicates the categorization of the image as “Cracked” or “Non-Cracked.” The current state of the VGG16 model is regulated so that the utmost potential in performance and computational efficiency can be achieved, so it is the most suitable for precise crack detection operation.

4.3.4 LSTM

In our project, we have implemented a model composed of an LSTM network to recognize images and use them for binary classification as “Cracked” or “Non-Cracked.” Popularly known for their performance with sequential data, an LSTM network [64] has been modified especially to cope with the image-to-sequence problem. This is done through intermediate steps of images to sequences conversion. The sequence is processed by standard LSTM with a hidden space of 128, and it allows the model to capture the time series of the data. At the next step, the last hidden state from the LSTM layer is fed to a dense classification layer to make the final decision more tailored. This network includes a dense layer with 256 units, using ReLU as the activation function for introducing nonlinearity and dropout with a rate of 50% to overcome overfitting. The concluding part is a one-node layer with sigmoid activation making the estimation of the probability of the input image being “Cracked.” The idea presents a mix of a sequential model and a classification model, where the LSTM’s strengths have been used for data integration to yield and become a new and proficient binary classification model for crack detection in images.

Motivation for including LSTM

Although LSTM models are traditionally applied to sequential or time-series data, we included LSTM in this study to evaluate whether its ability to capture long-term dependencies could offer benefits when 2D crack images are reshaped into sequential input vectors. This exploratory inclusion tests the limits of sequential learning on spatially encoded features, especially since cracks often exhibit line-like or progressive structures. Our experimental results, however, indicate that LSTM underperforms compared to convolution-based models, confirming its limitations for spatially driven image classification tasks.

5.1.1 CNN model loss curves

Figure 1 with the training and validation loss lines for the CNN model over six datasets, it is evident that loss curves in most of the cases consistently show falling values for both training and testing. They show a good leaning shape of the CNN model. The plots for SDNET 2018 and CCI datasets display that the training and validation losses nearly become one, which tells us that the model is fitted well without overfitting. The model is thus able to generalize satisfactorily to unknown validation data. In the SCD and CDIBM datasets, as training continues, the validation loss tends to flatten out or fluctuate slightly indicating that the model is stable in learning but the validation data are noisy or suffers from some variability. In the CPC dataset, there is a significant drop in the training and validation losses after which the convergence shows that the learning has been efficient and the model has an appropriate capacity for this dataset. However, the varying nature of the validation losses of the HBC dataset at later epochs suggests that there might be some overfitting or the validation set became more complicated.

Figure 1

The progress of the CNN model in terms of the training and validation loss of (a) SDNET 2018, (b) SCD, (c) CPC, (d) CDIBM, (e) CCI, and (f) HBC.

Typically, the loss curves have little diversion which usually implies that the training and validation losses stay close. The good news is that the CNN model is finding a balance between not fitting the model enough and fitting it too much. The graphs seem to indicate that the CNN model is the most proper model for this job, although the differences found in datasets might be explained by the complexity of the data and the variations in class proper.

5.1.2 ResNet50 model loss curves

Figure 2 sequence demonstrates the evolution of the ResNet50 model’s training and validation loss on six various datasets. From the image, one can understand that the curves describe that both the training and validation losses are effectively reduced throughout the training process on all six datasets, which reveals that the ResNet50 is able to learn well the features during the training process. With SDNET 2018 dataset, the validation error follows a pattern of a reduction in the training loss error, hence it can be said that the model has good generalization. However, SCD and CPC data show a sharp decrease in the loss in the beginning, but an increase in the second period, which is indicative of efficient learning and convergence. Meanwhile, the changes in both the training and validation losses in CDIBM dataset are not so big and resemble two virtually identical lines, the deviation only in validation loss being very small, so the model seems not to be greatly overfitted and holds good stability. On the other hand, while the training loss of CCI dataset declines throughout, there is a clear fluctuation in the validation loss, which might be attributed to the presence of overfitting in the training data or some unusual states of the validation set. Likewise, in the case of the HBC dataset, the validation loss fluctuates inconsolably despite the fact that the training loss goes down steadily, mainly in the later stages thus reflecting possible sensitivity to the dataset’s complexity.

Figure 2

The progress of the Resnet50 model in terms of the training and validation loss of (a) SDNET 2018, (b) SCD, (c) CPC, (d) CDIBM, (e) CCI, and (f) HBC.

The ResNet50 model demonstrates strong learning capabilities across all datasets, with minimal divergence between training and validation loss curves. The observed fluctuations in some datasets could be attributed to variations in data quality or complexity, but the trends suggest that the model generalizes well to the validation data, particularly for datasets with smoother loss curves.

5.1.3 LSTM model loss curves

Based on the training and validation loss curves for the LSTM model across six datasets shown in Figure 3, it can be seen that the training and validation loss trends exhibit notable differences across the datasets, reflecting the challenges of adapting an LSTM model to image-based tasks. In SDNET 2018 dataset, both training and validation losses are relatively stable, with only minor improvements over epochs. This indicates limited learning and potential difficulty in modeling the dataset’s features using the sequential approach of the LSTM. SCD dataset shows fluctuations in validation loss despite a steady decline in training loss, suggesting overfitting or variability in the validation data. For CPC and CCI datasets, the training loss decreases smoothly, but validation loss exhibits significant fluctuations and even increases toward later epochs, indicating overfitting to the training data. This suggests that the LSTM model struggles with generalization on these datasets. The CDIBM dataset shows less fluctuation of the trends on training, as well as validation losses to converge, reflecting that the learning is effective and the model is stable. The case of HBC dataset demonstrates validation loss not in a steady state, it goes a little bit up and down in the trend of the training loss thus indicating not very good generalization with some space for improvement.

Figure 3

The progress of the LSTM model in terms of the training and validation loss of (a) SDNET 2018, (b) SCD, (c) CPC, (d) CDIBM, (e) CCI, and (f) HBC.

Overall, the LSTM model’s loss curves highlight its limited suitability for image-based tasks, with consistent signs of overfitting or inadequate feature extraction. The results suggest that this sequential modeling approach may not be optimal for crack classification tasks, especially when compared to CNN- or ResNet50-based architectures.

5.1.4 VGG16 model loss curves

Based on the training and validation loss curves for the VGG16 model across six datasets shown in Figure 4, it can be seen that the VGG16 model demonstrates a generally effective learning process, with most datasets showing a steady decline in both training and validation losses. In SDNET 2018 dataset, the validation loss follows the training loss closely, with slight fluctuations, indicating good generalization and stable learning. SCD dataset shows a more pronounced gap between training and validation losses, particularly in later epochs, suggesting potential overfitting, as the model performs better on the training data than on validation data. For CPC dataset, both losses decrease rapidly and converge closely, highlighting strong generalization and efficient learning on this dataset. Similarly, CCI dataset shows consistent decreases in both training and validation losses, with minor fluctuations, indicating robust performance. CDIBM dataset, however, exhibits notable divergence between training and validation losses, suggesting overfitting, potentially due to increased complexity or variability in the dataset. Finally, in HBC dataset, the validation loss closely tracks the training loss throughout the epochs, indicating stable training and good generalization.

Figure 4

The progress of the VGG16 model in terms of the training and validation loss of (a) SDNET 2018, (b) SCD, (c) CPC, (d) CDIBM, (e) CCI, and (f) HBC.

In general, the VGG16 model really can get most of the features of the datasets in a very effective way, revealing a very high performance with no or almost no overfitting in some points of the curve. Nevertheless, the tremendous disparity observed in certain datasets justifies the use of data augmentation or regularization as a methodology to increase further generalization. The model’s performance on different datasets definitely proves that it is the best for crack classification tasks especially when having well-prepared and balanced datasets.

The curves of the training and the validation losses of the CNN, ResNet50, LSTM, and VGG16 show the specific features and weaknesses of these models in the crack classification task. Both CNN and VGG16 exhibited consistent and powerful generalization skills manifested in almost identical values of training and validation losses. That made these two models the most superior for image-based classification. As far as the ResNet50 model is concerned, it showed a clear learning procedure and robust feature extraction although certain pieces of evidence from the graphs, occasional spikes in the validation loss for instance, indicated that the model was more sensitive to the dataset. The LSTM model, however, had trouble achieving the same kind of generalization, as the validation loss figures fluctuations and divergence, that is, the model could hardly be applied to the task, which is due to the natural processing of the sequential process. The VGG16 model turned out to be the most robust of the group, with the order of resilience being ResNet50 and CNN, while the LSTM model was not a good fit for the classification purpose and was rather the least effective. The next part of the article will present more detailed comparisons of the models’ performances using major indicators such as accuracy, precision, recall, and F1-score that help in the final assessment of their performance and generalizability.

5.3.1 Training on SDNET 2018 and test on remaining datasets

Table 6 presents the cross-testing accuracy for models trained on the SDNET 2018 dataset and tested on other datasets. These results provide insights into the generalization capabilities of the models when trained on a dataset with a specific distribution and applied to different domains.

The results highlight significant performance variations among the models and across datasets, revealing the challenges of achieving consistent accuracy in cross-domain scenarios. The VGG16 model demonstrates the highest level of generalizability, achieving the top accuracy across most datasets, including CPC (91%), SCD (82%), and HBC (87%). However, its accuracy drops on the CCI dataset (64%), reflecting potential domain-specific limitations. The ResNet50 model performs strongly on CPC (87%) and HBC (89%) but struggles with CDIBM (38%) and CCI (56%), indicating sensitivity to certain dataset characteristics. The CNN model shows mixed performance, with relatively strong results on CDIBM (96%) and HBC (82%) but poor generalization on SCD (56%) and CPC (55%). These inconsistencies highlight the CNN’s limited ability to adapt to unseen datasets compared to VGG16 and ResNet50. The LSTM model exhibits the lowest performance overall, with accuracy scores below 85% on all datasets and notably poor results on SCD (50%) and CCI (49%). This shows its limited ability to capture spatial features effectively. Table 7 provides precision, recall, and F1-score for models trained on the SDNET 2018 dataset and tested on other datasets, offering a deeper understanding of their performance in cross-domain scenarios.

The VGG16 model consistently achieves the best or near-best performance across all metrics and datasets. It excels in precision and recall for SCD (86% precision, 82% recall) and CPC (92% precision, 91% recall), resulting in high F1-scores of 82 and 91%, respectively. However, its performance drops slightly for CDIBM (F1-score of 47%) and CCI (F1-score of 59%), reflecting domain-specific challenges in generalization. The ResNet50 model also performs well, particularly for CPC (F1-score of 87%) and HBC (F1-score of 77%), showcasing its ability to adapt to some datasets. However, its performance significantly drops for CDIBM (F1-score of 28%) and CCI (F1-score of 52%), suggesting that while ResNet50 is precise in certain cases, it struggles to maintain consistent recall across datasets. The CNN model demonstrates moderate performance, with better results for HBC (F1-score of 51%) and CDIBM (F1-score of 59%), but lower scores for SCD and CCI, where its recall is particularly low, indicating difficulty in capturing true positives across these datasets. In contrast, the LSTM model struggles significantly, with the lowest F1-scores across all datasets. It achieves only 33% on SCD and 41% on CPC, highlighting its limitations in capturing strong spatial feature extraction.

5.3.2 Training on SCD and test on remaining datasets

Table 8 presents the cross-testing results, where models trained on SCD dataset are evaluated on datasets they were not trained on, offering insights into their generalizability.

The VGG16 model consistently demonstrates the highest accuracy across most datasets, particularly excelling on CPC (96%) and showing strong performance on SDNET 2018 (82%) and HBC (87%). However, its accuracy drops significantly on CDIBM (26%), highlighting a potential limitation in adapting to datasets with significantly different distributions. The ResNet50 model displays very high levels of performance in SDNET 2018 (84%) and HBC (88%) while it demonstrates a quite low 16% accuracy rate in CDIBM, which is in line with VGG16. Furthermore, a 64% accuracy on CCI is a signal that ResNet50 is also less stable in handling domain changes compared to VGG16. It can be seen that the CNN model is giving a good account of itself in CPC (86%) and HBC (85%) and it is only on CDIBM (15%) where a notable performance drop is observed, demonstrating a limited ability to transfer learning to more challenging datasets. The basic performance on SDNET 2018 (76%) and CCI (73%) is an additional proof of its niche and case-related effectiveness. Conversely, the LSTM model demonstrates its worst performance thus outclassing the others with the maximum accuracy rate of 85% only on CDIBM, which is mainly due to the inherent nature of the dataset that has a positive inclination toward the sequential architecture. The fact that the LSTM model can cope with the image-based tasks only on a very basic level is visible from its pretty poor performances in CPC (56%) and CCI (60%). Table 9 details out precision, recall, and F1-score for models trained from the SCD dataset to the other ones, providing a more in-depth knowledge of their performance across domains.

The VGG16 model demonstrates the strongest performance across most datasets. It achieves near-perfect precision, recall, and F1-scores on CPC (96%) and robust results on CCI (F1-score of 80%) and HBC (F1-score of 81%). However, its performance drops significantly for CDIBM, with a low F1-score of 22%, indicating difficulties in generalizing to datasets with distinct distributions. The ResNet50 model also shows strong performance, with high F1-scores for CPC (89%) and HBC (80%). However, it struggles with CDIBM (F1-score of 15%) and shows moderate performance on CCI (63%). These results suggest that while ResNet50 is capable of handling certain datasets, it faces challenges with datasets exhibiting significant domain differences. The CNN model achieves competitive performance on CPC (F1-score of 86%) but performs poorly on CDIBM (F1-score of 14%) and moderately on CCI (F1-score of 70%). Its low recall values for SDNET 2018 and CDIBM highlight limitations in capturing true positives across different datasets. The LSTM model demonstrates the weakest performance overall, with F1-scores below 60% for most datasets. While it achieves reasonable precision and recall on HBC (F1-score of 58%), it consistently struggles with datasets such as CDIBM (F1-score of 15%) and CPC (F1-score of 50%).

5.3.3 Training on CPC and test on remaining datasets

Table 10 presents the cross-testing accuracy for models trained on the CPC dataset and tested on other datasets. These results provide insights into the generalization capabilities of the models when trained on a highly specific dataset and applied to different domains.

The VGG16 model again demonstrates the strongest overall performance, achieving the highest accuracy on SCD (99%) and robust results on CCI (79%) and HBC (86%). However, its performance significantly drops on CDIBM (22%), reflecting challenges in adapting to datasets with distinct feature distributions. The ResNet50 model also performs well, with strong results on SCD (95%) and HBC (78%) but moderate performance on CCI (66%). Similar to VGG16, ResNet50 struggles with CDIBM, achieving only 23% accuracy, indicating domain-specific limitations in this dataset. The CNN model achieves competitive accuracy on SCD (94%) and moderate results on SDNET 2018 (78%) and CCI (68%). However, its performance on CDIBM is extremely low (10%), highlighting its limited ability to generalize to this dataset. In stark contrast, the LSTM model performs poorly across most datasets, with particularly low accuracy on CDIBM (49%) and HBC (27%). While it achieves reasonable accuracy on SDNET 2018 (21%) and CCI (52%), these results further emphasize the challenge for cross-domain generalization in image-based tasks. Table 11 provides a detailed comparison of precision, recall, and F1-score for models trained on the CPC dataset and tested on other datasets, offering a nuanced evaluation of the models’ cross-domain performance.

The VGG16 model demonstrates the strongest overall performance, achieving near-perfect precision, recall, and F1-scores on SCD (99%) and high F1-scores on CCI (79%) and HBC (81%). However, its performance drops on CDIBM, where its F1-score falls to 20%, indicating difficulties in adapting to datasets with distinct feature distributions. Nevertheless, its balanced precision and recall across most datasets affirm its robustness for generalization. The ResNet50 model performs competitively on SCD (F1-score of 95%) and HBC (F1-score of 78%), showcasing its ability to adapt to some datasets. However, like VGG16, its performance significantly drops on CDIBM, where it achieves an F1-score of only 20%. This trend highlights ResNet50’s domain-specific limitations, particularly when facing datasets with different distributions. The CNN model achieves good performance on SCD (F1-score of 94%) but struggles on other datasets, particularly CDIBM (F1-score of 10%) and HBC (F1-score of 57%). Its recall values for CCI (68%) and HBC (67%) indicate limitations in capturing true positives, further highlighting its restricted generalization capabilities. The LSTM model demonstrates the weakest performance, with F1-scores below 50% for most datasets. It achieves its highest F1-score on SCD (36%) but performs poorly on CDIBM (36%) and HBC (27%).

5.3.4 Training on CDIBM and test on remaining datasets

Table 12 summarizes the cross-testing accuracy for models trained on the CDIBM dataset and tested on other datasets. These results highlight the models’ generalization performance when trained on a dataset with distinct characteristics and evaluated on different domains.

The results show minimal variation across models, with CNN, ResNet50, and VGG16 all achieving identical accuracy scores on most datasets. For SDNET 2018, the models demonstrate strong generalization with an accuracy of 83%, reflecting their ability to adapt to this dataset’s features. However, performance on SCD, CPC, and CCI is notably weaker, with accuracy scores hovering around 50–54%, indicating significant challenges in cross-domain generalization for these datasets. All models perform well on HBC, achieving 81% accuracy, suggesting that this dataset’s characteristics align more closely with the training dataset (CDIBM). The LSTM model performs similar to the other architectures, achieving comparable accuracy across all datasets. However, this consistency at relatively low accuracy levels reinforces the general observation that LSTM struggles to extract meaningful spatial features for image classification tasks. To sum up, all models show almost the same performance (the trend is consistent), with SDNET 2018 and HBC being the ones giving the highest accuracy and SCD, CPC, and CCI the ones presenting the biggest generalization problems. It is that the data distribution among different datasets and the feature similarity have a significant impact on domain-generalization, regardless of the architecture of the model being applied. In these results, it can be inferred that the models such as VGG16, and ResNet50 that are usually regarded as the most stable ones, are not capable to cover the dissimilarity between the datasets in CDIBM training set to that extent that they can well generalize to other dissimilar data. Table 13 contains the details regarding model performance for precision, recall, and F1-score on the CDIBM dataset when the models were tested on the other datasets.

The results show that all models perform similarly across most datasets, reflecting challenges in generalizing from the CDIBM training dataset. Across all datasets, the F1-scores remain below 50%, indicating poor performance in cross-domain scenarios. VGG16 shows relatively higher precision compared to other models, particularly for CCI (76%) and CPC (73%). However, its recall is consistently low, leading to F1-scores below 50% across all datasets. This suggests that while VGG16 avoids false positives, it struggles to capture all positive cases when applied to unseen datasets. ResNet50 demonstrates balanced precision and recall for SDNET 2018 (50%) and HBC (47%) but falls short for other datasets, especially CCI (F1-score of 35%). This reflects limited adaptability to datasets with different feature distributions. CNN performs marginally better than ResNet50 on CPC (F1-score of 35%) but demonstrates significant weaknesses in recall for datasets like CCI (34%). Its F1-scores are low across the board, indicating challenges in achieving a balance between precision and recall. LSTM consistently underperforms, with its F1-scores rarely exceeding 45%. While it achieves moderate recall across most datasets, its precision is low, particularly on CPC (33%) and CCI (34%), leading to poor overall performance.

5.3.5 Training on CCI and test on remaining datasets

Table 14 presents the cross-testing accuracy for models trained on the CCI dataset and evaluated on other datasets. The results highlight significant variability in model generalization capabilities across datasets, indicating the challenges of training on a dataset with specific characteristics and applying the models to diverse domains.

The VGG16 model achieves the highest accuracy for SCD (78%) and CPC (78%), demonstrating its ability to generalize to datasets with somewhat similar features. However, its performance drops sharply on CDIBM (6%) and HBC (69%), reflecting its limitations when encountering datasets with distinct distributions. The LSTM model performs reasonably well on SDNET 2018 (76%) and HBC (73%) but struggles on CDIBM (35%), similar to other models. While its performance on certain datasets is competitive, its overall accuracy remains inconsistent. The ResNet50 model appears to be working well with datasets HBC (78%) and SCD (77%), which is why it is considered adaptable to such datasets. On the other hand, its performance for SDNET 2018 (39%) and CDIBM (4%) is a tell-tale sign that the model could not see the forest for the trees and is unable to reach high-level abstraction. On CPC, CNN produces the best accuracy (75%) but the situation is different with CDIBM (8%) and SDNET 2018 (41%) where its errors are far too high. Through these results, one can infer that CNN performs better when the datasets do not contain much different information. It is clear that VGG16 is very much successful at generalizing across the datasets SCD and CPC, but since the performance of all the models is not consistent with CDIBM, they inherently exhibit very low accuracy on the CDIBM dataset. The LSTM model, in particular, is capable of giving the other models a tough match. The results also point out that a correct approach to the choice of benchmark samples is crucial to attain a wider applicability. Tightly dedicated-hardware architectures can also have a contribution to the storage/retrieval issues of various datasets. Table 15 gives details of the precision, recall, and F1-score of all models, trained on CCI and tested on several datasets sorted from the top of the CCI group (Table 15).

Even though it dropped significantly when identifying CDIBM, the VGG16 model still outperformed all other models of the dataset on SCD (F1-score of 77%) and CPC (F1-score of 78%), suggesting that the model can generalize effectively to datasets with similar characteristics. Datasets that are largely different from the training set have become the reason for the model’s performance to be so low, thus the F1-score decreased dramatically to 6%. Its performance on HBC (F1-score of 65%) highlights a moderate level of adaptability. ResNet50 model performs competitively on SCD (F1-score of 76%) and HBC (F1-score of 70%) but struggles with CDIBM (F1-score 4%) and SDNET 2018 (F1-score of 38%). These results suggest that ResNet50 is effective in handling certain datasets but faces significant challenges with diverse domains, particularly those with different feature distributions. The CNN model performs moderately on CPC (F1-score of 75%) and SCD (F1-score of 50%) but exhibits poor results on CDIBM (F1-score of 8%). Its recall values for SDNET 2018 (55%) and HBC (59%) are higher than its precision, reflecting a tendency to identify more positives at the cost of false positives, which impacts its overall performance. The LSTM model achieves decent results on SCD (F1-score of 68%) and HBC (F1-score of 59%) but struggles significantly on CDIBM (F1-score of 28%) and CPC (F1-score of 58%). While it shows moderate precision and recall on some datasets, its overall F1-scores remain low.

5.3.6 Training on HBC and test on remaining datasets

Table 16 highlights the cross-testing accuracy for models trained on the HBC dataset and tested on the remaining datasets. This evaluation provides insights into how well the models generalize when trained on the HBC dataset.

VGG16 model achieves excellent accuracy on SCD (91%) and CPC (93%), demonstrating strong generalization to datasets with similar characteristics. However, its performance drops significantly on CDIBM (36%), indicating challenges with datasets that are highly divergent from HBC. The model performs well on CCI (86%), maintaining a high degree of adaptability. ResNet50 model performs strongly on SDNET 2018 (87%) and CPC (91%), showcasing its ability to generalize effectively to datasets with some shared features. Its accuracy on the SCD subset (81%) is also highly remarkable, although on CDIBM (71%) and CCI (64%), there is a loss in performance that means the model cannot adapt smoothly to all datasets. The CNN model is to a certain degree capable of generalizing and it can still achieve high accuracy in SDNET 2018 (74%) and CCI (78%). However, it is not able to accurately learn from SCD (56%) and CDIBM (53%) and the latter are significantly lower, which means that it can only use the same distribution of features in a series of datasets. The bad performance of the LSTM model on SCD (50%) and CPC (50%), which is its difficulty in extracting spatial features for cross-domain tasks, becomes apparent. In contrast, it excels at CDIBM (97%), the highest achievement, thus, demonstrating the effectiveness of this dataset for the model. It has got a low score on CCI (51%) which is inconsistent at the same time it is less to be found in the other datasets (Table 16). Precision, recall, and F1-score, which are key technical aspects of models trained by the HBC dataset and tested on the other datasets, are thoroughly covered in this analysis. This kind of analysis does not only make it easy to see which are the weaknesses and strengths of the model when applied to data not seen before, but also gives a more thorough insight into their generalization features (Table 17).

VGG16 model is known for consistently achieving the best results in datasets on which it was tested. Also, it reaches precision, recall, and F1 scores that indicate that it does best of all models on SCD and CPC with 91 and 95%, respectively. At the same time, the VGG16 model shows very poor results on CDIBM (28%). Its precision and recall are the main issues on this dataset (49 and 42%, respectively), which means that it is not able to capture features present in distributions that are too different. At last, VGG16 model keeps quite good accuracies on CCI (F1-score 85%) and SDNET 2018 (66%). ResNet50 model presents the example of good results, especially on CPC (91%) and SCD (80%) with very high precision and recall values. But its situation with CDIBM (F1-score 43%) is the same as VGG16’s and it performs not bad on CCI (64%) and SDNET 2018 (67%). From this we infer that ResNet50 fits well with datasets that share some common data but it is not able to perform well on those that have very different entities. The CNN model performs moderately, achieving its best F1-score on CCI (77%) and maintaining decent performance on CPC (60%) and SCD (53%). Its recall on CDIBM (39%) is particularly low, resulting in a poor F1-score (36%). Its overall performance indicates limited generalization capabilities compared to VGG16 and ResNet50. The LSTM model continues to underperform across most datasets, with F1-scores below 50% for all but CCI (34%). Its highest F1-score of 50% is on SDNET 2018, but it struggles significantly with spatial feature extraction, as seen in datasets like CPC (34%) and SCD (34%). Figure 6 shows samples of the misclassified images for cross-testing phase.

Figure 6

Samples of the misclassified images for cross-testing datasets.

The observed performance drops during cross-testing across the six datasets – SDNET 2018, SCD, CPC, CDIBM, CCI, and HBC – can be attributed to inherent variability and dataset-specific characteristics. Differences in resolution and image quality significantly impacted generalization, as models trained on high-resolution datasets like SCD (4,032 × 3,024) struggled with the lower-resolution datasets such as HBC (128 × 128 to 256 × 256), which lacked fine-grained details. Conversely, resizing higher-resolution images to smaller dimensions, as seen in CPC and CDIBM (227 × 227), resulted in a loss of critical crack details, especially for detecting narrow fractures. Surface variability further complicated model performance; datasets like SDNET 2018 encompassed diverse surfaces (bridge decks, walls, pavements), while more specialized datasets such as HBC and CDIBM focused on historical buildings or masonry walls with distinct textures. Dataset size and distribution were also critical factors. Small datasets, such as CCI (2,126 images) and HBC (3,886 images), lacked sufficient diversity, often leading to overfitting, while larger datasets like SDNET 2018 (56,000 images) offered broader variability. However, imbalanced distributions in SDNET 2018 introduced biases that impacted generalization. Environmental and contextual challenges, such as varying lighting conditions in SCD or outdoor noise in CDIBM, further hindered model adaptability, particularly for datasets collected under controlled conditions like HBC. Networks such as VGG16 and ResNet50 have shown excellent feature extraction and generalization capabilities on datasets with identifiable structures (e.g., SCD, CPC); however, the same networks were not able to perform well on more complex ones like CDIBM and CCI with reduced resolution and environmental inconsistency. Networks like CNN that were less complex had trouble generalizing, and LSTM, being weak at spatial feature extraction, performed poorly in all datasets.

These methods, despite being standard, brought about significant degradations of the test performance by all the models when they were evaluated on the new data. One of the reasons why this problem is so critical is that the most basic augmentation, such as random flipping and rotation, simply does not create variation and complexity to be found in the new sets of data in this cross-domain crack image detection problem. These changes in image resolution, surface texture, brightness, and noise in the picture and atmosphere within the various datasets became the obstacles that training samples with the help of augmentation could not fully overcome. Consequently, the deep learning models while working well in one situation tested themselves, failed in the generalization of new datasets structured or visual features of the other type. These results indicate that there is a dire need for domains of data to be able to utilize the techniques of domain adaptation or other forms of sophisticated augmentation to close the gap between datasets and allow the models to be robust in real-world applications.

5.3.7 Model complexity and deployment considerations

In addition to performance metrics, practical deployment scenarios – such as drone-based inspections or real-time edge inference – require evaluating model complexity and computational cost. Table 18 summarizes the architectural characteristics of each model, including total trainable parameters, average training time per epoch, inference time per image, and deployment-related remarks.

ResNet50 and VGG16 are considerably more computationally intensive than the custom CNN and LSTM. VGG16, in particular, has ∼138 million parameters, resulting in longer training times and higher memory consumption. While these models offer superior generalization, they may not be ideal for resource-constrained environments.

Conversely, CNN and LSTM architectures are significantly lighter. The CNN model, with only ∼1.2 M parameters, completes training in under 7 s per epoch and achieves inference times below 20 ms per image. Despite its lower complexity, CNN remains competitive in in-domain accuracy, though its cross-dataset robustness lags behind deeper models. LSTM demonstrates moderate efficiency in terms of computational cost but exhibits weaker performance due to its limited spatial feature extraction capabilities.

To complement the empirical evaluation, a theoretical comparison of the four deep learning models – CNN, ResNet50, VGG16, and LSTM – was conducted to assess architectural suitability for crack classification. CNNs, due to their spatial locality and efficiency, are well suited for fast deployment on embedded systems. ResNet50 offers a good compromise between depth and generalization, leveraging skip connections to mitigate vanishing gradient problems in deeper networks. VGG16 is effective in fine-grained tasks but computationally intensive. LSTM, while useful in sequential modeling, shows limited utility in static image classification.

While this study focused on benchmarking standard architectures, future work may explore hybrid models that combine spatial and temporal feature extraction – such as CNN-LSTM hybrids or attention-based encoders – to leverage the complementary strengths of different architectures for crack classification under diverse conditions.

5.3.8 Cross-domain performance limitations

The observed drop in performance during cross-testing is primarily attributed to domain shift factors such as image resolution mismatch, background texture complexity, and class imbalance. For instance, models trained on SCD (well-balanced, high-resolution) dropped up to 42% in F1-score when tested on HBC, which contains lower-resolution, noisy images with high background variability. Similarly, ResNet50 trained on CPC exhibited a 32% decrease in precision when tested on CDIBM, likely due to the introduction of occlusions and varying lighting conditions. Furthermore, class imbalance in target datasets amplified false negative rates. In the case of CCI, the prevalence of non-crack images led models trained on balanced datasets to overpredict the majority class.

The generalization performance of the evaluated models is thus closely influenced by dataset-specific attributes such as resolution, surface texture, background complexity, and environmental conditions. Structured, high-quality datasets enable strong in-domain performance but do not adequately prepare models for the variability encountered in unseen datasets. These patterns were consistently observed across all evaluated models, underscoring the need for more adaptive learning strategies. While transfer learning helped reduce initial training requirements and improve in-domain accuracy, its benefit in mitigating domain shift was limited unless fine-tuning was applied on the target dataset. This is evident from the fact that ResNet50 with transfer learning still failed to maintain high F1-scores during cross-domain testing.

Although this study did not involve controlled experiments isolating individual dataset characteristics (e.g., resolution-only or lighting-only shifts), the observed trends across cross-testing results strongly suggest that dataset diversity plays a central role in shaping model generalization. These findings reaffirm that high accuracy on individual datasets does not guarantee robustness across domains and highlight the importance of future work on domain-invariant feature extraction, style transfer, or meta-learning techniques for improved cross-dataset crack classification.