A comprehensive review of deep learning and machine learning techniques for early-stage skin cancer detection: Challenges and research gaps

1 Introduction

In the recent past, machine learning (ML) algorithms in computer vision have enhanced the development of computer-aided diagnosis and early detection system for dangerous cancerous skin diseases [1]. Skin cancer have been diagnosed and detected through human examination, probably a visual examination of the affected skin. These approaches of visual inspection and screening the lesion images by dermatologists are tedious and time taking, having complex methodologies, and most importantly, the results depend on the observational ability of the dermatologists and as such the conversion of the images into binary form can also have some errors [2]. This is mainly attributed to the fact that the skin lesion images can be very much detailed. Before analysis, interpretation, and understanding of skin lesion images, the lesion pixels must be uniquely defined. This may be difficult due to the following reasons: First, in skin lesion images, there are body hair, blood vessels, oils, bubbles, and other noise existence that affects segmentation process. Also low contrast between the surrounding skin and the lesion area complicates the process of segmentation of the lesion. Then, skin lesions can be median, macular, or popular and may be symmetrical, oval, and multiple, and the surface may have distinct colours, which makes the shape, size, and colour restriction lower the performance of the methods to a higher level of accuracy [3].

It has made a manual screening a problem for the dermatologist and has thus made an automatic computerized diagnostic system highly essential in the analysis of skin lesions; this will assist and help the dermatologist in quick decision making. This has enhanced the clinicians in reaching faster and accurate in detecting skin cancer cases. Melanoma cancer is very lethal and deadly, but then, it is very easy to eliminate if diagnosed early enough. In the recent past, there have been tremendous advances in the use of computerized methods that are used in the diagnosis of skin cancer. The traditional method for architectural features analysis of medical images is performed by means of a sequence of low-level pixel level operations [4].

The pre-processing stage includes the operations like contrast and intensity, binarization, morphological operation, colour grey-scaling, and data augmentation. In this stage, the noises and other interferences are attained from the images and the sizes are standardized to avoid complex computation [5]. After pre-processing, the image segmentation is carried out with an intent to separate regions of interest by eradicating healthy tissue of the body in case of diseased regions [5]. This step is useful for discerning between normal tissues when extracting features for the lesion for diagnosis. The skin lesion analysis used several forms of traditional segmentation methodology, which include the thresholding techniques, clustering, edge and region-based techniques, and the most recent approach known as ABCDE. However, such techniques tend to fail at correctly segmenting the melanoma region due to religious work at isolating the skin lesion and its complicated texture. In the recent past, more sophisticated segmentation techniques like fuzzy logic, genetic algorithms, artificial neural networks (ANNs), and more recent advanced deep learning (DL) have been used in segmentation of skin lesion with higher accuracy and reliability [6]. The lesion characteristics are obtained by applying feature extraction techniques which are very crucial in identifying the kind of skin cancer present in images. Some of the typical feature extraction techniques are template matching, image transformations, graph based, projection histograms, contour, Fourier descriptors, gradient features, and Gabor feature techniques [7]. These extracted features are then fed to the classification techniques which are used to determine the type of skin tumour present in the lesion. Some past approaches on skin lesion images classification involve decision trees (DTs), fuzzy classifiers, support vector machines (SVMs), and ANN [8]. In the recent past, DL approaches have gained higher accuracy score in skin lesion analysis to even diagnose skin diseases by photographs. Their successes are attributed by their capability to learn and acquire intricate and hierarchical features from image data sets. For example, deep convolutional neural networks, which is a widely used DL technique, is particularly good at dealing with general and highly variable tasks involved in fine-grained objects, which is one of the significant features of skin lesion images. This pre-eminent method can present the most obvious features of whole skin lesion images more accurately than manually crafted features. Convolutional neural networks (CNNs) are basically formed of varied layers that take input data in the form of images and produce outputs such as patterns of the disease [9].

In this study, we have provided a review of various types of methods applied in the melanoma skin lesion image analysis to critically appraise and compile the present status of DL and ML approaches for the screening of skin cancer at its initial stage. This study aims at assessing the different approaches that have been used, the issues that have been encountered in the application of these procedures, and the existing research voids. Also, we present valuable suggestions on further developments in diagnosis of skin cancer, which may improve specificity, sensitivity as well as practical utility when applied early in the skin cancer development process to promote better outcomes among patients. The main contributions of this review are as follows:

• This is important because early-stage skin cancer detection is one of the few applications of DL and ML that our review is focused on. In contrast to prior reviews that could consider a range of issues or various cancer types at large, or focus on a certain cancer type at large, with no specific reference to early diagnosis, our emphasis on early detection is aimed at emphasizing the role of early diagnosis for better outcomes for a patient.

• Our work thus presents an objective analysis of several methods, thereby helping compare the merits and demerits of the existing state of studies. This broad approach is crucial for purposes of defining research gaps and for guiding future research.

• In particular, the benefits of employing free-from heuristic, white-box, and hybrid explainable artificial intelligence (XAI) methods in clinical environment are described in relation to the usefulness of the approach in making the models more understandable to clinicians. This aspect has not been well explored in the literature, and our review will be useful for practicing clinicians who want to incorporate AI tools in practice.

• The study highlighted the key challenges of evaluating skin lesion segmentation and classification techniques, such as small sample size dataset, or selective and random image acquisition or even racial prejudice.

• Our study conducted a comprehensive analysis of various approaches used for skin lesion analysis in melanoma detection. We performed a comparative analysis of both traditional and DL-based techniques.

• A comprehensive review of DL and ML techniques for early-stage skin cancer detection: challenges and research gaps.

This article is organized as follows: Section 2 presents the human skin details. The pre-processing methods for skin cancer are presented in Section 3. Section 4 provides the details of segmentation methods of skin cancer. The feature extraction techniques used for skin detection are presented in Section 5. Section 6 provides the details of skin cancer classification. The types of computer-aided diagnostic (CAD) systems used for skin cancer detection are presented in Section 7. Section 8 presents the role of AI for skin cancer identification and types of datasets used. The research challenges for skin cancer detection are presented in Section 9. Section 10 presents the future directions and suggestion methods for skin cancer diagnosis. Finally, the conclusion of the review study and findings are presented in Section 11.

2 The human skin

There two major layers that make up the skin of humans, including the dermis and epidermis. The compositions of the dermis include elastic fibres as well as collagen, which is a kind of protein. There are two sub-layers in the dermis, which are the papillary dermis (thin layer) and the reticular dermis (thick layer). The papillary dermis acts as an “adhesive” holding the epidermis and dermis together, whereas the reticular dermis is made up of lymph and blood vessels, hair follicles, and sweat glands. In addition, this layer energize and nourishes the epidermis, and it is also crucial to healing, thermoregulation, as well as the sense of touch.

There are several stratified layers that make up the second layer, the epidermis. As a result of its layered appearance and protective function, it is also called stratified squamous epithelium. The epidermis consists of four distinct layers, or strata, which are arranged in a specific order according to their morphology: the basal, spinosum, granulosum, and corneum layers [3]. The epidermis consists of four kinds of cells, including melanocytes, Merkel cells, keratinocytes, and Langerhans’ cells. However, of all the cells, keratinocytes makes up the largest part (95%) in the epidermis and are chiefly responsible for constant skin renewal [3]. These cells are able to traverse for about 30 days from the basal layer (also known as the stratum basale) to the horny layer (stratum corneum) because of its ability to divide and differentiate. During the traversing period, the basal layer divides, thereby resulting in the production of the keratinocytes, and then the produced keratinocytes traverse through the next layers, which facilitates the transformation of their biochemistry and morphology. Consequently, corneocytes, the cells that make up the outermost layer of the epidermis, are flattened, nuclei free, and keratin filled. The last step in the differentiation process is desquamation, which occurs when the corneocytes lose their cohesiveness and detach from the surface. The skin’s continual regeneration happens through this process. Mechanosensory receptors, Merkel cells that likely originate from keratinocytes, respond to touch by forming tight connections with sensory nerve terminals [3]. Thus, Langerhans’ cells are dendritic cells that deposit themselves in local lymph nodes upon detecting foreign entities (antigens) that have penetrated the epidermis.

Among all the different kinds of skin cells, we are interested in the melanocytes. The melanocytes are dendritic cells located in the basal layer of the epidermis [3]. The melanocytes, in contrast to the Langerhans’ cells, have the ability to produce packages of melanin pigment, which are then distributed to their surrounding keratinocytes through the dendrites. The function of the melanin goes beyond the protection of the subcutaneous tissue from damage caused by ultra violet radiation, but it also include the pigmentation of skin, eyes, and hair. When there is an increase in ultraviolet radiation, more melanin is produced by the melanocytes, and this is why the skin gets tanned when exposed to sun.

However, the aesthetic outcome of melanocyte activity is not the centre of focus, but their malignant transformation potential. Although the cancer can occur in any cell within the body, there are specific cells that are more prone to being attacked by cancer, including the skin. The development of a large number of skin cancers occurs from squamous and non-pigmented keratinocytes. The outcomes of such transformations are squamous and basal cell carcinoma, respectively [1,4]. Nevertheless, when melanocytes are exposed to malignant transformation, the outcome is a rare, but more harmful and aggressive cancer called malignant melanoma. In the subsequent sections, the treatment and epidemiology of this kind of cancer is presented, alongside certain lesions that are regarded as its signs.

3 The pre-processing stage: image enhancement

Improving the quality and legibility of images is the primary goal of image enhancement. In the literature, this procedure is sometimes called image pre-processing. This step is implemented on images of skin cancer to enable the elimination of artefacts like hair as well as artificial artefacts. Through the process of image enhancement, the quality of contrast is improved and so the details can be seen clearly during exploration. Thus, several techniques and algorithms have been introduced and explored on different databases of skin lesion with the aim of compensating the inadequacy of image acquisition while eliminating a wide range of artefacts. The artefacts of dermoscopic images come in the form of black frames, unequal illumination, dermoscopic gel, air bubbles, rulers, and ink markings. More so, natural cutaneous characteristics like hairs, blood vessels, and skin lines can affect the detection of border [20]. Koroktov [21] provided two classes of artefacts including image enhancement and artefacts rejection. Artefacts rejection is made up of ink marking, air bubbles, hair, and specular reflections. On the other hand, the image enhancement includes correction of illumination, enhancements of edge and contrast, as well as correction and calibration of colour. Celebi et al. [22] focused on the relevance of the current step in detection of border. In their work, they placed emphasis of Gaussian and Median filters as the most relevant methods.

The artefact which is commonly found in dermoscopic images is the hair, which must be eliminated. For this reason, there are several algorithms and approaches that have been introduced to enable the removal of hair that is not shaved prior to the image acquisition step. Most of the methods and approaches used in removing hairs involve two major steps including:

• The hair detection: During this process, the various hairs present on the image are detected and removed through the detection algorithms. In the majority of the approaches used for this purpose, segmentation is used since the hair is one of the most important component among the other kinds of noises.

• The image restoration: During the process of restoration, which is also referred to as inpainting step, the space that was previously occupied by the eliminated hairs is filled with proper colour and intensity values. If the hair density is particularly high around the lesion’s borders, it could degrade the image quality. Furthermore, in certain cases, the texture of the pigment might be impacted by the dense hair. Thus, it is important to shave the hair at the affected area so that the diagnostic errors can be minimized.

There is a wide range of approaches that can be explored and used for removal of hair. However, the first of such approaches known as the DullRazor software [23] was introduced in 1997, and it gained the attention of many as it was widely used for hair removal [11]. Using a morphological closure operation, the DullRazor can pinpoint the exact position of dark hair. In addition to checking the hair pixel’s form, it employs bilinear interpolation and the median filter to make the restored pixels as smooth as possible. The DullRazor software was further improved by Wighton et al. [24] in 2008 by adding the inpainting step to the software. Apart from the interpolation which is utilized in the DullRazor software, more details were explored by the authors. One of such details explored is the border direction, which was done by using Laplacian for the control and measurement of the smoothness. In the study by Kiani and Sharafat [25] in 2011, an improved version of the DullRazor was proposed through the use of a wide range of methods. In their work, they employed the use of Prewitt filter to facilitate the detection of borders. They also introduced the use of Radon transform for predominant hair direction, and they also achieved the separation of hair from other noises by using a variety of masks.

Similarly, Toossi et al. [26] proposed a morphological operator which was equipped with adaptive canny edge detector to facilitate the detection of hair. In addition, they added the multiresolution coherence transform inpainting method for repair and flawless replacement of the emplacements of the hairs that have been removed. Their proposed method is tested on a database containing 50 images, and a diagnostic accuracy of 88.3% was achieved, whereas error segmentation of 9.9% was achieved. Nguyen et al. [27] used universal matched filtering kernel as well as local entropy thresholding in obtaining binary hair mask. Thus, the authors were able to refine and verify hair masks by combining Gaussian curve fitting with morphological thinning. Xie et al. [28] mainly focused on the repair of eliminated hair using PDE-based image inpainting. The three main components of their suggested procedure are as follows. In the first stage, enhancement, morphological closure is used. Then, in the second stage, hair segmentation, statistical thresholding is used and extracted by elongating related areas. Finally, in the third and final stage, PDE-based image inpainting is used to restore the image. Eighty photos were used to test the suggested method; 40 of those images included hairs and 40 did not. Their findings revealed a 5% mistake rate for the hairless group and an 18% error rate for the hairy photos. In a similar vein, Fiorese et al. [29] suggested a PDE-based image inpainting to facilitate restoration. A top-hat operator was provided for morphological improvement with the PDE-based image inpainting, and an Otzu threshold was used for hair detection. They discovered a 15.6% error rate when they tried their suggested strategy on 20 photos. Similar to how linear discriminant analysis (DA) helped with image restoration, multi-scale curvilinear matched filtering was utilized by Huang et al. [30] to identify hair. To accomplish hair identification, Abbas et al. [31] suggested a matched filtering method that incorporates the first derivative-of-Gaussian algorithm. Although the approach obtained improved accuracy, its implementation is complicated owing to the multiple factors it employs, according to their results. A diagnostic accuracy rate of 93.3% was recorded when the method was implemented on 100 dermoscopic images. In their next studies [32,33] which required the removal of hair, the same method was used. For the detection of hair, the use of bank of directional filters was employed by Barata et al. [34], while inpainting was done using PDE-based interpolation.

Gómez et al. [35] presented an unsupervised approach based on independent histogram Pursuit. To enhance the image particular structures, their suggested technique estimates a set of linear combinations of image bands. Their method was able to produce greater quality boundary identification, as shown by their testing findings. Celebi et al. [36] have attempted to improve contrast by suggesting a method for a specific RGB image input. Their goal in using the histogram bi-modality was to make the lesion stand out more against the backdrop. Madooei et al. [37] conducted research with an emphasis on improving images and removing artefacts. On the basis of the influence of light intensity on the edges, their studies were conducted on a set of 120. When applied to improved border identification, the experience yielded a sensitivity rate of 92% and a specificity rate of 88%. The recent work by Koehoorn et al. [38] suggests a new method that combines thresholding set decomposition with morphological analysis that use multiscale skeletons for gap identification. In the line of detection of hair in dermoscopic images, a new approach was proposed by Mizaarlian et al. [39]. The proposed method makes use of measurement of turbulence quaternation [40] alongside dual matched filter to suppress hair. Also, in their proposed technique, the interpolation which is employed in DullRazor software introduced by the Lee et al. [23] is applied for restoration. They tested the performance of their proposed method using a database which contains 94 synthetic images and 40 dermoscopic images. Their experimental results for hair segmentation showed that an accuracy rate of 86% was achieved for the dermoscopic images, while that of synthetic images was 85%.

There is a wide range of techniques that can be used for the removal of other artefacts like blood vessels and capillary. Using two colour parameter characteristics and a small collection of one curvilinear, Huang et al. [41] extracted capillaries from skin lesions. To train the system to identify the various capillaries, a SVM classifier was utilized. The suggested approach was evaluated using a database including 49 photos, with 28 of those images depicting invisible capillaries and 21 depicting visible ones. Their approach achieved the best possible scores in terms of accuracy (98.8%), sensitivity (90.5%), and specificity (89.3), according to the findings of their studies. Argenziano et al. [13] detailed the numerous vascular structures and their association with the different types of cancers, including both melanocytic and non-melanocytic tumours. Fisher test and χ ² tests were among the statistical methods employed by the writers. Dot vessels exhibited a 90% positive prediction on melanocytic lesions when their approach was evaluated on a database of 531 photos. The use of Fourier transform was employed by Abbas et al. [32] to achieve the reduction of specular reflection, while the reduction of dermoscopic gel or air bubbles was achieved by means of median filter. In the work done by Barata et al. [34], the reflection was minimized using a sub-band thresholding so that the quality of the images can be enhanced in pigment network detection.

The summary of number of techniques that have been employed in the detection of hair alongside the steps of used in achieving inpainting. Also, the results obtained for each of the techniques are also presented in the table. The results of majority of the methods presented in Table 1 have been compared with the results of the DullRazor software. Regardless of the fact that the contributions made by scholars in the area of hair segmentation and removal are huge, only few researchers have focused on the analysis of tubular vessels. The majority of the results in these publications are centred on visual analysis and comparison. It was also observed that most of the approaches proposed for the enhancement of skin image are based on thresholding. Even though, there are several methods that have been developed, and the use of small and private databases to test these methods makes it difficult to conclude on the actual performances of the approaches.

4 Segmentation

In terms of classification and diagnosis, it is important for the border of skin lesion to be accurately detected. There is a wide range of algorithms and techniques that have been developed and used on a variety of databases in the field of image processing. The detection of border is not as easy as one may think, as it is accompanied by some many complexities and limitations [21]. There are two key problems associated with the process of border detection; the first is the problem of ground truth presented treated by dermatologists, which computers find challenging to distinguish and humans find tough to replicate. In such a situation, the human eye is unable to detect or adequately investigate the blur or contrasts [42]. A second issue, as detailed in the work by Celebi et al. [22], pertains to the manual or automated segmentation of the morphological structure of the lesion. This is especially the case when the lesion’s border is indistinct or when there is little contrast between the tumour and normal skin. Selecting the most appropriate technique or approach for the detection of a lesion’s border is made difficult by the numerous development of lesion and how it appears in dermoscopic images [22].

Thus, the discussions in the current subsection are presented based on three main approaches that are generally used in the area of image segmentation, especially, for images of kin lesion. The first category is the technique that make use of total variation segmentation, then the second set consists of the techniques the employ the use of multi-resolution analysis, and the last category consists of the methods in which thresholding approaches are used. The discussion also includes approaches that do not fall into any of the aforementioned methods. In recent times, the detection of border lesions was reviewed by Celebi et al. [63], and afterwards, he categorized into 12 classes including active contours, clustering, and histogram thresholding.

5 Feature extraction

For physicians to be able achieve a correct diagnosis of PSL, or to label it as “suspicious”), they rely mostly on the so-called lesion characteristics. These characteristics are method dependent. A particular feature of the ABCD rule is the asymmetry of a lesion, whereas one characteristic of pattern analysis is the pigmented network. The classification of a lesion using computerized PSL analysis is aimed at extracting critical features from the images and representing them in a manner that is understandable to the computer, i.e., using image-processing features. The word “features” is used in this literature for the sake of clarity, while the term “feature descriptors” is used to denote image-processing features. As can be seen from the literature, PSL feature extraction has been the subject of several investigations. However, very few of them offer any kind of overview or critique of the feature descriptors utilized by CAD programs. In particular, Umbaugh et al. [104] provided an explanation of a computer application in 1997 that automatically extracts and analyses PSL properties. Their suggested feature descriptors were classified as colour, spectral, binary object, and histogram characteristics. Binary object attributes include things like area, perimeter, and aspect ratios. Statistical measurements of grey level distribution and co-occurrence matrix features are included in the histogram features. Colour features were represented using metrics gotten from colour variations, colour transforms, and normalized colours. Finally, metrics obtained from the images’ Fourier transform were represented using spectral features. Apart from providing a review of CAD system development through research, Zagrouba and Barhoumi [105] provided a brief insight on the algorithms for feature selection. One of the most crucial steps before lesion classification is reducing the number of extracted feature descriptors, as this helps improve efficiency and accuracy. The result is a decrease in the computational expense of categorization. But you should not take it for granted just yet, as eliminating duplication among feature descriptions could reduce their discriminating value. Various sets of retrieved feature descriptors can be employed according to a feature selection technique that some researchers have established [106,107,108,109]. Maglogiannis and Doukas [20] provided a solid review of CAD systems and feature descriptors in 2009. Along with a list of traditional feature descriptors used in the literature, these writers offered insights on methodologies of PSL diagnosis. By applying a variety of feature selection algorithms to a single set of dermoscopic images, the authors were able to compare the performance of many classifiers. The results showed that the classifiers’ performance was significantly affected by the feature descriptors used performance, demonstrating the importance of feature descriptors in PSL computer analysis.

• A more comprehensive system of classifying feature descriptors is suggested in this study. The extension links the feature descriptors with particular diagnosis method, differentiating between dermoscopic and clinical images, as well as differentiating references based on our literature classification. With this categorization, a reader can gain insight on the following: the extant methods in PSL feature description.

• The differences between representing dermoscopic features and clinical features, and ultimately.

• To compile an exhaustive list of references on the relevant descriptions.

The individual descriptors have been grouped into different groups along with other descriptors so as to make the presentation concise. By taking this tack, how each set of adjectives corresponds to the trait we were trying to characterize can be determined. While most authors noted in their research papers that a given feature was mimicked by the descriptor, others would describe a different feature using the descriptor or will even not link it to any specific feature. The tag “Lesion’s area & perimeter” makes this type of group very evident. The “Border irregularity/sharpness” features, which is present in the majority of published works, is ascribed to this set in this study, rather than as “Asymmetry” feature, as used by some authors [110,111]. It is noteworthy that the attribution of such group is a relatively complex task, given that as a shape or geometry parameter, it could be adequately used to provide the description of both features. When it came to the additional sets of feature descriptors, the same logic of majority was used, but a separate list was created containing the descriptors for which a particular clinical attribution could not be defined.

While the ABCDE factors were utilized for clinical photos, dermoscopic images were thought about using pattern analysis and the ABCD rule. The references of the works that were carried out for calculating descriptors for pattern analysis features [110]. Several of these focused on extracting features in accordance with the seven-point checklist approach to dermoscopy-based melanoma detection [112–118]. Capdehourat et al. [18] carried out a preliminary study with the aim of detecting certain dermoscopic structures including irregular pigmentation, atypical pigmented network, and summaries of features’ descriptors employed regarding dermoscopy’s ABCD rule and the ABCDE clinical requirements. The commonalities are specified, and the contrasts are highlighted through this split depiction of the automated description of the features. The two types of images comprise the greatest groupings of feature descriptors. On the other hand, the differences are observed in smaller groups and in some cases individual descriptors. For instance, scale-invariant feature transform (SIFT) descriptors, size functions [119–121], dermoscopic interest points [122], dermoscopic images are the only ones that employ the bag-of-features structure, while clinical images make use of a variety of methods for describing border abnormalities and a plethora of papers on the skin pattern analysis. While a set of textural feature descriptors known as Haralick parameters is rather big when applied to dermoscopic images, it is relatively tiny when applied to clinical images. This is because dermoscopic images provide more detail about the texture than macroscopic clinical images, which allows for more intricate examination [1,2,3].

6 Lesion classification

The lasts step involved in the schedule of automated examination of images representing one PSL is the classification of lesion. The output of lesion classification varies according to the kind of system; it could be ternary (nevus/common nevus, melanoma/dysplastic), binary (can detect cancerous or benign skin conditions), or nary, which may detect a wide variety of skin diseases. The system’s trained recognition of certain classes of PSLs is reflected in these outputs. The systems carry out the task of classification through the application of numerous methods of classification to feature descriptors that have been extracted in the previous stage. The selected classifier and extracted descriptors have a great influence on the efficiency of the methods. Thus, to obtain optimal results from the comparison of methods of classification, the same set of descriptors and dataset must be used for all approaches. A summary of classification results obtained by different authors of numerous CAD systems was presented in the work published by Maglogiannis and Doukas [20]. The authors also used a set of feature descriptors and a dataset containing 3,639 dermoscopic images to uniformly compare 11 classifier. This task also involved the application of various procedures of feature selection. The 11 classifiers used for the comparison comprised the most commonly used group of classifiers in the PSL computerized analysis, including DTs, regression analysis, and ANNs. Three sub-experiments were performed during the comparison. The initial two trials focused on the dysplastic/common nevus and melanoma/common nevus categories, respectively, but the third trial combined the two. Based on the comparison, the best result was yielded by SVM. However, a conclusion was drawn by the authors that the aspects that play critical roles in the classifiers’ performance were the procedure involved in learning as well as the chosen feature descriptors.

Several authors in the area of CAD systems and classification have done a comparison of two or more classifiers. Particularly, many authors have compared the performance of SVM and ANN [123], and their results revealed that SVM outperformed the ANN. Also, some authors have done a comparison between ANN and DA [124], while some have done a comparison between the SV and ANN [125] and have reported equivalent or slightly worse performance. Aljanabi et al. [126] carried out an evaluation of the performances of Bayesian classifier, SVM, k-nearest neighbours (kNN) and ANN [2]. The result showed that the ANN outperformed the Bayesian classier, while the Bayesian classifier performed better than the KNN algorithm. Although several of these comparisons have been done, it is difficult to ascertain the performances of the numerous classifiers used for PSLs in a hierarchical manner. This is because of the slight variation in the statistical evaluation results, as well as the basis on which the comparisons are performed; some of them are performed based on different features, classifier parameters, image datasets, and different learning procedures. However, a relative method of evaluation was used by Manne et al. [123], who reported the performances of the classifiers based on performance scales they created (performing well, very well, or not well suited). So their results showed that the kNN was rated “performing well,” then SVM, ANN, and logistic regression were rated “very well,” and the DT was rated as “not well suited” because of continuous input variables.

The research is categorized into three groups, focusing on studies that have used, developed, or tested methods for diagnosing PSL using clinical or dermoscopic images. Notably, most studies in the “Classification” category emphasize the specific details of the proposed lesion classification methods. On the other hand, the two other categories consist of researches in which the use of one of methods of classification has been employed in analyzing, proposing, or improving complete CAD systems. Thus, in general, the details provided on implementation of these systems in the literature are scanty, but the results of comparative performance are presented. In PSL CAD systems, classification does not rely solely on the raw image data but rather on how the extracted feature descriptors are interpreted. However, it can be argued that these descriptors inherently encode information based on the type of images used, making them unique to each imaging modality. Regardless of this difference, it is not possible to clearly classify grouping the feature description into two categories according to these images modalities; this is because the feature descriptor sets are rather similar.

The classification of the techniques was done based on their matching category, without considering any particular characteristics of implementation. For example, both ANN and DA are broad categories that include a number of different ways that might be considered “a type” of these more specific sets of approaches. Thus, Table 1 includes a few of the many publications that compare algorithms. By far the most used classification approach is ANN, followed by SVMs, DA, kNNs, and DTs. Kernel logistic partial least square regression and hidden Markov models are two other approaches that have been investigated and utilized according to the challenge. Supervised ML techniques clearly outperform their unsupervised counterparts, as seen in the table. Reasons for this include the complexity of the classifying issues and the wide variety of dermoscopy and medical features that might reveal whether a lesion is benign or cancerous. This means that the clinical and dermoscopy findings may be at odds with the biopsy-established diagnosis for a number of different sample lesions [126]. To train a classifier to identify these uncommon manifestations of cancerous tumours, this scenario often calls for the training/testing paradigm for the classification development. However, unsupervised learning approaches have shown encouraging results in understanding the association between PSL cancer and observable features [127].

7 CAD systems

An summary of the recent studies on the development and study of CAD diagnostic systems for PSLs is provided. Razmjooy et al. [127] were among the first researchers to publish articles, providing the summaries of advancements in the area of CAD usage for PSLs diagnosis; the two papers were published in 1995. Subsequently, other authors [128,129, 130] began to publish in the area of computer vision. Nevertheless, the majority of the articles were clinical research papers that focused on comparing the performances of different CAD systems. Usually, these authors present their comparative studies in tabular forms, highlighting the various characteristics of the systems like performance parameters (e.g. specificity, sensitivity) dataset’s size and distribution (malignant melanomas versus dysplastic nevi), methods of classification, type of image, and methods of classification. Even though such tables do not permit complete comparison between CAD systems, they enable the analysis and quantification of the various aspects of extant methods. Examples of these kinds of comparative analysis have been presented in previous studies [127,131,132]. Recently, there are some systems that have been developed for commercial computer-aided diagnosis of PSLs. There is sufficient literature on research works that have focused on the exploring and designing such commercial systems, and majority of this systems are developed on the basis of dermoscopy. There are a lot of whole sets of CAD systems that include dermoscopy equipment and analytical software. The majority of these CAD software programs are all-inclusive packages that include analysis software and acquisition instruments (dermoscopy).

The DB-Mipsr (Dell’Eva-Burroni Melanoma Image Processing Software) is one of the most cited CAD systems that is employed in the detection of melanoma. This system has a variety of names, which depend on the period it was developed, such as DEM-Mips, DB-DM-Mips, DBDermoMips, DM-Mips, and DDA-Mips. Based on the survey conducted by Vestergaard and Menzies on automated instruments for the diagnosis of cutaneous melanoma [75], drawing conclusions about the efficiency of the instruments is difficult because of the several classifiers that are used in the various studies, which vary by structure. Particularly, this is evident in two studies conducted by expert dermatologists [127,132]. The results of the study carried out in Mabrouk et al. [133] showed that higher accuracy was demonstrated by the classifier compared to that of expert clinicians, which only uses the epiluminescence approach. Nevertheless, in the other study, [134], a significantly low result was recorded for the system in terms of specificity. It is important to note that the same classifier (ANN) was utilized in the two studies. According to the authors, the results point to the huge variation in the ration of dysplastic nevi in the benign sets. Some of the systems that are available for commercial use include MelaFindr, SpectroShader, and MoleMate. In MelaFindr, images are acquired sing multispectral dermoscopy, which enables the acquisition of images 10 different spectral bands, ranging from blue (430 nm) to near infrared (950 nm) [135]. Siascopy (MoleMate – system) performs the analysis of information about the levels of melanin, haemoglobin and collagen contained in the skin. For the system to achieve this analysis, the wavelength combinations of the received light must be interpreted [136].

In the extant literature, the technical characteristics of digital dermoscopy analysis instruments have been overviewed and compared. The previous studies [137,138] have provided a summary of the microDerm, DB-Mips, Videocap, SolarScan, MoleMax II, and Dermogenius, while Zhou et al. [122] have summarized the overview and presented the comparison of microDerm, Fotoinder, and Dermogenius Ultra [122]. Zhou et al. [122] noted that expert dermoscopists/dermatologists have little or nothing to gain from the CAD systems that were reviewed in their study. In the study by Sevli [1], other systems have been described alongside their performance evaluation. Some of the CAD systems are designed in a manner that allows them perform the diagnosis of a PSL based on its physical similarities to images of lesions with known histopathology. These types of systems are referred to as content-based image retrieval (CBIR) systems, because it is solely aimed at searching a given database to identify images that look very much alike with a query image. This is carried out using a wide range of parameters that establish similarities between extracted lesion feature descriptors, including Euclidean distance, Bhattacharyya, or even Mahlanobis distance. Despite the existence of so many parameters, the selection of the most suitable one is determined by the nature of the feature descriptors. Therefore, in the works of Rahman and Bhattacharaya [139], the use of Euclidean distances and Bhattacharayya was employed for texture and colour features. On the other hand, the use of Manhattan distance was employed by Celebi and Aslandogan [107] for descriptors based on the lesion’s shape information. Although content-based retrieval systems can be evaluated by using so many measures, the most commonly used one is the precision-recall graph [139]. Currently, the results recorded for the retrieval of both dermoscopic [140] and clinical [141] images show that there is need for further improvement.

In the past two decades, prior to the introduction and adoption of digital imaging in medical practice in place of film photography, researchers were aware of the potential benefits that could be derived from its use in the area of dermatology [1]. Gradually, a large number of the benefits became manifest, including tele-diagnosis, objective documentation of skin lesions in a non-intrusive manner, creation of digital dermatological image archives, 3D reconstructions of clinical features of cutaneous lesions, as well as their quantitative descriptions. Even though, the automatic PSL diagnostic systems are accompanied by some flaws, they do not pose serious challenges because their most relevant functionality has been achieved. In this study, computerized analysis of dermatological images have been overviewed and presented based on certain aspects resultant from the integration of two different disciplines, which are computer vision and dermatology. Particularly, the overview hammers more on the following aspects:

• The variance that exists between acquiring clinical and dermoscopic images of individual PSLs, which dependent on the visualization of the structural details of a lesion. This is noteworthy during the application of border detection, pre-processing, or even feature detection algorithms to skin lesions’ images. More so, there is an association between the constant variations in the terminology used in the literature and the basic variance between the two modes of acquisition. Consequently, sometimes in the literature of computer vision, there has been wrong attribution of clinical methods of diagnosis to image types.

• Clear distinction of researches in which images of multiple and individual pigmented skin lesions have been analyzed. The inconsistency in the number of researches carried out on each subject is huge. This could be attributed to the fact that the adoption of total body skin imaging is minimal. There is a huge discrepancy on the tradeoff between the relevance of total body skin imaging in the detection of melanoma versus logistic limitations, and the cost implication associated with the use of this approach. [3]. Hence, there is higher preference for automated approaches to individual lesion analysis compared to that of total body screening.

• Generally, in terms of the analysis of images showing individual PSLs, the focus has been on the development of computer-aided diagnosis systems that are designed to automatically detect melanoma from both dermoscopic and clinical images. Generally, the workflow of most of these systems is the same, including pre-processing of images, lesion borders’ detection, and extracting and classifying clinical feature descriptors of a lesion. There are different methods that have been proposed to facilitate the implementation of all the steps involved in this workflow. However, of all the steps, the feature extraction and border detection steps have the highest number of articles published about them.

• There are limited publications on automated change detection for multiple and individual PSL images. Even though the major sign that points at the early stage of melanoma is the fast change in the size and lesion morphology, it can be seen in the literature that there is scarcity of researches in which the complete application of automated change assessment is made.

• In addition, in this work, several categories of computerized analysis of dermatological images have been created. Within the boundaries of this classification, a review of the categories that make up the workflow of a typical CAD system have been carried out and summarized in a tabular form. The summary shows the pre-processing methods used, techniques of extracting features, as well as the approaches used in classifying the PSL images. One more significant impact of this review is the extensive classification of extant dermoscopic and clinical feature descriptors. These descriptors have been categorized according to their relatedness to particular techniques of diagnosis, distinguishing dermoscopic and clinical images, and separating references based on the classifications in the literature. Given the importance of feature descriptors in the classification of PSL, there are some reasons why this categorization is relevant: it provides an overview of extant techniques used in the extraction of PSL, highlighting the variance between dermoscopic and clinical feature descriptors, and providing an aggregate list of the related important references.

So far, within the context of experiment, CAD systems have been found to perform well, thereby being accepted by patients. Nevertheless, presently, the ability of the system in terms of providing the best diagnostic results or replacing the intervention of hispathology or skill of the clinicians is yet to be proven. Regardless of this, the use of such systems has been employed for educational purpose of general practitioners, therefore providing professional clinicians with advanced knowledge, alongside second opinions during the process of screening [127,128]. In other words, “clinical diagnosis support system” could be a more appropriate word to describe CAD system for skin cancer at their present state of development. Finally, it is crucial to have a benchmark dataset that is publicly accessible for the evaluation of algorithms and new methods being developed. This can help in improving the quality of output produced by these systems. When the dataset has been created, each PSL image should be complemented with a ground truth definition of the lesion’s border alongside its diagnosis with extra dermoscopy reports from as many dermatologists as possible [130]. For a very long time now, this kind of dataset has been highly expected, and it is only in recent times that few of such databases with the aforementioned characteristics have been made available for public use [134].

8 The role of AI for skin cancer identification

Esteva et al. [142] were the first to record a significant advancement in this field; they used a DL technique on a combined skin dataset consisting of 129,450 images representing 2,032 distinct skin lesion diseases, drawn from dermoscopic and clinical images. The algorithm was tested on images and compared to 21 dermatologists who are board certified to see how well it differentiated and classified carcinomas, benign seborrheic keratoses, melanomas, and benign nevi. It was observed that the AI performed according to the performance expectations of dermatologists in terms of skin cancer classification. These authors performed tasks of diagnosis and classification of skin lesion through the use of three kinds of modalities, including histopathology images, dermoscopic images, and clinical images. First, skin lesion datasets that are publicly accessible are analyzed, and then solutions associated with AI are discussed in different sub-sections according to each kind of imaging modality. The challenges of ISIC are briefly discussed.

9 The research challenges

A comprehensive literature review identified several limitations. Training deep CNNs, especially complex architectures, requires substantial computational resources, leading to long training times. Additionally, representing data through quantum encoding in quantum CNNs presents significant challenges, making it difficult to integrate quantum principles into the encoding process. Moreover, CNNs struggle with tasks requiring long-range dependencies. Addressing these limitations can enhance the performance of quantum-dependent neural networks, ultimately improving results. The research challenges are summarized as follows:

• Comprehensive Training: Extensive training is necessary for neural network-based skin cancer detection methods, which is a big hurdle. The system must undergo a comprehensive training, which is time consuming and demands powerful hardware, to evaluate and understand dermoscopic images’ characteristics effectively.

• Lesion Size Variation: Different lesions might range in size, which adds another layer of difficulty. In the 1990s, a team of researchers from Italy and Austria gathered a large number of images of melanoma lesions, both benign and malignant [54]. Lesions may be accurately identified with a diagnostic accuracy of 95–96% [55]. Nevertheless, the diagnostic procedure was considerably more challenging and prone to errors when dealing with early-stage lesions that were 1 mm or 2 mm in size.

• Images of Light-Skinned People in Public Databases: Most persons included in the current standard dermoscopy databases are white, with a few exceptions from the United States, Australia, and Europe. For a neural network to accurately identify skin cancer in individuals with dark skin tones, it has to learn to consider skin colour [56]. Nevertheless, this can be accomplished only if the neural network trains on a significant enough dataset that includes photographs of individuals with dark skin. To improve the precision of skin cancer detection algorithms, it is essential to have datasets that include enough images of lesions on both light- and dark-skinned individuals.

• Very Minimal Interclass Dissimilarity in Skin Cancer Images: Medical images differ significantly from other types of images in terms of interclass variance; for example, compared to photographs of cats and dogs, there is substantially less fluctuation in the difference between images of melanoma and non-melanoma skin cancer lesions. In addition, melanoma and birthmarks are extremely hard to tell apart. Because certain diseases’ lesions seem so identical, diagnosing them can be challenging. Image processing and classification become an extremely complicated task due to this restricted variety.

• Datasets of Skin Cancer Without a Balance: Real-world datasets are highly imbalanced when diagnosing skin cancer. There is a significant disparity in the number of photos for each skin cancer type in imbalanced datasets. Because of limitations in the number of photos for less frequent skin cancer types compared to the number of images for more common ones, it is difficult to make broad conclusions based on the visual characteristics of dermoscopic images.

• Utilization of Diverse Optimization Methods: Automated skin cancer detection relies heavily on pre-processing and lesion edge detection. We can enhance the performance of these systems by investigating different optimization algorithms, such as particle swarm optimization, social spider optimization, ant colony optimization, and artificial bee colony algorithm.

• Acquiring Images for Dermatology: Acquiring photos in dermatology often involves taking close-up images of dermatoses or lesions. Since the surrounding structures are typically not included in these images, the anatomical context is sometimes lost when focusing on the lesion. In addition, dermatologists are using more and more photographs taken with smartphones, which aligns with the exponential growth of digital skin imaging apps. Isolated datasets, unique lighting settings, etc., are some greedy circumstances utilized to train the techniques proposed for identifying melanomas from inconsistent dermoscopic images. As a result, these systems give localized findings that cannot be employed generically. Along with these issues, the collected images suffer greatly in quality because of the fluctuating lighting conditions (e.g., specular reflection) throughout the acquisition period. One solution to the ever-present issue of colour inconsistency is suggested in [132]: a generative adversarial neural network. To overcome the problems with image capture, these technologies need to be more widely used, and more creative solutions for unpredictable dermoscopic images need to be developed.

• Several studies have been working on improving CAD methods for the classification of skin cancer in the last few years. Due to the advent of DL, ML methods were largely used to create CAD systems. These ML-based techniques, however, are limited in their ability to identify skin diseases because of the difficulties associated with feature engineering and the constraints of handcrafted features. Conversely, DL algorithms are more accurate and efficient at automatically extracting meaningful features from huge amounts of data. As a consequence, in recent years, a significant number of the classification of skin cancer challenges have been solved with excellent results using DL-based techniques like CNN.

• Limited Capacity to Generalize Across Domains: In the challenging task of classifying skin cancer, the model’s capacity to generalize is frequently less than that of a skilled dermatologist. First off, the overfitting issue persists even in cases when a substantial quantity of synthetically created data is comparable due to the limited size of skin image datasets. Second, most studies are limited to dermatological images (dermoscopic and histological images) obtained with standardized medical devices. Image data from various devices that pertain to dermatology has not been extensively studied. A trained model performs much worse when it is used to a new set of data in a different domain.

• The kind of image noise and heterogeneous devices: The reliability of algorithms for the process of identifying skin cancer is challenged by a variety of noises coming from heterogeneous sources and images of skin diseases. The DL model may equal or even outperform dermatologists in terms of diagnosis if trained on high-quality skin tumour datasets. However, when tested with diverse images, the skin cancer model for classification often fails to produce adequate classification results because it is dependent on samples taken with varied machines, brightness options, and backdrops. Classification is further complicated by the wide variations in magnification, perspective, and illumination found in digital images like those from smartphones).

• Moving Towards Quicker and More Effective Classification Frameworks: The computational cost of the model still has to be taken into account, considering that a growing several of DL methods have been effectively utilized for the classification of skin cancer with outstanding classification results. First off, a lot of high-quality images of skin diseases now feature huge pixels because of advancements in imaging technology. Histological scans, for instance, have a resolution of more than 50,000 × 50,000 pixels and consist of millions of pixels. It therefore requires more time and computer power to train them. Second, as the accuracy of the DL models increases, so does its computational complexity. This means that applying the model to different medical devices or mobile devices will become more expensive. Here, we provide the most recent approaches to creating a successful skin cancer network.

Nevertheless, after further review, it seems that the problems with skin cancer classification are not as simple as those found in the non-medical area such as ImageNet and other types. In the beginning, the variations between the various skin cancer classifications cause numerous datasets of skin images to be imbalanced, which raises the possibility of a false positive by the diagnosis method. Furthermore, many datasets only offer a small number of images due to the labour-intensive and highly specialized nature of correct annotation (for example, the ISIC dataset, which comprises approximately 13,000 skin images, is currently the biggest openly accessible skin disease dataset).

10 The future directions of research

Without fail, AI researchers will assert that their algorithms can detect skin cancer more accurately than dermatologists. However, this differs from how things work in the real world because these trials follow strict protocols in controlled environments. Given the several obstacles discussed earlier, it is clear that these performance assessments do not reflect the actual diagnostic work done by skin cancer specialists. Because they lack domain expertise and cannot conduct logical deductions to determine the link between various skin lesion types, DL algorithms are often seen as opaque because they learn solely from the pixel values of imaging datasets. However, the following opportunities suggest that DL may 1 day be useful for skin cancer diagnosis.

• Using AI and Next-Generation Sequencing to Improve Skin Cancer Diagnosis Diagnostics: Advances in next-generation sequencing (NGS) technology have made it possible to enhance data output while simultaneously improving associated efficiency. Reading lengths are used to classify NGS. Genome-wide or specific RNA or DNA area nucleotide order determination is a crucial application of NGS. Due to high volumes of DNA sequencing technologies and methodologies, new technologies can now be commercialized. Using less DNA and RNA input data while maintaining speed and accuracy is the objective of DNA sequencing technologies. Among all malignancies, SCC has a tumour mutation load that is among the greatest. The goal of comparing gene alterations in localized and metastatic high-risk SCCs using targeted NGS is to find essential distinctions and improve targeted therapy options. The development of molecular techniques has significantly expedited the discovery of new viruses, such as the papillomavirus. NGS can be used with enhanced procedures to aid in detecting known and undiscovered human papillomaviruses.

• It would help to have a balanced dataset and carefully choose the samples to get the most out of DL methods for classification tasks. Therefore, it is essential to have balanced datasets that include instances that fully represent the classification of that specific skin lesion; in this regard, the advice of seasoned dermatologists might be invaluable.

• Automated Skin Cancer Decision Support Systems Powered by AI Explanation: Skin cancer detection can be aided by decision support systems driven by AI, which are computer programs that help with decision-making and picking the correct course of action. Their hints to typical methods and looping patterns allow designers of DL classification methods choices for flexibility. Pre-trained models of deep neural networks with transfer learning (TL) can be used to initialize support systems for skin lesion categorization and localization. These days, automated DL techniques are a part of decision support systems. Through the use of TL on data that are skewed, these algorithms are taught and fine-tuned. Using an average pooling layer, the model retrieves features; nevertheless, these features are insufficient. To aid in decision-making, an improved genetic algorithm based on heuristics is used to extract essential and relevant traits, which are then passed on to a classifier. One possible alternative to invasive diagnostic procedures is using AI-powered decision support systems, which can aid physicians in diagnosing.

• Digital pathology computer-aided diagnosis: Computer vision and pathology groups have been interested in creating such a system due to the recent advances in whole-slide photography for digital image analysis and GPU clusters for robust computation. Using DL algorithms for sliding window classification and then aggregating those classifications to identify prevalent histopathological patterns is one of the most common AI approaches to dimensionality.

• Skin Cancer Diagnosis Based on Wearable Computing: Computing on human-wearable accessories is a new paradigm in wearable computing. Wearable computers are any little device that can process data and perform computations while worn on the body. The identification of cancer has made use of wearable computers. The high cost and awareness of wearable devices, along with their non-cooperative form factor, clinical inertia, improper connection, and high fragility and bendability, are some obstacles that have prevented their widespread clinical acceptance. Companies and researchers should work together in the future to find ways to use wearable computing as a viable alternative to traditional cancer detection methods while also lowering the associated costs. Computing effectiveness, efficiency, power consumption, adaptability, and real-time performance are all areas where these event-driven solutions shine. Incorporating these capabilities into wearable devices has the potential to improve performance significantly.

• Consistency of colour under different lighting conditions and with varying data types: The skin lesion images in the publicly accessible dermoscopy and clinical datasets were taken using varied lighting settings and acquisition equipment, which might compromise the AI systems’ performance. Shades of gray and max-RGB are colour constancy algorithms that have been shown in several research to increase the performance of ML techniques regarding multisource image classification. The shadow of gray method is a pre-processing technique to standardize the lighting effect and illumination on dermoscopic images of skin lesions.

• The generative adversarial network, or GAN, is one kind of DL architecture that is gaining interest in medical imaging. The primary use of GAN is to circumvent dataset limitations by producing high-quality false image data. GAN can be employed to produce lifelike synthetic images of skin lesions to avoid the shortage of annotated data related to skin cancer. Because patient prevalence biases the distribution of skin lesion classes in publically accessible datasets, GAN can be utilized to provide imaging data for under-represented skin lesion classes or uncommon skin cancer types as Kaposi sarcoma, sebaceous carcinoma, or Merkel cell carcinoma.

This comparison identifies the strengths and weaknesses of each method based on the various aspects such as sensitivity, specificity, computational time, and amount of interpretability required. This addition is intended to help the reader to better understand how each method works and the cost-benefit consideration associated with their use. Table 2 enables the researcher and clinician to distinguish the various methods based on their merits and demerits at a glance. For instance, newly sophisticated approaches such as CNNs and hybrid structure models exhibit the higher sensitivity and specificity, but at the same time require better computational power and are relatively harder to explain. While SVM and RF are less accurate and do require feature engineering they do provide for easy interpretation for big data sets.

Thus, this crowded comparison enriches the understanding of the differences in performance and interactivity of various ML and DL methodologies to help readers draw appropriate conclusions about which method is best to apply in skin – cancer detection situations.

11 Conclusion

This study has conducted a detailed review of the current nature of research concerning a fully automatic diagnosis system for detecting pigmented skin lesions. However, several issues should be addressed: the scarcity of large datasets of high quality, lack of methods for balancing datasets, and modelling the models’ performance for various populations and skin types. Other challenges common with such models include interpretability, computational cost, and how to incorporate the models into clinical practice among them. Some research limitations that have been highlighted in this review are as follows: more emphasis needs to be given to building more accurate and generalized models that can be of real help to dermatologists for early screening. Therefore, future research should focus on overcoming the limitations of current analysis to develop more effective Explainable AI (XAI) models for early skin cancer diagnosis, ultimately improving patient outcomes. Additionally, interdisciplinary and replication studies should be conducted to validate best practices and enhance advancements in this critical field.