A multi-crop disease identification approach based on residual attention learning

2.1 Architecture

The proposed architecture is based on the ResNet-18 network. The Attention mechanism is integrated into ResNet-18 architecture and a novel deep neural network (DNN) is developed. The attention modules are placed after every residual unit of ResNet-18. The attention mechanism blends the local and global information and learns the discriminative feature for classification [31]. The proposed architecture for plant disease identification is inspired by the Residual Attention (Res-ATTEN) network introduced in a previous study [32] and uses Attention-56 Network, as shown in Figure 1.

Figure 1

Architecture of the proposed approach for plant disease identification.

The architecture is structured by assembling various attention modules, which is an application of a special kind of attention, i.e., mixed attention (a fusion of spatial and channel attention) which makes the architecture capable of capturing different kinds of attention from the attention modules. A direct stacking up of attention modules might lead to a decline in the system’s performance. So, to avoid this issue, Res-ATTEN network came into the picture, which imitates the bottom-up fast feedforward process and top-down attention feedback in a single feedforward process that permits the creation of an end-to-end network that can be trained with top-down attention.

The first layer of the plant disease identification architecture denotes the input layer, which acquires an image with an input size of 224 × 224 . The second layer after the input layer was a convolutional layer containing 64 filters, kernel size 7 × 7, and stride value = 2, which provides an output of size 112 × 112 . After that, max-pooling is done using a 3 × 3 kernel with stride = 2, creating an output of size 56 × 56 . A residual unit contains three types of convolutional layers, i.e., 1 × 1 kernel with 64 filters, 3 × 3 kernel with 64 filters, and 1 × 1 kernel with 256 filters, producing an output of size 56 × 56 .

The residual unit is followed by an attention module, attention module stage1 producing an output of 56 × 56 . Then, another residual unit is placed with three types of convolutional layers, i.e., 1 × 1 kernel with 128 filters, 3 × 3 kernel with 128 filters, and 1 × 1 kernel with 512 filters, producing an output of size 28 × 28 . This residual unit is followed by an attention module, attention module stage2 producing an output of size 28 × 28 . After that, one more residual unit is placed with three types of convolutional layers, i.e., 1 × 1 kernel with 256 filters, 3 × 3 kernel with 256 filters, and 1 × 1 kernel with 1,024 filters, producing an output of size 14 × 14 . The third and last attention module is placed, namely, attention module stage3 producing an output of size 14 × 14 . The third attention module is followed by three more residual units, each with three different types of convolutional layers, i.e., 1 × 1 kernel with 512 filters, 3 × 3 kernel with 512 filters, and 1 × 1 kernel with 2,048 filters, producing an output of size 7 × 7 . After that, an average pooling is done using a 1 × 1 kernel with stride = 7, creating an output of size 1 × 1 . The fully connected layer was modified according to the number of classes required using transfer learning.

2.2 Residual learning units

The increase in the depth of the architectures leads to a saturation in the system accuracy. Eventually, it degrades at a point. This problem is known as the degradation problem. The degradation problem can be avoided by creating shallower architectures in combination with their deeper counterparts which contain the identity mappings copied from the layers of already learned shallower architecture, as illustrated in Figure 2. The desired residual mappings can be denoted as H ( h ) . The non-linear layers can be fitted into the following mapping in equation (1):

Figure 2

Residual learning block.

The original mapping can be re-casted as in equation (2).

The residual networks show tremendous results and better accuracy even in deeper networks [23], which shows that optimizing the residual mapping is much easier than optimizing the original or unreferenced mappings. Therefore, it was observed that it was easier to optimize residual networks than their counterparts, i.e., the plain networks since there is an increase in training error with an increase in depth.

2.3 Residual attention learning

Residual learning is combined with the attention mechanism, which developed the concept of the Res-ATTEN network. The mixed type of attention is used in the attention modules. The attention modules contain the residual units, convolutional layers, interpolation layers, and sigmoid function, as shown in Figures 3–5.

Figure 3

Attention module (stage 1).

Figure 4

Attention module (stage 2).

Figure 5

Attention module (stage 3).

The attention modules employed in the proposed architecture contain two types of branches (1) Trunk branch (Tk) and (2) Soft mask branch (Mk), as shown in Figure 6. The output of attention module ATT is as in equation (3).

Figure 6

Attention module: soft mask branch and trunk branch.

where i denotes all the spatial positions, and Ch ∈ { 1 , 2 , 3 , … , Ch } signifies the index of the channel. The attention mechanism is combined with the residual learning, and the formula was created similar to the concept of residual learning to modify the output ATT of the attention module as in equation (4).

where the range Mk ( h ) is between 0 and 1, with Mk ( h ) approximating to zero, ATT ( h ) approximates the original features H ( h ) , which is called as the residual attention mechanism.

Trunk branch: The trunk branch contains the operations which perform the feature extraction tasks. The soft mask branch imitates the fast feedforward and feedback attention process and employs the bottom-up and top-down structures to acquire a mask Mk ( h ) of the same size as the trunk branch Tk ( h ) with input h .

Soft mask branch: The soft mask branch comprises two operations – the first is the fast feedforward sweep that accumulates the global information from the entire image and the second is the top-down feedback process which interfuses the global information with the original feature maps.

The up-sampling of output is performed using linear interpolation layers after the processing from some residual units. The input coming from the previous layers gets entered into attention modules. After the processing in few residual unit, the max pooling operation is performed on the inputs numerous times for increasing the size of the receptive field rapidly. The global information expands by the help of a top-down architecture which is symmetrical in nature, as soon as the lowest resolution approaches. This process was intended to guide the input features in each position. After that, to ensure the output size is same as the input feature maps, the number of bilinear operations was kept the same as the number of max pooling layers.

The output then gets normalized using a sigmoid function as in mixed attention, there are no additional restrictions. It uses a simple sigmoid function, as illustrated in Figure 7, for each channel and spatial position. The function for the mixed type of attention is as in equation (5).

where i denotes the range of spatial positions, Ch denotes the range of channels, and h i , Ch signifies the feature vector at the i th spatial position.

Figure 7

Attention module: soft mask branch.

The element-wise cross-product of the output from the soft mask branch and the trunk branch was done. The output of this process was then summed up (element-wise) with the output of the trunk branch and further passed from the residual units. The output was passed from an average pooling layer to the fully connected layer, where the classification label gets generated. The number of pre-processing residual units is denoted as p RU , the number of trunk branch residual units is denoted as t RU . Moreover, the number of residual units between the pooling layer in the mask branch is denoted as res .

3.1 Datasets for disease identification

The experiments for the disease identification task employed four datasets from two different databases, i.e., PlantVillage Database and Rice Disease Database, as illustrated in Figure 8.

Figure 8

Sample images from the datasets for disease identification.

Apple, corn, and grape datasets are taken from the PlantVillage database [33]. Each dataset contains the leaf images of the mentioned plants. All the images are in jpeg format and have a resolution of 256 × 256 pixels, as summarized in Table 1.

The datasets are structured in the following manner: the first type of leaf images are the plant's healthy leaf images without any visible disease symptoms. The rest of the images are distributed as three different kinds of diseases. The apple dataset contains three types of diseases: apple scab (630 images), black rot (621 images), and cedar rust (275 images), along with healthy leaf images (1,645 images). Similarly, the corn dataset consisted of three types of diseases: Cercospora leaf spot (513 images), common rust (1,192 images), and northern leaf blight (985 images), along with healthy leaf images (1,162 images). The grape dataset has three types of diseases: black measles (1,383 images), black rot (1,180 images), and Isariopsis leaf spot, along with healthy leaf images (423 images).

The Rice leaf disease dataset is the second database used for disease identification [34]. There are a total of 5,932 images contained in this dataset. The images are captured in the rice crop fields and have four types of diseases, i.e., rice tungro (1,308 images), brown spot (1,600 images), rice blast (1,440 images), and bacterial blight (1,584 images). All the Rice leaf disease dataset images are in jpeg format with a resolution of 300 × 300 pixels.

3.2 System specifications

A Lenovo Legion 64-bit system with a 4GB NVIDIA Geforce GTX 1650 graphical processing unit (GPU) is employed for performing the experiments and obtaining the results. The system possessed an Intel(R) Core (TM) i5-9300H CPU with a clock speed of 2.40 GHz and 16 GB RAM.

3.3 Evaluation metrics

The testing accuracy has been used as an evaluation metric for the performance comparison of the classification models. The accuracy signifies the number of correct predictions the model provides over the total number of predictions [35]. The method to compute the accuracy of the model is as follows:

where TruePos refers to the case under True positive, where a leaf has a label x, and the model classifies this case as the label x. TrueNeg refers to the case under True Negatives, where a leaf is not having label x, and the model classifies this case as not label x. FalPos refers to a case under False Positives, where a leaf is not having label x, and the model classifies this case as label x. FalNeg refers to a case under False Negatives where a leaf has label x, and the model classifies this case as not label x.

4.1 Experimental setup for experiments

The four ResNets, ResNet-18, ResNet-50, ResNet-101, and Res-ATTEN, were employed for the classification task in disease identification. The mini-batch size was kept to eight to adapt to the system. The datasets were split into the ratio of 80:10:10, i.e., 80% of the data was employed for training purposes, 10% was used for validation purposes, and the rest of the 10% of data from each dataset was kept aside for testing purposes. ADAM optimizer and Cross entropy are chosen for computing loss function. The execution environment was set to be on GPU. The parameter setup is encapsulated in Table 2.

A learning rate (LR) scheduler was incorporated, which declines the value of the learning rate of each parameter group by a value known as gamma after every epoch [36]. This scheduler helps the DNN to determine the optimal LR for minimizing the loss in the network. For every five epochs (i.e., setting step size = 5), the learning rate will decay by half (i.e., gamma = 0.5). For example, if the initial LR was 0.01, then after five epochs, the learning rate value will be 0.01 divided by 0.5, which is 0.02.

4.2 Performance comparison of DNNs with Res-ATTEN for disease identification

Three types of experiments were performed on all datasets. Four types of DNNs, i.e., ResNet-18, ResNet-50, ResNet-101, and Res-ATTEN, are employed to compute the testing accuracy of the system, and a performance comparison between the DNNs was made accordingly. The LRs varied for the experiments in the following manner: 1st experiment with LR = 0.1, 2nd experiment with LR = 0.001, and 3rd experiment with LR = 0.0001. The performance comparison between the DNNs is made according to the highest testing accuracy. All three LRs are compared, and the best testing accuracy providing DNN was selected. The experiments on DNNs are run on categories, i.e., Epochs = 10 and 25. Various optimizing experiments select the optimal number of epochs to get the best accuracy.

4.2.1 Performance on PlantVillage (apple) dataset

The results of the experiments performed on PlantVillage (apple) dataset for the identification of the labels/classes: healthy, scab, black rot, and cedar rust are encapsulated in Table 4.

Table 4

Performance comparison of DNNs with Res-ATTEN for disease identification for PlantVillage (apple)

		Testing accuracy (%): PlantVillage – apple
		Batch size = 8, optimizer step size = 5, gamma = 0.5
DNN/testing accuracy		LR = 0.01	LR = 0.001	LR = 0.0001
ResNet-18	Epochs = 10	95	94	90
	Epochs = 25	92	92	93
ResNet-50	Epochs = 10	86	95	87
	Epochs = 25	95	95	90
ResNet-101	Epochs = 10	90	93	89
	Epochs = 25	95	94	93
Res-ATTEN	Epochs = 10	90	93	96
	Epochs = 25	96	99	97

The results show that the highest accuracy for PlantVillage (apple) dataset with ResNet-18 is LR = 0.01 with 95% at 10epochs. The highest accuracy ResNet-50 showed was 95% with LR = 0.01 at 10 epochs and LR = 0.001 at 10 and 25 epochs. The ResNet-101 DNN provided the highest accuracy of 95% with LR = 0.01 at 25 epochs, as illustrated in Figure 9. In comparison with all the DNNs, the proposed Res-ATTEN-based approach produced the highest accuracy of 99% with LR = 0.001 at 25epochs.

Figure 9

Graphs showing the comparison of testing accuracy among DNNs at (a) LR = 0.01, (b) LR = 0.001, and (c) LR = 0.0001 for PlantVillage (apple) dataset.

4.2.2 Performance on PlantVillage (corn) dataset

The results of the experiments performed on the PlantVillage (corn) dataset for the identification of the labels/classes: healthy, leaf spot, common, rust, and northern leaf blight, are encapsulated in Table 5.

Table 5

Performance comparison of DNNs with Res-ATTEN for disease identification for PlantVillage (corn)

		Testing accuracy (%): PlantVillage – corn
		Batch size = 8, optimizer step size = 5, gamma = 0.5
LR = 0.01	DNN/testing accuracy		LR = 0.001	LR = 0.0001
ResNet-18	Epochs = 10	83	86	87
	Epochs = 25	88	89	83
ResNet-50	Epochs = 10	85	89	90
	Epochs = 25	90	90	89
ResNet-101	Epochs = 10	84	87	87
	Epochs = 25	92	90	90
Res-ATTEN	Epochs = 10	83	89	94
	Epochs = 25	93	92	94

The results showed that the highest accuracy for PlantVillage (corn) dataset with ResNet-18 is 89% with LR = 0.001 at 25 epochs. The highest accuracy ResNet-50 showed is 90% with LR = 0.01 at 25 epochs, LR = 0.001 at 25 epochs, and LR = 0.0001 at 10 epochs. The ResNet-101 DNN provided the highest accuracy of 92% with LR = 0.01 at 25 epochs, as illustrated in Figure 10. In comparison with all the DNNs, the proposed Res-ATTEN-based approach produced the highest accuracy of 94% with LR = 0.0001 at 25 epochs.

Figure 10

Graphs showing the comparison of testing accuracy among DNNs at (a) LR = 0.01, (b) LR = 0.001, and (c). LR = 0.0001 for PlantVillage (corn) dataset.

4.2.3 Performance on PlantVillage (grape) dataset

The results of the experiments performed on the PlantVillage (grape) dataset for the identification of the labels/classes: healthy, black rot, black measles, and leaf spot are encapsulated in Table 6.

Table 6

Performance comparison of DNNs with Res-ATTEN for disease identification for PlantVillage (grape)

		Testing accuracy (%): PlantVillage – grape
		Batch size = 8, optimizer step size = 5, gamma = 0.5
DNN/testing accuracy		LR = 0.01	LR = 0.001	LR = 0.0001
ResNet-18	Epochs = 10	93	94	89
	Epochs = 25	93	88	90
ResNet-50	Epochs = 10	84	90	90
	Epochs = 25	94	91	89
ResNet-101	Epochs = 10	93	92	93
	Epochs = 25	93	92	94
Res-ATTEN	Epochs = 10	83	88	96
	Epochs = 25	96	95	97

The results showed that the highest accuracy for PlantVillage (grape) dataset with ResNet18 is LR = 0.001 with 94% at 10 epochs. The highest accuracy ResNet-50 showed is 94% with LR = 0.01 at 25 epochs. The ResNet-101 DNN provided the highest accuracy of 92% with LR = 0.01 at 25 epochs, as illustrated in Figure 11. In comparison with all the DNNs, the proposed Res-ATTEN-based approach produced the highest accuracy of 97% with LR = 0.0001 at 25 epochs.

Figure 11

Graphs showing the comparison of testing accuracy among DNNs at (a) LR = 0.01, (b) LR = 0.001, and (c) LR = 0.0001 for PlantVillage (grape) dataset.

4.2.4 Performance on Rice leaf disease dataset

The results for the experiments performed on the Rice leaf disease dataset for the identification of the labels/classes: Tungro, Brown Spot, Blast, and bacterial leaf blight, are encapsulated in Table 7.

Table 7

Performance comparison of DNNs with Res-ATTEN for disease identification for Rice leaf disease dataset

		Testing accuracy (%): Rice leaf disease dataset
		Batch size = 8, optimizer step size = 5, gamma = 0.5
DNN/testing accuracy		LR = 0.01	LR = 0.001	LR = 0.0001
ResNet-18	Epochs = 10	84	87	82
	Epochs = 25	87	87	84
ResNet-50	Epochs = 10	88	89	88
	Epochs = 25	92	92	78
ResNet-101	Epochs = 10	88	88	86
	Epochs = 25	90	90	88
Res-ATTEN	Epochs = 10	92	94	97
	Epochs = 25	98	99	99

The results showed that the highest accuracy for Rice leaf disease dataset with ResNet-18 is 87%, with LR = 0.01 at 25 epochs and LR = 0.001 at 10 and 25 epochs. The highest accuracy ResNet-50 showed was 92% with LR = 0.01 and LR = 0.001 at 25 epochs. The ResNet-101 DNN provided the highest accuracy of 90% with LR = 0.01 and LR = 0.001 at 25 epochs, as illustrated in Figure 12. In comparison with all the DNNs, the proposed Res-ATTEN-based approach produced the highest accuracy of 99% with LR = 0.001 and LR = 0.0001 at 25 epochs.

Figure 12

Graphs show the testing accuracy comparison among DNNs at (a) LR = 0.01, (b) LR = 0.001, and (c) LR = 0.0001 for Rice leaf disease dataset.

The overall highest testing accuracy achieved by each dataset at each DNN for the disease identification task is as follows: for PlantVillage (apple) dataset – 95% with ResNet-18, ResNet-50, and ResNet-101 and 99% with Res-ATTEN. For PlantVillage (corn) – 89% with ResNet-18, 90% with ResNet-50, 92% with ResNet-101, and 94% with Res-ATTEN. For PlantVillage (grape) – 94% with ResNet-18, ResNet-50, and ResNe-t101 and 97% with Res-ATTEN. For Rice leaf disease dataset – 87% with ResNet-18, 92% with ResNet-50, 90% with ResNet-101, and 99% with Res-ATTEN, as encapsulated in Table 8, and illustrated in Figure 13.

Table 8

Overall highest testing accuracy achieved by each dataset at each DNN

Datasets	Crop	Highest testing accuracy achieved (%)
		ResNet-18	ResNet-50	ResNet-101	Res-ATTEN
PlantVillage	Apple	95	95	95	99
	Corn	89	90	92	94
	Grape	94	94	94	97
Rice leaf disease	Rice	87	92	90	99

Figure 13

Highest accuracy achieved by datasets with different DNNs.

It was observed that the Res-ATTEN-based plant disease identification system provided the highest accuracy for every dataset compared to other ResNets.

4.2.5 Performance comparison of the proposed approach with traditional approaches

The performance metric for comparing the traditional and proposed approaches was chosen to be accuracy. Various researchers implemented the traditional approaches for plant disease and observed the accuracy as high as 88.46, 89.5, 90, 93.62, 93.67, 94.2996.46, 96.88, 97.47, 97.56, 98, 98.11, and 98.44%, as shown in Table 9. The proposed approach performed well with even small training data and provided the highest accuracy of 99%, as illustrated in Figure 14.

Table 9

Accuracy comparison between traditional approaches and the proposed approach

Approaches	Accuracy (%)
Mishra et al. [18]	88.46
Sujatha et al. [19]	89.5
Argüeso et al. [20]	90
Yin et al. [21]	93.62
Arsenovic et al. [22]	93.67
Wu et al. [23]	94.29
Geetharamani and Arun Pandian [24]	96.46
Gajjar et al. [25]	96.88
Thenmozhi and Srinivasulu [26]	97.47
Turkoglu et al. [27]	97.56
Picon et al. [28]	98
Shin et al. [29]	98.11
Li et al. [30]	98.44
Proposed approach	99

Figure 14

Comparison of the proposed approach with traditional approaches.