Research on Identification Method of Apple Diseases in Southern Xinjiang Based on Deep Learning and Its System Implementation

Peng QIN　Nannan ZHANG　Rong WU　Lijun GAO

AbstractApple disease samples were collected from the southern Xinjiang and annotated to design a convolutional neural network model based on deep learning. The accuracy and robustness of the model was improved through training and optimization algorithms, and a complete apple disease identification system was developed with the model as the core, and evaluated for its performance in terms of accuracy, recall rate and speed. This study provides a reliable AI-based apple disease diagnosis solution for the apple planting industry in the southern Xinjiang, hoping to help farmers better manage and protect crop health.

Key wordsDeep learning; Convolutional neural network; Apple disease identification; Southern Xinjiang; System implementation

DOI:10.19759/j.cnki.2164-4993.2023.05.018

Apple trees are one of the important cash crops in southern Xinjiang. However, apple diseases may cause a lot of losses. Therefore, this study aimed to provide an accurate, fast and automatic disease identification method for apple planting in southern Xinjiang based on deep learning (convolutional neural network) technique. We designed a system based on cloud servers, through which users can upload pictures, and the system will automatically detect and analyze diseases and provide corresponding diagnosis suggestions according to the results. The disease identification system will greatly improve the diagnosis efficiency of apple diseases, reduce the dependence on manual diagnosis, and lower the economic loss of apple planting.

Research Background and Significance

Research background

Apple trees are one of the important cash crops in southern Xinjiang, and its health is directly related to its yield. However, there are many common apple diseases in southern Xinjiang, such as rust, scab and anthracnose, which bring great challenges to apple planting. At present, the identification of apple diseases mainly relies on manual experience and professional knowledge, which is not only time-consuming and labor-intensive, but also prone to misdiagnosis. Therefore, developing an accurate, fast and automated method for identifying apple diseases is of great significance.

Research purpose and content

This study aimed to solve the problem of identifying apple diseases in southern Xinjiang based on deep learning (convolutional neural network) technique. We built a cloud server-based apple disease identification system, which can receive apple disease images uploaded by users and automatically analyze and identify disease types. Through this study, our goal was to improve the diagnostic accuracy and speed of apple diseases, provide practical decision support tools for farmers, and help them take timely and effective measures to protect the health of apple crops and reduce economic losses caused by diseases.

Main contributions

Main contributions of this study included:

(1) Design and optimization of apple disease identification model based on deep learning: We designed a convolutional neural network model, and improved the accuracy and generalization ability of the model through training and optimization on large-scale apple disease image datasets.

(2) Collection and annotation of apple disease image datasets: We collected the image data of apple diseases in southern Xinjiang and recorded the disease types in detail, which provided valuable data resources for model training (Fig. 1).

(3) Realization of apple disease identification system based on cloud servers: We developed a complete apple disease identification system. Users can take photos or upload images in photo albums through mobile phones. The system will automatically detect and analyze diseases and provide accurate diagnosis results and suggestions.

(4) System performance evaluation and experimental results analysis: We evaluated the accuracy, recall rate and speed of the system and compared it with other methods to verify the effectiveness and superiority of the system.

Through the results of this study, we hope to provide an efficient and convenient apple disease identification solution for apple planting in southern Xinjiang, so as to effectively reduce the impact of diseases on yield and quality, and promote sustainable development of the apple industry.

Introduction to Basic Theory and Techniques

Methods for identifying apple diseases

Apple disease identification is an important research direction in the agricultural field, aiming to accurately identify the types of apple diseases by analyzing and processing images of apple leaves, fruits, and other related materials. In the past few decades, scholars have proposed various methods to solve this problem.

Traditional apple disease identification methods are mainly based on image processing and feature extraction techniques. These methods usually include following steps: first, input apple images are preprocessed, such as denoising and enhancement. Next, the color, texture, shape and other features of the images are calculated, and relevant information is extracted. Finally, some empirical rules or classifiers (such as support vector machine, artificial neural network, etc.) are used for distinguishing and classification. However, traditional methods have some limitations in apple disease identification.  Firstly, the performance of traditional methods is usually limited by feature expression because they mainly rely on the features of manual design. Secondly, it is difficult for these methods to deal with the interference caused by complex image background and illumination changes. In addition, because traditional methods need a lot of manual participation and experience adjustment, their efficiency is low and it is not easy to popularize and apply them.

Application of deep learning in disease identification

In recent years, the rapid development of deep learning techniques has brought new opportunities for apple disease identification. Deep learning is a machine learning method based on artificial neural network, which learns the advanced representation of input data through multi-layer nonlinear transformation and realizes automatic classification and recognition of data. The application of deep learning in apple disease identification mainly focuses on Convolutional Neural Networks (CNN). CNN is composed of multi-layer convolution, pooling and a fully connected layer, and it can effectively extract features from images and has good image classification ability.

Theoretical basis of deep learning

Deep learning is a powerful technical tool in the task of apple disease identification. Based on the concept of artificial neural network, deep learning constructs models through multi-layer neuron tissue.  A neural network consists of an input layer, a hidden layer and an output layer, of which the hidden layer can be multiple hidden layers. Each neuron carries out weighting and summation on inputs and nonlinear transformation through function activation.

Design of convolutional neural network architecture

Convolutional Neural Network (CNN) is a network structure widely used in image processing tasks in deep learning. In apple disease identification, designing an appropriate convolutional neural network architecture can improve the performance of the model. The convolution layer extracts features from images through convolution operation, including multiple convolution kernels. Each convolution kernel performs sliding window convolution on the inputs and generates corresponding feature maps. The pooling layer is used to down-sample the feature maps, so as to reduce the number of parameters and the amount of calculation, while retaining main features. The fully connected layer flattens the feature maps output by the pooling layer and then connects them to the output layer. In order to avoid over-fitting, a dropout layer can be added to the convolutional neural network to randomly discard some activations of neurons.

Training and optimization of algorithms

SGD is a fundamental optimization algorithm that calculates gradients and updates parameters by randomly sampling small batches of data. It has the characteristics of high simplicity and efficiency. In the task of identifying apple diseases, commonly used loss functions include cross entropy loss function, quadratic loss function, etc., used for measuring the difference between the model output and the real label. To prevent overfitting, regularization techniques such as L1 regularization and L2 regularization can be used to control the complexity of model parameters (Fig. 2).

Method for Identifying Apple Diseases in Southern Xinjiang Based on Deep Learning

Collection of apple disease samples in southern Xinjiang

In order to build an accurate and reliable apple disease identification model, it is first necessary to collect a large number of representative apple disease samples. In southern Xinjiang, we can organize professional teams to conduct on-site investigations in apple planting bases, orchards, and other areas.   Different types of apple diseases, including leaves, fruits and tree trunks, were observed to collect samples.   Cooperative relationships were established with local farmers to obtain samples of diseases in apple trees they grew. Next, the collected apple disease samples should be annotated and preprocessed for subsequent deep learning training and model construction.

Method for identifying apple diseases in southern Xinjiang

In this study, Fast R-CNN was used as the object detection algorithm, which consists of three parts: a backbone feature extraction network, a region proposal network, and a detection sub-network (Fast R-CNN). We used the Split-Attention Networks (ResNeSt) as the feature extraction network, which divides the feature mapping into multiple subgroups by using the Split-Attention module, and determines the feature representation of each subgroup through weighted arrays. Such structure makes the network pay more attention to effective features, thus improving learned feature representations. Meanwhile, we also used the feature pyramid network to fuse features of different scales to enhance the feature extraction ability of the network (Fig. 3).

System Implementation and Evaluation

Design of system architecture

The architecture design of the apple disease identification system needs to consider the cooperation and interaction between various components of the system. The architecture provides an interface for users to upload pictures and view identification results, which can be a web application or a mobile application, which is responsible for receiving pictures uploaded by users and preprocessing the images, such as cropping, scaling or enhancing the images, to improve the recognition performance of the model. It contains a trained deep learning model for disease identification of images. Common convolutional neural network models, such as ResNet and VGG can be chosen, or new models can be designed and trained according to specific needs. The deep learning model is deployed to the cloud server or local server to provide reasoning service for image recognition. This module can be implemented by using tools such as TensorFlow Serving and Flask, which is used for storing images uploaded by users and related recognition results. A relational database or a NoSQL database can be chosen.

Image upload and processing

The task of the image uploading and processing module is to receive the images uploaded by users and preprocess them to improve the recognition performance of the model. The image uploading and processing module provides a user interface that allows users to select local image files to upload, or shoot and upload images through cameras. Next, it preprocesses uploaded images, such as cropping, scaling and denoising. The preprocessing operation can be adjusted according to specific situations to ensure that the input images meet the input requirements of the model. There are also optional steps, for enhancing the quality and identifiability of images. Common image enhancement methods include contrast enhancement, histogram equalization and filtering (Fig. 4).

Model deployment and cloud server configuration

Appropriate cloud service providers should be selected according to requirements, such as Alibaba Cloud, Tencent Cloud and Amazon AWS, to configure a cloud server. The trained deep

learning model is uploaded to the cloud server to build corresponding reasoning service. TensorFlow Serving, Flask, Django and other tools can be used to deploy the model and build the service. According to the concurrent requirements of the system and the workload of prediction, appropriate cloud server performance can be configured, including CPU, GPU, memory and storage. To ensure smooth communication between the cloud server and user interface, domain name resolution, network firewall and other techniques can be used to ensure the stability and security of the network.

Analysis of experimental results

In order to ensure the stability of the experimental results, we adopted a 5-fold cross-validation method to evaluate the performance of the model before and after improvement. Firstly, the training set was divided into five parts, four of which were used as training sets and the remaining one as a test set for performance verification. This process was repeated five times to ensure that the test sets used each time were mutually exclusive. Although there were some differences in performance on different data sets, the performance improvement of the improved model was relatively stable on the same dataset, and the average value of mAP@0.5 was increased by 4.9%.

In order to verify the final performance of the model, we used the whole training set to repeatedly train the apple disease identification model for three times, and used an independent test set to test the average accuracy (AP) of the model as the experimental result under the AP index. With the addition of FPN, the AP of the model on grey speck disease with small lesions, scab and cedar apple rust with irregular lesion areas increased significantly by 9.2%, 4.1% and 9.7% respectively. Compared with ResNet-FPN, ResNeSt-FPN had small increases of 1.4%, 0.4%, 1.1% and 2.2% in scab, grey speck, cedar apple rust and mosaic, which indicated that the attention separation mechanism played a positive role in improving the performance of the model. The cascade mechanism also significantly improved the recognition performance of various lesions. The overall AP increased by 2.8%, but the detection time increased by 2.2%. The improved model had AP of 91.5%, 87.2% and 93.7% on apple Alternaria leaf spot, grey speck and cedar apple rust, respectively. The boundaries of the lesion areas in the images of these three diseases were clear, so the model could effectively identify them. However, in scab and mosaic disease, the lesion areas were irregular and the boundaries were fuzzy, and the AP identified by the model only reached 78.3% and 80%, respectively. There is still room for improvement.

Conclusions

In this study, a new apple disease identification method was proposed, aiming to solve the problem of difficulties in monitoring images with complex background and small target disease spots. Based on the Faster R-CNN target detection framework, the method was improved to meet the special needs of images with small target lesions and complex background, so as to better adapt to practical application scenarios. ResNest was used as the backbone network, which has stronger feature extraction ability, and the feature pyramid network FPN was introduced to extract more robust features with richer semantic information. The cascade mechanism was introduced to realize a suggestion box with high quality, which enhanced the positioning accuracy of the improved network model to diseased targets. The experimental results showed that the improved model reached 0.862 in mAP, and the recognition accuracy of disease images reached 98.3%. It could effectively identify five kinds of apple diseases, and the average time to identify a picture under CPU was 1.396 s, which was practical. The mAP@0.5 of the improved model was 8.7% higher than that of the unimproved model, which verified the effectiveness of the method.

Through the research and experiment on the apple disease identification system, we came to following main conclusions: in the apple disease identification system, classical convolutional neural network models (such as ResNet and VGG) showed good performance and generalization ability. An appropriate dataset division ratio (such as 8：1：1 or 7：2：1) could improve the training effect and generalization ability of the model. Preprocessing of apple disease images (such as cropping, scaling and enhancement) could improve the identification performance and robustness of the model. The performance of the model could be evaluated in terms of accuracy, recall rate, precision and F1 score, and comparison was conducted with other most advanced methods to verify the effectiveness of the system.

References

［1］ CHEN P, FANG T, ZHANG J, et al. Classification method of apple leaf diseases based on deep learning and automatic identification device thereof: CN201910324553.5［P］. CN110097104A ［2023-09-08］. (in Chinese).

［2］ CHAO XF. Research on common apple leaf diseases identification and lesion segmentation based on deep learning［J］. ［2023-09-08］. (in Chinese).

［3］ XIAO JW. Identification and classification of apple diseases based on deep learning［D］. Xian: Xian Technological University, 2023. (in Chinese).

［4］ LI XR. Research on apple leaf disease detection method based on deeping learning［D］. Yangling: Northwest A&F University, 2023. (in Chinese).

［5］ LIU XY, DUAN N. An apple leaf disease detection method based on deep learning: CN202210835044.0［P］.CN202210835044.0［2023-09-08］. (in Chinese).

［6］ LIU XY, DUAN N. An apple leaf disease detection method based on deep learning: 202210835044［P］. ［2023-09-08］. (in Chinese).

［7］ ZHANG ZT, HAN JY, ZHANG DH, et al. Anomaly detection of potato, maize and apple leaf diseases based on color moments［J］. Acta Agriculturae Zhejiangensis, 20234(10): 10. (in Chinese).

［8］ WANG J, LIU XH. Pathological recognition of apple leaves based on deeply separable convolution［J］. Computer Systems & Applications, 2020(11): 6. (in Chinese).

［9］ WANG C, WANG CQ, LIU JM. Identification of maize leaf diseases based on deep learning［J］. Modern Agriculture Research, 2022(6): 028. (in Chinese).

［10］ WANG YL, WU JF, LAN P, et al. Apple disease identification using improved Faster R-CNN［J］. Journal of Forestry Engineering, 2022(1): 007. (in Chinese).

Editor: Yingzhi GUANGProofreader: Xinxiu ZHU

Received: July 29, 2023Accepted: September 30, 2023

Supported by Bingtuan Science and Technology Program (2021DB001; 2021BB023); Science and Technology Innovation Team of Tarim University (TDZKCX202102).

Peng QIN (1995-), male, P. R. China, teaching assistant, devoted to research about agricultural engineering and information technology.

*Corresponding author. E-mail: 929959182@qq.com.