Crop phenotyping studies with application to crop monitoring

Xiuliang Jin ,Wanneng Yang ,John H.Doonan ,Clement Atzerger

a Institute of Crop Sciences,Chinese Academy of Agricultural Sciences/Key Laboratory of Crop Physiology and Ecology,Ministry of Agriculture,Beijing 100081,China

b National Key Laboratory of Crop Genetic Improvement,National Center of Plant Gene Research,Huazhong Agricultural University,Wuhan 430070,Hubei,China

c The National Plant Phenomics Centre,Institute of Biological,Rural and Environmental Sciences (IBERS),Aberystwyth University,Aberystwyth,Ceredigion SY23 3FL,UK

d Institute of Geomatics,University of Natural Resources and Life Sciences (BOKU),Peter Jordan Straße 82,1190 Vienna,Austria

1.Introduction

Crop yield must urgently be sustainably increased to accommodate a rising global population and anticipated climate change in the coming decades,in the face of plant stresses and limited resources[1].Conventional crop breeding is limited by phenotypic selection and breeding efficiency.Crop phenotyping is defined as the application of protocols and methodologies to obtain a specific trait phenotype,ranging from whole plant or canopy level to cellular level,associated with plant biochemistry,function,or structure[2,3].Genomics-assisted breeding advances food security,but the crop breeding community needs more effective ways to study the relationship between phenotype and genotype.Although high-throughput genotyping is available at low cost,crop phenotyping and related data management and analysis remain relatively expensive.High-throughput crop phenotyping methods have received increasing attention for their potential for using genomic resources for the genetic improvement of crop yield.They provide powerful tools for measuring physiological and agronomic trait phenotypes,quantifying and monitoring large genetically defined populations in field experiments and breeding nurseries on multiple temporal and spatial scales [4-8].To do this,they apply advanced robotics,high-tech sensors,data processing systems,and images.Several new bioinformatic platforms include multi-dimensional,large-scale trait phenotype datasets and genotypic and omics information.Gene functions and environmental responses can now be dissected with unprecedented temporal and spatial resolution using combined genotyping,phenotyping,and multi-omics data.This ability will help to overcome the limitation of incremental improvements in crop yield.The aim of this special issue is to investigate the latest innovative research in remote sensing technologies,sensor development,technological platforms,and applications for estimating crop trait phenotype based on multisource data streams and imagery.The special issue titled‘‘Crop phenotyping studies with application to crop monitoring”is launched.Here we summarize these papers according to the classification of topics and add our perspectives.

2.Overview of contributions of this special issue

Contributions included in this special issue describe the estimation of crop phenotypes by sensors installed on various phenotyping platforms.They highlight the use of spectral analysis,image segmentation,and machine learning algorithms.

2.1.Sensors and platforms

High-throughput crop phenotyping technologies described in this special issue employ remote sensing phenotyping platforms including ground-based [9-20],aerial [21-23],indoor [24-27],and satellite-based [28-34] platforms.Twelve papers report results from near-ground platforms and three from aerial,four from indoor,and seven from satellite phenotyping platforms.For ground-based platforms,handheld-based field measuring system[9,11-18,20] and fixed scanning systems [10,19] were used to estimate traits for several crop types.For aerial phenotyping platforms,the recent development of unmanned aerial vehicles(UAVs)has made data acquisition more efficient with unprecedented temporal,spectral and spatial detail[2].Indoor phenotyping platforms have been used to acquire organ-scale traits associated with rice panicles[24],seed germination[25],pod length[26],and vascular bundles[27],because illumination can be well controlled.Because satellite phenotyping platforms have the advantage of regional scale,they have been successfully used for crop classification[12,28,30,32,34],yield estimation[29,31],and crop coefficient estimation [33].

In addition to phenotyping platforms,optical sensors play an important role in advanced phenotyping methods.Light,cheap sensors can be installed in these platforms to increase their data acquisition efficiency and quality.Light detection and ranging(LiDAR) sensors [10,19],hyperspectral sensors [9,11,12,14,15,17,20],thermal sensors [22],and RGB and multi-spectral imagery cameras [13,16,18,22-34] have been used for studying crop phenotyping.RGB images for segmentation,detec-tion,or classification and multispectral and thermal images and hyperspectral data for physical and biochemical crop traits are fully explored using optical radiative transfer models.LiDAR sensors loaded on UAV or ground platforms may be used to acquire three-dimensional crop structure information in future breeding programs.

2.2.Crop phenotyping traits

In this special issue,crop traits are divided into two types:biochemical traits and morphological/structural traits.Biochemical traits include canopy nitrogen and carbon content [9,20],leaf pigmentation [11],stripe rust disease [14],chlorophyll [15],wheat powdery mildew [17],yield [19,29,31],aboveground dry biomass[20,22,23],phenological stages [21],seed germination [25],and crop coefficient [33].These traits are estimated by multi/hyperspectral images or sensors,thermal images,or RGB images from ground,satellite,and UAV platforms.Li et al.[19] used multispectral data and LiDAR to identify the best machine learning model and growth stage for estimating yield in wheat.Li et al.[22] evaluated the performance of different data (acquired by RGB,multispectral,and thermal cameras) from a UAV for estimating biomass in sorghum.Morphological and structural traits included plant height [13],rice panicles [24],seed germination [25],pod length [26] and vascular bundle [27].These morphological and structural traits were measured from three-dimensional point clouds from LiDAR sensors or RGB images.Wu et al.[24]integrated supervoxel clustering and a deep convolutional neural network to model 3D rice panicles.Du et al.[27] developed a deep learningintegrated phenotyping pipeline to detect vascular bundles with computed tomography images.The special issue also presents studies on rice and wheat spike detection [16,18] and crop classification [30,32,34] under various ecological environmental conditions.

2.3.Data processing and analysis approaches

In recent years,in the era of big data,data processing and analysis approaches are critical for increasing the efficiency and quality of information extracted from crop phenotyping systems.Such approaches have been used to estimate crop phenotypes.They are classified into two types,corresponding to the abovementioned two kinds of crop traits.Approaches used for estimating biochemical traits include partial least-squares regression [9,25],lookup table [11],difference-in-differences algorithm [14],random forest[15,17,19,23],extreme learning machine[17],artificial neural network [17,19],support vector machine [17,19,22,23],hierarchical linear model [20],asymmetric Gaussian function [21],quadratic and cubic polynomials [29],data assimilation [31],and linear discriminant analysis models[25].These approaches are shown to be efficient for estimating various targeted crop traits.Optical radiative transfer models combined with an optimizing algorithm was applied by Sun et al.[11],who used radiative transfer models to invert leaf chlorophyll and carotenoid content.In contrast to biochemical traits,morphological and structural traits including plant height[13],rice panicles[24],pod length[26]and vascular bundle[27] are usually estimated using image detection and segmentation approaches.Qiu et al.[13] used RGB-D camera to capture depth information and color images for measuring maize plant height using a segmentation algorithm.Li et al.[26]used a feature pyramid network,principal component analysis and instance segmentation to measure pod length and width in soybean.These detection and segmentation approaches are used to reduce the influence of background information,and then are applied for accurate estimation of crop morphological and structural traits.

3.Summary and perspectives

Conventional crop phenotyping costs much time,effort,and resources.High-throughput crop phenotyping methods are complementary to such field work and allow high-throughput crop phenotyping using UAVs and advanced sensors (thermal infrared,multi/hyperspectral,LiDAR,and others).The integration of UAV with advanced sensors to acquire abundant spatial,temporal,and spectral information has been applied to crop phenotyping by many scientists[2].The unique advantages of UAV remote sensing not only increase the efficiency of data acquisition,but facilitate data standardization,reducing human subjective evaluation[35].Machine learning algorithms and image processing and analysis methods are rapidly advancing,including data preprocessing,deep learning algorithms,and platform or system development and testing.All these features contribute to estimating targeted crop traits using multi-source data [5,8].

These 26 papers presented in this special issue highlight the topic of estimation of crop traits using remote sensing technologies,sensors,technological platforms and machine learning algorithms.First,the special issue describes the importance of novel high-throughput crop phenotyping methods for improving crop breeding.Second,it investigates the application of sensors and platforms for high-throughput phenotyping of diverse crops in diverse growth environments.Finally,it provides guidelines to effectively combining data processing and analysis methods for improving crop phenotyping.

The special issue does not discuss the rapid development of ground phenotyping platforms (phenopoles,phenomobiles,and stationary platforms) and their applications for crop phenotyping traits under biotic and abiotic stresses in the field experiments.Phenotypic studies of crop roots and micro-scale crop phenotypes are not featured.This issue does not focus on genome-wide association study approaches or quantitative trait locus identification based on crop genomic and phenotype datasets.Scientists should pay more attention to these study directions in the future.

High-throughput crop phenotyping methods need to be further improved to yield more accurate estimates of crop traits.The combination of aerial and ground platforms and advanced sensors,such as thermal,hyperspectral and multispectral cameras,have resulted in a pressing need for advanced image processing algorithms.Deep-learning algorithms have shown advantages for crop phenotype detection and segmentation[6,7].The estimation accuracy of crop phenotyping traits is reduced because of crop growth environmental conditions that degrade the stability of optical sensors.Field crop phenotyping will benefit from refined and more stable optical sensors.Crop phenotyping platforms and sensors are expensive in most crop breeding studies,but the rapid development of mobile and miniaturized technologies will offer powerful and affordable micro-sensors for monitoring crop phenotypes via multi-temporal high-resolution images.Smaller and lighter sensors have been combined with phenotyping platforms to conduct the study of crop phenotyping[4,5,8].Various optical sensors have been used to estimate crop traits under multiple stress conditions.Integrating the data and image outputs of sensors to increase the accuracy of crop phenotype estimation remains a challenge for crop phenotyping research [2,3].Satellites acquiring relatively high-resolution temporal-spatial images offer the opportunity to estimate crop traits on a large regional scale according to international image processing standard protocols.Because images from ground-and aerial-based crop phenotyping systems cannot contain internationally uniform data analysis standards,the sharing of image datasets will be prohibited [2].

Multidisciplinary collaboration teams will build a more efficient crop phenotype data management and analysis system.This system will include a user-friendly data management and analysis interface that is combined with data or image preprocessing and analysis algorithms.Field weather and soil information should be input into the system to maintain the estimation stability and accuracy of phenotyping.In future,high-throughput crop phenotyping will increase the efficiency of crop trait identification and further find new crop traits in crop breeding studies with more advanced sensors,image processing algorithms,and platforms.Despite great progress in the field of crop phenotyping,there are still several opportunities for follow-up investigations about the field of crop phenotyping.In particular,we suggest more studies on the application and development of ground platforms and the creation of algorithms for multi-source data fusion.Deep learning algorithms linking functional structure models and optical radiative transfer models will better leverage the value of big data in the field of crop phenotyping.With fast development of image processing algorithms and sensor technology,we believe that crop phenotyping will receive more attention by the image processing,remote sensing,and crop breeding communities.Finally,highthroughput crop phenotyping methods will accelerate follow-up studies of precision agriculture.