Rotating machinery fault diagnosis based on convolutional neural network and infrared thermal imaging

Yongo LI, Xioqing DU, Fngyi WAN, Xinzhi WANG, Hungho YU

a MIIT Key Laboratory of Dynamics and Control of Complex Systems, Northwestern Polytechnical University, Xi’an 710072, China

b School of Mechanical Engineering, Northwestern Polytechnical University, Xi’an 710072, China

c Institute of Unmanned System, National University of Defense Technology, Changsha 410073, China

KEYWORDS Convolutional neural network;Feature extraction;Infrared thermography(IRT);Intelligent fault diagnosis;Rotating machinery

Abstract Rotating machinery is widely applied in industrial applications.Fault diagnosis of rotating machinery is vital in manufacturing system, which can prevent catastrophic failure and reduce financial losses. Recently, Deep Learning (DL)-based fault diagnosis method becomes a hot topic.Convolutional Neural Network (CNN) is an effective DL method to extract the features of raw data automatically.This paper develops a fault diagnosis method using CNN for InfRared Thermal(IRT)image.First,IRT technique is utilized to capture the IRT images of rotating machinery.Second, the CNN is applied to extract fault features from the IRT images. In the end, the obtained features are fed into the Softmax Regression (SR) classifier for fault pattern identification. The effectiveness of the proposed method is validated using two different experimental data. Results show that the proposed method has a superior performance in identification various faults on rotor and bearings comparing with other deep learning models and traditional vibration-based method.©2019 Chinese Society of Aeronautics and Astronautics.Production and hosting by Elsevier Ltd.This is an open access article under the CC BY-NC-ND license(http://creativecommons.org/licenses/by-nc-nd/4.0/).

1. Introduction

Rotating machinery plays an important role in aerospace applications, which has received increasing attention from both academic circles and industry, for example, in airplanes,aircraft engine, rotor system and spacecraft.1-6Subjected to the harsh operation condition,rotating machinery often suffer from various failures, resulting in unscheduled maintenance cost and financial losses. Therefore, it is highly desired to perform fault diagnosis of rotating machinery to prevent catastrophic accident and ensure the safety operation.7,8

To address this issue, various sensor signals have been applied in fault diagnosis of rotating machinery, including vibration signal,9current signal,10acoustic emission signal,11sound signal,12torque signal,13and rotating encoder signal14etc. Among these sensor signals, vibration-based diagnostic method is the most commonly researched because the vibration signals can directly represent the dynamic behavior of rotating machinery.15-17,14However, the existing methodologies had three major challenges for the fault diagnosis in real industrial applications as bellows:

(1) Most of the existing methodologies are inefficient to monitor the large-scale machinery, for example, aircraft engines. Because the long transfer path could weaken the fault signatures and mix more interference noises during machine operating process.

(2) Most of the existing methodologies are easily affected by the time-varying working conditions. It is difficult to extract the sensitive fault signatures that are unaffected by working conditions.

(3) The prior knowledge are required in the fault diagnosis of rotating machinery,which are not always available in real industrial applications.

For these reasons,it is vital to search for a new sensing technique to monitor the health condition of rotating machinery.14Recently, temperature signal gains noticeable attentions. The temperature signal is often collected using a thermocouple.Since the fault can cause the temperature of machinery increase,recent research has proven that the temperature signal carries rich diagnostic information,which offers a promising tool for the Health Condition Monitoring(HCM)of rotating machinery.18-20Thermal camera,as an IRT imagine measuring device,can measure the surface temperature of the object without contact. The IRT images contain rich fault information. By virtue of the advantage of IRT images,this paper presents a novel fault diagnosis method to achieve the fault diagnosis of rotating machinery. First, a Convolutional Neural Network (CNN)-based deep learning model is applied to extract the distinctive features from the IRT images. Then, softmax regression classifier is employed to classify various fault types of rotating machinery.The effectiveness of the proposed method is validated using the experimental data from a rotating machinery.

The rest of this paper is organized as follows: In Section 2,the related research work is reviewed. In Section 3, the framework and steps of the proposed method for fault diagnosis are briefly introduced.In Section 4,the validation experiment and data acquisition system are described in detail. In Section 5,the effectiveness of the proposed method is validated using experimental data. Finally, Section 6 draws the conclusion and gives the future work.

2. Review related works and their limitations

The main advantage of the IRT-based HCM is that IRT technique only requires an infrared camera,which can generate the video for displaying the IRT images. Due to its easy-to-implement, IRTbased fault diagnosis method has been successfully applied in the HCM of various objectives, for example, aerospace,18electronically components,21,22buildings,23medical,24freeze-drying,25and cultural heritage,26as shown in Fig.1.

Known that the IRT images contain rich diagnostic information,18-20which provides an alternative and non-invasive way to detect the potential failure of machines. Generally,the IRT-based fault diagnosis method consists of three steps:data collection,fault feature extraction and fault pattern identification. Among these steps, effective feature representation is crucial to enhance the classification performance.27In recent years,many studies have been done to develop the fault feature representation of IRT images aiming to automatic identify the various fault types of mechanisms.28This section focuses on the IRT-based fault diagnosis of rotating machinery and the state-of-art methods and challenges are reviewed.

The color histogram is a simple but effective method for feature extraction, Younus et al.29utilized color histogram to extract the fault information from the IRT images. Then, a Support Vector Machines(SVM)classifier is applied to recognize different health conditions of rotating machines. Widodo et al. used some indicators, such as kurtosis, mean, variance,entropy, central moments, standard deviation and maximum and minimum to distinguish the unbalance, misalignment,looseness,and rolling element bearing faults.30Glowacz Adam and Glowacz Zygfryd developed Method of Area Selection of States (MoASoS)31and Method of Areas Selection of Image Differences (MoASoID)32to recognize different fault types of the induction motor. Janssens et al. attempted to extract the fault features using standard deviation of the temperature,Gini coefficient, and the moment of light and then classify the health conditions of rolling bearings based on SVM.33Meanwhile, researchers have developed various bi-dimensional decomposition methods to preprocess the IRT imagines.Tran et al. explored the bi-dimensional empirical mode decomposition method to characterize the discriminative fault signatures of thermal images and then used a relevance vector machine for fault classification of rotating machinery.34Younus and Yang investigated a fault pattern classification system using the discrete wavelet decomposition to extract the fault signatures, and various fault modes of rotating machinery were identified.35

To date, the reported literature using the thermal images is still very limited and there are two existing problems as follows.

(1) Most of the existing IRT-based fault diagnosis methods depend on the supervised learning to acquire the discriminative features. However, these methods require a considerable amount of engineering skill and domain expertise, which is inefficient for the HCM of rotating machinery. It is necessary to introduce the deepfeature-learning methods,which can automatically learn from data using a general-purpose learning procedure via deep-architecture neural networks.36

(2) In real industrial applications, faults may occur under different temperature distribution. These different temperature conditions will cause different fault information distribution on thermal images. It is difficult to extract the discriminative features that are unaffected by the varying temperature condition. However, the existing IRT-based fault diagnosis methods do not consider the temperature fluctuation influence.

Fig. 1 Some applications of infrared thermography technology.

To fill this research gap, this paper proposed a deep learning model to extract the fault features from thermal images under different temperature operation conditions. Deep learning models have shown good performance in image processing.There are four common deep learning models,including Deep Belief Nets (DBNs),37Deep Neural Networks (DNN),38Stacked AutoEncoder(SAE)39and Convolutional Neural Networks(CNNs).40,41Among these models,CNN shows impressive feature representation capability for a wide range of image classifications. Therefore, CNN is employed to extract the fault features in this study.

2.1.1. Convolutional layer

As the key component of CNN, the convolutional layer is responsible for most of the computationally heavy work. The convolutional layer is composed of a series of filter kernels that can generate new feature maps from the local regions of the input maps.Here,we use kl∈RJ×D×Hrepresents the filter kernel in layer l,where J，D，H indicate the number,the depth and the height of the kernels,respectively.The jth output mapof layer l can be expressed as:

2.1. CNN method

CNN, as a special MultiLayer Perceptron (MLP) topology network, consists of some filter stages and one classification stage. CNN has been widely applied in multiple fields, including image analysis, video signal processing and speech signal processing.42CNN can extract the discriminative features from the IRT images directly, which avoids the information loss brought by artificial processing. Moreover, CNN performs local operation on the input raw data with nonlinear functions for data dimension reduction. Following subsections will provide detailed description for different layers of CNN. Fig. 2 illustrates the process of the CNN method: convolutional layer, batch normalization layer, activation layer and pooling layer.

2.1.2. Batch normalization layer

Batch normalization layer aims to reduce the shift of internal covariance and accelerate the training process of the deep neural network. Batch normalization layer is often added right after the convolutional layer and before the activation layer.The batch normalization process can be expressed as:

Fig. 2 Process of the convolutional neural networks.

Fig. 4 Flowchart of the proposed CNN-based fault diagnosis of rotating machinery.

Step 4. The training samples are utilized to train the deep learning model that includes the CNN layer and Softmax regression layer;

Step 5.The trained deep learning model is used to recognize the fault patterns using the test samples.

It is noteworthy that no expert knowledge of failure mechanisms and parameter setting are required beforehand for application of the proposed method. The proposed CNNbased method is easy for application and highly adaptive for the fault diagnosis of rotating machinery.

4. Experimental IRT images acquisition

In this paper, the IRT images are acquired from the SpectraQuest Machinery Fault Simulator(MFS)machine localized in Northwestern Polytechnical University lab. Fig. 5(a) illustrates the rotating machinery test rig and its corresponding layout is shown in Fig. 5(b). We can find the test rig contains a drive motor, two rolling bearings, an unbalanced rotor, and a data acquisition system. The bearing specification is MB ER-12K,which is marked in blue line in Fig.5(a).The parameters of the MFS set are given in Table 1. During this experiment, the thermal images are collected using the thermal camera Hawk-1384,as shown in Fig.5.The parameters about the thermal camera are given in Table 2.

In the experiments, nine fault types were introduced to the rotating machinery, including UNbalanced fault (UN), Inner Race Fault (IRF), Outer Race Fault (ORF), Ball Fault (BF),Combination bearing Fault (CF), Inner Race Fault with UNbalanced fault (IRFUN), Outer Race Fault with UNbalanced fault (ORFUN), Ball Fault with UNbalanced fault(BFUN), and Combination bearing Fault with UNbalanced fault (CFUN). Note that the unbalanced state is simulated by attaching weights on the unbalanced rotor. The bearing and rotor faults are shown in Fig. 6. The rotating speed is 3000 rpm and the IRT videos are recorded using thermal camera following the detailed steps in Algorithm1.

Algorithm 1. Steps for the thermal image acquisition Step 1. Select one fault type from the ten health conditions.Step 2. Initialize the temperature of MFS system to the environment temperature (18.9°C in this paper).Step 3. Conduct the thermal video acquisition under a constant speed of 3000 rpm.Step 4.Observe the highest temperature through the IRT camera(around the testing bearing). When the highest temperature reaches a relatively stable temperature (60°C in this test), stop the thermal video acquisition.Step 5. Cool down the MFS system to the environment temperature.Step 6.Repeat the above steps for another fault type until the end of experiment.Step 7.Extract the thermal images from the thermal videos under each health condition.

5. Experimental study

In this paper, we adopt four standard deep learning methods:CNN, DBN, DNN and SAE to process the IRT images. Two experiments: Case 1 (using Data subset 1) and Case 2 (using Data subset 2) of rotating machinery are chosen to verify the effectiveness of the proposed method. Data subset 1 consists of the IRT images measured under six temperature regions and Data subset 2 is composed of IRT images measured under fusing temperature regions.

5.1. Dataset details

Fig. 5 The test rig and its schematic diagram.

Table 1 Working parameters of rotating machinery.

Table 2 Configuration parameters setting of the thermal camera.

Fig. 6 The actual used testing unbalanced rator and rolling bearings.

Data subset 1 consists of six temperature regions of meshing area, including Region 1 (35-38°C), Region 2 (38-41°C),Region 3 (41-44°C), Region 4 (44-47°C), Region 5 (47-50°C), and Region 6 (50-53°C). Each temperature region consists of 1000 samples(100 samples×10 health conditions).For example, the IRT images under10 health conditions in Region 1 are shown in Fig. 7.

As we can be seen in Fig.7,it is hard to distinguish the fault types visually though observing the IRT images. This can be attributed to that the difference between the thermal images of each health condition is very small.

In this study, we use 35% of samples for training and the rest samples are utilized to test the performance of the each method. Note that Data subset 2 is composed of the IRT images measured under fusing temperature regions with 6000 samples (1000 samples×6 different temperature regions).The detailed description of numbers for Data subset 1 and Data subset 2 are illustrated in Tables 3 and 4, respectively.

5.2. Parameters setting of deep learning methods

The CNN method is utilized to extract the fault features of IRT images. In this paper, the LeNet-5 model is adopted in CNN method.The detailed parameters of CNN are as follows.The CNN contains four convolutional layers and pooling layers for input images with size of 100×100×3. The filter size contains 5×5 and 3×3.Then,the fully connection layers are followed to compute the class scores for 10 classifications.The detailed structure of CNN is given in Table 5.

In order to verify the superiority of the CNN approach,some recent deep learning methods are also considered,including DBN model, DNN model and SAE model. In DBN method,we choose the network with 5 layers.The network size is [30000, 512, 128, 64, 10]. In DNN method, we set the network with the size of[30000,200,100,10].SAE adopts four layers in which the last one is output layer. The size of these four layers is [30000 200 100 10] where the feature dimension is set 100. Moreover, the iteration number of these networks is set 100. They are all employed to recognize various fault types of rotating machinery.The detailed parameters of above models are summarized in Table 6.

5.3. Results and analysis

5.3.1. Case 1: condition identification using the IRT images measured under one temperature region

Fig. 7 Thermal images under ten health conditions.

Table 3 Detailed description of numbers for Data subset 1.

Table 4 Detailed description of numbers for Data subset 2.

Case 1 aims to test the performance of the proposed method using the IRT images measured under one temperature region with Data subset 1. The detailed description of numbers for Data subset 1 is shown in Table 7. For comparison, DBN method, DNN method and SAE method are also performed for analyzing each sub-experiment respectively.Meanwhile,20 rails are conducted for each experiment to reduce the effect of randomness. In the end, SR classifier is utilized to identify the health conditions. The obtained classification results of six sub-experiments using four deep learning methods are illustrated in Fig. 8 and Table 8, respectively.

It can be observed from Table 7 that the proposed CNN obtains the highest average classification accuracy (98.11%)and DNN method has the second highest average classification accuracy (83.60%). The SAE and DBN methods have even lower average classification accuracies with 75.53% and 60.84%, respectively. Meanwhile, the CNN method has the lowest standard deviation value of 0.0115 among the four deep learning methods.The comparison results reinforce the superiority of the proposed CNN method in extracting the fault features from the IRT images. Moreover, the comparison results demonstrate that CNN has certain advantage of antiinterference ability for temperature fluctuation.

This is because that CNN can extract the features from the IRT images directly,avoiding the information loss brought by artificial processing. Moreover, convolution and pooling are the key operations of CNN, in which convolution is used to extract local image features and pooling is employed to extract local features and reduce data dimension. After multiple convolutions and pooling, higher-level semantic information of images can be extracted. Therefore, CNN has a better ability for image feature extraction.

5.3.2. Case 2: condition identification using the IRT images measured under fusing temperature regions

In actual applications, the localized damage may occur under random temperatures.Motivated by this,Case 2 is designed to investigate the effectiveness of the proposed method under mixed temperature regions. The detailed description of numbers for Data subset 2 is shown in Table 4.To show the advantage of the proposed method, DBN, DNN, SAE, and CNN methods are all applied to recognize the various health conditions. Fig. 9 illustrates the obtained classification results of four methods using the confusion matrix.

As can be seen from Fig.9,CNN performs best in diagnosing ten faults of rotating machinery under fusing temperature regions, while DBN, DNN and SAE all misclassify the rotating machinery. The classification accuracy using DBN, DNN and SAE to identify the various health conditions is actually modest. Moreover, 20 rails are conducted to reduce the effect of randomness. The average testing accuracy and its corresponding standard deviation are given in Table 8.

It can be observed that the proposed CNN method has the highest testing accuracy of 98.64%,which is much higher than the second highest accuracy of 88.20% generated from SAE.DBN method has the lowest average testing accuracy of70.62%.Moreover,CNN has the smallest standard deviation,while DBN has the largest standard deviation. Overall, the proposed CNN method has the best feature extraction ability among the four deep learning models.

Table 6 Detailed description of parameters of compared methods.

Table 7 Classification results for Data subset 1.

To demonstrate the advantage of proposed IRT-based method, a comparison is conducted between the proposed method and highly cited vibration-based method using Permutation Entropy(PE).44The time domain waveforms of rotating machinery are shown in Fig. 10. First, the fault features are extracted from vibration signals using PE. Second, the obtained fault features are put into the SR to complete fault identification. For fair comparison, the vibration-based method also adopts 35% samples for training and 65% samples for testing. The classification result of vibration-based method is displayed in Fig.11. We can observe that classification accuracy of vibration-based method is 75.38% (489/650),which is much lower than the proposed IRT-based method.The comparison results demonstrate that the IRT-based method offers a promising tool for fault diagnosis of rotating machinery, which is more effective than vibration-based method.

Furthermore, to show the discriminative features extracted from IRT images and vibration signals,three-dimensional projections are used for visualizing using PCA, as shown in Fig.12. It can be observed that the samples are all distributed around the class center and the samples are discriminated clearly using IRT-based method, which indicates that the extracted features has a stronger capability of holding the local information and stronger distinguishing ability. However, the features extracted from vibration signal are distributed without a nice cluster, as shown in Fig. 12 (b).

6. Conclusions

Fig. 8 Diagnosis results of the six trails using CNN, DBN,DNN and SAE methods.

Table 8 Classification results for Data subset 2.

This paper develops a novel CNN-based fault diagnosis method of rotating machinery using IRT images. To validate the effectiveness of the proposed method, the IRT images are collected from a rotating machinery under ten health conditions. Meanwhile, the influence of the temperature is also investigated in this study. By comparing with DBN, DNN,and SAE, the results demonstrate that the proposed CNN-based method outperforms the DBN, DNN, and SAE in identifying ten health conditions of rotating machinery. In summary, we have three main contributions in this paper:

(1) A fault diagnosis strategy is proposed by combining

CNN and Softmax using IRT images.

Fig. 9 The detailed classification results using four methods.

Fig. 10 The time domains of ten health conditions.

Fig. 11 The classification results using vibration-based method.

(2) The IRT-based method offers a promising tool for fault diagnosis of rotating machinery, which is more effective than vibration-based method.

(3) Experimental signals demonstrate that the proposed method has a better fault diagnosis ability comparing with DBN, DNN, and SAE methods.

In this preliminary study,the proposed method was demonstrated to be effective for fault diagnosis of rotating machinery. The combination of IRT images and vibration signal for health monitoring of rotating machinery will be considered in our future work.

Acknowledgements

The research was supported by National Natural Science Foundation of China (No. 51805434); in part by the China Postdoctoral Innovative Talent Plan, China (No.BX20180257); in part by the Postdoctoral Science Funds,China (No. 2018M641021); and in part by the Key Research Program, Shaanxi Province.