Feature extraction of welding defect based on machine vision

Qi Jiyang and Li Jinyan

School of Mechanical Engineering, Jiangsu University of Science and Technology, Zhenjiang 212003, China

Abstract There are many flaws in welding images such as noise, low contrast, and blurred edges, which affect feature extraction from welding defect regions and impede classification and recognition of welding defects. To deal with the complexity of welding defect images, this paper proposes an effective method for extracting the features of welding defect regions. Firstly, image preprocessing, image segmentation and image background removal are carried out to a welding image in order to extract welding defect region; and then an 8-connected-component labeling method is used to mark defect regions. Finally, it extracts geometric characteristic parameters including perimeter, area, circularity and others. The experimental result shows that the method proposed in the paper can accurately extract the features of welding defect regions. It has good adaptability and practicability.

Key words welding defect, feature extraction, image processing, defect marking

0 Introduction

With the development of modern industrial technology, welding, which is an important processing technology, has been widely used in the machinery manufacturing industry. The quality of the welding product is determined by the quality of the welding. The welding components will inevitably have defects, due to the influence of the working environment in the welding process. The common defects are cracks, pores, slag, incomplete welding, etc[1]. If these defects cannot be detected accurately and quickly, the quality of the products will be seriously affected, and even the production safety may also be threatened. Therefore, it is of great significance to improve the quality of welding and ensure the safety of welding production by exploring the method of welding defect detection and identifying the types of welding defects.

In the process of welding defect detection, feature extraction from welding defects is an essential step. Feature extraction is the premise and basis of defect recognition and classification, which finds out several important features of the defects from a large number of welding defect’s features, so as to simplify the evaluation index and facilitate the classification and recognition of welding defects[2]. So far, many domestic and foreign scholars have proposed a variety of feature extraction methods. The common methods are: wavelet analysis[3-4], Fourier transform[5], and defect characteristic parameters representation[6]. In reference[3], the welding defect image is decomposed into low-frequency and high-frequency components by Wavelet-Contourlet transform. Then, improved total variation (TV) and Catte_PM/kernel anisotropic diffusion (KAD) are used to process the two kinds of components, respectively. Finally, the noise suppressed image is obtained by inverse wavelet-contourlet transform. In reference [6], the morphological close and open operation，labeling algorithm and false alarm eliminating algorithm are applied to process the binary image to obtain the defect detection result. Among them, the wavelet analysis method and the Fourier transform method belong to the frequency domain processing algorithm, which are not fit for the real-time detection of welding defects because of the complex process and the lengthy calculation time. The defect characteristic parameter representation is widely used in welding defect detection because of the diversity and flexibility of the selected parameters. Therefore, the method of defect characteristic parameter representation is used to extract the main characteristics of the welding defects in this paper, which provides a premise and basis for the type identification and real-time online detection of welding defects.

1 The composition of welding defect image acquisition system

In order to obtain the welding defect image, a set of image acquisition system is designed in this paper. The acquisition system uses IPC (industrial personal computer) as the main controller. The embedded image acquisition card is plugged into the motherboard of the IPC. The welding defect image acquisition is realized by embedded image acquisition card, the welding seam is imaged on the CCD camera after being beamed by transmission light source, and then the acquired digital image is transmitted to the computer through the image acquisition card.

2 Welding defect image feature extraction

The defect characteristic parameter representation method is to extract the main features on behalf of the defect through image processing, and then identify the defect types according to the characteristic parameters. The details of the image are blurred and the image contrast is reduced due to the influence of arc light, spatter and noise when acquiring the welding image. In order to identify the welding defects, the defect image needs a series of processing. There may be a number of defects in the welding image. In order to obtain the characteristics of each defect, the defects need to be marked. Finally, the suitable characteristic parameters are acquired from the marked defects. The flow chart of welding defect feature extraction is shown in Fig.1.

2.1 Image processing of welding defects

During the acquisition of welding defect image, the quality of the defect image will be decreased due to the influence of various factors[7]. The welding defect image cannot be used directly to extract the defect feature. Therefore it is necessary to conduct a series of image processing to the acquired welding defect image, including preprocessing, binary segmentation, and background removal. Now a more accurate defect region will be obtained, which is easier for the defect feature extraction.

Fig.1Flowchartoffeatureextractionforweldingdefectsimage

2.1.1 Welding defect image preprocessing

Image preprocessing is a process to improve the image quality, highlight and strengthen the region of interest and suppress the useless information. Acquired original welding defect images may not meet the requirements of the subsequent image feature extraction, so the welding defect image preprocessing is needed, which mainly includes smoothing filter and image enhancement, so as to achieve the effect of reducing noise and improving image contrast.

2.1.1.1 Image smoothing filter

The purpose of the image smoothing filter is to reduce or eliminate the influence caused by noise in the process of the acquisition of the image, and to improve the quality of the image for further image processing and analysis. The common filter methods are the spatial domain method and frequency domain method. Frequency domain method is based on the Fourier transform of an image, however the process is complex and time-consuming. The spatial domain method is a direct processing of the image pixels, including mean filtering, median filtering and Gauss filtering, etc. Although these methods can achieve the denoising effect, the median filtering method has better effect to maintain the image edge details. The median filtering method is used to denoise the image in this paper.

2.1.1.2 Image enhancement

Image enhancement is to highlight the useful information in the image and suppress the useless information, so that the details of the image will be highlighted and image contrast will be enhanced[8]. At present, the methods of image enhancement are histogram equalization[9]and fuzzy enhancement[10]. Histogram equalization is to improve the contrast of the image by changing the distribution range of pixels’ gray value, but it is prone to lead to the loss of tiny details. Fuzzy enhancement method can keep the details of the image well under the condition that the difference of fuzziness between the target and the background is large. During the feature extraction of welding defect image, the better the image details are kept, the more accurate the extracted characteristic parameters are. So the fuzzy enhancement method is used in this paper.

The basic idea of fuzzy enhancement algorithm is as follows. Firstly, the image gray value is mapped to a fuzzy matrix by a transformation function,and then fuzzy set theory is used for image enhancement processing in fuzzy space. Finally, the enhanced gray value is obtained by inverse transformation of the original transformation function.

Processing steps of the algorithm are as follows.

(1) Calculating the membership value of each pixel of the welding image,uij(gij).

Supposing that a pixel’s gray value isgij,the membership value is calculated according to the following formula.

(1)

wheregijis the gray value of the pixel (i,j),gcis the gray value of the crossover point, andlis the gray scale of the welding image.

The pixel whose membership value is 0.5 is defined as the crossover points.

(2) Image fuzzy enhancement is realized by using enhanced operatorT. Fuzzy enhancement operatorTis recursive and its definition is as follows.

Tr(uij)=T1(Tr-1(uij))，r=2,3,…,k

(2)

whereris the number of recursion calls. Enhanced operatorTmeets:

(3)

(3) After the inverse transformation of the membership function, the new gray value of the welding defect image after fuzzy enhancement is obtained. New gray value of the pixel (i,j), denoted bygij,can be obtained through the following formula:

(4)

2.1.2 Image segmentation

The segmentation of the enhanced welding defec timage is to separate the region of interest from the whole image. At present, the common method of image segmentation can be divided into three categories roughly: threshold method[11], edge detection method[12]and region tracking method[13]. The edge detection and region tracking have higher requirements for the quality of the image. Usually, the edge of the welding image is blurred and the image quality is affected because of the interference of arc light and splash in the process of image acquisition, so the two methods are not suitable. The threshold method becomes the common method of image segmentation because it has stable performance and high accuracy of the image segmentation. The basic idea of the algorithm is as follows. The whole image is divided into two classes through a threshold. One class represents the target and the other represents the background. The division result will differ based on different threshold, so the ratio of the interclass variance and intraclass variance of gray value will differ. The threshold which makes the largest ratio of the interclass variance and intraclass variance is determined as the best segmentation threshold of the image.

An image is assumed to havem×npixels, and the gray scale of the image isl. The range of the pixel gray value isG={0,1,2,…,l-1}. The initial image segmentation threshold is assumed to bet, then the image is divided into two regions, denoted by C0and C1: The gray value ranges of C0and C1are [0,t], [t+1,l-1] respectively.w0andw1are assumed to be the percentages of C0and C1in the whole image respectively.μ0andμ1are the average gray value in C0and C1region respectively.μis the average gray value of the whole image.

(5)

(6)

Threshold value is calculated based on the following formula[14]:

(7)

where,p(i) is the percentage of the pixels whose gray value equals toiin the image. The value oftvaries from 0 tol-1 gradually, thetwhich can make the above ratio maximum is the best segmentation thresholdt*.

2.1.3 Welding defect image background removal

When the welding defectregion is being extracted, a large region with black background on both sides of the welding seam is not needed and it will affect the extraction speed of the target region. In order to reduce the operation time of the algorithm and improve the real-time performance of welding defect detection, it is necessary to remove the large region with the black background in the image, leaving only the middle black region. The welding image is scanned from top to bottom in columns. When the pixel’s color changes from black to white, write down the column coordinate of the pixel and define it as the upper edge. All the black pixels are turned into white pixels (gray value is changed from 0 to 255) above the upper edge of each column, so that the black background above the upper edge is removed. Then, the defect image is scanned from the bottom to the top, and the black background below the lower edge is removed with the same method.

2.2 Target marking of welding defect image

The main aim of the target marking of the defect image is to distinguish the difference of the segmented defects. By marking the defect target in the binary image, each individual connected region is formed into a marked block. Further, the geometric parameters such as perimeter, area, circularity and so on can be obtained. This paper presents 8-connected-component labeling method[15]to mark the defect regions of the segmented binary image.

The 8-connected-component labeling method is to extract a set of pixels which are adjacent to each other and their gray value is 0 (black pixel point). The purpose is to mark all pixels belonging to the same target object with a unique tag value.

The procedure of the specific target defect region marking is as follows:

(1) First, all the pixels on the whole image are set to unlabeled and the initial value of tag is set to 1.

(2) The pixels are scanned from left to right, from top to bottom, once an unmarked black pixel is found, set it as the initial point and mark it.

(3) The adjacent right point, the lower right point, the lower point and the lower left point are judged in turn. If the pixel is black and unmarked, the point coordinates are pushed into the stack.

(4) Pop the stack and repeat step 3.

(5) End the iteration until the stack is empty, then return to step 2. In the meantime the tag value increases by 1.

(6) When all the pixels of the whole image whose gray value is 0 are marked, end the scan.

2.3 Characteristic parameter extraction of welding defect image

After the welding defectregions are marked, the geometric parameters of the defect regions can be measured further, so as to analyze the defects quantitatively and classify them. The predominant geometric characteristic parameters of welding defects are perimeter, area, circularity and so on. The extraction of these parameters will provide an accurate basis for the classification and recognition of the image.

Before calculating the perimeter, area and other parameters, the image size calibration method need to be defined, that is, how the extracted feature parameters in the image are converted into the actual size. The calibration result is assumed askμm/pixel, which means the actualkμm size is represented by 1 pixel in the image.

2.3.1 Defect region’s area

Counting the number of pixels in the defect region (including the defect boundary) is adopted to calculate the area of the defect region in this paper. The number of pixels, denoted byP, has been counted when the welding defect is marked previously. The actual area of the defect area, denoted byS, is calculated as follows:

S=P×k2

(8)

wherekis the calibration result.

2.3.2 Defect region’s perimeter

Perimeter is the contour length of the defect region. The defect perimeter is calculated by calculating the number of pixels on defect region’s boundary.

As shown in Fig.2, a sketch map of the discrete welding seam.

BA↑CJ←K→DI↓EHFG

Fig.2Sketchmapofdiscreteweldingseam

Since the welding defect regions have been marked, the number and location of the pixels of each defect region can be determined. The pixels of the welding defect region can be divided into two categories: interior points (such as K) and boundary points (such as A, B, C, …, J). All the pixels of each defect region are judged one by one. If a pixel point’ adjacent upper, lower, left and right points are all black points (the gray value is 0), it can be concluded that it is the interior point of the defect, which can be ignored, otherwise, it is the boundary point, which will be recorded. Count the number of pixels in the record, expressed byQ. Then the actual perimeterLof welding defect region is calculated as follows:

L=Q×k

(9)

2.3.3 Defect region’s circularity

The circularity of the defec tregion is used to describe the extent of the defect region approaching to a circle. Its calculation formula is:

(10)

where,Ris circularity with a range of (0, 1).Sis the defect region’s area.Lis the perimeter of the defect region. The closer the circularity is to 1, the closer the defect region’s shape is to a circle. The closer the circularity is to 0, the closer the defect region’s shape is to strip shape. The circularity of defects is of great significance to distinguish circular defects such as pores and sliver defects such as cracks.

3 Experimental results and analysis

3.1 Experimental results

In order to verify the validity and practicability of the image feature extraction algorithm,a test platform is established. An IPC with Windows 7 as its operating system is used as the main controller of the test platform. The IPC has 4 gigabytes of RAM. The software is developed with VB.net. One welding defect image is selected to test. The experimental results are shown in Fig.3.

Fig.3Processingresultsofweldingdefectsimage(a)Theoriginalweldingimage(b)Weldingimageaftermedianfiltering(c)Weldingimageafterfuzzyenhancement(d)Binarysegmentedimageofweldingimage(e)Weldingimageafterbackgroundremoval(f)Weldingdefectimageaftermarking

Fig.3a is the original welding image. Fig.3b is welding image after median filtering. The filtered image reduces the influence of noise on the image quality. Fig.3c is the welding image after fuzzy enhancement. Image contrast is enhanced and image details are clearer. Fig.3d is the binary segmented image of welding image. The image is divided into two regions: the defect region and the background region. Fig.3e is the welding image after background removal. The black background is removed and the defect region is extracted. Fig.3f is the welding defect image after marking. Every defect region is marked with a number.

3.2 Analysis of experimental results

From the experimental results in Fig.3 and Table 1, it can be seen that the welding defect feature can be extracted effectively by the method mentioned above. And the defect characteristic parameters, such as perimeter, area, circularity, coincide with the actual parameters within a certain range of error. So it is an effective feature extraction method.

Table1Characteristicparametersofweldingdefects

Area/mm2Extraction valueMeasured valueError (%)Perimeter/mmExtraction valueMeasured valueError (%)CircularityExtraction valueMeasured valueError (%)Defect 12.992.777.947.907.239.270.600.65-7.69Defect 22.743.01-8.977.187.90-9.110.670.619.84Defect 30.280.267.691.921.769.090.951.04-8.65Defect 40.410.44-6.822.332.52-7.540.940.878.05Defect 50.150.147.141.501.388.700.850.92-7.61

4 Conclusions

(1) The median filtering and fuzzy image enhancement are ideal ways to preprocess the welding defect image.

(2) The different welding defect regions can be identified through marking the defect regions of the segmented binary image by 8-connected-component labeling method.

(3) The geometric features such as perimeter, area, circularity, can be used to identify different kinds of welding defects such as pore, crack, and the size of the welding defect.