Retrieval of High Resolution Satellite Images Using Texture Features

Samia Bouteldja and Assia Kourgli

1. Introduction

In the past few years, with the development of imaging techniques, satellites with very high spatial resolution imaging systems have been launched, e.g. IKONOS,QuickBird, World-View-1, and GeoEye-1, which are providing more accurate earth observations. The increasing availability of high resolution images (HRSI) has led to the problem of managing the image databases. A content based image retrieval system (CBIR) is a system which based on a query image, retrieves the images from the image database. The query image is an image provided by the user,in which he is interested, and he wants to extract similar images from the image collection. “Content-based” means that the search will analyze the actual contents of the image rather than the metadata such as keywords and tags. In a typical CBIR, the features of the images in the database are extracted and indexed accordingly, as well as the image features of the query image. The system measures the similarity distance between the query image features and those from the database. Relevant images are retrieved from the database based on the best similarity distance.

The choice of indexing features of remote sensing images recives a great attention. These features depend on the type and resolution of the sensor, as well as the task we want to achieve. So many different techniques have been proposed in the literature to retrieve satellite images.

In the case of low-resolution satellite images, color,texture, and intensity cues have been proved to be efficient for recognition[1],[2]. Shape features have been particularly used in weather imagery to determine the type of clouds[3],[4]. With multi-spectral and hyper-spectral low resolution satellite images (Noaa, Landsat), efficient features issued from the combination of channels like the vegetation index (NDVI), brightness index (BI) or urban index (ISU) are used together with color, shape, and texture features[5]. With mid resolution images, textural and shape properties are known to be highly discriminating[6],[7].However, the indexing and retrieval of high resolution satellite images is still a new area of research. Recent work has shown the ability of texture description in segmenting and retrieving HRSI[8],[9], where the texture feature is generally combined with other features. In this paper, we investigate the performance of texture features alone in retrieving high resolution satellite imagery by using the local binary pattern (LBP) operator. The LBP operator is a texture operator which is gaining increasing interest in many areas of image processing and computer vision and has shown its effectiveness in a number of applications.

The paper is organized as follows. Section 2 will briefly describe the LBP operator. The preprocessing chain is detailed in Section 3. The block division scheme is provided in Section 4. Section 5 reports the experimental results and conclusions are drawn in Section 6.

2. LBP Texture Operator

LBP is a simple but a very efficient method for analyzing textures. Due to its discriminative power and its computational simplicity, the LBP texture operator has become a popular approach in various applications. Some of these applications include face recognition[10], biometric recognition[11], iris recognition[12], fingerprint recognition[13],inspection of surfaces[14], image classification[15], image segmentation[7], and image retrieval[16],[17]. The most important property of the LBP operator in real-world applications is its robustness to monotonic gray-scale changes, for example the changes caused by illumination variations. Another important property is its computational simplicity.

The LBP operator was first introduced in 1994 by Ojala et al.[18]. Originally, it operated with 3×3 rectangular neighborhood using the center pixel as a threshold. The LBP code associated with the central pixel was calculated by multiplying thresholded values by weights given by the power of two and adding the results as depicted by (1):

where and gcis the gray value of the central pixel, gpis the value of its neighbor, P is the number of neighbors, and R is the radius of the neighborhood. For a digital image, the coordinates of the neighbors gpare given by

where (xc, yc) are the coordinates of the center pixel. The values of neighbors that do not fall exactly on the pixel grid of the image are estimated by bilinear interpolation. Thus LBP8,1is approximately equivalent to 3×3 LBP. For an image of dimension M×N, and after computing the LBP code for each pixel (i, j), the LBP histogram can be built by using

where

and K is the maximal LBP pattern value. The K bin histogram of the obtained LBP image is then used as a texture descriptor for further image analysis. Each bin of the LBP histogram codifies the occurrence of a given texture primitive presented in the image. Texture primitives include different types of edges, spots, flat areas, line ends,corners, etc.

The LBP8,1operator produces 256 (28) different codes.When the image is rotated, the gray values gpwill correspondingly move along the perimeter of the circle around gc. Rotating a particular binary pattern naturally results in a different LBP value. This does not apply to patterns 00000000 and 11111111 which remain constant at all rotation angles. The LBP operator can be made rotation invariant by bitwise rotating the code so that the bits represent the minimum integer value, referring to (4):

where ROR(LBP, i) performs a circular bit-wise right shift on the LBP code i times until the code represents its minimum integer value. For the LBP8,1operator we have only 36 unique rotation invariant local binary patterns.With larger neighborhoods, the number of possible LBP codes increases exponentially. The increase in the number of codes by considering only a subset of the codes is called uniform patterns. Uniform patterns are LBPs that have a binary description with no more than two binary level transitions. For example, we can consider only those LBP codes where U=2 (U refers to the measure of uniformity,that is the number of 0/1 and 1/0 transitions in the circular binary code pattern).

It has been observed that uniform patterns are presented a lot more frequently than non-uniform patterns. Giving more weight to uniform patterns provides better discrimination of textures. This can be done by grouping all non-uniform patterns into a single bin of the distribution. In the case of a 3×3 operator () we get 58 codes instead of 256 codes with the remaining patterns accumulated to a single code. The image histogram becomes containing 59 bins. The histogram generated by an LBP operator can then be reduced without a significant loss in its discrimination capability. Uniform patterns describe fundamental features such as bright spots, flat area, dark spots, and edges. For example, when P=8, there are 9 uniform patterns out of 36 unique rotation invariant patterns.

Texture is distinguished not only by texture patterns,but also by another important property which is its contrast.However, the LBP operator by itself totally ignores the magnitude of gray level differences. In [18], Ojala et al.used the joint distribution of LBP codes and a local contrast measure VARP,Ras a texture descriptor. The local contrast is invariant against shifts in the gray scale and is rotation invariant. It is measured in a circularly neighborhood just like the LBP:

where

The variance VARP,Rhas continuous values and has to be quantized. In this work instead of computing the joint histogram of LBPP,Rand VARP,R, we use the LBP variance(LBPV) histogram proposed in [15]. The LBPV histogram is a simplified but efficient joint histogram computed by accumulating the variance measures of the neighborhood to the LBP bin. Simple LBP histogram calculation assigns the same weight 1 to each LBP pattern. Usually the high textured images will have higher variances and contribute more to the discrimination of texture images. Therefore,VARP,Rcan be used as an adaptive weight to adjust the contribution of the LBP code. Compared to the 2-dimensional joint histogram of LBPP,RVARP,R, the LBPV operator can reduce greatly the computational time.The LBPV histogram is computed by (6), where K is the maximal LBP pattern value:

3. Preprocessing

An RGB color image is an array of color pixels composed of triplets corresponding to the red, green, and blue components (RGB components). To keep only texture information within the images, the 3-dimensional RGB satellite images are converted into one-dimensional gray scale images by

A drawback of the LBP operator is that a small change in the input image would cause a change in the output. LBP may not work properly for noisy images. This is due to the thresholding scheme of the operator. In order to make the LBP more robust against these negligible changes in pixel values, we propose to reduce the color depth of the image.A series of experiments were conducted by using different quantification values (q=128, q=64, q=32, and q=16 colors),and the quantification was applied to the database images as well as the query image.

4. Block Based CBIR

The texture features for most of the current CBIR systems are calculated for full images even though texture appears usually only in some parts of the image. The full image approach is well justified as it generally keeps the size of the feature database reasonably low. However,extracting the feature vector for the image globally will result in an incorrect representation for the textures within the image, and hence will decrease the retrieval efficiency.In order to pay attention to local properties, the local features from the texture regions should be extracted instead of using global features. This can be achieved by using either image segmentation or image subdivision.Image segmentation is usually prone to errors as it is difficult to obtain an accurate segmentation, especially around the boundary of the textured region, which can later affect the retrieval result. The other way is to divide the image into areas without using any type of segmentation.

where B represents the total number of bins in the histograms, and it is related to the values of parameters of P and R used. For example (P, R)=(8, 1), B=10, for (P,R)=(16, 2), B=18, and for (P, R)=(24, 3), B=26. The query and database images are compared by calculating the Chi-square distance between the ith query and the jth database block histograms. The final image similarity distance D for retrieval is the sum of minimum distances depicted by (9):

In this paper the second approach is used. Each 512×512 pixels database image is divided into 16 non overlapping fixed size blocks. The LBP histogram is calculated for each block. In the query phase the same is done for the query image.

In the similarity measure, a distance measure is needed for comparing images through their LBP features. There are many different dissimilarity measures to choose from. In this study, the Chi-square distance is chosen to evaluate the similarity between two LBPV histograms H1and H2, due to its performance in terms of both speed and good retrieval rates, referring to (8):

5. Experiments and Discussion

In the experiments, we computed L BPwith (P, R)values of (8, 1), (16, 2), and (24, 3), respectively. The image retrieval system was implemented by using the interactive data language (IDL) which is an image processing language. Some visual results are shown in Fig.1 (a), (b), and (c), where the query image is in the first position. The results show that different kinds of textured and structured images can be correctly retrieved. The effectiveness of a CBIR system can be specified by several objective measures. A standard measure of performance named recall (Re) is defined as

In this section, we illustrate the recognition capability of our retrieval scheme for HRSI.

5.1 Testing Database

The experiments were carried out by considering the database used by Xia et al.[19]. The database has been extracted from Google Earth and is available at the author webpage. It includes 12 categories each containing 50 samples, and it has been selected so that it includes the low textured, highly textured, low structured, and highly structured regions with different illuminations and different spatial resolutions. We resized the images to 512×512 pixels in order to have 16 blocks of 128×128 pixels each.

5.2 Retrieval Results

where N is the number of retrieved relevant objects and Mt is the number of the total relevant objects.

Fig. 1. Visual results: (a) for pond category, (b) for viaduct category, and (c) for forest category.

To plot the recall curve, all the 600 images in the database were used as query images. An average recall was then computed to the number of retrieval images. In Fig. 2,the curves reveal that for the first 200 matches more than 70% of the images are correctly retrieved. By comparing these results with those using features extracted by the Gabor filter and reported in [19], the performance of our system for retrieving HRSI images outperforms considerably that using the Gabor filter by about 13.62%.In addition, the LBP feature extraction algorithm complexity is much simpler than that of the Gabor technique.

In order to evaluate the effect of P, R parameters on the retrieval results, the average recall has been computed for the whole database by using different values of P, R parameters: (8, 1), (16, 2), and (24, 3). (See Fig. 3).

We have noticed that the performance of (8, 1)neighborhood with its 8 bins histogram is poorer than that of the (16, 2) one, because it is too small to capture the texture patterns present in the image. The (24, 3)neighborhood with its 26 bins histogram improves slightly the discrimination capability. The (16, 2) neighborhood seems to be more adequate in terms of the feature length and performance. In addition, through the experimental analysis we have noticed that dividing the image into 16 blocks increases considerably computing time for large neighborhoods, e.g. (24, 3), and therefore decreasing the retrieval speed. This can be avoided by giving the opportunity to users to select only some significant parts within the image.

Different color depths also have been considered, such as {q=128, 64, 32, 16}. By comparing the recall curves without using quantification with those using the quantification of q=128, q=64, q=32, q=16 colors, we found that the performance improves slightly for using the quantification of 16 colors or greater than 16 colors. Fig. 4 shows the effect of 16 colors quantification on the retrieval results.

Fig. 2. Retrieval performance.

Fig. 3. Effect of (P, R) parameters on the retrieval results.

Fig.4. Effect of the quantification on the retrieval results.

6. Conclusions

In this paper, we have developed a CBIR systme for indexing high-resolution satellite images based on LBP and a block division scheme. The experimental results show that the LBP texture features in HRSI retrieval, although the HRSI are compacted, can provide good image retrieval performance. As future work and by considering the fact that HRSI contains both structures and textures, most likely the inclusion of some structural features would lead to higher performances.

[1] P. Maheswary and N. Srivastava, “Retrieval of remote sensing images using colour&texture attribute,” Int. Journal of Computer Science and Information Security, vol. 4, no. 1& 2, pp. 1-5, 2009.

[2] R. Tebourbi and Z. Belhadj, “A texture based multispectral images indexing,” in Proc. of the 12th IEEE Int. Conf. on Electronics, Circuits, and Systems, Gammarth, 2005, pp.1-4.

[3] A. L. Ma, “Indexing and retrieval of satellite images,” M.S.thesis, Oakland University, Rochester, 2005.

[4] D. Upreti, “Content-based satellite cloud image retrieval,”M.S. thesis, University of Delhi, New Delhi, 2011.

[5] P. Mmaheshwary and N. Srivastava, “Retrieval of remote sensing images using color, texture and spectral features,”Int. Journal of Engineering Science and Technology, vol. 2,no. 9, pp. 4306-4311, 2010.

[6] H. Maître, “Indexing and retrieval in large satellite image databases, ” Proc. of SPIE, doi: 10.1117/12.775018.

[7] A. Bhattacharya, M. Roux, H. Maître, I. Jermyn, X.Descombes, and J. Zerubia, “Indexing of mid-resolution satellite images with structural attributes,” in Proc. of Int.Society for Photogrammetry and Remote Sensing Congress,Beijing, 2008, pp. 187-192.

[8] S. Wang and A. Wang, “Segmentation of high-resolution satellite imagery based on feature combination,” in Proc. of Int. Society for Photogrammetry and Remote Sensing Congress, Beijing, 2008, doi: 10.1.1.158.6997.

[9] Q.-M. Wan, M. Wang, X.-Y. Zhang, and D.-Q. Zhang,“Two-stage high resolution remote sensing image retrieval combining semantic and visual features,” Proc. of SPIE, doi:10.1117/12.832727.

[10] C.-Y. Jo. Face detection using LBP features. [Online].Available: http://cs229.stanford.edu/proj2008/Jo-Face DetectionUsingLBPfeatures.pdf.

[11] K. Saxena and P. Misha, “Human gait recognition using local binary pattern variance,” Int. Journal of Advanced Engineering Sciences and Technologies, vol. 7, no. 2, pp.234-238, 2011.

[12] M. Z. Rashad, M. Y. Shams, O. Nomir, and R. M. El-Awady,“Iris recognition based on LBP and combined LVQ classifier,” Int. Journal of Computer Science and Information Technology, vol. 3, no. 5, pp. 6–7, 2011.

[13] L. Nanni and A. Lumini, “Local binary patterns for a hybrid fingerprint matcher,” Pattern Recognition, vol. 41, no. 11,pp. 3461–3466, 2008.

[14] M. Pietikainen, T. Nurmela, T. Maenpaa, and M. Turtinen,“View-based recognition of real-world textures,” Pattern Recognition, vol. 37, no. 2, pp. 313–323, 2004.

[15] Z.-H. Guo, L. Zhang, and D. Zhang, “Rotation invariant texture classification using LBP variance (LBPV) with global matching,” Pattern Recognition, vol. 43, no. 3, pp.706-719, 2010.

[16] D. Connah and G. D. Finlayson, “Using local binary pattern operators for colour constant image indexing,” in Proc. of the 3rd European Conf. on Colour in Graphics, Imaging and Vision, Leeds, 2006, pp. 1-5.

[17] V. Takala, T. Ahonen, and M. Pietikäinen, “Block-based methods for image retrieval using local binary patterns,”Image Analysis LNCS, doi: 10.1007/11499145_89.

[18] T. Ojala, M. Pietikäinen, and D. Harwood, “Performance evaluation of texture measures with classification based on Kullback discrimination of distributions,” in Proc. of the 12th IAPR Int. Conf. on Pattern Recognition,Jerusalem,1994, pp. 582-585.

[19] G.-S. Xia, W. Yang, J. Delon, Y. Gousseau, H. Sun, and H.Maître, “Structural high-resolution satellite image indexing,”in Prof. of Int. Society for Photogrammetry and Remote Sensing Congress, Vienna, doi: 10.1.1.400.9110.