Texture Feature Fusion for High Resolution Satellite Image Classification
Texture Feature Fusion for High Resolution Satellite Image Classification Yindi Zhao, Liangpei Zhang, Pingxiang Li LIESMARS, Wuhan University, Wuhan 430079, China [email protected], [email protected], [email protected] Abstract Multi-channel Gabor filters (MCGF) and Markov random fields (MRF) have been demonstrated to be quite effective for texture analysis. In this paper, MCGF and MRF features are respectively extracted from input texture images by means of the two above techniques. A MCGF/MRF feature fusion algorithm for texture classification is proposed. The fused MCGF/MRF features achieved by this novel algorithm have much higher discrimination than either the pure features or the combined features without selection, according to the Fisher criterion and classification accuracy. The stability and effectiveness of the proposed algorithm are verified on samples of Brodatz and QuickBird images.
1. Introduction With the advent of high resolution satellite images, such as QuickBird, texture analysis has received great attention in image classification. Texture reflects the local variability of grey levels in the spatial domain and reveals the information about the object structures in the natural environment. Many approaches of extracting texture features have been proposed over recent years including spatial frequency based techniques such as multi-channel Gabor filters (MCGF) [1], stochastic models such as Markov random fields (MRF) [2] , and statistical analysis methods. MCGF can obtain multi-scale texture information corresponding to different scales and orientations. MRF can capture the local spatial textural information in an image assuming that the intensity in an image depends on the intensities of only the neighboring pixels. Features derived from the two above methods are quite different in nature and have low inter-feature correlations [3]. The combined MCGF and MRF features are expected to provide richer texture information than either MCGF or MRF features alone.
Proceedings of the Computer Graphics, Imaging and Vision: New Trends (CGIV’05) 0-7695-2392-7/05 $20.00 © 2005 IEEE
However, the combined features without selection, which give more dimensions, may affect the performance of designed classifiers and result in even worse classification accuracy than the pure features. In this paper, a MCGF/MRF feature fusion algorithm for texture classification is introduced in order to guarantee improved classification results. The performance of the fused MCGF/MRF features achieved by the novel algorithm is compared with some other types of features: the pure features (MCGF or MRF features alone) and the combined MCGF and MRF features without optimization. The rest of the paper is organized as follows. In Section 2, MCGF and MRF texture features are presented. In Section 3, the MCGF/MRF feature fusion algorithm for texture classification is proposed. Comparative experiments are given in Section 4 and conclusions drawn in Section 5.
2. Texture feature extraction 2.1. Multi-channel Gabor filters (MCGF) Due to its appealing simplicity and optimum joint spatial/spatial-frequency localization, multi-channel Gabor filters (MCGF) are attractive for texture analysis. The Gabor function takes the form of a 2-D Gaussian modulated complex sinusoidal grating in the spatial domain [1], given by h( x, y ) g ( x c, y c) exp[2S (Ux Vy )] , (1) where ( x c, y c) ( x cos T y sin T , x sin T y cos T ) is the rotated spatial-domain rectilinear coordinate, (U , V ) defines the position of the filter in the frequency domain with a center frequency of F U 2 V 2 and an orientation of I arctan(V U ) . The function g ( x, y ) is the 2-D Gaussian: 2 º½ 2 ª 1 ° 1 «§ x · §¨ y ·¸ » ° ¨ ¸ g ( x, y ) , (2) exp® ¨ ¸ ¨ ¸ ¾ 2SV xV y ° 2 «¬© V x ¹ © V y ¹ »¼ ° ¯ ¿
where V x , V y characterize the spread in the x and y directions, respectively. Let T I 1 , then we can create multiple channel filters to cover the spatial frequency space by tuning the parameters ( F , T , V x , V y ) . In order to maximize
coverage of the frequency domain while minimizing the overlap between filters, another two important aspects of MCGF are the frequency bandwidth B F (measured in octave) and the orientation bandwidth BT (measured in radian), respectively defined by
BF
§ SFV x ln 2 2 · ¸, log 2 ¨ ¨ SFV ln 2 2 ¸ x © ¹
BT
2 arctan
(4)
and ln 2 2
SV y F
.
(5)
The frequency and orientation bandwidth B F , BT can be set to constant values that match psychovisual data [1] [4]. In particular, unit octave frequency bandwidth is found to perform well, and an orientation bandwidth of 30 degree is preferred to 45 degree [4]. The filtered image ih ( x, y ) can be expressed as the convolution of the input image i ( x, y ) with the filter response h( x, y ) : i h ( x , y ) i ( x , y ) h ( x, y ) , (6) where denotes the application of convolution. However, Filter outputs ih ( x, y ) by default are not appropriate for identifying key texture features [5]. Each filtered image should be subjected to a nonlinearity transformation with the following bounded linearity: 1 e 2Dx , (7) < ( x) tanh(Dx) 1 e 2Dx where D 0.25 is an empirical constant. Then we simply compute the average absolute deviation from the mean in small overlapping windows as texture measures. Let W xy be the window of size M x u M y centered at the pixel with the coordinate ( x, y ) , and the feature image f h ( x, y ) corresponding to the filtered image ih ( x, y ) is given by 1 f h ( x, y ) \ (ih ( x, y )) . (8) M x u M y (i , j )W
¦
xy
2.2 Markov random field (MRF) model Markov Random Fields (MRF) can specify the local dependence of image regions by defining a neighborhood system on the pixels of an image and a probability density function on the spectrum distribution of the pixels. When this distribution is Gaussian, the model is called Gaussian Markov random field (GMRF) model. Let i ( s ) represent the gray level intensity of a pixel s in a texture region R , GMRF for modeling the texture region is defined by the following conditional probability density function: 1 1 ½ p i ( s ) | R exp® [e( s )] 2 ¾ , (9) ¯ 2Q ¿ 2SQ where e( s ) is a zero-mean Gaussian noise with the variance of Q . The spatial interactions of the pixel s are given by i(s) u T (r ) u i ( s r ) u e( s) , (10)
¦
rK k
where u is the mean of variables i (s ) , T (r ) s are the model parameters and the subscribed K k set represents a k -order neighborhood system. Since the power spectrum associated with Equation 10 must be real and positive [6], we should have r K k r K k and T (r ) T (r ) . Let K k denote an asymmetric neighbor set, the relationship between K k and K k is K k ^r : r K k ` ^ r : r K k ` . (11) Therefore, Equation 10 can be rewritten as follows: i(s) u T ( r ) q r ( s ) e( s ) , (12)
¦
rK k
where q r ( s ) i ( s r ) u i ( s r ) u . There are many existing methods for estimating the GMRF parameters, but none of them can guarantee both consistency and stability together [7]. The choice of the least squares method here is motivated by this simplicity-stability tradeoff. Then the least square estimates of the unknown parameters are 1
șˆ
where șˆ
º ª 7 « q( s )q ( s )» »¼ «¬ sR col[Tˆ(r ) | r K ] ,
¦
k
1
Proceedings of the Computer Graphics, Imaging and Vision: New Trends (CGIV’05) 0-7695-2392-7/05 $20.00 © 2005 IEEE
¦
the hat over a quantity
indicates an estimate of that quantity, R is an interior of
the image, and q( s ) col[q r ( s ) | r K k ] . The estimate Qˆ of the noise variance is calculated by 2 1 Qˆ i(s) u șˆ 7 q( s) , (14) M R sR
¦>
It is usually convenient to consider filters whose modulating Gaussians have the same orientation as the complex sin grating.
º ª « q( s ) i ( s ) u » , (13) »¼ «¬ sR
@
where M R denotes the number of elements in R .
According to the GMRF model parameters Tˆ(r ) s, a new different set of texture features are derived by 1 i(s) u Tˆ(r )q r (s) . (15) f (r ) M R sR
¦>
@
f (r ) s have been proved to be more discriminatory than Tˆ(r ) s [2]. Hereby f (r ) s and the variance Qˆ are
employed as MRF features.
3. Feature fusion and classification The combined MCGF and MRF features can capture richer texture features than either MCGF or MRF features alone [3]. However, if the pooled set of MCGF and MRF features without selection were used in classification, both computational performance and classification accuracy would have been poor. Since the classification time increases linearly with the number of features, it is advantageous to discard “useless” features. Furthermore, discarding redundant information often enhances classification accuracy. From this point, a novel MCGF/MRF feature fusion algorithm for texture classification is proposed in order to guarantee the anticipated classification improvement. The strategy of feature fusion is first to combine various features and then perform feature selection to choose an optimal feature subset. Here the sequential floating forward search (SFFS) method is used in view of its good performance in both the quality of obtained feature subset and computation efficiency. SFFS, which can be understood as plus 1 minus x and minus 1 plus x , where x is dynamically changed according to the backward effect, can prevent the “nesting” effect in the sequential forward selection (SFS) method, and avoid the problem the problem of predefining l and r in the plus l / take away r (PTA) method. For more details of SFFS we refer to [8]. The classification process is based on the widely used multivariate Gaussian Bayes classifier [9]. The Kappa coefficient ( N ) is used as the criterion of feature selection. And the minimum and maximum Fisher criterion ( J ), which can express the separation ability between classes in the selected feature space, are recorded simultaneously. Below is a detailed description of our MCGF/MRF feature fusion system for texture classification. Firstly, texture features are respectively extracted from the input image by means of MCGF and MRF, according to Section 2. Then the correlation relationship between the two above different types of features is investigated. A low correlation coefficient (e.g., closer to zero), which suggests that the relationship between MCGF and
Proceedings of the Computer Graphics, Imaging and Vision: New Trends (CGIV’05) 0-7695-2392-7/05 $20.00 © 2005 IEEE
MRF features is weak or non-existent, shows the potential producing the higher classification accuracy by MCGF/MRF fused features. Secondly, feature normalization complies with a rule that each feature component should be treated equally for its contribution to the designed classifier. Thirdly, the SFFS method is used to select an optimal feature subset after the normalized MCGF and MRF features are pooled together, and the Kappa coefficient ( N ) is used as the criterion of feature selection. Thus the fused MCGF/MRF features are obtained, which can obtain satisfactory classification results.
4. Experimental results In this section, the experiments performed on Brodatz and QuickBird texture images are presented. For MCGF, the frequency bandwidth is set to unit octave, the orientation bandwidth 30 degree. The center frequency spacing is set to the frequency bandwidth, and the orientation spacing is equal to the orientation bandwidth. Just the three highest center frequencies are used because low frequencies are unhelpful for texture discrimination. And only half part [0, S ] is considered due to the symmetry of Fourier spectrum. The application of such a filter bank to an input image results in an 18-dimensional MCGF feature vector for each pixel of that image. For MRF, the fourth-order is used [10], yielding an 11dimensional MRF feature vector for each image pixel. To test the effectiveness of the fused MCGF/MRF features achieved by the proposed algorithm, comparative tests are conducted by using the following other types of features: the pure features (MCGF or MRF features alone) with selection or not and the combined MCGF and MRF features without selection. Performance evaluation is measured by Kappa coefficients ( N ) and overall accuracies.
4.1. Experiment on Brodatz textures Figure 1(a) shows the 256 u 256 image containing five different Brodatz textures. Its corresponding ground truth label map is shown in Figure 1(b), and labels 1 to 5 respectively correspond to D54 (beach pebble), D36 (lizard skin), D55 (straw matting), D9 (grass lawn), and D24 (pressed calf leather). The correlation coefficient between the MCGF and MRF features extracted from Figure 1(a) is 0.0739. The low correlation coefficient indicates that MRF features are not well correlated with MCGF features, and the combined features are expected to offer more texture information than the pure features. Figure 1(c)
2
1 5
4
3
(c) (d) (a) (b) Figure 1. Brodatz textures for experiment: (a) input image, (b) ground truth, (c) the classification result with the combined CMGF and MRF features without selection, (d) the classification result with the fused CMGF/MRF features.
displays the classification result with the combined MCGF and MRF features without feature selection, and its Kappa coefficient is 0.7972 with an overall accuracy of 83.78%. Figure 1(d) shows the classification result using the fused MCGF/MRF features. Using the proposed MCGF/MRF feature fusion algorithm, the Kappa coefficient goes up from 0.7972 to 0.9078, with the overall accuracy from 83.78% to 92.62%. The comparison results for the composite Brodatz texture images are summarized in Table 1. Without feature selection, the combined MCGF and MRF
features don’t bring any significant change; on the contrary, the Kappa coefficient actually drops, ranging from 0.8023 for MRF features to 0.7972. From Table 1, two important remarks are made as follows. For one thing, the features involving feature selection obtain superior classification performance to those without feature selection. For another, the fused MCGF/MRF features achieved by the MCGF/MRF feature fusion algorithm can produce better classification result, compared with the pure features (MCGF or MRF features alone) with selection or not and the combined features without selection.
Table 1. Comparison of Brodatz texture image classification results: NF-number of features, N -Kappa coefficient, OA-overall accuracy, J max and J min -maximum and minimum Fisher criterion.
Feature type
Without Feature selection
Using feature selection
NF
N
OA
NF
N
OA
J max
J min
MCGF features
18
0.6876
75.01%
4
0.7973
83.79%
40.04
4.28
MRF features
11
0.8023
84.19%
5
0.8514
88.11%
111.07
4.37
MCGF and MRF features
29
0.7972
83.78%
9
0.9078
92.62%
125.95
12.60
4.2. Experiment on QuickBird image Figure 2(a) is a patchwork of uniform region samples from the QuickBird image of a suburban area in Beijing, China (2002), consisting of three different textures. Figure 2(b) displays the gray level coded ground truth of Figure 2(a), and label 1 to 3 respectively denote bare soil, green fields, and clumpy trees. The correlation coefficient between the MCGF and MRF features extracted from Figure 2(a) is 0.1387, which indicates that there is strong complementary between the two different features. And the integrated
Proceedings of the Computer Graphics, Imaging and Vision: New Trends (CGIV’05) 0-7695-2392-7/05 $20.00 © 2005 IEEE
utilization of MCGF and MRF features is anticipated to provide a much more satisfactory result than the pure features. Figure 2(c) is the classified image using the combined MCGF and MRF features without selection. Its corresponding Kappa coefficient is 0.7876 with an overall accuracy of 87.16%. Figure 2(d) displays the classification result using the fused MCGF/MRF features, and the Kappa coefficient increases to 0.8924 with the overall accuracy 93.44%. Experemental results on Figure 2(a) are listed in Table 2. As we can see from Table 2, the features with selection outperform those without selection and the fused MCGF/MRF features perform best.
2
1
3 (a)
(c)
(b)
(d)
Figure 2. QuickBird image for experiment: (a) input image, (b) ground truth, (c) the classification
result with the combined CMGF and MRF features without selection, (d) the classification result with the fused CMGF/MRF features. Table 2. Comparison of QuickBird image classification results.
Feature type
Without Feature selection
Using feature selection
NF
N
OA
NF
N
OA
J max
J min
MCGF features
18
0.6711
80.58%
8
0.8022
88.21%
47.41
27.22
MRF features
11
0.7918
87.47%
4
0.8758
92.42%
93.15
23.69
MCGF and MRF features
29
0.7876
87.16%
9
0.8924
93.44%
561.52
21.10
5. Conclusion This paper presents a MCGF/MRF feature fusion algorithm for texture classification. The performance of this algorithm is investigated with Brodatz and QuickBird images. The fused MCGF/MRF features can provide higher classification accuracy, compared with the pure features (MCGF or MRF features alone) and the combined MCGF and MRF features without selection. The experimental results indicate that the proposed algorithm is stable, reliable and efficient to improve texture classification results.
Acknowledgements This work was supported by the 973 Project of the People’s Republic of China (Project Number 2003CB415205), and the National Natural Science Foundation of China (Project Number 40471088).
References [1] A.C. Bovik, M. Clark, and W.S. Geisler, “Multichannel Texture Analysis Using Localized Spatial Filters”, IEEE Transactions on Pattern Analysis and Machine Intelligence 12(1), 1990, pp. 55-73.
Proceedings of the Computer Graphics, Imaging and Vision: New Trends (CGIV’05) 0-7695-2392-7/05 $20.00 © 2005 IEEE
[2] E. Cesmeli and D.L. Wang, “Texture Segmentation Using Gaussian-Markov Random Fields and Neural Oscillator Networks”, IEEE Transactions on Neural Network, 12(2), 2001, pp. 394-404. [3] D.A. Clausi, “Comparison and Fusion of Co-occurrence, Gabor, and MRF Texture Features for Classification of SAR Sea Ice Imagery”, Atmosphere & Oceans, 39(4), 2001, pp. 183-194. [4] D.A. Clausi and M.Ed Jernigan, “Designing Gabor Filters for Optimal Texture Separability”, Pattern Recognition, 33(11), 2000, pp. 1835-1849. [5] A.K. Jain and F. Farrokhnia, “Unsupervised Texture Segmentation Using Gabor Filters”, Pattern Recognition, 24(12), 1991, pp. 1167-1186. [6] G. Sharma and R. Chellappa, “A Model Based Approach for the Estimation of 2-D Maximum Entropy Power Spectra”, IEEE Transaction on Information Theory, 31(1), 1985, pp. 90-99. [7] BS Manjunath and R. Chellappa, “Unsupervised Texture Segmentation Using Markov Random Field Models”, IEEE Trans. Pattern Analysis and Machine Intelligence, 13(5), 1991, pp. 478-482. [8] P. Pudil, J. Novovicova and J. Kittler, “Floating Search Methods in Feature Selection”, Pattern Recognition Letters, 15 (11), 1994, pp. 1119-1125. [9] Richard O. Duda, Peter E. Hart and David G. Stork, Pattern Recognition(Second Edition), New York :Wiley,2001. [10] R. Kashyap and R. Chellappa. “Estimation and Choice of Neighbors in Spatial Interaction Models of Images”, IEEE Transaction on Information Theory, 29(1), 1983, pp. 60-72.
1. Introduction With the advent of high resolution satellite images, such as QuickBird, texture analysis has received great attention in image classification. Texture reflects the local variability of grey levels in the spatial domain and reveals the information about the object structures in the natural environment. Many approaches of extracting texture features have been proposed over recent years including spatial frequency based techniques such as multi-channel Gabor filters (MCGF) [1], stochastic models such as Markov random fields (MRF) [2] , and statistical analysis methods. MCGF can obtain multi-scale texture information corresponding to different scales and orientations. MRF can capture the local spatial textural information in an image assuming that the intensity in an image depends on the intensities of only the neighboring pixels. Features derived from the two above methods are quite different in nature and have low inter-feature correlations [3]. The combined MCGF and MRF features are expected to provide richer texture information than either MCGF or MRF features alone.
Proceedings of the Computer Graphics, Imaging and Vision: New Trends (CGIV’05) 0-7695-2392-7/05 $20.00 © 2005 IEEE
However, the combined features without selection, which give more dimensions, may affect the performance of designed classifiers and result in even worse classification accuracy than the pure features. In this paper, a MCGF/MRF feature fusion algorithm for texture classification is introduced in order to guarantee improved classification results. The performance of the fused MCGF/MRF features achieved by the novel algorithm is compared with some other types of features: the pure features (MCGF or MRF features alone) and the combined MCGF and MRF features without optimization. The rest of the paper is organized as follows. In Section 2, MCGF and MRF texture features are presented. In Section 3, the MCGF/MRF feature fusion algorithm for texture classification is proposed. Comparative experiments are given in Section 4 and conclusions drawn in Section 5.
2. Texture feature extraction 2.1. Multi-channel Gabor filters (MCGF) Due to its appealing simplicity and optimum joint spatial/spatial-frequency localization, multi-channel Gabor filters (MCGF) are attractive for texture analysis. The Gabor function takes the form of a 2-D Gaussian modulated complex sinusoidal grating in the spatial domain [1], given by h( x, y ) g ( x c, y c) exp[2S (Ux Vy )] , (1) where ( x c, y c) ( x cos T y sin T , x sin T y cos T ) is the rotated spatial-domain rectilinear coordinate, (U , V ) defines the position of the filter in the frequency domain with a center frequency of F U 2 V 2 and an orientation of I arctan(V U ) . The function g ( x, y ) is the 2-D Gaussian: 2 º½ 2 ª 1 ° 1 «§ x · §¨ y ·¸ » ° ¨ ¸ g ( x, y ) , (2) exp® ¨ ¸ ¨ ¸ ¾ 2SV xV y ° 2 «¬© V x ¹ © V y ¹ »¼ ° ¯ ¿
where V x , V y characterize the spread in the x and y directions, respectively. Let T I 1 , then we can create multiple channel filters to cover the spatial frequency space by tuning the parameters ( F , T , V x , V y ) . In order to maximize
coverage of the frequency domain while minimizing the overlap between filters, another two important aspects of MCGF are the frequency bandwidth B F (measured in octave) and the orientation bandwidth BT (measured in radian), respectively defined by
BF
§ SFV x ln 2 2 · ¸, log 2 ¨ ¨ SFV ln 2 2 ¸ x © ¹
BT
2 arctan
(4)
and ln 2 2
SV y F
.
(5)
The frequency and orientation bandwidth B F , BT can be set to constant values that match psychovisual data [1] [4]. In particular, unit octave frequency bandwidth is found to perform well, and an orientation bandwidth of 30 degree is preferred to 45 degree [4]. The filtered image ih ( x, y ) can be expressed as the convolution of the input image i ( x, y ) with the filter response h( x, y ) : i h ( x , y ) i ( x , y ) h ( x, y ) , (6) where denotes the application of convolution. However, Filter outputs ih ( x, y ) by default are not appropriate for identifying key texture features [5]. Each filtered image should be subjected to a nonlinearity transformation with the following bounded linearity: 1 e 2Dx , (7) < ( x) tanh(Dx) 1 e 2Dx where D 0.25 is an empirical constant. Then we simply compute the average absolute deviation from the mean in small overlapping windows as texture measures. Let W xy be the window of size M x u M y centered at the pixel with the coordinate ( x, y ) , and the feature image f h ( x, y ) corresponding to the filtered image ih ( x, y ) is given by 1 f h ( x, y ) \ (ih ( x, y )) . (8) M x u M y (i , j )W
¦
xy
2.2 Markov random field (MRF) model Markov Random Fields (MRF) can specify the local dependence of image regions by defining a neighborhood system on the pixels of an image and a probability density function on the spectrum distribution of the pixels. When this distribution is Gaussian, the model is called Gaussian Markov random field (GMRF) model. Let i ( s ) represent the gray level intensity of a pixel s in a texture region R , GMRF for modeling the texture region is defined by the following conditional probability density function: 1 1 ½ p i ( s ) | R exp® [e( s )] 2 ¾ , (9) ¯ 2Q ¿ 2SQ where e( s ) is a zero-mean Gaussian noise with the variance of Q . The spatial interactions of the pixel s are given by i(s) u T (r ) u i ( s r ) u e( s) , (10)
¦
rK k
where u is the mean of variables i (s ) , T (r ) s are the model parameters and the subscribed K k set represents a k -order neighborhood system. Since the power spectrum associated with Equation 10 must be real and positive [6], we should have r K k r K k and T (r ) T (r ) . Let K k denote an asymmetric neighbor set, the relationship between K k and K k is K k ^r : r K k ` ^ r : r K k ` . (11) Therefore, Equation 10 can be rewritten as follows: i(s) u T ( r ) q r ( s ) e( s ) , (12)
¦
rK k
where q r ( s ) i ( s r ) u i ( s r ) u . There are many existing methods for estimating the GMRF parameters, but none of them can guarantee both consistency and stability together [7]. The choice of the least squares method here is motivated by this simplicity-stability tradeoff. Then the least square estimates of the unknown parameters are 1
șˆ
where șˆ
º ª 7 « q( s )q ( s )» »¼ «¬ sR col[Tˆ(r ) | r K ] ,
¦
k
1
Proceedings of the Computer Graphics, Imaging and Vision: New Trends (CGIV’05) 0-7695-2392-7/05 $20.00 © 2005 IEEE
¦
the hat over a quantity
indicates an estimate of that quantity, R is an interior of
the image, and q( s ) col[q r ( s ) | r K k ] . The estimate Qˆ of the noise variance is calculated by 2 1 Qˆ i(s) u șˆ 7 q( s) , (14) M R sR
¦>
It is usually convenient to consider filters whose modulating Gaussians have the same orientation as the complex sin grating.
º ª « q( s ) i ( s ) u » , (13) »¼ «¬ sR
@
where M R denotes the number of elements in R .
According to the GMRF model parameters Tˆ(r ) s, a new different set of texture features are derived by 1 i(s) u Tˆ(r )q r (s) . (15) f (r ) M R sR
¦>
@
f (r ) s have been proved to be more discriminatory than Tˆ(r ) s [2]. Hereby f (r ) s and the variance Qˆ are
employed as MRF features.
3. Feature fusion and classification The combined MCGF and MRF features can capture richer texture features than either MCGF or MRF features alone [3]. However, if the pooled set of MCGF and MRF features without selection were used in classification, both computational performance and classification accuracy would have been poor. Since the classification time increases linearly with the number of features, it is advantageous to discard “useless” features. Furthermore, discarding redundant information often enhances classification accuracy. From this point, a novel MCGF/MRF feature fusion algorithm for texture classification is proposed in order to guarantee the anticipated classification improvement. The strategy of feature fusion is first to combine various features and then perform feature selection to choose an optimal feature subset. Here the sequential floating forward search (SFFS) method is used in view of its good performance in both the quality of obtained feature subset and computation efficiency. SFFS, which can be understood as plus 1 minus x and minus 1 plus x , where x is dynamically changed according to the backward effect, can prevent the “nesting” effect in the sequential forward selection (SFS) method, and avoid the problem the problem of predefining l and r in the plus l / take away r (PTA) method. For more details of SFFS we refer to [8]. The classification process is based on the widely used multivariate Gaussian Bayes classifier [9]. The Kappa coefficient ( N ) is used as the criterion of feature selection. And the minimum and maximum Fisher criterion ( J ), which can express the separation ability between classes in the selected feature space, are recorded simultaneously. Below is a detailed description of our MCGF/MRF feature fusion system for texture classification. Firstly, texture features are respectively extracted from the input image by means of MCGF and MRF, according to Section 2. Then the correlation relationship between the two above different types of features is investigated. A low correlation coefficient (e.g., closer to zero), which suggests that the relationship between MCGF and
Proceedings of the Computer Graphics, Imaging and Vision: New Trends (CGIV’05) 0-7695-2392-7/05 $20.00 © 2005 IEEE
MRF features is weak or non-existent, shows the potential producing the higher classification accuracy by MCGF/MRF fused features. Secondly, feature normalization complies with a rule that each feature component should be treated equally for its contribution to the designed classifier. Thirdly, the SFFS method is used to select an optimal feature subset after the normalized MCGF and MRF features are pooled together, and the Kappa coefficient ( N ) is used as the criterion of feature selection. Thus the fused MCGF/MRF features are obtained, which can obtain satisfactory classification results.
4. Experimental results In this section, the experiments performed on Brodatz and QuickBird texture images are presented. For MCGF, the frequency bandwidth is set to unit octave, the orientation bandwidth 30 degree. The center frequency spacing is set to the frequency bandwidth, and the orientation spacing is equal to the orientation bandwidth. Just the three highest center frequencies are used because low frequencies are unhelpful for texture discrimination. And only half part [0, S ] is considered due to the symmetry of Fourier spectrum. The application of such a filter bank to an input image results in an 18-dimensional MCGF feature vector for each pixel of that image. For MRF, the fourth-order is used [10], yielding an 11dimensional MRF feature vector for each image pixel. To test the effectiveness of the fused MCGF/MRF features achieved by the proposed algorithm, comparative tests are conducted by using the following other types of features: the pure features (MCGF or MRF features alone) with selection or not and the combined MCGF and MRF features without selection. Performance evaluation is measured by Kappa coefficients ( N ) and overall accuracies.
4.1. Experiment on Brodatz textures Figure 1(a) shows the 256 u 256 image containing five different Brodatz textures. Its corresponding ground truth label map is shown in Figure 1(b), and labels 1 to 5 respectively correspond to D54 (beach pebble), D36 (lizard skin), D55 (straw matting), D9 (grass lawn), and D24 (pressed calf leather). The correlation coefficient between the MCGF and MRF features extracted from Figure 1(a) is 0.0739. The low correlation coefficient indicates that MRF features are not well correlated with MCGF features, and the combined features are expected to offer more texture information than the pure features. Figure 1(c)
2
1 5
4
3
(c) (d) (a) (b) Figure 1. Brodatz textures for experiment: (a) input image, (b) ground truth, (c) the classification result with the combined CMGF and MRF features without selection, (d) the classification result with the fused CMGF/MRF features.
displays the classification result with the combined MCGF and MRF features without feature selection, and its Kappa coefficient is 0.7972 with an overall accuracy of 83.78%. Figure 1(d) shows the classification result using the fused MCGF/MRF features. Using the proposed MCGF/MRF feature fusion algorithm, the Kappa coefficient goes up from 0.7972 to 0.9078, with the overall accuracy from 83.78% to 92.62%. The comparison results for the composite Brodatz texture images are summarized in Table 1. Without feature selection, the combined MCGF and MRF
features don’t bring any significant change; on the contrary, the Kappa coefficient actually drops, ranging from 0.8023 for MRF features to 0.7972. From Table 1, two important remarks are made as follows. For one thing, the features involving feature selection obtain superior classification performance to those without feature selection. For another, the fused MCGF/MRF features achieved by the MCGF/MRF feature fusion algorithm can produce better classification result, compared with the pure features (MCGF or MRF features alone) with selection or not and the combined features without selection.
Table 1. Comparison of Brodatz texture image classification results: NF-number of features, N -Kappa coefficient, OA-overall accuracy, J max and J min -maximum and minimum Fisher criterion.
Feature type
Without Feature selection
Using feature selection
NF
N
OA
NF
N
OA
J max
J min
MCGF features
18
0.6876
75.01%
4
0.7973
83.79%
40.04
4.28
MRF features
11
0.8023
84.19%
5
0.8514
88.11%
111.07
4.37
MCGF and MRF features
29
0.7972
83.78%
9
0.9078
92.62%
125.95
12.60
4.2. Experiment on QuickBird image Figure 2(a) is a patchwork of uniform region samples from the QuickBird image of a suburban area in Beijing, China (2002), consisting of three different textures. Figure 2(b) displays the gray level coded ground truth of Figure 2(a), and label 1 to 3 respectively denote bare soil, green fields, and clumpy trees. The correlation coefficient between the MCGF and MRF features extracted from Figure 2(a) is 0.1387, which indicates that there is strong complementary between the two different features. And the integrated
Proceedings of the Computer Graphics, Imaging and Vision: New Trends (CGIV’05) 0-7695-2392-7/05 $20.00 © 2005 IEEE
utilization of MCGF and MRF features is anticipated to provide a much more satisfactory result than the pure features. Figure 2(c) is the classified image using the combined MCGF and MRF features without selection. Its corresponding Kappa coefficient is 0.7876 with an overall accuracy of 87.16%. Figure 2(d) displays the classification result using the fused MCGF/MRF features, and the Kappa coefficient increases to 0.8924 with the overall accuracy 93.44%. Experemental results on Figure 2(a) are listed in Table 2. As we can see from Table 2, the features with selection outperform those without selection and the fused MCGF/MRF features perform best.
2
1
3 (a)
(c)
(b)
(d)
Figure 2. QuickBird image for experiment: (a) input image, (b) ground truth, (c) the classification
result with the combined CMGF and MRF features without selection, (d) the classification result with the fused CMGF/MRF features. Table 2. Comparison of QuickBird image classification results.
Feature type
Without Feature selection
Using feature selection
NF
N
OA
NF
N
OA
J max
J min
MCGF features
18
0.6711
80.58%
8
0.8022
88.21%
47.41
27.22
MRF features
11
0.7918
87.47%
4
0.8758
92.42%
93.15
23.69
MCGF and MRF features
29
0.7876
87.16%
9
0.8924
93.44%
561.52
21.10
5. Conclusion This paper presents a MCGF/MRF feature fusion algorithm for texture classification. The performance of this algorithm is investigated with Brodatz and QuickBird images. The fused MCGF/MRF features can provide higher classification accuracy, compared with the pure features (MCGF or MRF features alone) and the combined MCGF and MRF features without selection. The experimental results indicate that the proposed algorithm is stable, reliable and efficient to improve texture classification results.
Acknowledgements This work was supported by the 973 Project of the People’s Republic of China (Project Number 2003CB415205), and the National Natural Science Foundation of China (Project Number 40471088).
References [1] A.C. Bovik, M. Clark, and W.S. Geisler, “Multichannel Texture Analysis Using Localized Spatial Filters”, IEEE Transactions on Pattern Analysis and Machine Intelligence 12(1), 1990, pp. 55-73.
Proceedings of the Computer Graphics, Imaging and Vision: New Trends (CGIV’05) 0-7695-2392-7/05 $20.00 © 2005 IEEE
[2] E. Cesmeli and D.L. Wang, “Texture Segmentation Using Gaussian-Markov Random Fields and Neural Oscillator Networks”, IEEE Transactions on Neural Network, 12(2), 2001, pp. 394-404. [3] D.A. Clausi, “Comparison and Fusion of Co-occurrence, Gabor, and MRF Texture Features for Classification of SAR Sea Ice Imagery”, Atmosphere & Oceans, 39(4), 2001, pp. 183-194. [4] D.A. Clausi and M.Ed Jernigan, “Designing Gabor Filters for Optimal Texture Separability”, Pattern Recognition, 33(11), 2000, pp. 1835-1849. [5] A.K. Jain and F. Farrokhnia, “Unsupervised Texture Segmentation Using Gabor Filters”, Pattern Recognition, 24(12), 1991, pp. 1167-1186. [6] G. Sharma and R. Chellappa, “A Model Based Approach for the Estimation of 2-D Maximum Entropy Power Spectra”, IEEE Transaction on Information Theory, 31(1), 1985, pp. 90-99. [7] BS Manjunath and R. Chellappa, “Unsupervised Texture Segmentation Using Markov Random Field Models”, IEEE Trans. Pattern Analysis and Machine Intelligence, 13(5), 1991, pp. 478-482. [8] P. Pudil, J. Novovicova and J. Kittler, “Floating Search Methods in Feature Selection”, Pattern Recognition Letters, 15 (11), 1994, pp. 1119-1125. [9] Richard O. Duda, Peter E. Hart and David G. Stork, Pattern Recognition(Second Edition), New York :Wiley,2001. [10] R. Kashyap and R. Chellappa. “Estimation and Choice of Neighbors in Spatial Interaction Models of Images”, IEEE Transaction on Information Theory, 29(1), 1983, pp. 60-72.
