A Spatial Clustering Method With Edge Weighting for Image ...
1124
IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 10, NO. 5, SEPTEMBER 2013
A Spatial Clustering Method With Edge Weighting for Image Segmentation Nan Li, Hong Huo, Yu-ming Zhao, Xi Chen, and Tao Fang
Abstract—As one of the best image clustering methods, fuzzy local information C-means is often used for image segmentation. The effects of noise are avoided by utilizing the spatial relationship among pixels, but it often generates boundary zones for the mix pixels around the edges. This letter presents an image spatial clustering method, called fuzzy C-means with edge and local information (FELICM), which reduces the edge degradation by introducing the weights of pixels within local neighbor windows. The edges are extracted at first by Canny edge detection. During detection, two adaptive thresholds obtained by multi-Otsu method are used. Then, different weights are set according to whether the window center and the local neighbors are separated by an edge or not. Pixels, together with different weighted local neighbors, are clustered iteratively, until the final clustering result is obtained. The method can be directly applied to the image without any filter preprocessing, and the experimental results over remote sensing images show that FELICM not only effectively solves the problem of isolated and random distribution of pixels inside regions but also obtains high edge accuracies. Index Terms—Canny edge detection, local information, multiOtsu, spatial clustering.
I. I NTRODUCTION
I
MAGE clustering is an effective method for image original segmentation, but it usually regards the pixels as isolated samples and ignores the spatial relationships among pixels, so it is very sensitive to noise. In practice, some smooth filters [1]–[3] are used to reduce noise before clustering, while some other researchers [4], [5] use texture description or spectral reduction to decrease the differences among pixels in a region. However, these preprocessing methods may lose some important image details. Moreover, the effects of these methods
Manuscript received May 12, 2012; revised August 14, 2012 and October 29, 2012; accepted November 21, 2012. This work was supported in part by the National Key Basic Research and Development Program of the People’s Republic of China under Grants 2012CB719903 and 2011CB707105, by the National Natural Science Foundation of China under Grant 41071256, by the Ph.D. Programs Foundation of the Ministry of Education of China under Grant 20090073110018, by the Young Scientists Fund of the National Natural Science Foundation of China under Grant 41101386, by the Science Fund for Creative Research Groups of the National Natural Science Foundation of China under Grant 61221003, and by the Shanghai Jiao Tong University Scientific and Technological Innovation and Development Special Fund under Grant X198144. The authors are with the Department of Automation, Shanghai Jiao Tong University, Shanghai 200240, China, and also with the Key Laboratory of System Control and Information Processing, Ministry of Education of China, Shanghai 200240, China (e-mail: [email protected]; huohong@sjtu. edu.cn; [email protected]; [email protected]; [email protected]). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/LGRS.2012.2231662
often depend on some important parameters, and the robustness of clustering is weakened consequently. In order to make clustering more robust, many spatial clustering methods [6]–[9], which can deal with original image without filtering, have been proposed. Ahmed et al. [9] proposed bias-corrected fuzzy C-means (FCM) (BCFCM). In BCFCM, the label of a pixel is determined by both spectral features of the pixel and its mean-filtered neighbors, and a parameter α controls the effect of neighbors. Tarabalka et al. [6] proposed a spectral–spatial method for hyperspectral images. The homogeneous regions are obtained by combining the results of support vector machine and clustering using majority voting. Liew et al. [7] presented an adaptive FCM which utilizes local contextual information to impose local spatial continuity and allows the suppression of noise and resolves ambiguities. Dulyakarn and Rangsanseri [8] added a priori spatial information with FCM, and the remote sensing image experiments showed that the spatial information improved the segmentation results. As one of the best methods, Krinidis and Chatzis [10] presented a robust image clustering method called fuzzy local information C-means (FLICM). FLICM is a noise insensitive method without a priori knowledge. The clustering is dependent on both the spectral and local spatial information which cooperate by using a fuzzy factor. However, this method assumes that the label of one pixel is related to the labels of its spatial neighbors. Therefore, incorrect cluster labels may be assigned to the pixels near the edges of regions, and edges will be dislocated consequently. This letter presents an image spatial clustering method, called FCM with edge and local information (FELICM), which reduces edge degradation by introducing the weights of pixels within local neighbor windows. The image clustering procedure will be introduced in Section II. The third section discusses the results on the images. Finally, the conclusions are drawn. II. I MAGE C LUSTERING P ROCEDURE The framework for the image clustering procedure is described in Fig. 1. A gray image is first obtained from the original image by taking first the component of principal components analysis. Then, a lot of edges are extracted from the gray image by adjusting two thresholds (a high threshold and a low threshold) in Canny edge detection algorithm [11], and two thresholds are obtained adaptively by using multi-Otsu algorithm [12]. Through the analysis of edges, different weights are set to the neighbors within local windows. Pixels, together with different weighted local neighbors, are clustered by FELICM
1545-598X/$31.00 © 2013 IEEE
LI et al.: SPATIAL CLUSTERING METHOD WITH EDGE WEIGHTING FOR IMAGE SEGMENTATION
1125
2) A 5 × 5 window is used to scan edge map. If the center pixel of the window is a candidate edge pixel, the gradient of this pixel is set to n
Ti =
Tj
j=1
n
(1)
where Tj and n are the gradient of candidate edge pixel j and the number of candidate edge pixels, respectively, within the window. 3) A histogram is created by recording the gradients of edge pixels, and the statistical mean is l
u=
Fig. 1.
Framework for image clustering procedure.
using spectral and spatial information iteratively. Finally, the final clustering result can be obtained by processing the pixels on edges. A. Edge Extraction Canny operator is used to extract edges from the gray image. The basic steps of Canny edge detection are as follows. 1) Noise reduction. The gray image is convolved with a 5 × 5 Gaussian filter with standard deviation σ = 0.4. 2) Finding the intensity gradient of the image. 3) Nonmaximum suppression. 4) Tracing edges through the image and hysteresis thresholding. In step 2), Scharr operator instead of the common Sobel operator is used, as it has better rotational symmetry [13]. It is described as ⎡ ⎤⎡ ⎤ +3 +10 +3 +3 0 −3 ⎣ 0 0 0 ⎦ ⎣ +10 0 −10 ⎦ . −3 −10 −3 +3 0 −3 In step 4), multi-Otsu method is used to find the thresholds by analyzing gradient histogram, and the optimum threshold is calculated by separating the gradients into three types so that the combined spread is minimal. In this way, the adaptive thresholds for edge detection are obtained. Then, all gradients in the image are recorded in a histogram. This will result in a large histogram with many low count value cells. This kind of histogram is difficult to analyze, so we reduce the number of cells by combining cells with low count values. Then, a new histogram containing cells with high count values is obtained. In summary, the threshold setting can be described as follows. 1) After nonmaximum suppression, the thin-edge map with edges of one-pixel width is obtained.
i=1
l
Si (2)
where Si represents the count value of cell i which is greater than zero in the histogram and l is the number of cells whose count values are greater than zero. 4) The cells whose count values are lower than u are combined with their nearest cells to reduce the number of cells. 5) Two thresholds are calculated according to this histogram by multi-Otsu method. These two thresholds are the optimum threshold for Canny operator. In general, edge detection under these two thresholds will obtain more edges than correct boundaries of objects. However, Canny method in this letter is not used to obtain the accurate boundaries but to get the edges which are utilized to set the weights of pixels within neighbor window, and the edges inside the same region will be eliminated after clustering. Therefore, no matter how complex the image is, the edges will be detected adaptively with Canny operator. B. Neighborhood Weighting In contrast with FLICM, our method considers that pixels are usually impacted by only part of spatial neighbors which belong to the same region as the center pixel of the window. This idea is different from the Kuwahara filter [3] in which a window is partitioned into four identical square subwindows, and one subwindow is chosen by local standard deviation for filtering. This filter does not guarantee that the pixels in the selected subwindow belong to the same region as the center pixel, and the subwindows of the adjacent pixels may not overlap. These drawbacks will result in clustering errors. According to the method used in [14], we make a simple judgment: If the straight line between two pixels is cut off by an edge, these two pixels belong to “different regions.” Here, the “different regions” are used to find the most appropriate spatial neighbors, but they are not really different regions. Pixels in different regions will set weights which are different from those of pixels in the same regions. The weights for pixels of local neighbors can be set by the following method. 1) Suppose that the pixels in n × n neighbor window may be affected by the label of the center pixel pi , i.e., the size
1126
IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 10, NO. 5, SEPTEMBER 2013
defined as J=
N c
[Qki + Gki ]
(3)
i=1 k=1
Fig. 2. Three cases of the windows setting, where the dash–dot lines represent the edges detected by Canny operator and S1i and S2i are the spatial neighbor windows with the same center pixel pi .
of spatial neighbor window S1i is n × n, the subscript i means that the window center is pi . 2) Get a m × m spatial neighbor window S2i which has the same center as window S1i , where m = 2 × n + 1. 3) Count the number of neighbor pixels which are not separated from the center pixel pi by edges in window S2i , and set the total number as t. 4) If t is more than n2 , the weight of pixel pj within window S1i is set as 0.33, if pi and pj are separated by the edge wij = 1, Otherwise. If pi and pj are separated by edges, the weight of pixel pj should be set as zero. Here, we set the weight as 0.33 for weakening the possible errors in edge detection. 5) If t is less than n2 , the weight of pixel pj within window S1i is wij = 1. Based on the aforementioned settings, there exist three cases. Case 1) There is no edge within window S1i , as shown in Fig. 2(a). The weights of all the pixels in S1i are set as one. Case 2) One edge exists in window S1i , and the total number t in window S2i is more than n2 , as shown in Fig. 2(b). Different weights will be set for the neighbor pixels in S1i . As the neighbor pixel pj and center pixel pi are separated by the edge, these two pixels may belong to different regions, so the weight of neighbor pixel pj is set as 0.33, and its impact on center pixel pi has also been reduced. Case 3) The edges exist in window S1i , and the total number t in window S2i is less than n2 , as shown in Fig. 2(c). This case may be caused by the noise or the edge error, and the weights of all pixels in window S1i are set as one, so the effect of errors can be weakened by the spatial neighborhood. According to the aforementioned weight setting method, different weights will be assigned to all pixels in the image, and then, the edges of regions can be retained well by using the weighted windows. C. FELICM For an image of N pixels P = {p1 , p2 , . . . , pN }, FELICM is an iterative clustering method, and its objective function can be
where N is the number of pixels, c is the number of clusters, and Qki is the distance between pixel pi and the center of cluster k and is defined as 2 Qki = um ki pi − vk
(4)
where m is the exponent of weight and it has been set to three in the back experiments from practical experience, uki is the membership degree of pixel pi to the cluster k, and vk is the prototype of the center of cluster k. The distance Gki between the spatial neighbors of pixel pi and the center of cluster k is used to control the impact of noises and is defined as Gki =
j∈S1i i=j
1 [wij (1 − ukj )]m pj − vk 2 dij + 1
(5)
where pixel pj belongs to the window S1i whose center is pixel pi , dij is the spatial Euclidean distance between pi and pj , and wij is the weight of pixel pj in the window S1i . Local minimal extreme J is obtained iteratively as follows. 1) Set c, m, and the stopping condition ε. 2) Initialize the fuzzy partition matrix, and set all elements as 1/c. Set the loop counter b = 0. 3) Scan the image, and the cluster prototypes are calculated as N
vk =
um ki pi
i=1 N
i=1
.
(6)
um ki
4) The degree of membership is obtained by uki =
1
1 . c pi −vk 2 +Gki m−1 j=1
(7)
pi −vj 2 +Gji
5) If {U (b) − U (b+1) } < ε, then stop; otherwise, set b = b + 1 and go to 3). 6) Pixel pi is clustered into the cluster Ci = argk {max{uki }}. From aforementioned descriptions, it can be seen that Gki can be obtained without any parameter. It varies automatically according to the spectral feature of pixels in the spatial window and the fuzzy membership of pixels to the clusters. In other words, the local fuzzy membership balances the relationship between pixels and their neighbors. This preserves the details and edges in the image and improves the accuracy of clustering ultimately. After the clustering, all pixels on the edges obtained by Canny operator will be assigned to their nearest adjacent regions, and the final clustering result is achieved.
LI et al.: SPATIAL CLUSTERING METHOD WITH EDGE WEIGHTING FOR IMAGE SEGMENTATION
1127
land, some small buildings, and a river. Fig. 3(b) is the reference data obtained by artificial segmentation. LUV color space is used in this experiment. In order to keep a balance between efficiency and effect, the neighbor window size (S1i ) is 11 × 11. Fig. 3(c) is the result of FCM. It shows that different agricultural regions have been distinguished, but there are many isolated and randomly distributed pixels inside regions. Fig. 3(d) is the result of MeanShift [1]. The filtering effect of MeanShift is very good, and a lot of noise is eliminated, but it focuses on the probability distribution of neighbors rather than spatial relationship between neighbors, so the isolated regions still occur in some areas. Fig. 3(e) is the result of FLICM. The number of isolated regions has been reduced greatly, and the regions are in agreement with visual sense, but there are still some errors in the edges, for example, the area between the planted forest and agricultural land has a clear “buffer” in the bottom part of the image. Fig. 3(f) is the result of FELICM. Due to redundant edges extracted by Canny operator, the pixels, which are separated within local spatial windows, are not only located in the vicinity of the edges of objects but also distributed within the regions. Although these pixels may result in isolated regions, the number of isolated regions is decreased greatly. Moreover, the “buffers” are eliminated, and FELICM obtains more accurate edges than FLICM. Fig. 3(g) shows the comparison results in detail. The edges of FLICM results are degenerated, while the edges of FELICM results are distinguished. B. Quantitative Evaluation For the complex image, many different objects have similar features, and the objects cannot be distinguished well by only increasing the number of clusters. Moreover, the reference data are not always available. Therefore, we use the reference segmentation results which are obtained by artificial segmentation to evaluate the different clustering results. Two accuracies are defined for the quantitative evaluation of image clustering methods: regional accuracy and edge accuracy. First, the regional accuracy is defined as n
Fig. 3. Experiment of aerial image. (a) Aerial image. (b) Reference data. (c) Result of FCM. (d) Result of MeanShift. (e) Result of FLICM. (f) Result of FELICM. (g) Comparison results in detail (FLICM results in the upper row, FELICM results in the lower row, and the black line as the reference edge).
III. E XPERIMENTS Many images are tested to evaluate the performance of the proposed FELICM method. We illustrate its effectiveness using only one aerial image. A. Experiment of Aerial Image Fig. 3(a) is an aerial image of a rural area which is located at Chongming Island in the eastern of China. Its size is 512 × 512 pixels. It includes several different types of agricultural
max{Aik }
i=1
RA =
N
k ∈ [1, c]
(8)
where N is the overall number of pixels in the image, n is the number of objects in the reference data, Aik is the number of pixels of cluster k in object i, and max{·} is used to obtain the maximum number of pixels of all clusters in the object. Then, we use the accuracy of pixels around edges to represent the accuracies of edges. The edge accuracy can be described as n
EA =
i=1
AiK
N
K = arg max {Aik } k∈[1,c]
(9)
where N is the number of pixels which are obtained by dilating the reference edges with a window, AiK is the number of pixels
1128
IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 10, NO. 5, SEPTEMBER 2013
TABLE I R EGIONAL ACCURACIES
IV. C ONCLUSION The traditional image clustering methods usually regard image pixels as isolated samples, which usually result in isolated regions. FLICM uses local information to guarantee noise insensitiveness, but it often produces boundary zones due to the mix pixels near the edges of different regions. In this letter, FELICM is proposed, and it improves FLICM by introducing the weights for pixels within local neighbor windows. The experiments show that FELICM method is insensitive to the isolated regions and obtains more accurate edges than FLICM. R EFERENCES
Fig. 4. Comparison of edge accuracies.
of cluster K in object i within the range of dilating area, and K is the cluster which has the maximum number of pixels in the object. Table I shows the regional accuracies of the aerial image experiment. It can be seen that the accuracies of FCM and MeanShift are low for their strong isolated regions. FLICM and FELICM methods have higher accuracies for their better resolving noise problems than those of FCM and MeanShift. For all experiments, the regional accuracies of FLICM and FELICM are similar and higher than those of FCM of about 7%–14%. Fig. 4 shows the edge accuracies of the aerial image experiment. It can be seen that the accuracies of all methods in the small dilation window are lower than those in the larger dilation window. This means that the accuracies of pixels near the edges are lower than those within the regions. However, FELICM and MeanShift retain the edges better than the other two methods, so their edge accuracies are higher than those of FCM and FLICM. For all experiments, the edge accuracies of FELICM are about 2%–8% higher than those of FLICM. For the time complexity, both FLICM and FELICM spend most time on their iterative processing, and the numbers of their iterations are usually similar. Under the same termination conditions, FELICM needs 17 iterations, while FLICM needs 19 iterations.
[1] D. Comaniciu, “Mean shift: A robust approach toward feature space analysis,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 5, pp. 603–619, May 2002. [2] D. E. Ilea and P. F. Whelan, “CTex—An adaptive unsupervised segmentation algorithm based on color–texture coherence,” IEEE Trans. Image Process., vol. 17, no. 10, pp. 1926–1939, Oct. 2008. [3] K. H. M. Kuwahara, S. Ehiu, and M. Kinoshita, “Processing of riangiocardiographic images,” in Digital Processing of Biomedical Images. New York: Plenum, 1976, pp. 187–203. [4] G. Dong and M. Xie, “Color clustering and learning for image segmentation based on neural networks,” IEEE Trans. Neural Netw., vol. 16, no. 4, pp. 925–936, Jul. 2005. [5] Z. Xiangrong, L. Jiao, F. Liu, L. Bo, and M. Gong, “Spectral clustering ensemble applied to SAR image segmentation,” IEEE Trans. Geosci. Remote Sens., vol. 46, no. 7, pp. 2126–2136, Jul. 2008. [6] Y. Tarabalka, J. A. Benediktsson, and J. Chanussot, “Spectral–spatial classification of hyperspectral imagery based on partitional clustering techniques,” IEEE Trans. Geosci. Remote Sens., vol. 47, no. 8, pp. 2973– 2987, Aug. 2009. [7] A. W. C. Liew, H. Yan, and N. F. Law, “Image segmentation based on adaptive cluster prototype estimation,” IEEE Trans. Fuzzy Syst., vol. 13, no. 4, pp. 444–453, Aug. 2005. [8] P. Dulyakarn and Y. Rangsanseri, “Fuzzy C-means clustering using spatial information with application to remote sensing,” in Proc. 22nd Asian Conf. Remote Sens., Singapore, 2001. [9] M. N. Ahmed, S. M. Yamany, N. Mohamed, A. A. Farag, and T. Moriarty, “A modified fuzzy C-means algorithm for bias field estimation and segmentation of MRI data,” IEEE Trans. Med. Imag., vol. 21, no. 3, pp. 193– 199, Mar. 2002. [10] S. Krinidis and V. Chatzis, “A robust fuzzy local information C-means clustering algorithm,” IEEE Trans. Image Process., vol. 19, no. 5, pp. 1328–1337, May 2010. [11] J. Canny, “A computational approach to edge detection,” IEEE Trans. Pattern Anal. Mach. Intell., vol. PAMI-8, no. 6, pp. 679–698, Nov. 1986. [12] P. S. Liao, T.-S. Chen, and P.-C. Chung, “A fast algorithm for multilevel thresholding,” J. Inf. Sci. Eng., vol. 17, no. 5, pp. 713–727, Sep. 2001. [13] H. Scharr, “Optimal filters for extended optical flow,” in Proc. 1st Int. Conf. Complex Motion, 2007, vol. 3417, pp. 14–29, 235. [14] J. Malik, S. Belongie, T. Leung, and J. Shi, “Contour and texture analysis for image segmentation,” Int. J. Comput. Vis., vol. 43, no. 1, pp. 7–27, Jun. 2001.
IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 10, NO. 5, SEPTEMBER 2013
A Spatial Clustering Method With Edge Weighting for Image Segmentation Nan Li, Hong Huo, Yu-ming Zhao, Xi Chen, and Tao Fang
Abstract—As one of the best image clustering methods, fuzzy local information C-means is often used for image segmentation. The effects of noise are avoided by utilizing the spatial relationship among pixels, but it often generates boundary zones for the mix pixels around the edges. This letter presents an image spatial clustering method, called fuzzy C-means with edge and local information (FELICM), which reduces the edge degradation by introducing the weights of pixels within local neighbor windows. The edges are extracted at first by Canny edge detection. During detection, two adaptive thresholds obtained by multi-Otsu method are used. Then, different weights are set according to whether the window center and the local neighbors are separated by an edge or not. Pixels, together with different weighted local neighbors, are clustered iteratively, until the final clustering result is obtained. The method can be directly applied to the image without any filter preprocessing, and the experimental results over remote sensing images show that FELICM not only effectively solves the problem of isolated and random distribution of pixels inside regions but also obtains high edge accuracies. Index Terms—Canny edge detection, local information, multiOtsu, spatial clustering.
I. I NTRODUCTION
I
MAGE clustering is an effective method for image original segmentation, but it usually regards the pixels as isolated samples and ignores the spatial relationships among pixels, so it is very sensitive to noise. In practice, some smooth filters [1]–[3] are used to reduce noise before clustering, while some other researchers [4], [5] use texture description or spectral reduction to decrease the differences among pixels in a region. However, these preprocessing methods may lose some important image details. Moreover, the effects of these methods
Manuscript received May 12, 2012; revised August 14, 2012 and October 29, 2012; accepted November 21, 2012. This work was supported in part by the National Key Basic Research and Development Program of the People’s Republic of China under Grants 2012CB719903 and 2011CB707105, by the National Natural Science Foundation of China under Grant 41071256, by the Ph.D. Programs Foundation of the Ministry of Education of China under Grant 20090073110018, by the Young Scientists Fund of the National Natural Science Foundation of China under Grant 41101386, by the Science Fund for Creative Research Groups of the National Natural Science Foundation of China under Grant 61221003, and by the Shanghai Jiao Tong University Scientific and Technological Innovation and Development Special Fund under Grant X198144. The authors are with the Department of Automation, Shanghai Jiao Tong University, Shanghai 200240, China, and also with the Key Laboratory of System Control and Information Processing, Ministry of Education of China, Shanghai 200240, China (e-mail: [email protected]; huohong@sjtu. edu.cn; [email protected]; [email protected]; [email protected]). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/LGRS.2012.2231662
often depend on some important parameters, and the robustness of clustering is weakened consequently. In order to make clustering more robust, many spatial clustering methods [6]–[9], which can deal with original image without filtering, have been proposed. Ahmed et al. [9] proposed bias-corrected fuzzy C-means (FCM) (BCFCM). In BCFCM, the label of a pixel is determined by both spectral features of the pixel and its mean-filtered neighbors, and a parameter α controls the effect of neighbors. Tarabalka et al. [6] proposed a spectral–spatial method for hyperspectral images. The homogeneous regions are obtained by combining the results of support vector machine and clustering using majority voting. Liew et al. [7] presented an adaptive FCM which utilizes local contextual information to impose local spatial continuity and allows the suppression of noise and resolves ambiguities. Dulyakarn and Rangsanseri [8] added a priori spatial information with FCM, and the remote sensing image experiments showed that the spatial information improved the segmentation results. As one of the best methods, Krinidis and Chatzis [10] presented a robust image clustering method called fuzzy local information C-means (FLICM). FLICM is a noise insensitive method without a priori knowledge. The clustering is dependent on both the spectral and local spatial information which cooperate by using a fuzzy factor. However, this method assumes that the label of one pixel is related to the labels of its spatial neighbors. Therefore, incorrect cluster labels may be assigned to the pixels near the edges of regions, and edges will be dislocated consequently. This letter presents an image spatial clustering method, called FCM with edge and local information (FELICM), which reduces edge degradation by introducing the weights of pixels within local neighbor windows. The image clustering procedure will be introduced in Section II. The third section discusses the results on the images. Finally, the conclusions are drawn. II. I MAGE C LUSTERING P ROCEDURE The framework for the image clustering procedure is described in Fig. 1. A gray image is first obtained from the original image by taking first the component of principal components analysis. Then, a lot of edges are extracted from the gray image by adjusting two thresholds (a high threshold and a low threshold) in Canny edge detection algorithm [11], and two thresholds are obtained adaptively by using multi-Otsu algorithm [12]. Through the analysis of edges, different weights are set to the neighbors within local windows. Pixels, together with different weighted local neighbors, are clustered by FELICM
1545-598X/$31.00 © 2013 IEEE
LI et al.: SPATIAL CLUSTERING METHOD WITH EDGE WEIGHTING FOR IMAGE SEGMENTATION
1125
2) A 5 × 5 window is used to scan edge map. If the center pixel of the window is a candidate edge pixel, the gradient of this pixel is set to n
Ti =
Tj
j=1
n
(1)
where Tj and n are the gradient of candidate edge pixel j and the number of candidate edge pixels, respectively, within the window. 3) A histogram is created by recording the gradients of edge pixels, and the statistical mean is l
u=
Fig. 1.
Framework for image clustering procedure.
using spectral and spatial information iteratively. Finally, the final clustering result can be obtained by processing the pixels on edges. A. Edge Extraction Canny operator is used to extract edges from the gray image. The basic steps of Canny edge detection are as follows. 1) Noise reduction. The gray image is convolved with a 5 × 5 Gaussian filter with standard deviation σ = 0.4. 2) Finding the intensity gradient of the image. 3) Nonmaximum suppression. 4) Tracing edges through the image and hysteresis thresholding. In step 2), Scharr operator instead of the common Sobel operator is used, as it has better rotational symmetry [13]. It is described as ⎡ ⎤⎡ ⎤ +3 +10 +3 +3 0 −3 ⎣ 0 0 0 ⎦ ⎣ +10 0 −10 ⎦ . −3 −10 −3 +3 0 −3 In step 4), multi-Otsu method is used to find the thresholds by analyzing gradient histogram, and the optimum threshold is calculated by separating the gradients into three types so that the combined spread is minimal. In this way, the adaptive thresholds for edge detection are obtained. Then, all gradients in the image are recorded in a histogram. This will result in a large histogram with many low count value cells. This kind of histogram is difficult to analyze, so we reduce the number of cells by combining cells with low count values. Then, a new histogram containing cells with high count values is obtained. In summary, the threshold setting can be described as follows. 1) After nonmaximum suppression, the thin-edge map with edges of one-pixel width is obtained.
i=1
l
Si (2)
where Si represents the count value of cell i which is greater than zero in the histogram and l is the number of cells whose count values are greater than zero. 4) The cells whose count values are lower than u are combined with their nearest cells to reduce the number of cells. 5) Two thresholds are calculated according to this histogram by multi-Otsu method. These two thresholds are the optimum threshold for Canny operator. In general, edge detection under these two thresholds will obtain more edges than correct boundaries of objects. However, Canny method in this letter is not used to obtain the accurate boundaries but to get the edges which are utilized to set the weights of pixels within neighbor window, and the edges inside the same region will be eliminated after clustering. Therefore, no matter how complex the image is, the edges will be detected adaptively with Canny operator. B. Neighborhood Weighting In contrast with FLICM, our method considers that pixels are usually impacted by only part of spatial neighbors which belong to the same region as the center pixel of the window. This idea is different from the Kuwahara filter [3] in which a window is partitioned into four identical square subwindows, and one subwindow is chosen by local standard deviation for filtering. This filter does not guarantee that the pixels in the selected subwindow belong to the same region as the center pixel, and the subwindows of the adjacent pixels may not overlap. These drawbacks will result in clustering errors. According to the method used in [14], we make a simple judgment: If the straight line between two pixels is cut off by an edge, these two pixels belong to “different regions.” Here, the “different regions” are used to find the most appropriate spatial neighbors, but they are not really different regions. Pixels in different regions will set weights which are different from those of pixels in the same regions. The weights for pixels of local neighbors can be set by the following method. 1) Suppose that the pixels in n × n neighbor window may be affected by the label of the center pixel pi , i.e., the size
1126
IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 10, NO. 5, SEPTEMBER 2013
defined as J=
N c
[Qki + Gki ]
(3)
i=1 k=1
Fig. 2. Three cases of the windows setting, where the dash–dot lines represent the edges detected by Canny operator and S1i and S2i are the spatial neighbor windows with the same center pixel pi .
of spatial neighbor window S1i is n × n, the subscript i means that the window center is pi . 2) Get a m × m spatial neighbor window S2i which has the same center as window S1i , where m = 2 × n + 1. 3) Count the number of neighbor pixels which are not separated from the center pixel pi by edges in window S2i , and set the total number as t. 4) If t is more than n2 , the weight of pixel pj within window S1i is set as 0.33, if pi and pj are separated by the edge wij = 1, Otherwise. If pi and pj are separated by edges, the weight of pixel pj should be set as zero. Here, we set the weight as 0.33 for weakening the possible errors in edge detection. 5) If t is less than n2 , the weight of pixel pj within window S1i is wij = 1. Based on the aforementioned settings, there exist three cases. Case 1) There is no edge within window S1i , as shown in Fig. 2(a). The weights of all the pixels in S1i are set as one. Case 2) One edge exists in window S1i , and the total number t in window S2i is more than n2 , as shown in Fig. 2(b). Different weights will be set for the neighbor pixels in S1i . As the neighbor pixel pj and center pixel pi are separated by the edge, these two pixels may belong to different regions, so the weight of neighbor pixel pj is set as 0.33, and its impact on center pixel pi has also been reduced. Case 3) The edges exist in window S1i , and the total number t in window S2i is less than n2 , as shown in Fig. 2(c). This case may be caused by the noise or the edge error, and the weights of all pixels in window S1i are set as one, so the effect of errors can be weakened by the spatial neighborhood. According to the aforementioned weight setting method, different weights will be assigned to all pixels in the image, and then, the edges of regions can be retained well by using the weighted windows. C. FELICM For an image of N pixels P = {p1 , p2 , . . . , pN }, FELICM is an iterative clustering method, and its objective function can be
where N is the number of pixels, c is the number of clusters, and Qki is the distance between pixel pi and the center of cluster k and is defined as 2 Qki = um ki pi − vk
(4)
where m is the exponent of weight and it has been set to three in the back experiments from practical experience, uki is the membership degree of pixel pi to the cluster k, and vk is the prototype of the center of cluster k. The distance Gki between the spatial neighbors of pixel pi and the center of cluster k is used to control the impact of noises and is defined as Gki =
j∈S1i i=j
1 [wij (1 − ukj )]m pj − vk 2 dij + 1
(5)
where pixel pj belongs to the window S1i whose center is pixel pi , dij is the spatial Euclidean distance between pi and pj , and wij is the weight of pixel pj in the window S1i . Local minimal extreme J is obtained iteratively as follows. 1) Set c, m, and the stopping condition ε. 2) Initialize the fuzzy partition matrix, and set all elements as 1/c. Set the loop counter b = 0. 3) Scan the image, and the cluster prototypes are calculated as N
vk =
um ki pi
i=1 N
i=1
.
(6)
um ki
4) The degree of membership is obtained by uki =
1
1 . c pi −vk 2 +Gki m−1 j=1
(7)
pi −vj 2 +Gji
5) If {U (b) − U (b+1) } < ε, then stop; otherwise, set b = b + 1 and go to 3). 6) Pixel pi is clustered into the cluster Ci = argk {max{uki }}. From aforementioned descriptions, it can be seen that Gki can be obtained without any parameter. It varies automatically according to the spectral feature of pixels in the spatial window and the fuzzy membership of pixels to the clusters. In other words, the local fuzzy membership balances the relationship between pixels and their neighbors. This preserves the details and edges in the image and improves the accuracy of clustering ultimately. After the clustering, all pixels on the edges obtained by Canny operator will be assigned to their nearest adjacent regions, and the final clustering result is achieved.
LI et al.: SPATIAL CLUSTERING METHOD WITH EDGE WEIGHTING FOR IMAGE SEGMENTATION
1127
land, some small buildings, and a river. Fig. 3(b) is the reference data obtained by artificial segmentation. LUV color space is used in this experiment. In order to keep a balance between efficiency and effect, the neighbor window size (S1i ) is 11 × 11. Fig. 3(c) is the result of FCM. It shows that different agricultural regions have been distinguished, but there are many isolated and randomly distributed pixels inside regions. Fig. 3(d) is the result of MeanShift [1]. The filtering effect of MeanShift is very good, and a lot of noise is eliminated, but it focuses on the probability distribution of neighbors rather than spatial relationship between neighbors, so the isolated regions still occur in some areas. Fig. 3(e) is the result of FLICM. The number of isolated regions has been reduced greatly, and the regions are in agreement with visual sense, but there are still some errors in the edges, for example, the area between the planted forest and agricultural land has a clear “buffer” in the bottom part of the image. Fig. 3(f) is the result of FELICM. Due to redundant edges extracted by Canny operator, the pixels, which are separated within local spatial windows, are not only located in the vicinity of the edges of objects but also distributed within the regions. Although these pixels may result in isolated regions, the number of isolated regions is decreased greatly. Moreover, the “buffers” are eliminated, and FELICM obtains more accurate edges than FLICM. Fig. 3(g) shows the comparison results in detail. The edges of FLICM results are degenerated, while the edges of FELICM results are distinguished. B. Quantitative Evaluation For the complex image, many different objects have similar features, and the objects cannot be distinguished well by only increasing the number of clusters. Moreover, the reference data are not always available. Therefore, we use the reference segmentation results which are obtained by artificial segmentation to evaluate the different clustering results. Two accuracies are defined for the quantitative evaluation of image clustering methods: regional accuracy and edge accuracy. First, the regional accuracy is defined as n
Fig. 3. Experiment of aerial image. (a) Aerial image. (b) Reference data. (c) Result of FCM. (d) Result of MeanShift. (e) Result of FLICM. (f) Result of FELICM. (g) Comparison results in detail (FLICM results in the upper row, FELICM results in the lower row, and the black line as the reference edge).
III. E XPERIMENTS Many images are tested to evaluate the performance of the proposed FELICM method. We illustrate its effectiveness using only one aerial image. A. Experiment of Aerial Image Fig. 3(a) is an aerial image of a rural area which is located at Chongming Island in the eastern of China. Its size is 512 × 512 pixels. It includes several different types of agricultural
max{Aik }
i=1
RA =
N
k ∈ [1, c]
(8)
where N is the overall number of pixels in the image, n is the number of objects in the reference data, Aik is the number of pixels of cluster k in object i, and max{·} is used to obtain the maximum number of pixels of all clusters in the object. Then, we use the accuracy of pixels around edges to represent the accuracies of edges. The edge accuracy can be described as n
EA =
i=1
AiK
N
K = arg max {Aik } k∈[1,c]
(9)
where N is the number of pixels which are obtained by dilating the reference edges with a window, AiK is the number of pixels
1128
IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 10, NO. 5, SEPTEMBER 2013
TABLE I R EGIONAL ACCURACIES
IV. C ONCLUSION The traditional image clustering methods usually regard image pixels as isolated samples, which usually result in isolated regions. FLICM uses local information to guarantee noise insensitiveness, but it often produces boundary zones due to the mix pixels near the edges of different regions. In this letter, FELICM is proposed, and it improves FLICM by introducing the weights for pixels within local neighbor windows. The experiments show that FELICM method is insensitive to the isolated regions and obtains more accurate edges than FLICM. R EFERENCES
Fig. 4. Comparison of edge accuracies.
of cluster K in object i within the range of dilating area, and K is the cluster which has the maximum number of pixels in the object. Table I shows the regional accuracies of the aerial image experiment. It can be seen that the accuracies of FCM and MeanShift are low for their strong isolated regions. FLICM and FELICM methods have higher accuracies for their better resolving noise problems than those of FCM and MeanShift. For all experiments, the regional accuracies of FLICM and FELICM are similar and higher than those of FCM of about 7%–14%. Fig. 4 shows the edge accuracies of the aerial image experiment. It can be seen that the accuracies of all methods in the small dilation window are lower than those in the larger dilation window. This means that the accuracies of pixels near the edges are lower than those within the regions. However, FELICM and MeanShift retain the edges better than the other two methods, so their edge accuracies are higher than those of FCM and FLICM. For all experiments, the edge accuracies of FELICM are about 2%–8% higher than those of FLICM. For the time complexity, both FLICM and FELICM spend most time on their iterative processing, and the numbers of their iterations are usually similar. Under the same termination conditions, FELICM needs 17 iterations, while FLICM needs 19 iterations.
[1] D. Comaniciu, “Mean shift: A robust approach toward feature space analysis,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 5, pp. 603–619, May 2002. [2] D. E. Ilea and P. F. Whelan, “CTex—An adaptive unsupervised segmentation algorithm based on color–texture coherence,” IEEE Trans. Image Process., vol. 17, no. 10, pp. 1926–1939, Oct. 2008. [3] K. H. M. Kuwahara, S. Ehiu, and M. Kinoshita, “Processing of riangiocardiographic images,” in Digital Processing of Biomedical Images. New York: Plenum, 1976, pp. 187–203. [4] G. Dong and M. Xie, “Color clustering and learning for image segmentation based on neural networks,” IEEE Trans. Neural Netw., vol. 16, no. 4, pp. 925–936, Jul. 2005. [5] Z. Xiangrong, L. Jiao, F. Liu, L. Bo, and M. Gong, “Spectral clustering ensemble applied to SAR image segmentation,” IEEE Trans. Geosci. Remote Sens., vol. 46, no. 7, pp. 2126–2136, Jul. 2008. [6] Y. Tarabalka, J. A. Benediktsson, and J. Chanussot, “Spectral–spatial classification of hyperspectral imagery based on partitional clustering techniques,” IEEE Trans. Geosci. Remote Sens., vol. 47, no. 8, pp. 2973– 2987, Aug. 2009. [7] A. W. C. Liew, H. Yan, and N. F. Law, “Image segmentation based on adaptive cluster prototype estimation,” IEEE Trans. Fuzzy Syst., vol. 13, no. 4, pp. 444–453, Aug. 2005. [8] P. Dulyakarn and Y. Rangsanseri, “Fuzzy C-means clustering using spatial information with application to remote sensing,” in Proc. 22nd Asian Conf. Remote Sens., Singapore, 2001. [9] M. N. Ahmed, S. M. Yamany, N. Mohamed, A. A. Farag, and T. Moriarty, “A modified fuzzy C-means algorithm for bias field estimation and segmentation of MRI data,” IEEE Trans. Med. Imag., vol. 21, no. 3, pp. 193– 199, Mar. 2002. [10] S. Krinidis and V. Chatzis, “A robust fuzzy local information C-means clustering algorithm,” IEEE Trans. Image Process., vol. 19, no. 5, pp. 1328–1337, May 2010. [11] J. Canny, “A computational approach to edge detection,” IEEE Trans. Pattern Anal. Mach. Intell., vol. PAMI-8, no. 6, pp. 679–698, Nov. 1986. [12] P. S. Liao, T.-S. Chen, and P.-C. Chung, “A fast algorithm for multilevel thresholding,” J. Inf. Sci. Eng., vol. 17, no. 5, pp. 713–727, Sep. 2001. [13] H. Scharr, “Optimal filters for extended optical flow,” in Proc. 1st Int. Conf. Complex Motion, 2007, vol. 3417, pp. 14–29, 235. [14] J. Malik, S. Belongie, T. Leung, and J. Shi, “Contour and texture analysis for image segmentation,” Int. J. Comput. Vis., vol. 43, no. 1, pp. 7–27, Jun. 2001.
