Fuzzy Clustering Based on Generalized Entropy ... - Semantic Scholar
Fuzzy Clustering Based on Generalized Entropy and Its Application to Image Segmentation Kai Li and Yu Wang School of Mathematics and Computer, Hebei University, Baoding 071002, China {likai_njtu,wangyu}@163.com
Abstract. Fuzzy clustering based on generalized entropy is studied. By introducing the generalized entropy into objective function of fuzzy clustering, a unified model is given for fuzzy clustering in this paper. Then fuzzy clustering algorithm based on the generalized entropy is presented. At the same time, by introducing the spatial information of image into the generalized entropy fuzzy clustering algorithm, an image segmentation algorithm is presented. Finally, experiments are conducted to show effectiveness of both clustering algorithm based on generalized entropy and image segmentation algorithm. Keywords: Generalized entropy, unified model, fuzzy clustering, image segmentation.
1 Introduction Clustering is an important tool for data analysis. It groups data objects into multiple classes or clusters, wherein data have much high similarity in same cluster whereas they have great difference in different cluster. One of the most commonly used methods is k-means clustering algorithm[1].Since Zadeh proposed the concept of fuzzy in 1965, people began to study fuzzy clustering algorithm. Ruspin first proposed Fuzzy C partition in 1969. In 1974, Dunn proposed a weighted index of m = 2 fuzzy C-means algorithm. And Bezdek extended to m> 1 of the fuzzy C-means algorithm in 1981. In reality, fuzzy clustering has been widely studied and applied in various fields, for example, image processing, pattern recognition, medical, etc. However, the fuzzy clustering algorithm (FCM) only deal with the points of the data set which have the same weight, number of data points in each cluster with no difference, and the loss of anti-noise properties, etc. In order to solve these problems, the researchers proposed many clustering algorithms. In 1979, Gustafson and Kessel proposed the clustering algorithm for clusters of different shapes by using the Mahalanobis distance. Dave extended the FCM by proposing the FCS algorithm which is used for the detection curve boundary in 1990; in 1991, Borowski and Bezdek introduced different norms to the FCM algorithm. In 1993, Yang put forward a kind of punished FCM (PFCM) based on fuzzy classification of maximum likelihood. In 1994, Karayiannis proposed the maximum entropy clustering algorithm H. Deng et al. (Eds.): AICI 2011, Part II, LNAI 7003, pp. 640–647, 2011. © Springer-Verlag Berlin Heidelberg 2011
Fuzzy Clustering Based on Generalized Entropy and Its Application
641
[2]; In 1995, aiming at the uncertainty problem, Li and others introduced the maximum entropy inference method and proposed the maximum entropy clustering algorithm by combining the loss function of the point to the center [3]; Krishnapuram and Keller proposed possibilistic C-means algorithm by relaxing the constraint that the sum of fuzzy C partition is equal to 1, to overcome the flaw of FCM under the noisy environment. In 2000, Tran and Wagner proposed the fuzzy entropy clustering algorithm using the objective function proposed by Li; In 2002, Wei and Fahn proposed a fuzzy bidirectional associative clustering network which solves the problem of fuzzy clustering [4]; In the same year, Wu and Yang proposed AFCM by using the index distance; in 2004, Wang and others developed a feature weighted FCM based on the feature selection methods; In 2005, Yu and Yang studied FCM and its expanded algorithm, and established a unified model of GFCM; In 2010 Graves and Pedrycz made a comparative experimental study of fuzzy C-means and kernelbased fuzzy C-means clustering algorithm, and obtained the conclusion that kernelbased clustering algorithm is sensitive to the kernel parameters. As for the image segmentation, spatial information is introduced into the clustering algorithm to improve the performance of image segmentation and the anti-noise performance of the algorithm. In 2002, Ahmed and others proposed BCFCM algorithm and successfully applied to the segmentation of MRI data by adjusting the objective function of FCM. Later, in order to reduce the spending time, Chen and Zhang proposed an improved BCFCM algorithm which is applied the kernel theory in BCFCM in 2004 and then they proposed the KFCM algorithm with spatial constraints. In the same year, Zhang and Chen proposed a new kernel-based fuzzy Cmeans algorithm which is applied to the medical image segmentation; in 2008, Yang and Tsai proposed the GKFCM algorithm which is successfully applied to image segmentation by further studying kernel-based clustering algorithm with a spatial bias correction [6]. In 2010, Kannan and others proposed fuzzy C-means based kernel function in segmenting medical images [7]. In this year, Swagatam and others researched Kernel-induced fuzzy clustering of image segmentation [8]. Fuzzy clustering based on entropy is mainly studied in this paper. We present a unified objective function based on generalized entropy in FCM. At the same time, fuzzy clustering algorithm based on generalized entropy is obtained, and they are applied to image segmentation. This paper is organized as follows. In section 2, we describe the proposed unified model in detail. In the following section, fuzzy clustering algorithm and image segmentation algorithm based on generalized entropy are given. We evaluate our method on IRIS data set and some chosed images. At the same time, we compare it with the classical FCM in section 4. Some conclusions are given in the final section.
2 The Unified Model Based on Generalized Entropy Fuzzy clustering algorithm is common method. It has widely been applied to many real problems. However, as it exist some problems, researchers present some clustering algorithm by revising objective function. In 1994, Karayiannis proposed the Maximum entropy clustering algorithm, which makes the clustering of data points from the maximum uncertainty gradually transformed into a deterministic process by
642
K. Li and Y. Wang
the introduction of the entropy of the membership and the distance to the center point in the clustering process. The objective function is defined as n
c
J( μ,a)=αμij log μij + j =1 i =1
1− α n c μij || xj − ai ||2 . n j=1 i=1
Now, by reducing the objective function (1) and let β =
(1)
1−α , 0 < α < 1 , above nα
equation is written as n
c
n
c
J( μ ,a) = μij log μij + β μij || x j − ai ||2 . j =1 i =1
(2)
j=1 i=1
In 1995, aiming at the uncertainty problem, Li and others introduced the maximum entropy inference method and proposed the maximum entropy clustering algorithm by combining the loss function of the point to the center. Its objective function is defined as n
c
n
c
J( μ ,a)= β −1 μij log μij + μij || x j − ai ||2 . j =1 i =1
(3)
j=1 i=1
Actually, the objective function (2) and (3) above is consistent. In 2000, Tran and Wagner proposed the fuzzy entropy clustering algorithm by using the objective function proposed by Li. In 2002, Wei and Fahn proposed a fuzzy bidirectional associative clustering network to solve the problem of fuzzy clustering with the objective function as follows n
c
n
c
J( μ ,a) = β μ ij log μ ij + μ ijm || x j − ai ||2 . j =1 i =1
(4)
j=1 i=1
Above objective functions are shown that they are basically similar each other in different clustering algorithms. In the following, we give a unified objective function n
c
n
c
JG ( μij ,v)= μij m || x j − vi ||2 +δ (21−α −1)−1 (μijα −1) j=1 i=1
j =1 i =1
.
(5)
α > 0,α ≠ 1 It is seen that when m = 1 and α → 1 , objective function above is become that proposed by Karayiannis, Li,Tran and Wagner. When α → 1 , objective function above is become that proposed by Wei and Fahn. We can see that main difference in the two cases above is the second term of the unified model, written it as H ( μ , α ) and named it as the generalized entropy. It can be proved when α → 1 , H ( μ , α ) is Shannon entropy.
Fuzzy Clustering Based on Generalized Entropy and Its Application
643
3 Fuzzy Clustering and Image Segmentation Based on Generalized Entropy From the unified objective function（5）we know that m and α can take the different value. When m ≠ α , its analytic solution was very difficult to obtain. Here, we mainly consider the objective function with m= α , namely n
c
n
c
JG ( μij ,v)= μij m || x j − vi ||2 +δ (21−m −1)−1 (μij m −1)
(6)
j =1 i =1
j=1 i=1
3.1 Fuzzy Clustering Based on the Generalized Entropy
The problem of fuzzy clustering based on the generalized entropy is as follows:
m i n J G ( μ ij , v ) c
s .t .
i =1
(7)
μ ij = 1, j = 1, 2 , " , n
Now we use the Lagrange method to obtain the fuzzy degree of membership and the cluster center. The Lagrange function for optimization problem (7) is
L ( μ ij ,v , λ1 , " , λ n ) =
n
c
μ
n
m ij
j= 1 i= 1
|| x j − v i ||2 + δ
c
(2
j =1 i =1
c
c
i =1
i =1
1− m
− 1) − 1 ( μ ij m − 1)
+ λ1 ( μ i1 − 1) + " + λ n ( μ in − 1) Using
.
∂L ∂L = 0 , we obtain = 0 and ∂vi ∂μik
1
μik =
|| xk − vi || +δ (2 − 1) 2 1− m − 1) −1 j =1 k − v j || +δ (2 2
c
|| x
1− m
−1
1 m −1
, (8)
i = 1, 2, " , c; k = 1, 2, " , n n
vi =
j =1 n
μ ij m x j
j =1
, i = 1, 2 , " , c
μ ij m
The generalized entropy fuzzy clustering algorithm (GEFCM) is as follows:
(9)
644
K. Li and Y. Wang
，
， ， ε and the
Step 1 Fix the number of Clusters c assign proper m δ maximum iterations maxIter Step 2 Initialize the cluster centers vi (i = 1, 2,", c) . Step 3 while j<maxIter Update degree of membership using 8 ; Update the cluster centers using 9 ; Calculate Err =|| v j − v j −1 || break if Err< ε or else let j=j+1 end
（） （）
，
，
；
3.2 Image Segmentation Based on Generalized Entropy
By introducing the spatial information of image into the generalized entropy fuzzy clustering algorithm, we can use it for the segmentation of image, the specific objective functions is as follows: n
c
μ
J G S ( μ ij ,v) =
n
|| x j − v i ||2 + δ
m ij
+ β
c
μ
1− m
j = 1 i =1
j= 1 i= 1 n
c
(2
m ij
− 1) − 1 ( μ ij m − 1)
|| x j − v i ||2
.
j= 1 i= 1
c
s .t .
μ i =1
ij
= 1, j = 1, 2, " , n
Where x j is the mean of samples in xj’s window. The Lagrange function is as follows: n
c
n
c
L( μij ,v, λ1 , " , λn ) = μij m || x j − vi ||2 +δ (21− m − 1) −1 ( μij m − 1) j =1 i =1
j=1 i=1
n
c
c
c
i =1
i =1
+ β μij m || x j − vi ||2 + λ1 ( μ i1 − 1) + " + λn ( μ in − 1) j=1 i=1
Let
.
∂L ∂L = 0 , then = 0 and ∂vi ∂ μ ik 1
μik =
|| xk − vi || +δ (2 − 1) + β || x k − vi || 2 1− m − 1) −1 + β || x k − v j || j =1 k − v j || +δ (2 1− m
2
c
|| x
−1
1 m −1
, ,
(10)
i = 1, 2, " , c; k = 1, 2, " , n n
vi =
j =1
μ ij m ( x j + β x j ) n
(1 + β ) μ ij m j =1
, i = 1, 2 , " , c .
(11)
Fuzzy Clustering Based on Generalized Entropy and Its Application
645
In this paper, the method for image segmentation using (10) and (11) is named as GEFCMS.
4 Experimental Study In following experiments, we study two groups of experiments to test and verify the effectiveness of the generalized entropy clustering algorithm: (i) we mainly select the IRIS data set in UCI; (ii) we mainly selected the Coin, BinImage, Lena and MRI images and some images were added salt-and-pepper noise. The fixed algorithm [7] was used to select the initial center in both groups. In unsupervised learning, IRIS data set is often used to evaluate the performance of learning algorithm, there are 150 points and four characteristics in the data set that can be divided into three categories. One of the categories and the other two are completely separated, the data points of remaining two categories have some overlap. The study showed that the best performance of the existing clustering algorithm is with approximately 15 error clustering points [9]. Table 1 shows results of experiment on Iris data based on generalized entropy fuzzy clustering algorithm (GEFCM). NofErr is expressed as the number of error data point in clustering algorithm. The experimental results show that presented algorithm GEFCM has better performance and the number of error clustering are smaller than 15. At the same time, in the experimental study, we obtained better clustering results by selecting randomly initial center, where the number of error clustering data is only 7. Unfortunately the experimental results are not steady. Table 1. The performance of using different value m with GEFCM
m 4 6 11 13
δ 5 5 5 5
NofErr 14 13 12 11
In the experimental study of image segmentation, BinImage image is artificially generated binary image, Coin and MRI images are from Matlab itself and Lena image is chose from network. In the experiment, most images are added salt-and-pepper noises, the size of window is 3×3, and xj itself is not included when we calculate the sample mean x j in the window. The results are shown from Fig. 2 to Fig. 5. In the experimental results below, (a) is the original image or it is added salt-and-pepper noises. Figures with (b) and (c) are the results of image segmentation with fuzzy clustering algorithm and GEFCMS algorithm, respectively. The results of experiment show that we can obtain more satisfactory results for noisy images and noise-free images by using either GEFCMS, but noise cannot be removed by using FCM algorithm. Of course, the method in this paper involves some parameter setting, and different parameter may have different influence to the results.
646
K. Li and Y. Wang
(a)
(b)
(c)
Fig. 2. Results of the segmentation of BinImage using different methods
(a)
(b)
(c)
Fig. 3. Results of the segmentation of Coin image using different methods
(a)
(b)
(c)
Fig. 4. Results of the segmentation of MRI image using different methods
(a)
(b)
(c)
Fig. 5. Results of the segmentation of Lena image using different methods
5 Conclusions In this paper, we mainly propose the unified objective function of fuzzy clustering algorithm based on generalized entropy and present the generalized entropy fuzzy clustering algorithm. At the same time, we obtain image segmentation algorithm GEFCMS based on generalized entropy by introducing the spatial information of
Fuzzy Clustering Based on Generalized Entropy and Its Application
647
image into GEFCM. We study with experiment the effectiveness of clustering algorithm and image segmentation algorithm based on generalized entropy, and obtain satisfactory results. In the future, we need to further study generalized clustering algorithm in order to get some better results by choosing suitable parameters. Acknowledgments. This work is support by Natural Science Foundation of China (61073121) and Nature Science Foundation of Hebei Province (F2009000236).
References 1. Anil, K.J.: Data clustering: 50 years beyond K-means. Pattern Recognition Letters 31, 651– 666 (2010) 2. Karayiannis, N.B.: MECA maximum entropy clustering algorithm. In: Third IEEE Conference On Fuzzy Systems, pp. 630–635. IEEE Press, New York (1994) 3. Li, R.P., Mukaidon, M.: A maximum entropy approach to fuzzy clustering. In: Proceedings of Fourth IEEE International Conference on Fuzzy System, Yokohama, Japan, pp. 2227–2232 (1995) 4. Wei, C., Fahn, C.: The multisynapse neural network and its application to fuzzy clustering. Transactions on Neural Networks 13, 600–618 (2002) 5. Daniel, G., Witold, P.: Kernel-based fuzzy clustering and fuzzy clustering: A comparative experimental study. Fuzzy Sets and Systems 161, 522–543 (2010) 6. Yang, M.S., Tsai, H.S.: A gaussian kernel-based fuzzy c-means algorithm with a spatial bias correction. Pattern Recognition Letters 29, 1713–1725 (2008) 7. Kannan, S.R., Ramathilagam, S., Sathya, A., et al.: Effective fuzzyc-means based kernel function in segmenting medical images. Computers in Biology and Medicine 40, 572–579 (2010) 8. Swagatam, D., Sudeshna, S.: Kernel-induced fuzzy clustering of image pixels with an improved differential evolution algorithm. Information Sciences 180, 1237–1256 (2010) 9. Pal, N.R., Bezdek, J.C., Tsao, E.C.K.: Generalized clustering networks and kohonen’s selforganizing scheme. IEEE Trans. on Neural Networks 4, 549–557 (1993)
：
Abstract. Fuzzy clustering based on generalized entropy is studied. By introducing the generalized entropy into objective function of fuzzy clustering, a unified model is given for fuzzy clustering in this paper. Then fuzzy clustering algorithm based on the generalized entropy is presented. At the same time, by introducing the spatial information of image into the generalized entropy fuzzy clustering algorithm, an image segmentation algorithm is presented. Finally, experiments are conducted to show effectiveness of both clustering algorithm based on generalized entropy and image segmentation algorithm. Keywords: Generalized entropy, unified model, fuzzy clustering, image segmentation.
1 Introduction Clustering is an important tool for data analysis. It groups data objects into multiple classes or clusters, wherein data have much high similarity in same cluster whereas they have great difference in different cluster. One of the most commonly used methods is k-means clustering algorithm[1].Since Zadeh proposed the concept of fuzzy in 1965, people began to study fuzzy clustering algorithm. Ruspin first proposed Fuzzy C partition in 1969. In 1974, Dunn proposed a weighted index of m = 2 fuzzy C-means algorithm. And Bezdek extended to m> 1 of the fuzzy C-means algorithm in 1981. In reality, fuzzy clustering has been widely studied and applied in various fields, for example, image processing, pattern recognition, medical, etc. However, the fuzzy clustering algorithm (FCM) only deal with the points of the data set which have the same weight, number of data points in each cluster with no difference, and the loss of anti-noise properties, etc. In order to solve these problems, the researchers proposed many clustering algorithms. In 1979, Gustafson and Kessel proposed the clustering algorithm for clusters of different shapes by using the Mahalanobis distance. Dave extended the FCM by proposing the FCS algorithm which is used for the detection curve boundary in 1990; in 1991, Borowski and Bezdek introduced different norms to the FCM algorithm. In 1993, Yang put forward a kind of punished FCM (PFCM) based on fuzzy classification of maximum likelihood. In 1994, Karayiannis proposed the maximum entropy clustering algorithm H. Deng et al. (Eds.): AICI 2011, Part II, LNAI 7003, pp. 640–647, 2011. © Springer-Verlag Berlin Heidelberg 2011
Fuzzy Clustering Based on Generalized Entropy and Its Application
641
[2]; In 1995, aiming at the uncertainty problem, Li and others introduced the maximum entropy inference method and proposed the maximum entropy clustering algorithm by combining the loss function of the point to the center [3]; Krishnapuram and Keller proposed possibilistic C-means algorithm by relaxing the constraint that the sum of fuzzy C partition is equal to 1, to overcome the flaw of FCM under the noisy environment. In 2000, Tran and Wagner proposed the fuzzy entropy clustering algorithm using the objective function proposed by Li; In 2002, Wei and Fahn proposed a fuzzy bidirectional associative clustering network which solves the problem of fuzzy clustering [4]; In the same year, Wu and Yang proposed AFCM by using the index distance; in 2004, Wang and others developed a feature weighted FCM based on the feature selection methods; In 2005, Yu and Yang studied FCM and its expanded algorithm, and established a unified model of GFCM; In 2010 Graves and Pedrycz made a comparative experimental study of fuzzy C-means and kernelbased fuzzy C-means clustering algorithm, and obtained the conclusion that kernelbased clustering algorithm is sensitive to the kernel parameters. As for the image segmentation, spatial information is introduced into the clustering algorithm to improve the performance of image segmentation and the anti-noise performance of the algorithm. In 2002, Ahmed and others proposed BCFCM algorithm and successfully applied to the segmentation of MRI data by adjusting the objective function of FCM. Later, in order to reduce the spending time, Chen and Zhang proposed an improved BCFCM algorithm which is applied the kernel theory in BCFCM in 2004 and then they proposed the KFCM algorithm with spatial constraints. In the same year, Zhang and Chen proposed a new kernel-based fuzzy Cmeans algorithm which is applied to the medical image segmentation; in 2008, Yang and Tsai proposed the GKFCM algorithm which is successfully applied to image segmentation by further studying kernel-based clustering algorithm with a spatial bias correction [6]. In 2010, Kannan and others proposed fuzzy C-means based kernel function in segmenting medical images [7]. In this year, Swagatam and others researched Kernel-induced fuzzy clustering of image segmentation [8]. Fuzzy clustering based on entropy is mainly studied in this paper. We present a unified objective function based on generalized entropy in FCM. At the same time, fuzzy clustering algorithm based on generalized entropy is obtained, and they are applied to image segmentation. This paper is organized as follows. In section 2, we describe the proposed unified model in detail. In the following section, fuzzy clustering algorithm and image segmentation algorithm based on generalized entropy are given. We evaluate our method on IRIS data set and some chosed images. At the same time, we compare it with the classical FCM in section 4. Some conclusions are given in the final section.
2 The Unified Model Based on Generalized Entropy Fuzzy clustering algorithm is common method. It has widely been applied to many real problems. However, as it exist some problems, researchers present some clustering algorithm by revising objective function. In 1994, Karayiannis proposed the Maximum entropy clustering algorithm, which makes the clustering of data points from the maximum uncertainty gradually transformed into a deterministic process by
642
K. Li and Y. Wang
the introduction of the entropy of the membership and the distance to the center point in the clustering process. The objective function is defined as n
c
J( μ,a)=αμij log μij + j =1 i =1
1− α n c μij || xj − ai ||2 . n j=1 i=1
Now, by reducing the objective function (1) and let β =
(1)
1−α , 0 < α < 1 , above nα
equation is written as n
c
n
c
J( μ ,a) = μij log μij + β μij || x j − ai ||2 . j =1 i =1
(2)
j=1 i=1
In 1995, aiming at the uncertainty problem, Li and others introduced the maximum entropy inference method and proposed the maximum entropy clustering algorithm by combining the loss function of the point to the center. Its objective function is defined as n
c
n
c
J( μ ,a)= β −1 μij log μij + μij || x j − ai ||2 . j =1 i =1
(3)
j=1 i=1
Actually, the objective function (2) and (3) above is consistent. In 2000, Tran and Wagner proposed the fuzzy entropy clustering algorithm by using the objective function proposed by Li. In 2002, Wei and Fahn proposed a fuzzy bidirectional associative clustering network to solve the problem of fuzzy clustering with the objective function as follows n
c
n
c
J( μ ,a) = β μ ij log μ ij + μ ijm || x j − ai ||2 . j =1 i =1
(4)
j=1 i=1
Above objective functions are shown that they are basically similar each other in different clustering algorithms. In the following, we give a unified objective function n
c
n
c
JG ( μij ,v)= μij m || x j − vi ||2 +δ (21−α −1)−1 (μijα −1) j=1 i=1
j =1 i =1
.
(5)
α > 0,α ≠ 1 It is seen that when m = 1 and α → 1 , objective function above is become that proposed by Karayiannis, Li,Tran and Wagner. When α → 1 , objective function above is become that proposed by Wei and Fahn. We can see that main difference in the two cases above is the second term of the unified model, written it as H ( μ , α ) and named it as the generalized entropy. It can be proved when α → 1 , H ( μ , α ) is Shannon entropy.
Fuzzy Clustering Based on Generalized Entropy and Its Application
643
3 Fuzzy Clustering and Image Segmentation Based on Generalized Entropy From the unified objective function（5）we know that m and α can take the different value. When m ≠ α , its analytic solution was very difficult to obtain. Here, we mainly consider the objective function with m= α , namely n
c
n
c
JG ( μij ,v)= μij m || x j − vi ||2 +δ (21−m −1)−1 (μij m −1)
(6)
j =1 i =1
j=1 i=1
3.1 Fuzzy Clustering Based on the Generalized Entropy
The problem of fuzzy clustering based on the generalized entropy is as follows:
m i n J G ( μ ij , v ) c
s .t .
i =1
(7)
μ ij = 1, j = 1, 2 , " , n
Now we use the Lagrange method to obtain the fuzzy degree of membership and the cluster center. The Lagrange function for optimization problem (7) is
L ( μ ij ,v , λ1 , " , λ n ) =
n
c
μ
n
m ij
j= 1 i= 1
|| x j − v i ||2 + δ
c
(2
j =1 i =1
c
c
i =1
i =1
1− m
− 1) − 1 ( μ ij m − 1)
+ λ1 ( μ i1 − 1) + " + λ n ( μ in − 1) Using
.
∂L ∂L = 0 , we obtain = 0 and ∂vi ∂μik
1
μik =
|| xk − vi || +δ (2 − 1) 2 1− m − 1) −1 j =1 k − v j || +δ (2 2
c
|| x
1− m
−1
1 m −1
, (8)
i = 1, 2, " , c; k = 1, 2, " , n n
vi =
j =1 n
μ ij m x j
j =1
, i = 1, 2 , " , c
μ ij m
The generalized entropy fuzzy clustering algorithm (GEFCM) is as follows:
(9)
644
K. Li and Y. Wang
，
， ， ε and the
Step 1 Fix the number of Clusters c assign proper m δ maximum iterations maxIter Step 2 Initialize the cluster centers vi (i = 1, 2,", c) . Step 3 while j<maxIter Update degree of membership using 8 ; Update the cluster centers using 9 ; Calculate Err =|| v j − v j −1 || break if Err< ε or else let j=j+1 end
（） （）
，
，
；
3.2 Image Segmentation Based on Generalized Entropy
By introducing the spatial information of image into the generalized entropy fuzzy clustering algorithm, we can use it for the segmentation of image, the specific objective functions is as follows: n
c
μ
J G S ( μ ij ,v) =
n
|| x j − v i ||2 + δ
m ij
+ β
c
μ
1− m
j = 1 i =1
j= 1 i= 1 n
c
(2
m ij
− 1) − 1 ( μ ij m − 1)
|| x j − v i ||2
.
j= 1 i= 1
c
s .t .
μ i =1
ij
= 1, j = 1, 2, " , n
Where x j is the mean of samples in xj’s window. The Lagrange function is as follows: n
c
n
c
L( μij ,v, λ1 , " , λn ) = μij m || x j − vi ||2 +δ (21− m − 1) −1 ( μij m − 1) j =1 i =1
j=1 i=1
n
c
c
c
i =1
i =1
+ β μij m || x j − vi ||2 + λ1 ( μ i1 − 1) + " + λn ( μ in − 1) j=1 i=1
Let
.
∂L ∂L = 0 , then = 0 and ∂vi ∂ μ ik 1
μik =
|| xk − vi || +δ (2 − 1) + β || x k − vi || 2 1− m − 1) −1 + β || x k − v j || j =1 k − v j || +δ (2 1− m
2
c
|| x
−1
1 m −1
, ,
(10)
i = 1, 2, " , c; k = 1, 2, " , n n
vi =
j =1
μ ij m ( x j + β x j ) n
(1 + β ) μ ij m j =1
, i = 1, 2 , " , c .
(11)
Fuzzy Clustering Based on Generalized Entropy and Its Application
645
In this paper, the method for image segmentation using (10) and (11) is named as GEFCMS.
4 Experimental Study In following experiments, we study two groups of experiments to test and verify the effectiveness of the generalized entropy clustering algorithm: (i) we mainly select the IRIS data set in UCI; (ii) we mainly selected the Coin, BinImage, Lena and MRI images and some images were added salt-and-pepper noise. The fixed algorithm [7] was used to select the initial center in both groups. In unsupervised learning, IRIS data set is often used to evaluate the performance of learning algorithm, there are 150 points and four characteristics in the data set that can be divided into three categories. One of the categories and the other two are completely separated, the data points of remaining two categories have some overlap. The study showed that the best performance of the existing clustering algorithm is with approximately 15 error clustering points [9]. Table 1 shows results of experiment on Iris data based on generalized entropy fuzzy clustering algorithm (GEFCM). NofErr is expressed as the number of error data point in clustering algorithm. The experimental results show that presented algorithm GEFCM has better performance and the number of error clustering are smaller than 15. At the same time, in the experimental study, we obtained better clustering results by selecting randomly initial center, where the number of error clustering data is only 7. Unfortunately the experimental results are not steady. Table 1. The performance of using different value m with GEFCM
m 4 6 11 13
δ 5 5 5 5
NofErr 14 13 12 11
In the experimental study of image segmentation, BinImage image is artificially generated binary image, Coin and MRI images are from Matlab itself and Lena image is chose from network. In the experiment, most images are added salt-and-pepper noises, the size of window is 3×3, and xj itself is not included when we calculate the sample mean x j in the window. The results are shown from Fig. 2 to Fig. 5. In the experimental results below, (a) is the original image or it is added salt-and-pepper noises. Figures with (b) and (c) are the results of image segmentation with fuzzy clustering algorithm and GEFCMS algorithm, respectively. The results of experiment show that we can obtain more satisfactory results for noisy images and noise-free images by using either GEFCMS, but noise cannot be removed by using FCM algorithm. Of course, the method in this paper involves some parameter setting, and different parameter may have different influence to the results.
646
K. Li and Y. Wang
(a)
(b)
(c)
Fig. 2. Results of the segmentation of BinImage using different methods
(a)
(b)
(c)
Fig. 3. Results of the segmentation of Coin image using different methods
(a)
(b)
(c)
Fig. 4. Results of the segmentation of MRI image using different methods
(a)
(b)
(c)
Fig. 5. Results of the segmentation of Lena image using different methods
5 Conclusions In this paper, we mainly propose the unified objective function of fuzzy clustering algorithm based on generalized entropy and present the generalized entropy fuzzy clustering algorithm. At the same time, we obtain image segmentation algorithm GEFCMS based on generalized entropy by introducing the spatial information of
Fuzzy Clustering Based on Generalized Entropy and Its Application
647
image into GEFCM. We study with experiment the effectiveness of clustering algorithm and image segmentation algorithm based on generalized entropy, and obtain satisfactory results. In the future, we need to further study generalized clustering algorithm in order to get some better results by choosing suitable parameters. Acknowledgments. This work is support by Natural Science Foundation of China (61073121) and Nature Science Foundation of Hebei Province (F2009000236).
References 1. Anil, K.J.: Data clustering: 50 years beyond K-means. Pattern Recognition Letters 31, 651– 666 (2010) 2. Karayiannis, N.B.: MECA maximum entropy clustering algorithm. In: Third IEEE Conference On Fuzzy Systems, pp. 630–635. IEEE Press, New York (1994) 3. Li, R.P., Mukaidon, M.: A maximum entropy approach to fuzzy clustering. In: Proceedings of Fourth IEEE International Conference on Fuzzy System, Yokohama, Japan, pp. 2227–2232 (1995) 4. Wei, C., Fahn, C.: The multisynapse neural network and its application to fuzzy clustering. Transactions on Neural Networks 13, 600–618 (2002) 5. Daniel, G., Witold, P.: Kernel-based fuzzy clustering and fuzzy clustering: A comparative experimental study. Fuzzy Sets and Systems 161, 522–543 (2010) 6. Yang, M.S., Tsai, H.S.: A gaussian kernel-based fuzzy c-means algorithm with a spatial bias correction. Pattern Recognition Letters 29, 1713–1725 (2008) 7. Kannan, S.R., Ramathilagam, S., Sathya, A., et al.: Effective fuzzyc-means based kernel function in segmenting medical images. Computers in Biology and Medicine 40, 572–579 (2010) 8. Swagatam, D., Sudeshna, S.: Kernel-induced fuzzy clustering of image pixels with an improved differential evolution algorithm. Information Sciences 180, 1237–1256 (2010) 9. Pal, N.R., Bezdek, J.C., Tsao, E.C.K.: Generalized clustering networks and kohonen’s selforganizing scheme. IEEE Trans. on Neural Networks 4, 549–557 (1993)
：
