Entropy Operator for Membership Function of ... - Semantic Scholar
Entropy Operator for Membership Function of Uncertain Set Kai Yao1 , Hua Ke2∗ 1. School of Management, University of Chinese Academy of Sciences, Beijing 100190, China 2. School of Economics and Management, Tongji University, Shanghai 200092, China [email protected], ∗ corresponding author: [email protected]
Abstract Similar to fuzzy set on a possibility space, uncertain set is a set-valued function on an uncertainty space, and attempts to model unsharp concepts. Entropy provides a quantitative measurement of the uncertainty associated with an uncertain set. This paper presents a formula for calculating the entropy of an uncertain set via its inverse membership function. Based on the formula, the entropy operator is shown to satisfy positive linearity property. In addition, this paper proposes a concept of relative entropy to describe the divergence between the membership functions of two uncertain sets. Keywords: uncertain set; membership function; entropy; relative entropy
1
Introduction
Entropy is used to measure the degree of possibility associated with an indeterminacy phenomena. It was first presented by Shannon [32] for a random variable in 1949. In order to model the divergence between two random variables, a concept of relative entropy was proposed by Kullback and Leibler [17] in 1951. As we know, when the expected value and variance of a random variable are given, we can find many probability distributions for the random variable. In 1957, Jaynes [13] suggested to choose the one with the maximum entropy as the probability distribution of the random variable in practice, that is the maximum entropy principle nowadays. Although probability has been used to model indeterminacy phenomena for a long time, due to the incompleteness of the available information, sometimes it is difficult to get the precise probability distribution of an indeterminacy quantity. In this case, we have to rely on the experts’ belief degree that each event will occur. So far, many theories has been proposed to deal with the belief degree, such as fuzzy theory (Zadeh [40]), Dempster-Shafer theory (Dempster [7], Shafer [31]), and rough theory (Pawlak [29]). Entropy was first introduced to fuzzy set by Zadeh [41] to quantify the fuzziness in 1968, in which he defined the entropy of a fuzzy set as a weighted Shannon entropy. After that, based on four
1
requirements that an entropy should satisfy, De Luca and Termini [12] refined the fuzzy entropy in form of functions of Shannon entropy. In 1975, Kaufmann [14] attempted to measure the fuzziness of a fuzzy set via the distance between the fuzzy set and its closest classic set. After that, Yager [34, 35] studied the fuzziness contained in a fuzzy proposition. In 1986, Kosko [16] employed fuzzy entropy to measure the information in a fuzzy set. For a survey of fuzzy entropy, please refer to Pal and Bezdek [28]. The concept of entropy in the framework of Dempster-Shafer theory was first proposed by Yager [36] in 1983. Then in 1987, Yager [37] discussed the entropy of belief structure. After that, Klir and Ramer [15] and Harmanec and Klir [11] proposed a measure of discord to describe the uncertain evidence, and discussed its main properties. An application of entropy in Dempster-Shafer theory to multi-sensor information fusion was given by Basir and Yuan [1] in 2007. In 1998, D¨ untsch and Gediga [8] and Beaubouef et al. [2] proposed two different methods to measure the uncertainty of rough sets. Then Liang et al. [18, 19] defined conditional entropy and rough entropy for the rough information. An application of rough entropy to feature selection and recognition was given by Swiniarski and Skowron [33] in 2003. In 2007, an uncertainty theory was founded by Liu [20] to deal with the uncertainty associated with the experts’ belief degree. Then in 2010, Liu [22] refined the uncertainty theory based on normality, duality, subadditivity and product axioms. In order to model the uncertain quantities, Liu [20] proposed a concept of uncertain variable. Then Peng and Iwamura [27] gave a sufficient and necessary condition for a function being the uncertainty distribution of an uncertain variable. You [39] discussed the convergence of a sequence of uncertain variables. Qin and Kar [30] gave an application in inventory model. In 2009, Liu [21] provided a concept of entropy for uncertain variable in the form of logarithm function. The maximum entropy principle for uncertain variable was proposed by Chen and Dai [4] in 2011, and the properties of entropy for uncertain variables were investigated by Dai and Chen [6] in 2012. In order to measure the difference between two uncertain variables, Chen et al. [5] introduced relative entropy to uncertain variables. In addition, Yao et al. [38] proposed entropy in the form of sine function for uncertain variables. Uncertain set, as a generalization of uncertain variable, was proposed by Liu [23] in 2010 as a setvalued function on an uncertainty space. Inspired by fuzzy set, a membership function was defined to describe the uncertain measure that a real number belongs to an uncertain set. In 2010, Gao et al. [9] proposed some inference rules with multiple antecedents based on uncertain sets, which was further applied to balance the inverted pendulum by Gao [10]. Besides, Liu [24] applied the uncertain sets to the field of linguistic summarizer. In 2012, Liu [25] recast the concept of uncertain set. The concept of entropy for uncertain sets was proposed by Liu [24] in 2011. In this paper, we will give a formula to calculate the entropy of uncertain set via inverse membership function, based on which we will also verify the positive linearity of the entropy operator. In addition, we will propose a concept
2
of relative entropy to describe the divergence between the membership functions of two uncertain sets. The rest of this paper is structured as follows. The next section intends to introduce some concepts about uncertain set. After that, we introduce the entropy for uncertain set in Section 3. In section 4, we provide a formula for calculating the entropy of an uncertain set via inverse membership function, based on which we verify the positive linearity of entropy in Section 5. The relative entropy for uncertain sets is presented in Section 6. At last, some remarks are made in Section 7.
2
Preliminary
This section will introduce some basic concepts and theorems about uncertain set that will used throughout the paper. In order to provide a quantitative measurement that an uncertain phenomenon will occur, an uncertain measure was defined as below. Definition 1. (Liu [20]) Let L be a σ-algebra on a nonempty set Γ. A set function M : L → [0, 1] is called an uncertain measure if it satisfies the following axioms: Axiom 1: (Normality Axiom) M{Γ} = 1 for the universal set Γ. Axiom 2: (Duality Axiom) M{Λ} + M{Λc } = 1 for any event Λ. Axiom 3: (Subadditivity Axiom) For every countable sequence of events Λ1 , Λ2 , · · · , we have (∞ ) ∞ X [ M {Λi } . M Λi ≤ i=1
i=1
The triple (Γ, L, M) is called an uncertainty space. The product uncertain measure on the product σalgebra L is defined by Liu [21] as follows, Axiom 4: (Product Axiom) Let (Γk , Lk , Mk ) be uncertainty spaces for k = 1, 2, · · · Then the product uncertain measure M on the product σ-algebra satisfies (∞ ) ∞ Y ^ M Λk = Mk {Λk } i=1
k=1
where Λk are arbitrarily chosen events from Lk for k = 1, 2, · · · , respectively. For example, assume λ(x) is a nonnegative function on the real number set < such that sup(λ(x) + λ(y)) = 1. x6=y
For each event Λ in the Borel set B, the set function sup λ(x), x∈Λ M{Λ} = 1 − supc λ(x), x∈Λ
if sup λ(x) < 0.5 x∈Λ
if sup λ(x) ≥ 0.5 x∈Λ
is an uncertain measure, and (
Abstract Similar to fuzzy set on a possibility space, uncertain set is a set-valued function on an uncertainty space, and attempts to model unsharp concepts. Entropy provides a quantitative measurement of the uncertainty associated with an uncertain set. This paper presents a formula for calculating the entropy of an uncertain set via its inverse membership function. Based on the formula, the entropy operator is shown to satisfy positive linearity property. In addition, this paper proposes a concept of relative entropy to describe the divergence between the membership functions of two uncertain sets. Keywords: uncertain set; membership function; entropy; relative entropy
1
Introduction
Entropy is used to measure the degree of possibility associated with an indeterminacy phenomena. It was first presented by Shannon [32] for a random variable in 1949. In order to model the divergence between two random variables, a concept of relative entropy was proposed by Kullback and Leibler [17] in 1951. As we know, when the expected value and variance of a random variable are given, we can find many probability distributions for the random variable. In 1957, Jaynes [13] suggested to choose the one with the maximum entropy as the probability distribution of the random variable in practice, that is the maximum entropy principle nowadays. Although probability has been used to model indeterminacy phenomena for a long time, due to the incompleteness of the available information, sometimes it is difficult to get the precise probability distribution of an indeterminacy quantity. In this case, we have to rely on the experts’ belief degree that each event will occur. So far, many theories has been proposed to deal with the belief degree, such as fuzzy theory (Zadeh [40]), Dempster-Shafer theory (Dempster [7], Shafer [31]), and rough theory (Pawlak [29]). Entropy was first introduced to fuzzy set by Zadeh [41] to quantify the fuzziness in 1968, in which he defined the entropy of a fuzzy set as a weighted Shannon entropy. After that, based on four
1
requirements that an entropy should satisfy, De Luca and Termini [12] refined the fuzzy entropy in form of functions of Shannon entropy. In 1975, Kaufmann [14] attempted to measure the fuzziness of a fuzzy set via the distance between the fuzzy set and its closest classic set. After that, Yager [34, 35] studied the fuzziness contained in a fuzzy proposition. In 1986, Kosko [16] employed fuzzy entropy to measure the information in a fuzzy set. For a survey of fuzzy entropy, please refer to Pal and Bezdek [28]. The concept of entropy in the framework of Dempster-Shafer theory was first proposed by Yager [36] in 1983. Then in 1987, Yager [37] discussed the entropy of belief structure. After that, Klir and Ramer [15] and Harmanec and Klir [11] proposed a measure of discord to describe the uncertain evidence, and discussed its main properties. An application of entropy in Dempster-Shafer theory to multi-sensor information fusion was given by Basir and Yuan [1] in 2007. In 1998, D¨ untsch and Gediga [8] and Beaubouef et al. [2] proposed two different methods to measure the uncertainty of rough sets. Then Liang et al. [18, 19] defined conditional entropy and rough entropy for the rough information. An application of rough entropy to feature selection and recognition was given by Swiniarski and Skowron [33] in 2003. In 2007, an uncertainty theory was founded by Liu [20] to deal with the uncertainty associated with the experts’ belief degree. Then in 2010, Liu [22] refined the uncertainty theory based on normality, duality, subadditivity and product axioms. In order to model the uncertain quantities, Liu [20] proposed a concept of uncertain variable. Then Peng and Iwamura [27] gave a sufficient and necessary condition for a function being the uncertainty distribution of an uncertain variable. You [39] discussed the convergence of a sequence of uncertain variables. Qin and Kar [30] gave an application in inventory model. In 2009, Liu [21] provided a concept of entropy for uncertain variable in the form of logarithm function. The maximum entropy principle for uncertain variable was proposed by Chen and Dai [4] in 2011, and the properties of entropy for uncertain variables were investigated by Dai and Chen [6] in 2012. In order to measure the difference between two uncertain variables, Chen et al. [5] introduced relative entropy to uncertain variables. In addition, Yao et al. [38] proposed entropy in the form of sine function for uncertain variables. Uncertain set, as a generalization of uncertain variable, was proposed by Liu [23] in 2010 as a setvalued function on an uncertainty space. Inspired by fuzzy set, a membership function was defined to describe the uncertain measure that a real number belongs to an uncertain set. In 2010, Gao et al. [9] proposed some inference rules with multiple antecedents based on uncertain sets, which was further applied to balance the inverted pendulum by Gao [10]. Besides, Liu [24] applied the uncertain sets to the field of linguistic summarizer. In 2012, Liu [25] recast the concept of uncertain set. The concept of entropy for uncertain sets was proposed by Liu [24] in 2011. In this paper, we will give a formula to calculate the entropy of uncertain set via inverse membership function, based on which we will also verify the positive linearity of the entropy operator. In addition, we will propose a concept
2
of relative entropy to describe the divergence between the membership functions of two uncertain sets. The rest of this paper is structured as follows. The next section intends to introduce some concepts about uncertain set. After that, we introduce the entropy for uncertain set in Section 3. In section 4, we provide a formula for calculating the entropy of an uncertain set via inverse membership function, based on which we verify the positive linearity of entropy in Section 5. The relative entropy for uncertain sets is presented in Section 6. At last, some remarks are made in Section 7.
2
Preliminary
This section will introduce some basic concepts and theorems about uncertain set that will used throughout the paper. In order to provide a quantitative measurement that an uncertain phenomenon will occur, an uncertain measure was defined as below. Definition 1. (Liu [20]) Let L be a σ-algebra on a nonempty set Γ. A set function M : L → [0, 1] is called an uncertain measure if it satisfies the following axioms: Axiom 1: (Normality Axiom) M{Γ} = 1 for the universal set Γ. Axiom 2: (Duality Axiom) M{Λ} + M{Λc } = 1 for any event Λ. Axiom 3: (Subadditivity Axiom) For every countable sequence of events Λ1 , Λ2 , · · · , we have (∞ ) ∞ X [ M {Λi } . M Λi ≤ i=1
i=1
The triple (Γ, L, M) is called an uncertainty space. The product uncertain measure on the product σalgebra L is defined by Liu [21] as follows, Axiom 4: (Product Axiom) Let (Γk , Lk , Mk ) be uncertainty spaces for k = 1, 2, · · · Then the product uncertain measure M on the product σ-algebra satisfies (∞ ) ∞ Y ^ M Λk = Mk {Λk } i=1
k=1
where Λk are arbitrarily chosen events from Lk for k = 1, 2, · · · , respectively. For example, assume λ(x) is a nonnegative function on the real number set < such that sup(λ(x) + λ(y)) = 1. x6=y
For each event Λ in the Borel set B, the set function sup λ(x), x∈Λ M{Λ} = 1 − supc λ(x), x∈Λ
if sup λ(x) < 0.5 x∈Λ
if sup λ(x) ≥ 0.5 x∈Λ
is an uncertain measure, and (
