A Trainable Transparent Universal Approximator For ... - IEEE Xplore
304
IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 6, NO. 2, MAY 1998
A Trainable Transparent Universal Approximator for Defuzzification in Mamdani-Type Neuro-Fuzzy Controllers Saman K. Halgamuge
Abstract— A novel technique of designing application specific defuzzification strategies with neural learning is presented. The proposed neural architecture considered as a universal defuzzification approximator is validated by showing the convergence when approximating several existing defuzzification strategies. The method is successfully tested with fuzzy controlled reverse driving of a model truck. The transparent structure of the universal defuzzification approximator allows to analyze the generated customized defuzzification method using the existing theories of defuzzification. The integration of universal defuzzification approximator instead of traditional methods in Mamdani-type fuzzy controllers can also be considered as an addition of trainable nonlinear noise to the output of the fuzzy rule inference before calculating the defuzzified crisp output. Therefore, nonlinear noise trained specifically for a given application shows a grade of confidence on the rule base, providing an additional opportunity to measure the quality of the fuzzy rule base. The possibility of modeling a Mamdani-type fuzzy controller as a feed-forward neural network with the ability of gradient descent training of the universal defuzzification approximator and antecedent membership functions fulfill the requirement known from multilayer preceptrons in finding solutions to nonlinear separable problems.
Fig. 1. Standard defuzzification strategies.
• Center of Gravity (COG)—the center of gravity of the area
Index Terms—Defuzzification, fuzzy control, fuzzy neural networks, mechatronics.
(1) I. INTRODUCTION
S
INCE the initial theoretical work of Zadeh [1] and early practical applications of fuzzy control [2], numerous defuzzification strategies have been proposed by several authors [3], [4]. The result of fuzzy rule inference and composition known as the fuzzy output is transformed back to a crisp output by defuzzification [5]. Although most of the methods are proposed in connection with Mamdani-type fuzzy systems [2], defuzzification is an essential step in all fuzzy system models such as Takagi–Sugeno–Kang (TSK) [6], [7], FuNe I [8], Tsukamoto [9] with the exception of classifier-type [10] fuzzy systems in which the fuzzy output is taken as the class membership. and denoting Assuming normalized membership values as the finite set of possible normalized output values of a where Mamdani-type fuzzy controller represent the discrete set of corresponding membership values. The resulting output obtained by applying several existing defuzzification schemes to an example can be shown in Fig. 1.
• Midpoint of Area (MOA)—the middle of the area where (2) • Mean of Maxima (MoM), the center of gravity of the area under the maxima of fuzzy output
(3) where
• Center of Mean (COM), the middle of the area under the maxima of fuzzy output where
Manuscript received September 28, 1995; revised May 7, 1997. The author is with the Department of Mechanical and Manufacturing Engineering, Mechatronics Research Group, The University of Melbourne, Parkville, Vic 3052, Australia. Publisher Item Identifier S 1063-6706(98)03276-7. 1063–6706/98$10.00 1998 IEEE
and (4)
HALGAMUGE: TRAINABLE TRANSPARENT UNIVERSAL APPROXIMATOR FOR DEFUZZIFICATION
From the neural network perspective defuzzification is a multidimensional function approximation problem that can be solved. But the neural networks trained to solve are mostly of blackbox type. If representative data sets exist, such neural networks can be trained to approximate any of above mentioned explicitly known standard defuzzification methods [11]. For example a transparent nontrainable neural structure is used to implement the method COG in [12] in constructing a prototype neuro-fuzzy system. Another approach is to use a conventional-type neural network to map fuzzy output into crisp output without considering the formal methods of defuzzification [8], [13]. Those neural networks are universal defuzzification approximators with the strong drawback of blackbox nature. The main contribution of this paper is the introduction of a universal defuzzification approximator that is parameterizable and transparent ,with the capability of approximating the defuzzification to an application specific solution or to select the best one from the known methods. Problems caused in applying existing defuzzification schemes are outlined in Section II. Section III presents the new trainable universal defuzzification approximator giving more emphasis to its implementation as a transparent neural network. The validation of the proposed method by training it to approximate different existing defuzzification methods and the application to a control example are described in Sections IV and V. The integration of the proposed method to the fuzzy neural controller FuNe II is outlined in Section VI. Section VII summarizes the paper. II. APPLICATION OF EXISTING DEFUZZIFICATION STRATEGIES The defuzzification methods MOA and COM are easily implementable in software as well as in hardware [14] using and covering the output range starting two pointers from the lower and the upper limit, respectively, and moving and stepwise toward the middle calculating the left area the right area as depicted in Fig. 2 for MOA. However, the possible error (half of the discrete step selected) should be taken into account. For a more accurate solution in an analog hardware or software implementation with continuous crisp output range, two different cases need to be considered for nonzero fuzzy output between the pointers in the final step. 1) Both pointers correspond to the same membership value [Fig. 3(a)]. 2) Pointers correspond to two different membership values and [Fig. 3(b)]. The following equations can be obtained considering both cases of MOA in Fig. 3 and only the first case for COM: (5) (6) (7) (8)
305
Fig. 2. MOA defuzzification.
where is the step size, , and , are
is the correction to be added to
For the case where the fuzzy output is zero (or for COM) between the pointers, either of the pointers could be selected as the output. The widely used COG method is not appropriate in some conflicting situations. An example is shown in Fig. 1. In this case, rules are active to the left and the right from the center. One can imagine this case as moving a vehicle toward possible paths in the left and the right with an obstacle in the middle. COG and MoM defuzzification methods deliver the most inappropriate solutions somewhere in the middle, whereas MOA and COM deliver though not similar but acceptable solutions. In case of having two equal possibilities to the left and to the right, i.e., both membership functions are equal, then either the right corner of the “left” membership block or the left corner of the “right” membership block has to be taken as the defuzzified value according to COM and MOA. Even MOA delivers an inappropriate crisp output along with COG and MoM if the obstacle is replaced by a less favorable swampy path causing a inference strength of “ ” at the output where in Fig. 1. The winner in this example is COM still delivering an acceptable solution, but is undesirable as a universal method due to its high small signal amplification similar to that of MoM as described in [15]. Even though the vehicle can move straight forward in the swampy path, the desirable way would be to the left or to the right. These situations can be avoided by restricting the membership value within the curve at least equal to the swamp constant. Apart from the basic defuzzification strategies mentioned above, a number of extended methods have been proposed by several authors. Most of them do not provide algorithms for application specific parameter training, but poses advantages in hardware implementation [16] and show improved static and dynamic properties [15], [17]. The parameterizable extended methods proposed extended center of area (XCOA) [15] and basic defuzzification distributions (BADD) [18] use the nonnegative parameter ,
306
IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 6, NO. 2, MAY 1998
(a)
(b)
Fig. 3. Calculation of MOA. (a) Both pointers correspond to the same membership value values 1 and 2 .
1 .
(b) Pointers correspond to two different membership
“confidence measure” [19] (9) where
(10)
A fuzzy set to the power of is a “concentration” if and a “dilation” if [20]. The COG method is extended in BADD and the MOA in XCOA, both replacing by in (1) and (2), respectively. It is obvious that (11) (12) (13) (14)
Fig. 4. Different regions in the confidence chart.
It must be noted that in case of reaching no confidence (15) is the defuzzified output by any of the methods where described above, when the universe of discourse is equally possible ( constant). The application of different defuzzification methods may lead to completely different crisp outputs depending on the shape of the fuzzy outputs (e.g., Fig. 1). In some situations none of the mentioned defuzzification methods delivers satisfactory results. Even the existing extended strategies are not successful in solving the problem with swampy path with the exemption of COM which has several other undesirable properties. Therefore, tuning to an application or customization of the defuzzification strategy is an interesting alternative to the conventional trial and error methods used. III. A TRAINABLE UNIVERSAL DEFUZZIFICATION APPROXIMATOR The proposed trainable method customisable basic defuzzification distributions (CBADD) is a useful extension of the
well-known BADD strategy that can be validated as a universal defuzzification approximator by neural network training. We define the set variable confidence measure , where is a real value. The confidence on the fuzzy output values can be divided into different regions as shown in Fig. 4. CBADD strategy can be described as
(16)
used in XCOA In contrast to the confidence measure and BADD, variable confidence measure can be negative. is negative, . It can be found an Whenever alternative nonnegative , in an effort by scaling to bring back the maximum of below a certain level (in the normalized case below one) so that function provides the . Scaling the fuzzy output means same defuzzified output as a complete change in the rule inference. Therefore, a trained CBADD neural network containing weights in the negative
HALGAMUGE: TRAINABLE TRANSPARENT UNIVERSAL APPROXIMATOR FOR DEFUZZIFICATION
307
Fig. 5. CBADD transparent neural network.
confidence region indicates the inconsistency of the existing rule base, hence, a warning to the user to check the rule base. in BADD and the set in The inclusion of constant CBADD can also be considered as an addition of nonlinear noise to the fuzzy output curve before defuzzification. The extension in CBADD is more flexible and eases the neural network based training.
inverted sum calculated in the lower neuron by using a neuron. The network can be resolved mathematically to match the CBADD equation due to its transparent structure. The activation functions of neurons employed in CBADDnetwork (linear , inverse , exponential , and logarithmic ) must satisfy a few rules [21] especially in the ranges of usage to assure convergence
A. Implementation as a Transparent Neural Network
(17)
CBADD uses a special transparent neural network with number of inputs (netwidth), each of them feeding a , the th discrete value of the fuzzy output, where in order to approximate the crisp output for . It consists of four consecutive layers of neurons, each layer consisting of different types of neurons concerning the calculation of net input and activation function (Fig. 5). The first three layers have summing net inputs and only the output neuron has multiplying inputs. The first layer consists of neurons having logarithmic activation functions and the connections to the following layer are weighted by the components of variable confidence . They are the only trainable weights in the CBADD network. Each neuron in the second layer has an exponential activation function, which partly eliminates the logarithm of the first layer and delivers . Within the first two layers there are no cross connections, but only connections among the neurons of the same net index. In the third neuron layer all outputs are fed into two neurons that build a weighted sum. The incoming connections to the upper neuron with linear activation is weighted by , the discrete values of the possible range of outputs (in Fig. 5 is 0.0, 0.1, 0.2, 0.9, 1.0 ). The lower neuron has unity incoming for weights and an inverting activation function. In the last layer, the resulting sum of the upper neuron is multiplied with the
(18) (19) (20) The linear activation function is only used as a “dummy” , the weight and it does not play a role in calculating change for . The inverse activation function is used with the fact that net input to the neuron is always nonzero (see Table I). The exponential activation function is inappropriate for gradient descent learning whenever net input is very high negative. Since this is not the case in the CBADD network, the exponential activation function can be used. To assure that net input to the neurons with logarithmic activation functions is always nonzero, a minimum limit for the fuzzy output is defined such that B. Alternative Neuron Models in CBADD Network In building the CBADD network, different types of neurons were used. Apart from the standard-type neurons with linear or sigmoid functions as activation functions and summation to calculate the weighted net input (net), neurons with multiplying net inputs , and those having
308
IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 6, NO. 2, MAY 1998
FULFILLMENT
TABLE I OF THE REQUIREMENTS
Fig. 6. Gradient descent learning.
and
for neurons with multiplying net inputs, i.e.,
(26) logarithmic , exponential , or inverse activation functions are used. Fig. 6 shows three consecutive neuron layers and having neuron indexes and respectively. The connection weight of the th neuron in the th layer and the th neuron in the layer is denoted as . Let us consider as the output layer and others as hidden layers. Using the least-mean square method (LMS) and the gradient descent algorithm [21], the total error (E) at the output can be calculated as (21)
Calculation of for an output neuron is independent of the calculation of net input, from (22) we conclude that hence (27) Calculating the partial derivative for hidden neurons depends on , , and the net input calculation method of the following neurons with index
(22) (28) where is the error occurred for a single pattern, is the desired output and is the current output of neuron . Assuming that can be a neuron in any of the layers, the by definition [21] is weight change
Assuming neurons in the layer we obtain
use summing net inputs,
(23) Considering , the learning rate, as the constant of proportionality, we can write (24)
(29) and assuming neurons in the layer inputs, we can write
use multiplying net
The function used for calculation of net inputs of the th neuron ( or ) influences the calculation of the partial for neurons with summing net inputs, so that derivative
(25)
(30)
HALGAMUGE: TRAINABLE TRANSPARENT UNIVERSAL APPROXIMATOR FOR DEFUZZIFICATION
309
The validation of the CBADD network is done by training the network with pattern sets generated with the defuzzification strategies COG and MoM [22]. Similar to (11) and (12), it is obvious that can be reduced to by setting all to one and it can be reduced to the elements in set by setting those elements virtually to . We approximate these mathematically exact solutions with neural network training and consider it as a validation for the convergence of the system. A. Validation with Known Defuzzification Methods
(a)
(b) Fig. 7. Sample pattern sets for CBADD training. (a) Pattern set 1. (b) Pattern set 2.
IV. CONVERGENCE OF THE CBADD NETWORK The convergence of the CBADD network is assured by the fulfilment of the requirements discussed in Section III-A and the derivations for the alternative neuron models found in CBADD networks as described in Section III-B. Denoting the layers in Fig. 5 from one (left most layer) to four (right most layer), starting the index for the neurons from the top neuron (neuron 1), applying (27) for the output neuron of the CBADD network (31) and simplifying (30) we can obtain (32) (33) It can be derived from (29) that
(34) Applying (25) and (34) in (24) we obtain (35)
inputs in Several sample pattern sets are used with the range from ALMOST ZERO to 1.0 and one output range . We use 2000 different patterns that from 0.0 to are applied over 1000 sweeps; that is 1000 times. Furthermore, we use single-mode backpropagation and update the weights after each single pattern. Each pattern represents one epoch. The system has been tested with different pattern sets, each defuzzified with the strategy of interest. This paper reports the simulation results only for two of them: pattern sets 1 and 2, which are similar to real fuzzy output variables (Fig. 7). The general strategy was to train the network with data sets having 32 inputs and an output representing the discrete values of fuzzy output values and the crisp output calculated according to one of the standard defuzzification algorithms. The trained CBADD network should show in case of convergence good results on an entirely different test data evolved must be identical set and the confidence values with the expected values. The randomly generated patterns in set 1 are evenly distributed and, therefore, have their and values often close to the middle of the output range. The pattern set 2 has fuzzy output curves composed of five random heights that are bounded in the vertical direction by the limits of the fuzzy output range and in the horizontal direction by two randomly selected edge values between 0.0 and . The resulting pattern sets can be of a rectangular shape as well as of a triangular or multi-trapezoidal shape. Therefore, and values are almost randomly distributed. Exploiting the advantages of the transparency of the CBADD-network, the confidence charts obtained using trained weights can be analyzed. The confidence charts of different existing defuzzification methods for pattern set 2 shown in Fig. 8 are similar to those of pattern set 1. 1) Validation with COG: The training of patterns defuzzified with COG strategy converges successfully, leading to weights in the range of 1.0 0.01 for pattern set 1 and set 2 [Fig. 8(a)]. Pattern set 1 shows a slightly slower convergence than pattern set 2. After 1000 sweeps the RMSerror for sets 1 and 2 are 0.004% and 0.0002%, respectively. The mathematically correct solution for is 1.0. Therefore, we see that CBADD approximates the set for the COG in the training phase. When trained over a longer period of time (10 000 sweeps) the tendency for the weights is toward an asymptote at one and an root—mean square error of zero. 3) Validation with MoM: When the CBADD network is trained with pattern sets defuzzified by the MoM strategy, the
310
IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 6, NO. 2, MAY 1998
(a)
(b)
(c)
(d)
Fig. 8. Confidence chart for pattern set 2. (a) COG strategy. (b) MoM strategy. (c) MOA strategy. (d) COM strategy.
weights are distributed in the range of for pattern set 1 and in the range of for pattern set 2 [Fig. 8(b)]. The RMS error for training the first set is 1.46% and that for the second is 0.49%. In this case, the mathematically correct is , a value that cannot be reached in solution for a computer simulation. Therefore, we conclude that CBADD can sufficiently approximate MOM since confidence values are well above 20. As shown by the simulation results, CBADD can successfully represent the defuzzification strategies COG and MOM and the implementation has herewith shown to be correct. The trained CBADD approximations for MOM and COG are successfully tested with different data sets. V. APPLICATION
OF
CBADD NETWORK
Due to the flexibility of the CBADD concept, any of the existing defuzzification method can be approximated, i.e., a can be found either mathematically as discrete set of shown in Section IV-A or by training the CBADD network
Fig. 9. Truck and trailer.
with numerical data. Furthermore, data sets obtained from an model truck application are also used for creating customized defuzzification methods.
HALGAMUGE: TRAINABLE TRANSPARENT UNIVERSAL APPROXIMATOR FOR DEFUZZIFICATION
Fig. 10.
Trained weights
i
311
(a)
(b)
(c)
(d)
for three reverse driving paths.
A. Approximation of Other Standard Defuzzification Methods The resulting weights or confidence values for the MOA and COM defuzzification strategies cannot be mathematically obtained as for COG and MoM. Therefore, training of the CBADD network with sample data for both MOA and COM can be considered as application examples. The importance in showing this using simulations is related to the universal approximation capability of the network. In this paper, the simulation results given are limited only for two methods. 1) Approximation of MOA: For the MOA method, the varireaches values varying from 0.0 able confidence measure
at both ends of the output range to 2.5 in the middle. The rms errors for pattern set 1 and for pattern set 2 [Fig. 8(c)] are 2.24 and 2.20%, respectively. The trained CBADD-net successfully approximates MOA. 2) Approximation of COM: The training of COM defuzzibetween 23.0 and 41.0 fied patterns leads to weights for for pattern set 1 and to weights between 19.0–29.0 for pattern set 2 [Fig. 8(d)]. The RMS error for the first set is 9.96% and are somewhat for the second set is 3.53%. The weights for similar to those of MoM, which is not a surprise as a single block of maximum height will have the same crisp output with both strategies. But the tests conducted show that COM
312
Fig. 11.
IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 6, NO. 2, MAY 1998
Consequent membership functions.
test have the least error with the COM trained network and not with the MoM trained network. Therefore, the resulting weights represent the new defuzzification strategy COM. B. Fuzzy Controlled Model Truck and Trailer Driving a vehicle with a trailer in the reverse direction is a difficult task for a beginner. Even experienced drivers have to undertake a “trial and error” approach, i.e., if the trailer comes to a position where the angle between the longitudinal axes of the vehicle and the trailer cannot be increased even by maximum angle of the steering wheel in reverse driving, then the driver has to change the gear and drive forward in order to avoid further bending (critical angle). Since this is a nonlinear problem, a support system is designed employing a Mamdani-type fuzzy controller in a model truck and trailer (Fig. 9) in an efficient manner to overcome this situation [23]. Inputs to the fuzzy controller are the driver command to the wheels of the truck at the front and the current rare wheel position of the trailer. The output is the fuzzy controlled command to the front wheels of the truck. The truck is slowly but correctly driven without fuzzy support by an experienced human and crisp input/output data are recorded. Applying the same crisp input data to the fuzzy controller the fuzzy output can be recorded. The application data for training CBADD network contains the recorded fuzzy output and the crisp output. The training of application data obtained from a fuzzy controlled model truck for three drives: “reverse turn to left bend (fl),” “straight line reversing (fs),” and “reverse turn to right bend (fr)” shows a new distribution of weights (Fig. 10).
However, confidence charts of the first two data sets (fl and fr) are similar to that of COM. It is on-line fuzzy controlled in order to enhance the reverse dynamics so that even an unexperienced driver would encounter hardly any problems in fast reverse driving the truck with a trailer. Either all three data sets can be applied to a single CBADD network creating a single defuzzification approximation or two different defuzzification schemes (one for fl and fr and another for fs) can be trained. Since the model truck contains a hierarchical fuzzy controller with two subknowledge bases for two situations, it is better to employ two different methods. The defuzzification method for left or right bend can be COM. But for the straight line reverse driving it is better to use the newly created defuzzification with CBADD network. Another method of tuning CBADD in this application is the use of crisp input/output data to train a neuro-fuzzy controller, which includes the CBADD network as shown in Fig. 12. Optimal results for the fuzzy system can be obtained this way due to the inclusion of membership functions and the rules to an integrated training network with defuzzification. VI. INTEGRATION INTO NEURO-FUZZY CONTROLLER Most of the neuro-fuzzy systems proposed for generation of fuzzy rules from numerical data do not include traditional defuzzification techniques since they generate classifier-type fuzzy systems [10], [24]–[28]. The FuNe I fuzzy system generator proposed in [8] was successful in employing several real-world applications [25]. It possesses the ability of fuzzy rule generation from sample data without compulsory expert
HALGAMUGE: TRAINABLE TRANSPARENT UNIVERSAL APPROXIMATOR FOR DEFUZZIFICATION
313
and max composition. Min inference can be implemented by using soft min [30] instead of multiplication in weighting. Together with the weights in antecedent membership functions created as scaled, shifted, and reflected sigmoid functions [8], weights in consequent membership functions (that can also be trained for tuning the membership functions) and trainable variable confidence measure tuning the CBADD network the fuzzy classifier generator FuNe I is successfully extended to operate as a FuNe II fuzzy controller, as shown in Fig. 12. Only the dark arrows in fuzzification, composition, and defuzzification blocks in Fig. 12 represent variable weights and other connection have fixed unity weights. The dark solid circles in the fuzzification layer represent sigmoid neurons. The dark dashed and dark dotted circles in the CBADD layer represent neurons with logarithmic (Ln) and exponential (E) activation functions, respectively. The elliptic neuron in the defuzzification has the inverse function as the activation and the output neuron has multiplication as the net input calculation. The white circles with have soft-min (soft-max) net input calculation and other neurons have summing inputs. The FuNe II neuro-fuzzy controller is used in training the antecedent membership functions together with CBADD network expanding the application results shown in Section VB for the reverse driving of the truck with the trailer.
VII. CONCLUSIONS Fig. 12.
Mamdani neuro-fuzzy controller with CBADD network.
opinion and the capability of integrating existing incomplete knowledge and parameter tuning. Unlike neuro-fuzzy classifier generators, it uses a blackbox-type defuzzification typical for FuNe I-type fuzzy systems. Mamdani-type fuzzy controller with CBADD defuzzification method is called FuNe II fuzzy system [29]. FuNe II fuzzy system generator is an extension of FuNe I fuzzy system generator for application to control problems with representative training data for efficient tuning. The training procedure includes the tuning of CBADD network together with antecedent membership functions. The fuzzy rules are either extracted (for an example with FuNe I) or human formulated. The simultaneous customization of defuzzification and the tuning of antecedent membership functions fulfill the requirement for a two-layer perceptron of neural network theory, i.e., the existence of at least two feed-forward neuron layers with trainable weights, acquiring the ability of finding solutions to nonlinear separable problems. FuNe II neuro-fuzzy network contains bell-shaped antecedent membership functions and trapezoidal (or triangular) consequent membership functions. The consequent membership functions are initialized with the weights shown in Fig. 11. The weighting method between two neuron layers (either product or soft min [30]) depends on the inference method applied. The net input functions of the neurons in both layers must be the same and must represent the composition method. Assuming each weight is multiplied by the incoming signal and considering as the soft max [8] net input function, the fuzzy output curve shown is the result of product inference
Existing defuzzification methods are outlined and analyzed together with several application dependent situations. Therefore, a trainable defuzzification method can be useful in many applications. A parameterizable universal defuzzification must have the capability of approximating all the standard defuzzification methods. Conventional neural networks are universal defuzzification approximators. However, they are not transparent and, therefore, not helpful in comparing with standard defuzzification methods. The CBADD network is presented as a universal transparent trainable approximator for defuzzification in fuzzy control. In comparison to other parameterizable defuzzification methods such as BADD, where only a limited number of standard defuzzification methods can be approximated, it is shown that CBADD network can be trained to approximate all the well-known defuzzification methods. The method is validated and applied to a real fuzzy control problem creating a customized defuzzification. Knowing fuzzy rules, the CBADD network can be used to generate an application dependent defuzzification method. Either the generated method can be used or the most desirable existing defuzzification method can be suggested using confidence chart analysis. However, the major motivation behind the development of CBADD network is the extension of FuNe I fuzzy system generator to a FuNe II fuzzy controller generator, training the confidence values of application specific defuzzification, the fuzzy rules, and the antecedent membership functions. After training a FuNe II network, an evaluation of fuzzy rules can also be done with respect to a traditional Mamdani-type fuzzy controller with COG defuzzification. If a trained variable confidence value is negative, the fuzzy rules containing the consequent membership function corresponding
314
IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 6, NO. 2, MAY 1998
to that value has to be checked for the correctness. The integration of CBADD network instead of COG defuzzification can also be considered as an addition of trainable noise to the fuzzy output before calculating the crisp output. Essentially the trained values of the set variable confidence measure serves as a quality measure for the existing rule base if otherwise.
(36)
The accepted range of the quality measure for the rule base . If the rule base need to be redesigned. would be The trained FuNe II Neuro-Fuzzy controller can either be left as it is, ignoring even the lack of confidence shown on several fuzzy rules since it is a functioning neural network ignoring the erroneous rule base and approximating a solution to the application considered, or the network can be retrained after the fuzzy rules are modified, providing a functioning solution together with a more reliable and correct fuzzy rule base. ACKNOWLEDGMENT The author would like to thank Dr. T. Runkler, Dr. W. Poechmueller, and Prof. M. Glesner for their support. He would also like to thank T. Wagner for the implementation of the algorithm in software and M. Seidel and R. Kothe for the collection of application data in early stages of this research at Technische Universit¨at Darmstadt, Germany, and the reviewers for their useful comments. REFERENCES [1] L. Zadeh, “Fuzzy sets,” in Inform. Contr., vol. 8, pp. 338–353, 1965. [2] E. H. Mamdani and S. Assilian, “An experiment in linguistic synthesis with a fuzzy logic controller,” Int. J. Man-Mach. Studies, vol. 7, pp. 1–13, 1975. [3] C. C. Lee, “Fuzzy logic in control systems: Fuzzy logic controller—Part II,” IEEE Trans. Syst., Man, Cybern., vol. 20, Mar./Apr. 1990. [4] M. Mizumoto, “Improvement methods of fuzzy controls,” in 3rd Int. Fuzzy Syst. Assoc. Congress, 1989, pp. 60–62. [5] D. Driankov, H. Hellendoorn, and M. Reinfrank, An Introduction to Fuzzy Control. New York: Springer-Verlag, 1993. [6] M. Sugeno and G. T. Kang, “Structure identification of fuzzy model,” Int. J. Fuzzy Sets Syst., vol. 28, pp. 15–33, 1988. [7] T. Takagi and M. Sugeno, “Fuzzy identification of systems and its application to modeling and control,” IEEE Trans. Syst., Man, Cybern., vol. 15, pp. 116–132, 1985. [8] S. K. Halgamuge and M. Glesner, “Neural networks in designing fuzzy systems for real world applications,” Int. J. Fuzzy Sets Syst., vol. 65, pp. 1–12, 1994. [9] Y. Tsukamoto, “An approach to fuzzy reasoning method,” in Advances in Fuzzy Set Theory and Applications, M. M. Gupta, R. K. Ragade, and R. Yager, Eds. Amsterdam, The Netherlands: North-Holland, 1979.
[10] S. K. Halgamuge, W. P¨ochm¨uller, and M. Glesner, “An alternative approach for generation of membership functions and fuzzy rules based on radial and cubic basis function networks,” Int. J. Approximate Reasoning, vol. 12, no. 3/4, pp. 279–298, Apr./May 1995. [11] Q. Song and G. Bortolan, “Some properties of defuzzification neural networks,” Int. J. Fuzzy Sets Syst., vol. 61, pp. 83–89, 1994. [12] A. Kawamura, N. Watanabe, H. Okada, and K. Asakawa, “A prototype of neuro-fuzzy cooperation system,” in IEEE Int. Conf. Fuzzy Syst., San Diego, CA, Mar. 1992, pp. 1275–1282. [13] S. K. Halgamuge, T. A. Runkler, and M. Glesner, “On the neural defuzzification methods,” in Int. Conf. Fuzzy Syst., New Orleans, LA, Sept. 1996, pp. 463–469. [14] T. Hollstein, S. K. Halgamuge, and M. Glesner, “Computer aided fuzzy system design based on generic VHDL specifications,” IEEE Trans. Fuzzy Syst., vol. 4, pp. 403–417, Nov. 1996. [15] T. A. Runkler and M. Glesner, “Defuzzification with improved static and dynamic behavior: Extended center of area,” in Eur. Congress Fuzzy Intell. Technol., Aachen, Germany, Sept. 1993, pp. 845–851. , “DECADE—Fast centroid approximation defuzzification for [16] real time fuzzy control applications,” in ACM Symp. Appl. Comput., Phoenix, AZ, Mar. 1994, pp. 161–165. , “A set of axioms for defuzzification strategies—Toward a theory [17] of rational defuzzification operators,” in 2nd IEEE Int. Conf. Fuzzy Syst., San Francisco, CA, Mar./Apr. 1993, pp. 1161–1166. [18] D. P. Filev and R. R. Yager, “A generalized defuzzification method via bad distributions,” Int. J. Intell. Syst., vol. 6, 1991. [19] R. R. Yager, “Fuzzy sets and approximate reasoning in decision and control,” in IEEE Int. Conf. Fuzzy Syst., San Diego, CA, Mar. 1992, pp. 418–428. [20] L. Zadeh, “Outline of a new approach to the analysis of complex systems and decision processes,” IEEE Trans. Syst., Man, Cybern., vol. SMC-3, pp. 28–44, 1973. [21] D. E. Rumelhart and J. L. McClelland, Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Cambridge, MA: MIT Press, 1986. [22] S. K. Halgamuge, T. Wagner, and M. Glesner, “Validation and application of an adaptive transparent defuzzification strategy for fuzzy control,” in IEEE Int. Conf. Fuzzy Syst., Orlando, FL, June 1994, pp. 1642–1645. [23] S. K. Halgamuge, T. A. Runkler, and M. Glesner, “A hierarchical hybrid fuzzy controller for realtime reverse driving support of vehicles with long trailers,” in IEEE Int. Conf. Fuzzy Syst., Orlando, FL, June 1994, pp. 1207–1210. [24] S. K. Halgamuge, “Self evolving neural networks for rule based data processing,” IEEE Trans. Signal Processing, vol. 45, pp. 2766–2773, Nov. 1997. , Advanced Methods for Fusion of Fuzzy Systems and Neural [25] Networks in Intelligent Data Processing. D¨usseldorf, Germany: Inst. Eng. Germany Press (VDI Verlag), 1996. [26] D. Nauck and R. Kruse, “NEFCLASS—A neuro-fuzzy approach for the classification of data,” in ACM Symp. Appl. Comput., Nashville, TN, Feb. 1995, pp. 461–465. [27] N. Tschichold-G¨urman, “Generation and improvement of fuzzy classifiers with incremental learning using fuzzy ruleNet,” in ACM Symp. Appl. Comput., Nashville, TN, Feb. 1995, pp. 466–470. [28] S. K. Halgamuge and M. Glesner, “Fuzzy neural networks: Between functional equivalence and applicability,” Inst. Elect. Eng. Int. J. Neural Syst., vol. 6, no. 2, pp. 185–196, June 1995. , “The fuzzy neural controller FuNe II with a new adaptive [29] defuzzification strategy based on CBAD distributions,” in Eur. Congress Fuzzy Intell. Technol., Aachen, Germany, Sept. 1993, pp. 852–855. [30] H. R. Berenji and P. Khedkar, “Learning and tuning fuzzy logic controllers through reinforcements,” IEEE Trans. Neural Networks, vol. 3, pp. 724–740, May 1992.
IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 6, NO. 2, MAY 1998
A Trainable Transparent Universal Approximator for Defuzzification in Mamdani-Type Neuro-Fuzzy Controllers Saman K. Halgamuge
Abstract— A novel technique of designing application specific defuzzification strategies with neural learning is presented. The proposed neural architecture considered as a universal defuzzification approximator is validated by showing the convergence when approximating several existing defuzzification strategies. The method is successfully tested with fuzzy controlled reverse driving of a model truck. The transparent structure of the universal defuzzification approximator allows to analyze the generated customized defuzzification method using the existing theories of defuzzification. The integration of universal defuzzification approximator instead of traditional methods in Mamdani-type fuzzy controllers can also be considered as an addition of trainable nonlinear noise to the output of the fuzzy rule inference before calculating the defuzzified crisp output. Therefore, nonlinear noise trained specifically for a given application shows a grade of confidence on the rule base, providing an additional opportunity to measure the quality of the fuzzy rule base. The possibility of modeling a Mamdani-type fuzzy controller as a feed-forward neural network with the ability of gradient descent training of the universal defuzzification approximator and antecedent membership functions fulfill the requirement known from multilayer preceptrons in finding solutions to nonlinear separable problems.
Fig. 1. Standard defuzzification strategies.
• Center of Gravity (COG)—the center of gravity of the area
Index Terms—Defuzzification, fuzzy control, fuzzy neural networks, mechatronics.
(1) I. INTRODUCTION
S
INCE the initial theoretical work of Zadeh [1] and early practical applications of fuzzy control [2], numerous defuzzification strategies have been proposed by several authors [3], [4]. The result of fuzzy rule inference and composition known as the fuzzy output is transformed back to a crisp output by defuzzification [5]. Although most of the methods are proposed in connection with Mamdani-type fuzzy systems [2], defuzzification is an essential step in all fuzzy system models such as Takagi–Sugeno–Kang (TSK) [6], [7], FuNe I [8], Tsukamoto [9] with the exception of classifier-type [10] fuzzy systems in which the fuzzy output is taken as the class membership. and denoting Assuming normalized membership values as the finite set of possible normalized output values of a where Mamdani-type fuzzy controller represent the discrete set of corresponding membership values. The resulting output obtained by applying several existing defuzzification schemes to an example can be shown in Fig. 1.
• Midpoint of Area (MOA)—the middle of the area where (2) • Mean of Maxima (MoM), the center of gravity of the area under the maxima of fuzzy output
(3) where
• Center of Mean (COM), the middle of the area under the maxima of fuzzy output where
Manuscript received September 28, 1995; revised May 7, 1997. The author is with the Department of Mechanical and Manufacturing Engineering, Mechatronics Research Group, The University of Melbourne, Parkville, Vic 3052, Australia. Publisher Item Identifier S 1063-6706(98)03276-7. 1063–6706/98$10.00 1998 IEEE
and (4)
HALGAMUGE: TRAINABLE TRANSPARENT UNIVERSAL APPROXIMATOR FOR DEFUZZIFICATION
From the neural network perspective defuzzification is a multidimensional function approximation problem that can be solved. But the neural networks trained to solve are mostly of blackbox type. If representative data sets exist, such neural networks can be trained to approximate any of above mentioned explicitly known standard defuzzification methods [11]. For example a transparent nontrainable neural structure is used to implement the method COG in [12] in constructing a prototype neuro-fuzzy system. Another approach is to use a conventional-type neural network to map fuzzy output into crisp output without considering the formal methods of defuzzification [8], [13]. Those neural networks are universal defuzzification approximators with the strong drawback of blackbox nature. The main contribution of this paper is the introduction of a universal defuzzification approximator that is parameterizable and transparent ,with the capability of approximating the defuzzification to an application specific solution or to select the best one from the known methods. Problems caused in applying existing defuzzification schemes are outlined in Section II. Section III presents the new trainable universal defuzzification approximator giving more emphasis to its implementation as a transparent neural network. The validation of the proposed method by training it to approximate different existing defuzzification methods and the application to a control example are described in Sections IV and V. The integration of the proposed method to the fuzzy neural controller FuNe II is outlined in Section VI. Section VII summarizes the paper. II. APPLICATION OF EXISTING DEFUZZIFICATION STRATEGIES The defuzzification methods MOA and COM are easily implementable in software as well as in hardware [14] using and covering the output range starting two pointers from the lower and the upper limit, respectively, and moving and stepwise toward the middle calculating the left area the right area as depicted in Fig. 2 for MOA. However, the possible error (half of the discrete step selected) should be taken into account. For a more accurate solution in an analog hardware or software implementation with continuous crisp output range, two different cases need to be considered for nonzero fuzzy output between the pointers in the final step. 1) Both pointers correspond to the same membership value [Fig. 3(a)]. 2) Pointers correspond to two different membership values and [Fig. 3(b)]. The following equations can be obtained considering both cases of MOA in Fig. 3 and only the first case for COM: (5) (6) (7) (8)
305
Fig. 2. MOA defuzzification.
where is the step size, , and , are
is the correction to be added to
For the case where the fuzzy output is zero (or for COM) between the pointers, either of the pointers could be selected as the output. The widely used COG method is not appropriate in some conflicting situations. An example is shown in Fig. 1. In this case, rules are active to the left and the right from the center. One can imagine this case as moving a vehicle toward possible paths in the left and the right with an obstacle in the middle. COG and MoM defuzzification methods deliver the most inappropriate solutions somewhere in the middle, whereas MOA and COM deliver though not similar but acceptable solutions. In case of having two equal possibilities to the left and to the right, i.e., both membership functions are equal, then either the right corner of the “left” membership block or the left corner of the “right” membership block has to be taken as the defuzzified value according to COM and MOA. Even MOA delivers an inappropriate crisp output along with COG and MoM if the obstacle is replaced by a less favorable swampy path causing a inference strength of “ ” at the output where in Fig. 1. The winner in this example is COM still delivering an acceptable solution, but is undesirable as a universal method due to its high small signal amplification similar to that of MoM as described in [15]. Even though the vehicle can move straight forward in the swampy path, the desirable way would be to the left or to the right. These situations can be avoided by restricting the membership value within the curve at least equal to the swamp constant. Apart from the basic defuzzification strategies mentioned above, a number of extended methods have been proposed by several authors. Most of them do not provide algorithms for application specific parameter training, but poses advantages in hardware implementation [16] and show improved static and dynamic properties [15], [17]. The parameterizable extended methods proposed extended center of area (XCOA) [15] and basic defuzzification distributions (BADD) [18] use the nonnegative parameter ,
306
IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 6, NO. 2, MAY 1998
(a)
(b)
Fig. 3. Calculation of MOA. (a) Both pointers correspond to the same membership value values 1 and 2 .
1 .
(b) Pointers correspond to two different membership
“confidence measure” [19] (9) where
(10)
A fuzzy set to the power of is a “concentration” if and a “dilation” if [20]. The COG method is extended in BADD and the MOA in XCOA, both replacing by in (1) and (2), respectively. It is obvious that (11) (12) (13) (14)
Fig. 4. Different regions in the confidence chart.
It must be noted that in case of reaching no confidence (15) is the defuzzified output by any of the methods where described above, when the universe of discourse is equally possible ( constant). The application of different defuzzification methods may lead to completely different crisp outputs depending on the shape of the fuzzy outputs (e.g., Fig. 1). In some situations none of the mentioned defuzzification methods delivers satisfactory results. Even the existing extended strategies are not successful in solving the problem with swampy path with the exemption of COM which has several other undesirable properties. Therefore, tuning to an application or customization of the defuzzification strategy is an interesting alternative to the conventional trial and error methods used. III. A TRAINABLE UNIVERSAL DEFUZZIFICATION APPROXIMATOR The proposed trainable method customisable basic defuzzification distributions (CBADD) is a useful extension of the
well-known BADD strategy that can be validated as a universal defuzzification approximator by neural network training. We define the set variable confidence measure , where is a real value. The confidence on the fuzzy output values can be divided into different regions as shown in Fig. 4. CBADD strategy can be described as
(16)
used in XCOA In contrast to the confidence measure and BADD, variable confidence measure can be negative. is negative, . It can be found an Whenever alternative nonnegative , in an effort by scaling to bring back the maximum of below a certain level (in the normalized case below one) so that function provides the . Scaling the fuzzy output means same defuzzified output as a complete change in the rule inference. Therefore, a trained CBADD neural network containing weights in the negative
HALGAMUGE: TRAINABLE TRANSPARENT UNIVERSAL APPROXIMATOR FOR DEFUZZIFICATION
307
Fig. 5. CBADD transparent neural network.
confidence region indicates the inconsistency of the existing rule base, hence, a warning to the user to check the rule base. in BADD and the set in The inclusion of constant CBADD can also be considered as an addition of nonlinear noise to the fuzzy output curve before defuzzification. The extension in CBADD is more flexible and eases the neural network based training.
inverted sum calculated in the lower neuron by using a neuron. The network can be resolved mathematically to match the CBADD equation due to its transparent structure. The activation functions of neurons employed in CBADDnetwork (linear , inverse , exponential , and logarithmic ) must satisfy a few rules [21] especially in the ranges of usage to assure convergence
A. Implementation as a Transparent Neural Network
(17)
CBADD uses a special transparent neural network with number of inputs (netwidth), each of them feeding a , the th discrete value of the fuzzy output, where in order to approximate the crisp output for . It consists of four consecutive layers of neurons, each layer consisting of different types of neurons concerning the calculation of net input and activation function (Fig. 5). The first three layers have summing net inputs and only the output neuron has multiplying inputs. The first layer consists of neurons having logarithmic activation functions and the connections to the following layer are weighted by the components of variable confidence . They are the only trainable weights in the CBADD network. Each neuron in the second layer has an exponential activation function, which partly eliminates the logarithm of the first layer and delivers . Within the first two layers there are no cross connections, but only connections among the neurons of the same net index. In the third neuron layer all outputs are fed into two neurons that build a weighted sum. The incoming connections to the upper neuron with linear activation is weighted by , the discrete values of the possible range of outputs (in Fig. 5 is 0.0, 0.1, 0.2, 0.9, 1.0 ). The lower neuron has unity incoming for weights and an inverting activation function. In the last layer, the resulting sum of the upper neuron is multiplied with the
(18) (19) (20) The linear activation function is only used as a “dummy” , the weight and it does not play a role in calculating change for . The inverse activation function is used with the fact that net input to the neuron is always nonzero (see Table I). The exponential activation function is inappropriate for gradient descent learning whenever net input is very high negative. Since this is not the case in the CBADD network, the exponential activation function can be used. To assure that net input to the neurons with logarithmic activation functions is always nonzero, a minimum limit for the fuzzy output is defined such that B. Alternative Neuron Models in CBADD Network In building the CBADD network, different types of neurons were used. Apart from the standard-type neurons with linear or sigmoid functions as activation functions and summation to calculate the weighted net input (net), neurons with multiplying net inputs , and those having
308
IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 6, NO. 2, MAY 1998
FULFILLMENT
TABLE I OF THE REQUIREMENTS
Fig. 6. Gradient descent learning.
and
for neurons with multiplying net inputs, i.e.,
(26) logarithmic , exponential , or inverse activation functions are used. Fig. 6 shows three consecutive neuron layers and having neuron indexes and respectively. The connection weight of the th neuron in the th layer and the th neuron in the layer is denoted as . Let us consider as the output layer and others as hidden layers. Using the least-mean square method (LMS) and the gradient descent algorithm [21], the total error (E) at the output can be calculated as (21)
Calculation of for an output neuron is independent of the calculation of net input, from (22) we conclude that hence (27) Calculating the partial derivative for hidden neurons depends on , , and the net input calculation method of the following neurons with index
(22) (28) where is the error occurred for a single pattern, is the desired output and is the current output of neuron . Assuming that can be a neuron in any of the layers, the by definition [21] is weight change
Assuming neurons in the layer we obtain
use summing net inputs,
(23) Considering , the learning rate, as the constant of proportionality, we can write (24)
(29) and assuming neurons in the layer inputs, we can write
use multiplying net
The function used for calculation of net inputs of the th neuron ( or ) influences the calculation of the partial for neurons with summing net inputs, so that derivative
(25)
(30)
HALGAMUGE: TRAINABLE TRANSPARENT UNIVERSAL APPROXIMATOR FOR DEFUZZIFICATION
309
The validation of the CBADD network is done by training the network with pattern sets generated with the defuzzification strategies COG and MoM [22]. Similar to (11) and (12), it is obvious that can be reduced to by setting all to one and it can be reduced to the elements in set by setting those elements virtually to . We approximate these mathematically exact solutions with neural network training and consider it as a validation for the convergence of the system. A. Validation with Known Defuzzification Methods
(a)
(b) Fig. 7. Sample pattern sets for CBADD training. (a) Pattern set 1. (b) Pattern set 2.
IV. CONVERGENCE OF THE CBADD NETWORK The convergence of the CBADD network is assured by the fulfilment of the requirements discussed in Section III-A and the derivations for the alternative neuron models found in CBADD networks as described in Section III-B. Denoting the layers in Fig. 5 from one (left most layer) to four (right most layer), starting the index for the neurons from the top neuron (neuron 1), applying (27) for the output neuron of the CBADD network (31) and simplifying (30) we can obtain (32) (33) It can be derived from (29) that
(34) Applying (25) and (34) in (24) we obtain (35)
inputs in Several sample pattern sets are used with the range from ALMOST ZERO to 1.0 and one output range . We use 2000 different patterns that from 0.0 to are applied over 1000 sweeps; that is 1000 times. Furthermore, we use single-mode backpropagation and update the weights after each single pattern. Each pattern represents one epoch. The system has been tested with different pattern sets, each defuzzified with the strategy of interest. This paper reports the simulation results only for two of them: pattern sets 1 and 2, which are similar to real fuzzy output variables (Fig. 7). The general strategy was to train the network with data sets having 32 inputs and an output representing the discrete values of fuzzy output values and the crisp output calculated according to one of the standard defuzzification algorithms. The trained CBADD network should show in case of convergence good results on an entirely different test data evolved must be identical set and the confidence values with the expected values. The randomly generated patterns in set 1 are evenly distributed and, therefore, have their and values often close to the middle of the output range. The pattern set 2 has fuzzy output curves composed of five random heights that are bounded in the vertical direction by the limits of the fuzzy output range and in the horizontal direction by two randomly selected edge values between 0.0 and . The resulting pattern sets can be of a rectangular shape as well as of a triangular or multi-trapezoidal shape. Therefore, and values are almost randomly distributed. Exploiting the advantages of the transparency of the CBADD-network, the confidence charts obtained using trained weights can be analyzed. The confidence charts of different existing defuzzification methods for pattern set 2 shown in Fig. 8 are similar to those of pattern set 1. 1) Validation with COG: The training of patterns defuzzified with COG strategy converges successfully, leading to weights in the range of 1.0 0.01 for pattern set 1 and set 2 [Fig. 8(a)]. Pattern set 1 shows a slightly slower convergence than pattern set 2. After 1000 sweeps the RMSerror for sets 1 and 2 are 0.004% and 0.0002%, respectively. The mathematically correct solution for is 1.0. Therefore, we see that CBADD approximates the set for the COG in the training phase. When trained over a longer period of time (10 000 sweeps) the tendency for the weights is toward an asymptote at one and an root—mean square error of zero. 3) Validation with MoM: When the CBADD network is trained with pattern sets defuzzified by the MoM strategy, the
310
IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 6, NO. 2, MAY 1998
(a)
(b)
(c)
(d)
Fig. 8. Confidence chart for pattern set 2. (a) COG strategy. (b) MoM strategy. (c) MOA strategy. (d) COM strategy.
weights are distributed in the range of for pattern set 1 and in the range of for pattern set 2 [Fig. 8(b)]. The RMS error for training the first set is 1.46% and that for the second is 0.49%. In this case, the mathematically correct is , a value that cannot be reached in solution for a computer simulation. Therefore, we conclude that CBADD can sufficiently approximate MOM since confidence values are well above 20. As shown by the simulation results, CBADD can successfully represent the defuzzification strategies COG and MOM and the implementation has herewith shown to be correct. The trained CBADD approximations for MOM and COG are successfully tested with different data sets. V. APPLICATION
OF
CBADD NETWORK
Due to the flexibility of the CBADD concept, any of the existing defuzzification method can be approximated, i.e., a can be found either mathematically as discrete set of shown in Section IV-A or by training the CBADD network
Fig. 9. Truck and trailer.
with numerical data. Furthermore, data sets obtained from an model truck application are also used for creating customized defuzzification methods.
HALGAMUGE: TRAINABLE TRANSPARENT UNIVERSAL APPROXIMATOR FOR DEFUZZIFICATION
Fig. 10.
Trained weights
i
311
(a)
(b)
(c)
(d)
for three reverse driving paths.
A. Approximation of Other Standard Defuzzification Methods The resulting weights or confidence values for the MOA and COM defuzzification strategies cannot be mathematically obtained as for COG and MoM. Therefore, training of the CBADD network with sample data for both MOA and COM can be considered as application examples. The importance in showing this using simulations is related to the universal approximation capability of the network. In this paper, the simulation results given are limited only for two methods. 1) Approximation of MOA: For the MOA method, the varireaches values varying from 0.0 able confidence measure
at both ends of the output range to 2.5 in the middle. The rms errors for pattern set 1 and for pattern set 2 [Fig. 8(c)] are 2.24 and 2.20%, respectively. The trained CBADD-net successfully approximates MOA. 2) Approximation of COM: The training of COM defuzzibetween 23.0 and 41.0 fied patterns leads to weights for for pattern set 1 and to weights between 19.0–29.0 for pattern set 2 [Fig. 8(d)]. The RMS error for the first set is 9.96% and are somewhat for the second set is 3.53%. The weights for similar to those of MoM, which is not a surprise as a single block of maximum height will have the same crisp output with both strategies. But the tests conducted show that COM
312
Fig. 11.
IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 6, NO. 2, MAY 1998
Consequent membership functions.
test have the least error with the COM trained network and not with the MoM trained network. Therefore, the resulting weights represent the new defuzzification strategy COM. B. Fuzzy Controlled Model Truck and Trailer Driving a vehicle with a trailer in the reverse direction is a difficult task for a beginner. Even experienced drivers have to undertake a “trial and error” approach, i.e., if the trailer comes to a position where the angle between the longitudinal axes of the vehicle and the trailer cannot be increased even by maximum angle of the steering wheel in reverse driving, then the driver has to change the gear and drive forward in order to avoid further bending (critical angle). Since this is a nonlinear problem, a support system is designed employing a Mamdani-type fuzzy controller in a model truck and trailer (Fig. 9) in an efficient manner to overcome this situation [23]. Inputs to the fuzzy controller are the driver command to the wheels of the truck at the front and the current rare wheel position of the trailer. The output is the fuzzy controlled command to the front wheels of the truck. The truck is slowly but correctly driven without fuzzy support by an experienced human and crisp input/output data are recorded. Applying the same crisp input data to the fuzzy controller the fuzzy output can be recorded. The application data for training CBADD network contains the recorded fuzzy output and the crisp output. The training of application data obtained from a fuzzy controlled model truck for three drives: “reverse turn to left bend (fl),” “straight line reversing (fs),” and “reverse turn to right bend (fr)” shows a new distribution of weights (Fig. 10).
However, confidence charts of the first two data sets (fl and fr) are similar to that of COM. It is on-line fuzzy controlled in order to enhance the reverse dynamics so that even an unexperienced driver would encounter hardly any problems in fast reverse driving the truck with a trailer. Either all three data sets can be applied to a single CBADD network creating a single defuzzification approximation or two different defuzzification schemes (one for fl and fr and another for fs) can be trained. Since the model truck contains a hierarchical fuzzy controller with two subknowledge bases for two situations, it is better to employ two different methods. The defuzzification method for left or right bend can be COM. But for the straight line reverse driving it is better to use the newly created defuzzification with CBADD network. Another method of tuning CBADD in this application is the use of crisp input/output data to train a neuro-fuzzy controller, which includes the CBADD network as shown in Fig. 12. Optimal results for the fuzzy system can be obtained this way due to the inclusion of membership functions and the rules to an integrated training network with defuzzification. VI. INTEGRATION INTO NEURO-FUZZY CONTROLLER Most of the neuro-fuzzy systems proposed for generation of fuzzy rules from numerical data do not include traditional defuzzification techniques since they generate classifier-type fuzzy systems [10], [24]–[28]. The FuNe I fuzzy system generator proposed in [8] was successful in employing several real-world applications [25]. It possesses the ability of fuzzy rule generation from sample data without compulsory expert
HALGAMUGE: TRAINABLE TRANSPARENT UNIVERSAL APPROXIMATOR FOR DEFUZZIFICATION
313
and max composition. Min inference can be implemented by using soft min [30] instead of multiplication in weighting. Together with the weights in antecedent membership functions created as scaled, shifted, and reflected sigmoid functions [8], weights in consequent membership functions (that can also be trained for tuning the membership functions) and trainable variable confidence measure tuning the CBADD network the fuzzy classifier generator FuNe I is successfully extended to operate as a FuNe II fuzzy controller, as shown in Fig. 12. Only the dark arrows in fuzzification, composition, and defuzzification blocks in Fig. 12 represent variable weights and other connection have fixed unity weights. The dark solid circles in the fuzzification layer represent sigmoid neurons. The dark dashed and dark dotted circles in the CBADD layer represent neurons with logarithmic (Ln) and exponential (E) activation functions, respectively. The elliptic neuron in the defuzzification has the inverse function as the activation and the output neuron has multiplication as the net input calculation. The white circles with have soft-min (soft-max) net input calculation and other neurons have summing inputs. The FuNe II neuro-fuzzy controller is used in training the antecedent membership functions together with CBADD network expanding the application results shown in Section VB for the reverse driving of the truck with the trailer.
VII. CONCLUSIONS Fig. 12.
Mamdani neuro-fuzzy controller with CBADD network.
opinion and the capability of integrating existing incomplete knowledge and parameter tuning. Unlike neuro-fuzzy classifier generators, it uses a blackbox-type defuzzification typical for FuNe I-type fuzzy systems. Mamdani-type fuzzy controller with CBADD defuzzification method is called FuNe II fuzzy system [29]. FuNe II fuzzy system generator is an extension of FuNe I fuzzy system generator for application to control problems with representative training data for efficient tuning. The training procedure includes the tuning of CBADD network together with antecedent membership functions. The fuzzy rules are either extracted (for an example with FuNe I) or human formulated. The simultaneous customization of defuzzification and the tuning of antecedent membership functions fulfill the requirement for a two-layer perceptron of neural network theory, i.e., the existence of at least two feed-forward neuron layers with trainable weights, acquiring the ability of finding solutions to nonlinear separable problems. FuNe II neuro-fuzzy network contains bell-shaped antecedent membership functions and trapezoidal (or triangular) consequent membership functions. The consequent membership functions are initialized with the weights shown in Fig. 11. The weighting method between two neuron layers (either product or soft min [30]) depends on the inference method applied. The net input functions of the neurons in both layers must be the same and must represent the composition method. Assuming each weight is multiplied by the incoming signal and considering as the soft max [8] net input function, the fuzzy output curve shown is the result of product inference
Existing defuzzification methods are outlined and analyzed together with several application dependent situations. Therefore, a trainable defuzzification method can be useful in many applications. A parameterizable universal defuzzification must have the capability of approximating all the standard defuzzification methods. Conventional neural networks are universal defuzzification approximators. However, they are not transparent and, therefore, not helpful in comparing with standard defuzzification methods. The CBADD network is presented as a universal transparent trainable approximator for defuzzification in fuzzy control. In comparison to other parameterizable defuzzification methods such as BADD, where only a limited number of standard defuzzification methods can be approximated, it is shown that CBADD network can be trained to approximate all the well-known defuzzification methods. The method is validated and applied to a real fuzzy control problem creating a customized defuzzification. Knowing fuzzy rules, the CBADD network can be used to generate an application dependent defuzzification method. Either the generated method can be used or the most desirable existing defuzzification method can be suggested using confidence chart analysis. However, the major motivation behind the development of CBADD network is the extension of FuNe I fuzzy system generator to a FuNe II fuzzy controller generator, training the confidence values of application specific defuzzification, the fuzzy rules, and the antecedent membership functions. After training a FuNe II network, an evaluation of fuzzy rules can also be done with respect to a traditional Mamdani-type fuzzy controller with COG defuzzification. If a trained variable confidence value is negative, the fuzzy rules containing the consequent membership function corresponding
314
IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 6, NO. 2, MAY 1998
to that value has to be checked for the correctness. The integration of CBADD network instead of COG defuzzification can also be considered as an addition of trainable noise to the fuzzy output before calculating the crisp output. Essentially the trained values of the set variable confidence measure serves as a quality measure for the existing rule base if otherwise.
(36)
The accepted range of the quality measure for the rule base . If the rule base need to be redesigned. would be The trained FuNe II Neuro-Fuzzy controller can either be left as it is, ignoring even the lack of confidence shown on several fuzzy rules since it is a functioning neural network ignoring the erroneous rule base and approximating a solution to the application considered, or the network can be retrained after the fuzzy rules are modified, providing a functioning solution together with a more reliable and correct fuzzy rule base. ACKNOWLEDGMENT The author would like to thank Dr. T. Runkler, Dr. W. Poechmueller, and Prof. M. Glesner for their support. He would also like to thank T. Wagner for the implementation of the algorithm in software and M. Seidel and R. Kothe for the collection of application data in early stages of this research at Technische Universit¨at Darmstadt, Germany, and the reviewers for their useful comments. REFERENCES [1] L. Zadeh, “Fuzzy sets,” in Inform. Contr., vol. 8, pp. 338–353, 1965. [2] E. H. Mamdani and S. Assilian, “An experiment in linguistic synthesis with a fuzzy logic controller,” Int. J. Man-Mach. Studies, vol. 7, pp. 1–13, 1975. [3] C. C. Lee, “Fuzzy logic in control systems: Fuzzy logic controller—Part II,” IEEE Trans. Syst., Man, Cybern., vol. 20, Mar./Apr. 1990. [4] M. Mizumoto, “Improvement methods of fuzzy controls,” in 3rd Int. Fuzzy Syst. Assoc. Congress, 1989, pp. 60–62. [5] D. Driankov, H. Hellendoorn, and M. Reinfrank, An Introduction to Fuzzy Control. New York: Springer-Verlag, 1993. [6] M. Sugeno and G. T. Kang, “Structure identification of fuzzy model,” Int. J. Fuzzy Sets Syst., vol. 28, pp. 15–33, 1988. [7] T. Takagi and M. Sugeno, “Fuzzy identification of systems and its application to modeling and control,” IEEE Trans. Syst., Man, Cybern., vol. 15, pp. 116–132, 1985. [8] S. K. Halgamuge and M. Glesner, “Neural networks in designing fuzzy systems for real world applications,” Int. J. Fuzzy Sets Syst., vol. 65, pp. 1–12, 1994. [9] Y. Tsukamoto, “An approach to fuzzy reasoning method,” in Advances in Fuzzy Set Theory and Applications, M. M. Gupta, R. K. Ragade, and R. Yager, Eds. Amsterdam, The Netherlands: North-Holland, 1979.
[10] S. K. Halgamuge, W. P¨ochm¨uller, and M. Glesner, “An alternative approach for generation of membership functions and fuzzy rules based on radial and cubic basis function networks,” Int. J. Approximate Reasoning, vol. 12, no. 3/4, pp. 279–298, Apr./May 1995. [11] Q. Song and G. Bortolan, “Some properties of defuzzification neural networks,” Int. J. Fuzzy Sets Syst., vol. 61, pp. 83–89, 1994. [12] A. Kawamura, N. Watanabe, H. Okada, and K. Asakawa, “A prototype of neuro-fuzzy cooperation system,” in IEEE Int. Conf. Fuzzy Syst., San Diego, CA, Mar. 1992, pp. 1275–1282. [13] S. K. Halgamuge, T. A. Runkler, and M. Glesner, “On the neural defuzzification methods,” in Int. Conf. Fuzzy Syst., New Orleans, LA, Sept. 1996, pp. 463–469. [14] T. Hollstein, S. K. Halgamuge, and M. Glesner, “Computer aided fuzzy system design based on generic VHDL specifications,” IEEE Trans. Fuzzy Syst., vol. 4, pp. 403–417, Nov. 1996. [15] T. A. Runkler and M. Glesner, “Defuzzification with improved static and dynamic behavior: Extended center of area,” in Eur. Congress Fuzzy Intell. Technol., Aachen, Germany, Sept. 1993, pp. 845–851. , “DECADE—Fast centroid approximation defuzzification for [16] real time fuzzy control applications,” in ACM Symp. Appl. Comput., Phoenix, AZ, Mar. 1994, pp. 161–165. , “A set of axioms for defuzzification strategies—Toward a theory [17] of rational defuzzification operators,” in 2nd IEEE Int. Conf. Fuzzy Syst., San Francisco, CA, Mar./Apr. 1993, pp. 1161–1166. [18] D. P. Filev and R. R. Yager, “A generalized defuzzification method via bad distributions,” Int. J. Intell. Syst., vol. 6, 1991. [19] R. R. Yager, “Fuzzy sets and approximate reasoning in decision and control,” in IEEE Int. Conf. Fuzzy Syst., San Diego, CA, Mar. 1992, pp. 418–428. [20] L. Zadeh, “Outline of a new approach to the analysis of complex systems and decision processes,” IEEE Trans. Syst., Man, Cybern., vol. SMC-3, pp. 28–44, 1973. [21] D. E. Rumelhart and J. L. McClelland, Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Cambridge, MA: MIT Press, 1986. [22] S. K. Halgamuge, T. Wagner, and M. Glesner, “Validation and application of an adaptive transparent defuzzification strategy for fuzzy control,” in IEEE Int. Conf. Fuzzy Syst., Orlando, FL, June 1994, pp. 1642–1645. [23] S. K. Halgamuge, T. A. Runkler, and M. Glesner, “A hierarchical hybrid fuzzy controller for realtime reverse driving support of vehicles with long trailers,” in IEEE Int. Conf. Fuzzy Syst., Orlando, FL, June 1994, pp. 1207–1210. [24] S. K. Halgamuge, “Self evolving neural networks for rule based data processing,” IEEE Trans. Signal Processing, vol. 45, pp. 2766–2773, Nov. 1997. , Advanced Methods for Fusion of Fuzzy Systems and Neural [25] Networks in Intelligent Data Processing. D¨usseldorf, Germany: Inst. Eng. Germany Press (VDI Verlag), 1996. [26] D. Nauck and R. Kruse, “NEFCLASS—A neuro-fuzzy approach for the classification of data,” in ACM Symp. Appl. Comput., Nashville, TN, Feb. 1995, pp. 461–465. [27] N. Tschichold-G¨urman, “Generation and improvement of fuzzy classifiers with incremental learning using fuzzy ruleNet,” in ACM Symp. Appl. Comput., Nashville, TN, Feb. 1995, pp. 466–470. [28] S. K. Halgamuge and M. Glesner, “Fuzzy neural networks: Between functional equivalence and applicability,” Inst. Elect. Eng. Int. J. Neural Syst., vol. 6, no. 2, pp. 185–196, June 1995. , “The fuzzy neural controller FuNe II with a new adaptive [29] defuzzification strategy based on CBAD distributions,” in Eur. Congress Fuzzy Intell. Technol., Aachen, Germany, Sept. 1993, pp. 852–855. [30] H. R. Berenji and P. Khedkar, “Learning and tuning fuzzy logic controllers through reinforcements,” IEEE Trans. Neural Networks, vol. 3, pp. 724–740, May 1992.
