Neuro-fuzzy Modeling and Fuzzy Rule Extraction ... - Tshilidzi Marwala
Neuro-fuzzy Modeling and Fuzzy Rule Extraction Applied to Conflict Management Thando Tettey and Tshilidzi Marwala University of the Witwatersrand School of Electrical and Information Engineering Private Bag 3 Wits 2050, Johannesburg, South Africa {t.tettey, t.marwala}@ee.wits.ac.za
Abstract. This paper outlines all the computational methods which have been applied to the conflict management. A survey of all the pertinent literature relating to conflict management is also presented. The paper then introduces the Takagi-Sugeno fuzzy model for the analysis of interstate conflict. It is found that using interstate variables as inputs, the Takagi-Sugeno fuzzy model is able to forecast conflict cases with an accuracy of 80.36%. Furthermore, it found that the fuzzy model offers high levels of transparency in the form of fuzzy rules. It is then shown how these rules can be translated in order to validate the fuzzy model. The Takagi-Sugeno model is found to be suitable for interstate modeling as it demonstrates good forecasting ability while offering a transparent interpretation of the modeled rules.
1
Introduction
With the frequency at which wars are occurring, it has become imperative that more research effort be directed towards conflict management. The focus of this study over the years has been centered on finding improved approaches to conflict forecasting at the same time not neglecting causal intepretation of interstate interactions [1]. Understanding the reasons why countries go to war is just as significant as forecasting the onset of war because it proposes the steps that can be taken to avoid conflict. Therefore a successful interstate conflict tool is one which is able to forecast dispute outcomes fairly accurately at the same time allowing for an intuitive causal intepretation of interstate interactions. International conflict has been studied using mainly techniques found in the fields of Statistics and Computational intelligence. These techniques have been applied on quantitative measures which have been collected over the years. It is widely accepted that improvements in conflict forecasting can mainly be acheived in two ways [1]. The first improvement that can be made is with the data and measures of interstate interactions [2] and secondly by improving the forecasting of interstate conflict involves finding models which approximate interstate interactions better. I. King et al. (Eds.): ICONIP 2006, Part III, LNCS 4234, pp. 1087–1094, 2006. c Springer-Verlag Berlin Heidelberg 2006
1088
T. Tettey and T. Marwala
In this paper a survey of the work performed on international conflict is presented. The paper focuses mainly on the techniques applied to analyse quantitative descriptions of interstate interactions. An outline of the methods together with their shortcomings is discussed. Finally, neuro-fuzzy modeling is presented as an alternative technique which addresses the current shortcomings that exist in international conflict studies.
2
Background and Literature Survey
Militarised interstate dispute (MID) is defined as a set of interactions between or among states that can result in the actual use, display or threat of using military force in an explicit way [3]. Projects such as the Correlates of War (COW) facilitate the collection, dissemination and use of accurate and reliable quantitative data in international relations [4]. The collected data, called interstate variables, are used to study the conditions associated with MID. The measures used in MID studies are People Oriented Government, Dependency, Capability, Alliance, Contiguity, Distance and Major power. Any set of measures describing a particular context has a dispute outcome attached to it. The dispute outcome is either a peace or conflict situation. Statistical methods such as logit and probit have been used in the analysis of interstate variables. However, it has been found that these methods have several shortcomings [1]. Some of the problems associated with the use of logit and probit is that they require the use of a priori knowledge usually obtained from the analyst. A problem then arises when the analyst pushes their data analyses extremely hard in search of effects they believe exist but are difficult to discover [1]. The consequence of this is that the results vary from researcher to researcher and are therefore not exactly repeatable. The other problem, as one might expect, is that conflict cases occur far less frequently than peace cases. Interstate conflict is therefore a rare event and the processes which drive it are likely to be different from those found elsewhere. This has led quantitative researchers to conclude that the relationship between the interstate variables and dispute outcomes is highly nonlinear and highly correlated [1]. The conclusion is further confirmed by the studies performed by Lagazio and Russet [5]. This means that statistical techniques, linear-normal models in particular, would perform poorly at modeling the relationship between interstate disputes and their outcomes. The neural network has also been applied to interstate conflict modeling and forecasting. The neural network was first introducted by Schrodt [6] in 1995 and by Zeng [7] in 1999 as a method of analysing conflict without the need for the researcher to incorporate qualitative a priori knowlege or make assumptions about the problem space. The neural network was presented as a function approximator which is able to model highly nonlinear and interdependant relationships. However the neural network itself suffers similar problems to statistical methods in that a model selection technique must be considered. In recent studies, Beck et al [1] make use of a neural network, which is trained using the Bayesian framework outlined in [8]. The Bayesian training of neural networks involves
Neuro-fuzzy Modeling and Fuzzy Rule Extraction
1089
the use of Bayesian framework to identify the optimal weights and biases in a neural network model. It is found that the use of neural networks yields results expressed in the form of classification accuracy. This interpretation of the results is found to be unambigious compared to previous methods. However, the resulting neural network model is regarded as a black box due to the fact that it does not provide a way of obtaining a causal interpretion of dispute outcomes. The weights extracted from the neural network offer no understanding as to why countries go to war. In [9], Marwala and Lagazio propose the use of Automatic Relevance Detection (ARD) as a means to making the neural network more transparent. The result of ARD reveals that the importance of the interstate variable in predicting dispute outcomes is as follows (listed in decreasing relavance): Peoples Oriented Government, Capability, Dependancy, Allies, Contiguity, Distance and Major power. From this work on neural networks we can conclude that neural network models have a fairly strong forecasting ability but only a limited amount of knowledge can be extracted. In [10], Habtemariam et al introduce support vector machines (SVMs) to the study of conflict management. It is found that SVMs offer an improved forecasting ability over neural networks. However, a sensitivity analysis which aims to determine the influence of each variable on a dispute outcome reveals that results obtained from neural networks are much more intuitive. Therefore, while SVMs offer better forecasting ability they lack the ability to give an intuitive causal interpretation of the results. As stated earlier on in the paper, the main focus of studies in international conflict has been on the ability of a model to accurately forecast dispute outcomes while at the same time allow the analyst to extract knowledge from the model. In the next section a Neuro-fuzzy model is proposed as a method of modeling interstate interaction which has a fairly accurate forecasting ability and at the same time offers intuitive causal explainations of disputes obtained from a fuzzyrule extraction process.
3
Fuzzy Systems and Neuro-fuzzy Modeling
Fuzzy logic concepts provide a method of modelling imprecise models of reasoning, such as common sense reasoning, for uncertain and complex processes. Fuzzy set theory resembles human reasoning in its use of approximate information and uncertainty to generate decisions. In fuzzy systems, the evaluation of the output is performed by a computing framework called the fuzzy inference system. The fuzzy inference system is a computing framework that maps fuzzy or crisp inputs to the output - which is usually a fuzzy set [11]. The inference system performs a composition of the inputs using fuzzy set theory, fuzzy if-then rules and fuzzy reasoning to arrive at the output. More specifically, the fuzzy inference involves the fuzzification of the input variables, evaluation of rules, aggregation of the rule outputs and finally the defuzzification of the result. The are two popular fuzzy models: the Mamdani model and the Takagi-Sugeno (TS) model [12].
1090
T. Tettey and T. Marwala
A fuzzy rule-based system can be viewed as a layered network similar to Radial basis function (RBF) artificial neural networks [12]. When training RBF networks several parameters of the kernel have to be optimised. Similarly when setting up a Fuzzy rule-based system we are required to optimise parameters such as membership functions and consequent parameters. In order to optimise these parameters, the neuro-fuzzy system relies on training algorithms inherited from artificial neural networks such as gradient descent [12]. There are two approaches to training neuro-fuzzy models [12]: 1. Fuzzy rules may be extracted from expert knowledge and used to create an initial model. The parameters of the model can then be fine tuned using data collected from the operational system being modelled. 2. The number of rules can be determined from collected numerial data using a model selection technique. The parameters of the model are also optimised using the existing data. The Takagi-Sugeno model is mostpopular when it comes to data-driven identification and has been proven to be a universal approximator [11]. The major motivation for using a neuro-fuzzy model in this work is that it is considered by many to be a ‘gray-box’ [12]. Unlike neural networks, once the neuro-fuzzy model has been optimised it is possible to extract the fuzzy rules and perhaps interpret the obtained results in an intuitive and qualitative manner. With neural networks e.g. the multilayer perceptron, it is not possible to intuitively explain input-output relationships using the weights of the network. In this study the neuro-fuzzy hybrid system is used to model the relationship between interstate (input) variables and the outcome of a dispute (output) from MID data.
4
Conflict Forecasting Using a Neuro-fuzzy Model
In our study, the TS neuro-fuzzy model has been optimised to map the relationship between the input variables (interstate variables) and the output i.e. the dispute outcome. The neuro-fuzzy model is trained using a balanced set of 500 peace cases and 500 conflict cases. The remaining 26845 peace instances and 392 conflict instance are then used for testing the forecasting ability of the model. The model selection process involves selecting the optimum number of rules of the TS model. This is done by creating models with rules ranging from 2 to 7 and evaluating them using 5 fold cross-validation. It is found that the optimum model consists of 2 fuzzy rules. The forecasting ability of the neuro-fuzzy model is then tested using the test examples. The prediction ability of the neuro-fuzzy model is evaluated based on how well it is able to predict both conflict and peace outcomes. The receiver operating characteristic (ROC) curve in fig 1 is used to illustrate the results. As the output of the neuro-fuzzy is expressed as a decision value, an optimum threshold needs to be determined as the decision point which allows the overall
Neuro-fuzzy Modeling and Fuzzy Rule Extraction
1091
Fig. 1. A ROC curve illustrating the performance of the neuro-fuzzy model Table 1. Results for the TS neuro-fuzzy model
Correctly predicted Incorrectly predicted
Conflict cases Peace cases 315 17967 77 8378
maximum peace and conflict prediction accuracy. This threshold is found to be 0.488 and the results are expressed as a confusion matrix shown in Table 1. The results show the TS neuro-fuzzy model predict conflict cases with an accuracy of 80.36% while predicting peace cases with an accuracy of 66.93%. In the work done by Habtemariam et al [10], it was found that SVM predicts peace and conflict with 79% and 75%, respectively. From these results it is clear that the TS neuro-fuzzy model predicts conflict with a much higher accuracy but sacrifices its performance on peace outcomes. This biased result will be accepted as the prediction of conflict is considered more important than the prediction of peace.
5
Fuzzy Rule Extraction
The TS neuro-fuzzy model used for forecasting in the previous section can also be used for rule extraction. Two fuzzy rules can be extracted from the model and they are shown below. 1. If u1 is A11 and u2 is A12 and u3 is A13 and u4 is A14 and u5 is A15 and u6 is A16 and u7 is A17 then y(k) = −1.86 · 10−1 u1 − 1.33 · 10−1 u2 + 0.00 · 100 u3 − 6.05 · 10−1 u4 − 1.26 · 10−1 u5 − 1.33 · 100 u6 + 4.71 · 10−1 u7 + 8.95 · 10−1
1092
T. Tettey and T. Marwala
Fig. 2. A ROC illustrates the performance degradation of the neuro-fuzzy model when several inputs are pruned
2. If u1 is A21 and u2 is A22 and u3 is A23 and u4 is A24 and u5 is A25 and u6 is A26 and u7 is A27 then y(k) = −2.79 · 10−1 u1 + 6.26 · 10−2 u2 + 2.47 · 10−1 u3 − 7.56 · 10−1 u4 − 8.85 · 10−1 u5 − 9.04 · 100 u6 + 0.00 · 100 u7 + 3.73 · 10−1 The symbols from u1 to u7 are the input vector which consists of People oriented Government, Dependancy, Capability, Alliance, Contiguity, Distance and Major power. The rest of the symbols are as previously defined. It is clear that the rules are quite complex and need to be simplified in order to obtain a didactic interpretation. In fact it is often found that when automated techniques are applied to obtaining fuzzy models, unnecessary complexity is often present [13]. Setnes et al [13] present a similarity measure for fuzzy rule based simplification. Two methods of simplifying rules that are proposed are the removal and/or merging of similar fuzzy sets. Removal of a fuzzy set is proposed in the event that it is similar to the universal set i.e. μ ≈ 1 and the merging of fuzzy sets is proposed in the event that fuzzy sets from different rules but belonging to the same premise are similar. In our case the TS fuzzy model contains only two fuzzy rules. The removal of a fuzzy sets similar to the universal set leaves only one remaining fuzzy set. This results in the input being partitioned into only one fuzzy set and therefore introduces difficulty when expressing the premise in linguistic terms. To simplify the fuzzy rules and avoid the redundant fuzzy sets the number of inputs into the TS neuro-fuzzy model have been pruned down to four variables. These variables are People oriented Government, Dependancy, Alliance and Contiguity. Fig 2 illustrates how the output deteriorates when three of the inputs are pruned. The ROC curve shows that the performance degradation is minimal. The rules extracted can then be converted so that they are represented in the commonly used linguistic terms. However it is only possible to translate the
Neuro-fuzzy Modeling and Fuzzy Rule Extraction
1093
antecedent of the fuzzy statement into english. The consequent part together with the firing strength of the rule are still expressed mathematically. The translated fuzzy rules with the firing strengths omitted can be written as shown below. 1. If Government orienting towards people is low and Alliance is strong and Contiguity is true then y(k) = −3.87 · 10−1 u1 − 9.19 · 10−1 u3 − 7.95 · 10−1 u4 + 3.90 · 10−1 2. If Government orienting towards people is high and Alliance is weak and Contiguity is false then y(k) = −1.25 · 10−1 u1 − 5.62 · 10−1 u3 − 2.35 · 10−1 u4 + 4.23 · 10−1 To validate the model we can then apply expert knowledge of the problem domain. For instance if the level of people oriented Government of two countries is low, they have a weak alliance and they share a border there is a reasonable chance that the countries can find themselves in a conflict situation. If we find values of Majority rule, Alliance and Contiguity which have a membership value of one, we can then use these as inputs to the model to see if it confirms our existing knowledge. It is found that by using these values and and an arbitrary Dependency value the model gives a prediction of 0.6743 which is above the conflict threshold of 0.5360 calculated from the ROC curve. By validating the model with similar statements, we can get a feel for how much confidence we can put in the system. The neuro-fuzzy model therefore offers an improved method of forecasting international conflict as it allows for accurate prediction of dispute outcomes while also catering for the cases where causal interpretations are required.
6
Conclusion and Recommendataion
A background on conflict management has been presented in the form of a literature survey. The methods that have been used in the past to predict interstate disputed have been highlighted. The performance criteria that is expected of a forecasting model has been stated i.e. a satisfactory model must be able to accurately predict dispute outcome and at the same time be able to give a causal explaination of the results. It has been found that previous models have either lacked a sufficient prediction ability or transparency. A Takagi-Sugeno fuzzy model which is trained using concepts from neural network learning is then proposed. It is shown that the TS model is able to predict conflict cases with an accuracy of 80.36%. Further, the TS model is expressed as a set of fuzzy rules which are made readable by expressing them using common linguistic terms. The TS model therefore is able to meet the specified peformance criteria. However, we recommend the Takagi-Sugeno model for prediction only as it is able to predict the conflict cases accurately. The transparency of the fuzzy model allows for improved validation and therefore increases the chances of user acceptance. It is not advisable for the system to be used to source expert knowledge because of its accuracy. A prediction accuracy of 80.36% is considered good
1094
T. Tettey and T. Marwala
especially on a rare-event prediction problem. However, this means that under certain conditions the fuzzy rules may offer incorrect knowledge. To increse the validity of the model we also recommend that the way some of the interstate variables are expressed be reviewed. For instance, variable such as contiguity have values of either true or false. In the fuzzy domain these are considered exceptions and are commonly termed as fuzzy singleton. Therefore, expressing these fuzzy sets as Gaussian membership functions might introduce flaws into the model.
References 1. N. Beck, G.K., Zeng, L.: Improving quantitative studies of international conflict: A conjecture. American Political Science Review 94(1) (2000) 21–35 2. Jones, D., Bremer, S., Singer, J.: Militarized interstate disputes, 1816-1992 rationale, coding rules and empirical patterns. Conflict Management and Peace Science 15(2) (1996) 585–615 3. Gochman, C., Maoz, Z.: Militarized interstate disputes 1816-1976. Conflict resolution 28(4) (1984) 585–615 4. : Correlates of war. (Internet Listing) URL: http://www.correlatesofwar.org/. 5. Lagazio, M., Russett, B.: A Neural Network Analysis of Militarised Disputes, 1885-1992: Temporal Stability and Causal Complexity. In: Temporal Stability and Causal Complexity. University of Michigan Press, New Jersey (2003) 28 –62 6. Schrodt, P.: Patterns, rules and learning: Computational models of international behaviour. (Unpublished manuscript available at) URL: http://www.ku.edu/simkeds/papers.dir/Schrodt.PRL.2.0.pdf. 7. Oneal, J., Russet, B.: Prediction and classification with neural network models. Sociological Methods and Research 4(3) (1999) 499–524 8. Mackay, D.: A practical bayesian framework for backpropagation networks. Neural Computing 4(3) (1992) 448–472 9. Marwala, T., Lagazio, M.: Modeling and controlling interstate conflict. In: Proceedings of the IEEE International Joint Conference on Neural Networks, Budapest, Hungary, IEEE (2004) 1233–1238 10. Habtemariam, E.A., Marwala, T.: Artificial intelligence for conflict management. In: Proceedings of the IEEE International Joint Conference on Neural Networks, Montreal, Canada, IEEE (2005) 2583–2588 11. Jang, J., Sun, C., Mizutani, E.: Neuro-fuzzy and soft computing: A computational approach to Learning and Machine Intelligence. First edn. Prentice Hall, New Jersey (1997) 12. Babuska, R., Verbruggen, H.: Neuro-fuzzy methods for nonlinear sytem identification. Annual Reviews in Control 27 (2003) 73–85 13. Sentes, M., ska, R.B., Kaymak, U., van Nauta Lemke, H.: Similarity measures is fuzzy rule based simplification. IEEE Transactions on Systems, Man and Cybernatics-Part B: Cybernatics 28(3) (1998) 376–386
Abstract. This paper outlines all the computational methods which have been applied to the conflict management. A survey of all the pertinent literature relating to conflict management is also presented. The paper then introduces the Takagi-Sugeno fuzzy model for the analysis of interstate conflict. It is found that using interstate variables as inputs, the Takagi-Sugeno fuzzy model is able to forecast conflict cases with an accuracy of 80.36%. Furthermore, it found that the fuzzy model offers high levels of transparency in the form of fuzzy rules. It is then shown how these rules can be translated in order to validate the fuzzy model. The Takagi-Sugeno model is found to be suitable for interstate modeling as it demonstrates good forecasting ability while offering a transparent interpretation of the modeled rules.
1
Introduction
With the frequency at which wars are occurring, it has become imperative that more research effort be directed towards conflict management. The focus of this study over the years has been centered on finding improved approaches to conflict forecasting at the same time not neglecting causal intepretation of interstate interactions [1]. Understanding the reasons why countries go to war is just as significant as forecasting the onset of war because it proposes the steps that can be taken to avoid conflict. Therefore a successful interstate conflict tool is one which is able to forecast dispute outcomes fairly accurately at the same time allowing for an intuitive causal intepretation of interstate interactions. International conflict has been studied using mainly techniques found in the fields of Statistics and Computational intelligence. These techniques have been applied on quantitative measures which have been collected over the years. It is widely accepted that improvements in conflict forecasting can mainly be acheived in two ways [1]. The first improvement that can be made is with the data and measures of interstate interactions [2] and secondly by improving the forecasting of interstate conflict involves finding models which approximate interstate interactions better. I. King et al. (Eds.): ICONIP 2006, Part III, LNCS 4234, pp. 1087–1094, 2006. c Springer-Verlag Berlin Heidelberg 2006
1088
T. Tettey and T. Marwala
In this paper a survey of the work performed on international conflict is presented. The paper focuses mainly on the techniques applied to analyse quantitative descriptions of interstate interactions. An outline of the methods together with their shortcomings is discussed. Finally, neuro-fuzzy modeling is presented as an alternative technique which addresses the current shortcomings that exist in international conflict studies.
2
Background and Literature Survey
Militarised interstate dispute (MID) is defined as a set of interactions between or among states that can result in the actual use, display or threat of using military force in an explicit way [3]. Projects such as the Correlates of War (COW) facilitate the collection, dissemination and use of accurate and reliable quantitative data in international relations [4]. The collected data, called interstate variables, are used to study the conditions associated with MID. The measures used in MID studies are People Oriented Government, Dependency, Capability, Alliance, Contiguity, Distance and Major power. Any set of measures describing a particular context has a dispute outcome attached to it. The dispute outcome is either a peace or conflict situation. Statistical methods such as logit and probit have been used in the analysis of interstate variables. However, it has been found that these methods have several shortcomings [1]. Some of the problems associated with the use of logit and probit is that they require the use of a priori knowledge usually obtained from the analyst. A problem then arises when the analyst pushes their data analyses extremely hard in search of effects they believe exist but are difficult to discover [1]. The consequence of this is that the results vary from researcher to researcher and are therefore not exactly repeatable. The other problem, as one might expect, is that conflict cases occur far less frequently than peace cases. Interstate conflict is therefore a rare event and the processes which drive it are likely to be different from those found elsewhere. This has led quantitative researchers to conclude that the relationship between the interstate variables and dispute outcomes is highly nonlinear and highly correlated [1]. The conclusion is further confirmed by the studies performed by Lagazio and Russet [5]. This means that statistical techniques, linear-normal models in particular, would perform poorly at modeling the relationship between interstate disputes and their outcomes. The neural network has also been applied to interstate conflict modeling and forecasting. The neural network was first introducted by Schrodt [6] in 1995 and by Zeng [7] in 1999 as a method of analysing conflict without the need for the researcher to incorporate qualitative a priori knowlege or make assumptions about the problem space. The neural network was presented as a function approximator which is able to model highly nonlinear and interdependant relationships. However the neural network itself suffers similar problems to statistical methods in that a model selection technique must be considered. In recent studies, Beck et al [1] make use of a neural network, which is trained using the Bayesian framework outlined in [8]. The Bayesian training of neural networks involves
Neuro-fuzzy Modeling and Fuzzy Rule Extraction
1089
the use of Bayesian framework to identify the optimal weights and biases in a neural network model. It is found that the use of neural networks yields results expressed in the form of classification accuracy. This interpretation of the results is found to be unambigious compared to previous methods. However, the resulting neural network model is regarded as a black box due to the fact that it does not provide a way of obtaining a causal interpretion of dispute outcomes. The weights extracted from the neural network offer no understanding as to why countries go to war. In [9], Marwala and Lagazio propose the use of Automatic Relevance Detection (ARD) as a means to making the neural network more transparent. The result of ARD reveals that the importance of the interstate variable in predicting dispute outcomes is as follows (listed in decreasing relavance): Peoples Oriented Government, Capability, Dependancy, Allies, Contiguity, Distance and Major power. From this work on neural networks we can conclude that neural network models have a fairly strong forecasting ability but only a limited amount of knowledge can be extracted. In [10], Habtemariam et al introduce support vector machines (SVMs) to the study of conflict management. It is found that SVMs offer an improved forecasting ability over neural networks. However, a sensitivity analysis which aims to determine the influence of each variable on a dispute outcome reveals that results obtained from neural networks are much more intuitive. Therefore, while SVMs offer better forecasting ability they lack the ability to give an intuitive causal interpretation of the results. As stated earlier on in the paper, the main focus of studies in international conflict has been on the ability of a model to accurately forecast dispute outcomes while at the same time allow the analyst to extract knowledge from the model. In the next section a Neuro-fuzzy model is proposed as a method of modeling interstate interaction which has a fairly accurate forecasting ability and at the same time offers intuitive causal explainations of disputes obtained from a fuzzyrule extraction process.
3
Fuzzy Systems and Neuro-fuzzy Modeling
Fuzzy logic concepts provide a method of modelling imprecise models of reasoning, such as common sense reasoning, for uncertain and complex processes. Fuzzy set theory resembles human reasoning in its use of approximate information and uncertainty to generate decisions. In fuzzy systems, the evaluation of the output is performed by a computing framework called the fuzzy inference system. The fuzzy inference system is a computing framework that maps fuzzy or crisp inputs to the output - which is usually a fuzzy set [11]. The inference system performs a composition of the inputs using fuzzy set theory, fuzzy if-then rules and fuzzy reasoning to arrive at the output. More specifically, the fuzzy inference involves the fuzzification of the input variables, evaluation of rules, aggregation of the rule outputs and finally the defuzzification of the result. The are two popular fuzzy models: the Mamdani model and the Takagi-Sugeno (TS) model [12].
1090
T. Tettey and T. Marwala
A fuzzy rule-based system can be viewed as a layered network similar to Radial basis function (RBF) artificial neural networks [12]. When training RBF networks several parameters of the kernel have to be optimised. Similarly when setting up a Fuzzy rule-based system we are required to optimise parameters such as membership functions and consequent parameters. In order to optimise these parameters, the neuro-fuzzy system relies on training algorithms inherited from artificial neural networks such as gradient descent [12]. There are two approaches to training neuro-fuzzy models [12]: 1. Fuzzy rules may be extracted from expert knowledge and used to create an initial model. The parameters of the model can then be fine tuned using data collected from the operational system being modelled. 2. The number of rules can be determined from collected numerial data using a model selection technique. The parameters of the model are also optimised using the existing data. The Takagi-Sugeno model is mostpopular when it comes to data-driven identification and has been proven to be a universal approximator [11]. The major motivation for using a neuro-fuzzy model in this work is that it is considered by many to be a ‘gray-box’ [12]. Unlike neural networks, once the neuro-fuzzy model has been optimised it is possible to extract the fuzzy rules and perhaps interpret the obtained results in an intuitive and qualitative manner. With neural networks e.g. the multilayer perceptron, it is not possible to intuitively explain input-output relationships using the weights of the network. In this study the neuro-fuzzy hybrid system is used to model the relationship between interstate (input) variables and the outcome of a dispute (output) from MID data.
4
Conflict Forecasting Using a Neuro-fuzzy Model
In our study, the TS neuro-fuzzy model has been optimised to map the relationship between the input variables (interstate variables) and the output i.e. the dispute outcome. The neuro-fuzzy model is trained using a balanced set of 500 peace cases and 500 conflict cases. The remaining 26845 peace instances and 392 conflict instance are then used for testing the forecasting ability of the model. The model selection process involves selecting the optimum number of rules of the TS model. This is done by creating models with rules ranging from 2 to 7 and evaluating them using 5 fold cross-validation. It is found that the optimum model consists of 2 fuzzy rules. The forecasting ability of the neuro-fuzzy model is then tested using the test examples. The prediction ability of the neuro-fuzzy model is evaluated based on how well it is able to predict both conflict and peace outcomes. The receiver operating characteristic (ROC) curve in fig 1 is used to illustrate the results. As the output of the neuro-fuzzy is expressed as a decision value, an optimum threshold needs to be determined as the decision point which allows the overall
Neuro-fuzzy Modeling and Fuzzy Rule Extraction
1091
Fig. 1. A ROC curve illustrating the performance of the neuro-fuzzy model Table 1. Results for the TS neuro-fuzzy model
Correctly predicted Incorrectly predicted
Conflict cases Peace cases 315 17967 77 8378
maximum peace and conflict prediction accuracy. This threshold is found to be 0.488 and the results are expressed as a confusion matrix shown in Table 1. The results show the TS neuro-fuzzy model predict conflict cases with an accuracy of 80.36% while predicting peace cases with an accuracy of 66.93%. In the work done by Habtemariam et al [10], it was found that SVM predicts peace and conflict with 79% and 75%, respectively. From these results it is clear that the TS neuro-fuzzy model predicts conflict with a much higher accuracy but sacrifices its performance on peace outcomes. This biased result will be accepted as the prediction of conflict is considered more important than the prediction of peace.
5
Fuzzy Rule Extraction
The TS neuro-fuzzy model used for forecasting in the previous section can also be used for rule extraction. Two fuzzy rules can be extracted from the model and they are shown below. 1. If u1 is A11 and u2 is A12 and u3 is A13 and u4 is A14 and u5 is A15 and u6 is A16 and u7 is A17 then y(k) = −1.86 · 10−1 u1 − 1.33 · 10−1 u2 + 0.00 · 100 u3 − 6.05 · 10−1 u4 − 1.26 · 10−1 u5 − 1.33 · 100 u6 + 4.71 · 10−1 u7 + 8.95 · 10−1
1092
T. Tettey and T. Marwala
Fig. 2. A ROC illustrates the performance degradation of the neuro-fuzzy model when several inputs are pruned
2. If u1 is A21 and u2 is A22 and u3 is A23 and u4 is A24 and u5 is A25 and u6 is A26 and u7 is A27 then y(k) = −2.79 · 10−1 u1 + 6.26 · 10−2 u2 + 2.47 · 10−1 u3 − 7.56 · 10−1 u4 − 8.85 · 10−1 u5 − 9.04 · 100 u6 + 0.00 · 100 u7 + 3.73 · 10−1 The symbols from u1 to u7 are the input vector which consists of People oriented Government, Dependancy, Capability, Alliance, Contiguity, Distance and Major power. The rest of the symbols are as previously defined. It is clear that the rules are quite complex and need to be simplified in order to obtain a didactic interpretation. In fact it is often found that when automated techniques are applied to obtaining fuzzy models, unnecessary complexity is often present [13]. Setnes et al [13] present a similarity measure for fuzzy rule based simplification. Two methods of simplifying rules that are proposed are the removal and/or merging of similar fuzzy sets. Removal of a fuzzy set is proposed in the event that it is similar to the universal set i.e. μ ≈ 1 and the merging of fuzzy sets is proposed in the event that fuzzy sets from different rules but belonging to the same premise are similar. In our case the TS fuzzy model contains only two fuzzy rules. The removal of a fuzzy sets similar to the universal set leaves only one remaining fuzzy set. This results in the input being partitioned into only one fuzzy set and therefore introduces difficulty when expressing the premise in linguistic terms. To simplify the fuzzy rules and avoid the redundant fuzzy sets the number of inputs into the TS neuro-fuzzy model have been pruned down to four variables. These variables are People oriented Government, Dependancy, Alliance and Contiguity. Fig 2 illustrates how the output deteriorates when three of the inputs are pruned. The ROC curve shows that the performance degradation is minimal. The rules extracted can then be converted so that they are represented in the commonly used linguistic terms. However it is only possible to translate the
Neuro-fuzzy Modeling and Fuzzy Rule Extraction
1093
antecedent of the fuzzy statement into english. The consequent part together with the firing strength of the rule are still expressed mathematically. The translated fuzzy rules with the firing strengths omitted can be written as shown below. 1. If Government orienting towards people is low and Alliance is strong and Contiguity is true then y(k) = −3.87 · 10−1 u1 − 9.19 · 10−1 u3 − 7.95 · 10−1 u4 + 3.90 · 10−1 2. If Government orienting towards people is high and Alliance is weak and Contiguity is false then y(k) = −1.25 · 10−1 u1 − 5.62 · 10−1 u3 − 2.35 · 10−1 u4 + 4.23 · 10−1 To validate the model we can then apply expert knowledge of the problem domain. For instance if the level of people oriented Government of two countries is low, they have a weak alliance and they share a border there is a reasonable chance that the countries can find themselves in a conflict situation. If we find values of Majority rule, Alliance and Contiguity which have a membership value of one, we can then use these as inputs to the model to see if it confirms our existing knowledge. It is found that by using these values and and an arbitrary Dependency value the model gives a prediction of 0.6743 which is above the conflict threshold of 0.5360 calculated from the ROC curve. By validating the model with similar statements, we can get a feel for how much confidence we can put in the system. The neuro-fuzzy model therefore offers an improved method of forecasting international conflict as it allows for accurate prediction of dispute outcomes while also catering for the cases where causal interpretations are required.
6
Conclusion and Recommendataion
A background on conflict management has been presented in the form of a literature survey. The methods that have been used in the past to predict interstate disputed have been highlighted. The performance criteria that is expected of a forecasting model has been stated i.e. a satisfactory model must be able to accurately predict dispute outcome and at the same time be able to give a causal explaination of the results. It has been found that previous models have either lacked a sufficient prediction ability or transparency. A Takagi-Sugeno fuzzy model which is trained using concepts from neural network learning is then proposed. It is shown that the TS model is able to predict conflict cases with an accuracy of 80.36%. Further, the TS model is expressed as a set of fuzzy rules which are made readable by expressing them using common linguistic terms. The TS model therefore is able to meet the specified peformance criteria. However, we recommend the Takagi-Sugeno model for prediction only as it is able to predict the conflict cases accurately. The transparency of the fuzzy model allows for improved validation and therefore increases the chances of user acceptance. It is not advisable for the system to be used to source expert knowledge because of its accuracy. A prediction accuracy of 80.36% is considered good
1094
T. Tettey and T. Marwala
especially on a rare-event prediction problem. However, this means that under certain conditions the fuzzy rules may offer incorrect knowledge. To increse the validity of the model we also recommend that the way some of the interstate variables are expressed be reviewed. For instance, variable such as contiguity have values of either true or false. In the fuzzy domain these are considered exceptions and are commonly termed as fuzzy singleton. Therefore, expressing these fuzzy sets as Gaussian membership functions might introduce flaws into the model.
References 1. N. Beck, G.K., Zeng, L.: Improving quantitative studies of international conflict: A conjecture. American Political Science Review 94(1) (2000) 21–35 2. Jones, D., Bremer, S., Singer, J.: Militarized interstate disputes, 1816-1992 rationale, coding rules and empirical patterns. Conflict Management and Peace Science 15(2) (1996) 585–615 3. Gochman, C., Maoz, Z.: Militarized interstate disputes 1816-1976. Conflict resolution 28(4) (1984) 585–615 4. : Correlates of war. (Internet Listing) URL: http://www.correlatesofwar.org/. 5. Lagazio, M., Russett, B.: A Neural Network Analysis of Militarised Disputes, 1885-1992: Temporal Stability and Causal Complexity. In: Temporal Stability and Causal Complexity. University of Michigan Press, New Jersey (2003) 28 –62 6. Schrodt, P.: Patterns, rules and learning: Computational models of international behaviour. (Unpublished manuscript available at) URL: http://www.ku.edu/simkeds/papers.dir/Schrodt.PRL.2.0.pdf. 7. Oneal, J., Russet, B.: Prediction and classification with neural network models. Sociological Methods and Research 4(3) (1999) 499–524 8. Mackay, D.: A practical bayesian framework for backpropagation networks. Neural Computing 4(3) (1992) 448–472 9. Marwala, T., Lagazio, M.: Modeling and controlling interstate conflict. In: Proceedings of the IEEE International Joint Conference on Neural Networks, Budapest, Hungary, IEEE (2004) 1233–1238 10. Habtemariam, E.A., Marwala, T.: Artificial intelligence for conflict management. In: Proceedings of the IEEE International Joint Conference on Neural Networks, Montreal, Canada, IEEE (2005) 2583–2588 11. Jang, J., Sun, C., Mizutani, E.: Neuro-fuzzy and soft computing: A computational approach to Learning and Machine Intelligence. First edn. Prentice Hall, New Jersey (1997) 12. Babuska, R., Verbruggen, H.: Neuro-fuzzy methods for nonlinear sytem identification. Annual Reviews in Control 27 (2003) 73–85 13. Sentes, M., ska, R.B., Kaymak, U., van Nauta Lemke, H.: Similarity measures is fuzzy rule based simplification. IEEE Transactions on Systems, Man and Cybernatics-Part B: Cybernatics 28(3) (1998) 376–386
