Comparison of Crisp Systems and Fuzzy Systems in
Comparison of Crisp Systems and Fuzzy Systems in Agent-Based Simulation A case study of Soccer Penalties Tuong Manh Vu
Peer-Olaf Siebers
Christian Wagner
School of Computer Science University of Nottingham Nottingham, UK [email protected]
School of Computer Science University of Nottingham Nottingham, UK [email protected]
School of Computer Science University of Nottingham Nottingham, UK [email protected]
Abstract— The Belief-Desire-Intention (BDI) software model is an example of a reasoning architecture for a bounded rational software agent. In our research we plan to expand the application of the BDI software model to the area of simulating human behaviour in social and socio-technical systems. To this effect, in this paper we explore the differences in using a classical crisp rule-based approach and a fuzzy rule-based approach for the reasoning within the BDI system. As a test case we have chosen a football penalty shootout. We have kept the case study example deliberately simple so that we can focus on the effects the different BDI implementations have on the decisions made. Our experiments highlight that the crisp system can result in unwanted "preferred" actions because of sudden leaps or drops between different ranges of decision variables, while the fuzzy system results have smoother transitions which results in more consistent decisions. The behaviour, as showcased in this simple context, underlines that a change from crisp to fuzzy rule based systems as the underlying reasoning model in BDI systems can provide the path to a superior approach for the simulation of human behaviour, which we will explore further in the future. Keywords— agent based simulation, belief-desire-intention, fuzzy system, soccer penalty
I.
INTRODUCTION
Agent-Based Simulation (ABS) is a powerful technique that has been growing recently. In ABS, a model contains many autonomous agents, which can observe the environment and make decisions based on their states and sets of rules. However, using agent decision making to simulate human behaviour is difficult, since it involves many different and complex factors such as feelings, mood, intention, surrounding area, interaction with other agents, and what other agents are doing. These factors, which constitute categories whose boundaries are not easy to sharply define, are fuzzy in nature. To handle decision making in such a fuzzy environment, Bellman [1] suggested using a fuzzy system rather than a classical crisp system. Recently, with the help of software packages, one can design a simple fuzzy system without indepth knowledge about the classical fuzzy theory. Still, there are only a few applications of fuzzy logic systems used for agent decision making.
For implementing agent decision making, different researchers from many disciplines have come up with different methods of modelling human behaviour leading to a variety of frameworks such as Belief-Desire-Intention (BDI) [2], CaseBased Reasoning (CBR) [3], Qualitative Decision Theory (QDT) [4], Markov Decision Process (MDP) [5]. In this paper, the BDI framework will be used to perform decision making of agents because it allows the modelling of human behaviour with a high degree of fidelity. BDI is a framework based on theory of the rational actions proposed by Bratman et al [6]. The BDI framework imitates human thinking process with belief (what agents believe), desire (what agents want), and intention (what agents try to do). The BDI model is also open to embed other models into its components. One example is that a fuzzy system has been embedded into BDI agents of BattleCity game for better handling of the uncertainty of a complex virtual environment, which a classical rule-based system often fails to obtain [7]. Using the BDI framework as a base, this project analyses the influence of embedding fuzzy rule-based systems in contrast to crisp rule-based systems. Moreover, in order to test the applicability of two rulebased systems, we use a case study of “soccer penalty”, in which a kicker will try to score a goal against a goalkeeper. There is a comparison of fuzzy and crisp decision rules in System Dynamics simulation by Polat [8], and he concluded that the superiority of one of these systems depends on the situation. But in this paper, the investigation of the two different decision making systems is in ABS. The goal of the research is to examine how different methods of modelling decision-making affect agent behaviours. The rest of paper is organized as follows. Section II provides background information on Agent-Based Simulation, the BDI model, and fuzzy system. In section III, the implementation of soccer penalty is described in detail. Section IV compares and discusses the results of the crisp systems and fuzzy systems. Finally, section V provides a summary of results and several directions for future work.
II.
BACKGROUND
A. Agent-Based Simulation Simulation is an imitation of a system, which involves designing the model and performing experiment to have better understanding of the system. While the model is an abstract representation of the real system, the simulation represents the operation of the system. The nature of simulation is predicting what will happen to the system if certain actions are taken. So it is the job of a user to vary the inputs, perform actions to see the changes in operations and get a better understanding of the system. In a way, simulation is a support decision making system because after gaining the sufficient understanding, user can decide on how improve the real system. Agent Based Simulation is one of main paradigms in simulation. It is an individual-centric, decentralized approach. When designing the model, we identify the entities/agents, define their behaviours, put them in an environment, establish connections, and run the simulation. In ABS, a system is modelled as a collection of agents, which are autonomous decision-making unit with diverse characteristics. The interaction of much individual behaviour of agents results in the global behaviour of the system. ABS has become a powerful technique with a number of applications: flows simulation (evacuation, traffic, and customer flow management), organizations simulation (operational risk and organizational design), markets simulation (stock market, and strategic simulation), and diffusion simulation (diffusion of innovation and adoption dynamics) [9]. ABS is suited for the simulation of human-centric system [10]. Human are heterogeneous, autonomous and proactive entities. An agent is a very good representation for a human, because agents have following properties [11]: • Discrete entities: with their own behaviour, goals, thread of control • Autonomous: be able to adapt and modify their behaviour • Proactive: adjust action depending on agent’s internal state. B. Decision making using Belief-Desire-Intention model For decision making different approaches can be used, one of which is the BDI model. BDI model is based on rational actions and originally proposed by Bratman [6]. Later, BDI model has been well studied [2] and possibly the best known and studied model of practical reasoning agent [12]. In BDI architecture, the rational agent has three main mental attitudes: • Belief: agent’s knowledge environment and itself.
about
the
state
Belief Intention Desire Fig. 1. Belief-Desire-Intention framework
In addition, the information about the actions of achieving a certain goal, and options available to agent is represented as plan. With the help of plan library, the agent does not need to search through the entire space of possible actions. In the next section, we will revise the definition of crisp set, then introduce fuzzy set concept. C. Crisp Set versus Fuzzy Set Classical Sets, also called Crisp Sets, can be described by a characteristic function: 1, 0, Set A (in a universe of discourse X) can be defined as A = {x | x meets some conditions}. Set A is mathematically equivalent to its characteristic function above. For example x|a . A fuzzy set F is a generalisation of a crisp set. It is characterized by the membership function μF(x) that takes on values in the interval [0,1]. A membership function provides a measure of the degree of membership of an element in the universal of X to the fuzzy set. Related to the crisp set example, let A = x is roughly in [a,b], then the membership function will be: 0,1 A fuzzy set F in X may be represented as a set of ordered pairs of a generic element x and its grade of membership function μF(x). | , With fuzzy sets, we can construct a fuzzy logic rule system. D. Fuzzy Logic Rule System In general, a Fuzzy Logic Rule System (FLRS), also called fuzzy control system, has structure as Fig. 2 [13]:
of
• Desire: motivational state of the system that the agent wants to accomplish. • Intention: current course of action that the agent chooses to perform.
Action
Fig. 2. Fuzzy rule system
The most commonly used FLRS is Mamdani FLRS, created by Professor Mamdani when building a fuzzy rule system to control a steam engine and boiler [14]. The fuzzy rule system will take one crisp input x and return one crisp output y. The system contains four components: rules, fuzzifier, inference engine, defuzzifier. • The fuzzifier maps the crisp input x to a fuzzy set. • Rules can be provided by experts or extracted from numerical data. The rules are collections of one or many IF-THEN statements.
B. Implementation The model is implemented in AnyLogic (a multi-method simulation modelling tool developed by XJ Technologies [18]) and simulates a soccer penalty where a kicker tries to score a goal against a goalkeeper. In the model, there are two Fuzzy Logic Systems which were implemented using Juzzy, a Javabased toolkit for type-1 and type-2 fuzzy logic systems [19]. An online version of the model with simple control is available at RunTheModel website [20].
• The inference engine handles which rules are activated and how to combine the rule. Through the rules, the inference engine maps input to output sets.
The model is a 2D simulation with bird’s eye view and contains two agents (one kicker, one keeper), a ball, and a goal. Only the width of the goal, which is 7.32m, is simulated. The ball is 11m from the goal and the kicker has a short run of 3.5m before kicking the ball.
• The defuzzifier processes the fuzzy set to obtain crisp output y.
For simplification, a number of assumptions has been made:
III.
SOCCER PENALTY CASE STUDY
In order to test the different methods of decision making, we have decided to use a case study approach. The soccer penalty kick situation has been chosen as a case study because two players can be modelled very well using agents. And their decision making processes are complicated enough for implementing the BDI framework as a platform to explore crisp versus fuzzy control based inference. In addition, because this case study is simple to understand, we can focus more on decision making mechanism. Once we implement and have better understanding, we can go to more complex cases. A. Soccer Penalty In soccer penalty, Greg Wood [15] defined kicking strategies in terms of the relationship between the kicker’s last fixation on the target prior to shooting, and resultant ball destination: • A last fixation on the goalkeeper (who is within the center of the goal) coupled with a shot to any location was classified as the Keeper-Dependent (KD) strategy. • A last fixation close to the goal post (to the left or right of the center) coupled with a shot to the same side was classified as the Keeper-Independent (KI) strategy. • If the last fixation was to one side of the goal and the ball was shot to the other, this was classified as the Opposite-Independent (OI) strategy. In order to investigate the interaction between two players in decision making, we only consider the case of KeeperDependent strategy, in which the kicker looks at the keeper and decides the direction of the shoot as the keeper dives. Additionally, Kuhn [16] suggests that 70% of all shots utilise the KD strategy. But using the KD strategy means that the kicker has to change the kick direction in the last minute of the run-up. Van Der Kamp [17] reports that overall a kicker requires at least 400ms to alter kick direction. The report also concluded that if the kicker redirects the ball late in the run-up, this was at the expense of shot accuracy.
• Straight shots (no curved ball). • Wind does not affect the ball trajectory. • Ignore the stress caused by audience or team score. • Goalkeeper does not try to guess shoot direction, but randomly dives left or right. • If goalkeeper dives in the correct direction, he has a 20% chance of stopping the ball. • The kicker never shoots over the crossbar, and the ball never hits the goal post or crossbar. The model consists of two agents: Goalkeeper and Kicker. We use state charts (Fig. 3) to illustrate all the states agents can attain as well as transitions between states. • The kicker first waits, then starts to run and shoot. The decision making of the kicker will happen in the Wait state and RunUp state. In RunUp state, the kicker also reconsiders the decision using self-transition. • The goalkeeper will be waiting until the kicker gets close to the ball and dive. For simplification, the keeper will dive 400ms before the kicker reaches the ball. The decision making and reconsidering of the goalkeeper happens when waiting to dive.
Fig. 3. Statecharts of the kicker and the goalkeeper agent
C. Decision Making System 1) Decisions making of the kicker Since the goalkeeper will randomly dive, we can focus on analysing the complex decision of the kicker. The kicker has three states: Wait, RunUp, and Shoot. After the “Wait” state, he makes decision on how the shot will be carried out. That decision is made based on three input variables:
• Displacement [0,7.32]: determined by distance between gaze direction and target location. • Accuracy [0,1]: distance between target location and ball placement. • Target Area [0,7.32]: deducted from target location. • Risk [0,1]: the risk of the shot.
• Gaze direction: where the kicker look before the run-up.
Each variable is divided into many categories:
• Target location: where the kicker shoot the ball.
• Anxiety: Low, Medium, High
• Anxiety: anxiety level of the kicker.
• Displacement: Close, Average, Far
2) Decision making progress: • The kicker has a list of alternative decisions which generated based on direction of keeper diving. For example, if the keeper dives right, the target locations in the decision list will be on the left side of the goal. The gaze and target locations of alternative decisions are generated from 0 to 7.32 with the step of 0.1.
• Accuracy: Low, Medium, High
• For each decision, the decision making system takes three input variables and calculates the risk of that decision. The risk of a decision is evaluated by either crisp rule-based system or fuzzy rule-based system. • Finally, for the kicker to kick the ball, one of possible decisions is chosen using Roulette Wheel Selection: The risk value is used to associate a probability of selection with each decision. If ri is the risk of decision i in the list of n decisions, its probability is being selected . With ri=[0,1], the decision with lower is ∑ risk will have larger probability and be more likely to be selected. The advantage of this method is that even a decision with lower risk is less likely to be selected, there still is a chance that they might be.
Generate decision list
• Target Area: Area 1, Area 2, ... Area 8, Area 9 • Risk: Low, Medium, High For the crisp system, a category is represented by a crisp set. For instance, in crisp system, Anxiety value is Low if it is between 0 and 0.25 (Fig. 5). On the other hand, the fuzzy system uses a membership function to represent a category. For example, in fuzzy logic system, the Medium category of Anxiety variable is a triangular membership function with three parameters {0.1, 0.5, 0.9} (Fig. 6). The categories of variables are designed so that they are as similar as possible in both two systems. The bound of a category in crisp system is the middle point of “the last parameter of one membership function” and “the first parameter of the next membership function”. For example, in the crisp system, Medium category of Anxiety variable has the lower bound of 0.25, which is the middle point of: • First parameter of Medium membership function: 0.1 • Last parameter of Low membership function: 0.4
Evaluate each risk Crisp system
or
Fuzzy system
Roulette wheel selection One final decision Fig. 4. Decision making progress
3) Modelling of variables Both the crisp system and the fuzzy logic system use five variables: • Anxiety [0,1]: anxiety level of kicker.
Fig. 5. Variables in the crisp system
TABLE II. Target area
Fig. 6. Variables in the fuzzy system
4) Risk evaluation To evaluate the risk for each decision, the decision making system receives three inputs: Anxiety, Gaze Direction, and Target Location. Then the Risk is calculated using two subsystems: • Sub-system 1: Calculate the Accuracy using the Displacement and the Anxiety level. o
Inputs: Displacement, Anxiety level
o
Output: Accuracy
o
Inputs: Target Area, Accuracy
o
Output: Risk
Displacement Close Close Close Average Average Average Far Far Far
RULE LIST OF SUB-SYSTEM 1
Anxiety Low Medium High Low Medium High Low Medium High
Accuracy Low Medium High Low Medium High Low Medium High Low Medium High Low Medium High Low Medium High Low Medium High Low Medium High Low Medium High
Risk High High Medium High Medium Low High Medium Low High Medium Medium High High Medium High Medium Medium High Medium Low High Medium Low High High Medium
Overall risk (1=highest) 1
3
3
2
1
2
3
3
1
Rule Table I and II are used for calculation the Accuracy and Risk of each system. One example of the first rule from Table I: IF displacement IS Close AND anxiety IS Low THEN Accuracy is High. Further detail on calculating using a rule is explained in next sections for both systems. The rule tables have four columns: • The first two columns are categories of inputs.
• Sub-system 2: Take the two inputs of Target Area and Accuracy (output of sub-system 1), and calculate the Risk of the decision.
TABLE I.
Area1 Area1 Area1 Area2 Area2 Area2 Area3 Area3 Area3 Area4 Area4 Area4 Area5 Area5 Area5 Area6 Area6 Area6 Area7 Area7 Area7 Area8 Area8 Area8 Area9 Area9 Area9
RULE LIST OF SUB-SYSTEM 2
Accuracy High High Medium Medium Medium Low High Medium Low
Overall accuracy (1=highest) 1
2
3
• The third column is the category of output. • The fourth column summaries the average value of the outputs, based on first variable. For instance, in table I the Accuracy is higher for Close Displacement than Average Displacement. In rule table I, the closer the displacement is, the higher the accuracy is. In rule table II, areas near the goal post and in the middle are the most dangerous (area 1, 5, 9); areas close to the goalkeeper (area 4, 6) are less dangerous; areas 2, 3, 7, 8 are the safest. 5) Crisp rule-based systems The crisp system evaluates the risk as follows: • Sub-system C1 takes Displacement and Anxiety level as inputs and determines which categories they belong to. Use the rule table I to find out the category of Accuracy (output). Return the mean value in the range of that category as output of C1. • Sub-system C2 takes the Target Area and Accuracy (output of C1) as inputs and determines which categories they belong to. Use the rule table II to find out the category of Risk. Return a mean value in the
range of that category as output of C22. That output will be the risk of a decision.
IV.
EXPERIMENTTATION AND RESULTS
Because of the randomisaation when selecting the final decision using Roulette Wheel Selection, the simulation model had been run for 10,000 timees for 10 steps of anxiety level varied from 0 to 1. For every shot, the final decision with all related information is saved to a file for later analysis. A. Experiment 1: Successful shhots related to Anxiety The first experiment invesstigates how the percentage of successful shots of both system ms varies according to the anxiety variable (Fig. 9). With the fuzzy system, thee successful shots remain stable between 0 and 0.6, and start to drop slightly from anxiety level 0.7, and reach the lowest poinnt at anxiety 0.8. With the crisp system, the successful shots sttay constant between 0 and 0.7, and then fall considerably at annxiety level 0.8.
6) Fuzzy rule-based systems To evaluate the risk, the fuzzy logic systeem uses two fuzzy inference sub-systems (Fig. 8). Each fuzzy sub-system will calculate the output using Mamdani method: • Take two crisp inputs • Fuzzification of crisp inputs: find out the value of membership function of each input. • Rule evaluation: check the rule table to see which rules are fired. • Aggregation of rule outputs: aggregatte the fuzzy output into one fuzzy set. o Gravity” of the • Defuzzification: return the “Center of aggregated fuzzy set as output.
The similar drop happens with w the fuzzy system, but since the High membership functioon of Anxiety variable starting from 0.6, the successful shotss start to drop from that point. Moreover, because the degreee of High anxiety is increasing gradually from 0.6, the numbeer of successful shots decreases accordingly (not a sudden drop d when comparing to crisp system). So the first conclusion is thhat the result of crisp system can have a sudden change when variable is changing from one category/range to another. On the other hand, the result of the fuzzy system will be affectedd by how fast the degree of a membership function changes.
%goal
Fig. 7. Crisp system
For the crisp system, the suudden drop at anxiety 0.8 can be explained by the anxiety level categories and rule tables. The anxiety level is categorized as “High” if it is greater than 0.75. And with high level of anxiety,, the rule table shows a low level of accuracy which leads to more m misses, even if the kicker chose to shoot in the safer zonee.
89 87 85 83 81 79
Crisp Fuzzy 0
0.2 0.4 0.6 0 0.8
1
Anxiety Fig. 9. Percentage of goal
B. Experiment 2: Kicker’s targget location Now, consider how the kickker targets the ball location. Fig. 10 shows the distribution of target t locations over the 7.32m width of the goal. The distributions are highlyy related to the rule tables: • For both systems, there is less focus on the middle and two areas near the goaal posts (area 1, 5, 9: the most Fig. 8. Fuzzy system
Number of times
• When using the crisp rule-based system, there are two peaks at location 2.2 and 5.1.
Accuracy
dangerous area in Rule Table II), and more focus on the safer areas (area 2, 3, 7, 8).
4000 3000
1 0.8 0.6 0.4 0.2 0
Crisp Fuzzy 0
2000 1000
Crisp
0
Fuzzy 0
1
2
3
4
5
6
Risk Fig. 10. Distribution of ball target location
• Displacement is Close, which results in the highest overall accuracy • Area is 3 or 7, which results in the lowest overall risk The locations with Close Displacement (highest overall accuracy) are less than 1.5m from the middle point 3.66, which are from 2.16 to 5.16. For all the locations between 2.16 and 5.16, we can find out which points belong to the areas with lowest overall risk (area 2, 3, 7, 8): •
Between 2.16 and 2.29 (area 3)
•
Between 5.03 and 5.16 (area 7)
And because in this experiment, the target locations in the decision list are generated from 0 to 7.32 with the step of 0.1, there are only two locations (2.2 and 5.1) which satisfied the above conditions. In order to explain how the drops of risk value happen, we need to look at the risk value in more details. In Fig. 13, at target location 2.2, the fuzzy system always has risk output of 0.5, while crisp system gives risk 0.125 (Low category) for anxiety from 0 to 0.7 and risk 0.5 (Medium category) for anxiety from 0.8. Since the low risks cover most of the anxiety ranges, the average risk is small.
3
4
5
6
7
Fig. 11. Accuracy
Target location
1 0.8 0.6 0.4 0.2 0
Crisp Fuzzy 0
1
2
3
4
5
6
7
Target location Fig. 12. Risk
Risk
The two drop of risk variables in the crisp system can be explained by the two rule tables and their relation with the Displacement variable. For both location 2.2 and 5.1:
2
Target location
7
To explain the peaks (which means the kicker chooses to target these locations more), let’s have a closer look at the accuracy and the risk of different target location of KD strategy (Fig. 11 and Fig. 12). In Fig. 11, the accuracy of both systems is similar. But in Fig. 12, at location 2.2 and 5.1, the risk of the crisp system spectacularly falls to the lowest points. And because of such dramatic low risks, target location 2.2 and 5.1 have been chosen more frequently by Roulette Wheel Selection.
1
1 0.8 0.6 0.4 0.2 0
Crisp Fuzzy 0
0.2 0.4 0.6 0.8
1
Anxiety Fig. 13. Risk at target position 2.2
When using the crisp system, it can be said that, the reasons for the dramatic drop is there are a few locations which can lead to outputs from the Low category. Actually, the similar sudden change happens with areas near the goal post, they have a very high risk, a big jump comparing to the average risk. This is because those areas resulted in risks of High category. The sudden change does not occur in the fuzzy system because of the membership functions. In this case, a target location can belong to two membership functions (two areas), which results in the aggregated fuzzy set of risk outputs being wider. Therefore, the value of defuzzification (centre of gravity) will be closer to the middle point, comparing to mean value of crisp system’s output. Also it creates a smoother transition when a variable is changing between different areas. In conclusion, the crisp rule-based system is not suitable for a fuzzy environment because of the sudden leaps or drops of crisp results between different ranges of variable. This
sometimes leads to unwanted behaviours, which a modeller needs to spend time to recalibrate. Meanwhile, the fuzzy ruledbased system is better at handling fuzziness with smooth transition in output and more consistent decisions. In order to achieve the similar behaviours with the fuzzy system, the crisp system needs to be redesigned with more categories of variables and more rules, resulted in a more complex crisp system. V.
[3]
[4] [5] [6] [7]
CONCLUSION AND FUTURE WORK
This paper has investigated the topic of two different rulebased systems in Agent-Based Simulation: a fuzzy system and a crisp system. This was achieved by comparing the performance and behaviour of BDI agents in a soccer penalty case study. The implementation has demonstrated the openness of BDI framework in embedding other models within its components. The experimentation results show that the results of fuzzy system are related to how fast the degree of membership function changes. In addition, the fuzzy system can be used to better handle uncertainty of a fuzzy environment with smooth transition of output which results in more consistent decisions. On the other hand, the classical crisp rulebased system can result in unwanted preferred actions because of the sudden leaps or drops between different ranges of variables. The behaviour in the simple case study indicates shifting reasoning model in BDI systems from crisp to fuzzy rule based system can provide the path to a superior approach for the simulation of human behaviour. For future work, we can investigate these decision making methods on additional case studies, for instance, electricity used in a shared house. It is also interesting to compare other approaches with BDI such as Case-Based Reasoning (CBR), Markov Decision Process (MDP). Another direction for future work is to embed different methods into BDI frameworks at different components. There could be many ways to embed a method into BDI, or we can combine several methods together to see whether it is effective doing so.
[8]
[9]
[10]
[11]
[12]
[13] [14]
[15]
[16] [17]
REFERENCES [1] [2]
Bellman, Richard E., and Lotfi Asker Zadeh. "Decision-making in a fuzzy environment." Management science 17, no. 4 (1970): B-141. Rao, Anand S., and Michael P. Georgeff. "BDI agents: From theory to practice." In Proceedings of the first international conference on multiagent systems (ICMAS-95), pp. 312-319. 1995.
[18] [19]
[20]
Aamodt, Agnar, and Enric Plaza. "Case-based reasoning: Foundational issues, methodological variations, and system approaches." AI communications 7, no. 1 (1994): 39-59. Boutilier, Craig. "Toward a logic for qualitative decision theory." Proc. of KR 94 (1994): 75-86. Bellman, Richard. "A Markovian decision process." No. P-1066. RAND CORP SANTA MONICA CA, 1957. Bratman, Michael E. "Intention, plans, and practical reason." (1999). Li, Yifan, Petr Musilek, and Loren Wyard-Scott. "Fuzzy logic in agentbased game design." In Fuzzy Information, 2004. Processing NAFIPS'04. IEEE Annual Meeting of the, vol. 2, pp. 734-739. IEEE, 2004. Polat, Seçkin, and Cafer Erhan Bozda . "Comparison of fuzzy and crisp systems via system dynamics simulation." European Journal of Operational Research 138, no. 1 (2002): 178-190. Bonabeau, Eric. "Agent-based modeling: Methods and techniques for simulating human systems." Proceedings of the National Academy of Sciences of the United States of America 99, no. Suppl 3 (2002): 72807287. Siebers, Peer-Olaf, and Uwe Aickelin. "A first approach on modelling staff proactiveness in retail simulation models." Journal of Artificial Societies and Social Simulation, 14(2) 2. (2011). Wooldridge, Michael, and Nicholas R. Jennings. "Intelligent agents: Theory and practice." Knowledge engineering review 10, no. 2 (1995): 115-152. Georgeff, Michael, Barney Pell, Martha Pollack, Milind Tambe, and Michael Wooldridge. "The belief-desire-intention model of agency." In Intelligent Agents V: Agents Theories, Architectures, and Languages, pp. 1-10. Springer Berlin Heidelberg, 1999. Mendel, Jerry M. "Uncertain rule-based fuzzy logic system: introduction and new directions." (2001). Mamdani, Ebrahim H., and Sedrak Assilian. "An experiment in linguistic synthesis with a fuzzy logic controller." International journal of man-machine studies 7, no. 1 (1975): 1-13. Wood, Greg, and Mark R. Wilson. "Gaze behaviour and shooting strategies in football penalty kicks: implications of a'keeper-dependent approach."International Journal of Sport Psychology 41, no. 3 (2010): 293-312. Kuhn, W. "Penalty-kick strategies for shooters and goalkeepers." Science and football (1988): 489-492. Van Der Kamp, John. "A field simulation study of the effectiveness of penalty kick strategies in soccer: Late alterations of kick direction increase errors and reduce accuracy." Journal of sports sciences 24, no. 05 (2006): 467-477. Multimethod Simulation Software and Solutions. “AnyLogic”. Accessed May 30, 2013. http://www.anylogic.com/ C. Wagner, "Juzzy – A Java based Toolkit for Type-2 Fuzzy Logic", Proceedings of the IEEE Symposium Series on Computational Intelligence, Singapore, April 2013. Vu, Tuong. “Football Penalty Shootout”. Accessed August 15, 2013. http://www.runthemodel.com/models/1267/
Peer-Olaf Siebers
Christian Wagner
School of Computer Science University of Nottingham Nottingham, UK [email protected]
School of Computer Science University of Nottingham Nottingham, UK [email protected]
School of Computer Science University of Nottingham Nottingham, UK [email protected]
Abstract— The Belief-Desire-Intention (BDI) software model is an example of a reasoning architecture for a bounded rational software agent. In our research we plan to expand the application of the BDI software model to the area of simulating human behaviour in social and socio-technical systems. To this effect, in this paper we explore the differences in using a classical crisp rule-based approach and a fuzzy rule-based approach for the reasoning within the BDI system. As a test case we have chosen a football penalty shootout. We have kept the case study example deliberately simple so that we can focus on the effects the different BDI implementations have on the decisions made. Our experiments highlight that the crisp system can result in unwanted "preferred" actions because of sudden leaps or drops between different ranges of decision variables, while the fuzzy system results have smoother transitions which results in more consistent decisions. The behaviour, as showcased in this simple context, underlines that a change from crisp to fuzzy rule based systems as the underlying reasoning model in BDI systems can provide the path to a superior approach for the simulation of human behaviour, which we will explore further in the future. Keywords— agent based simulation, belief-desire-intention, fuzzy system, soccer penalty
I.
INTRODUCTION
Agent-Based Simulation (ABS) is a powerful technique that has been growing recently. In ABS, a model contains many autonomous agents, which can observe the environment and make decisions based on their states and sets of rules. However, using agent decision making to simulate human behaviour is difficult, since it involves many different and complex factors such as feelings, mood, intention, surrounding area, interaction with other agents, and what other agents are doing. These factors, which constitute categories whose boundaries are not easy to sharply define, are fuzzy in nature. To handle decision making in such a fuzzy environment, Bellman [1] suggested using a fuzzy system rather than a classical crisp system. Recently, with the help of software packages, one can design a simple fuzzy system without indepth knowledge about the classical fuzzy theory. Still, there are only a few applications of fuzzy logic systems used for agent decision making.
For implementing agent decision making, different researchers from many disciplines have come up with different methods of modelling human behaviour leading to a variety of frameworks such as Belief-Desire-Intention (BDI) [2], CaseBased Reasoning (CBR) [3], Qualitative Decision Theory (QDT) [4], Markov Decision Process (MDP) [5]. In this paper, the BDI framework will be used to perform decision making of agents because it allows the modelling of human behaviour with a high degree of fidelity. BDI is a framework based on theory of the rational actions proposed by Bratman et al [6]. The BDI framework imitates human thinking process with belief (what agents believe), desire (what agents want), and intention (what agents try to do). The BDI model is also open to embed other models into its components. One example is that a fuzzy system has been embedded into BDI agents of BattleCity game for better handling of the uncertainty of a complex virtual environment, which a classical rule-based system often fails to obtain [7]. Using the BDI framework as a base, this project analyses the influence of embedding fuzzy rule-based systems in contrast to crisp rule-based systems. Moreover, in order to test the applicability of two rulebased systems, we use a case study of “soccer penalty”, in which a kicker will try to score a goal against a goalkeeper. There is a comparison of fuzzy and crisp decision rules in System Dynamics simulation by Polat [8], and he concluded that the superiority of one of these systems depends on the situation. But in this paper, the investigation of the two different decision making systems is in ABS. The goal of the research is to examine how different methods of modelling decision-making affect agent behaviours. The rest of paper is organized as follows. Section II provides background information on Agent-Based Simulation, the BDI model, and fuzzy system. In section III, the implementation of soccer penalty is described in detail. Section IV compares and discusses the results of the crisp systems and fuzzy systems. Finally, section V provides a summary of results and several directions for future work.
II.
BACKGROUND
A. Agent-Based Simulation Simulation is an imitation of a system, which involves designing the model and performing experiment to have better understanding of the system. While the model is an abstract representation of the real system, the simulation represents the operation of the system. The nature of simulation is predicting what will happen to the system if certain actions are taken. So it is the job of a user to vary the inputs, perform actions to see the changes in operations and get a better understanding of the system. In a way, simulation is a support decision making system because after gaining the sufficient understanding, user can decide on how improve the real system. Agent Based Simulation is one of main paradigms in simulation. It is an individual-centric, decentralized approach. When designing the model, we identify the entities/agents, define their behaviours, put them in an environment, establish connections, and run the simulation. In ABS, a system is modelled as a collection of agents, which are autonomous decision-making unit with diverse characteristics. The interaction of much individual behaviour of agents results in the global behaviour of the system. ABS has become a powerful technique with a number of applications: flows simulation (evacuation, traffic, and customer flow management), organizations simulation (operational risk and organizational design), markets simulation (stock market, and strategic simulation), and diffusion simulation (diffusion of innovation and adoption dynamics) [9]. ABS is suited for the simulation of human-centric system [10]. Human are heterogeneous, autonomous and proactive entities. An agent is a very good representation for a human, because agents have following properties [11]: • Discrete entities: with their own behaviour, goals, thread of control • Autonomous: be able to adapt and modify their behaviour • Proactive: adjust action depending on agent’s internal state. B. Decision making using Belief-Desire-Intention model For decision making different approaches can be used, one of which is the BDI model. BDI model is based on rational actions and originally proposed by Bratman [6]. Later, BDI model has been well studied [2] and possibly the best known and studied model of practical reasoning agent [12]. In BDI architecture, the rational agent has three main mental attitudes: • Belief: agent’s knowledge environment and itself.
about
the
state
Belief Intention Desire Fig. 1. Belief-Desire-Intention framework
In addition, the information about the actions of achieving a certain goal, and options available to agent is represented as plan. With the help of plan library, the agent does not need to search through the entire space of possible actions. In the next section, we will revise the definition of crisp set, then introduce fuzzy set concept. C. Crisp Set versus Fuzzy Set Classical Sets, also called Crisp Sets, can be described by a characteristic function: 1, 0, Set A (in a universe of discourse X) can be defined as A = {x | x meets some conditions}. Set A is mathematically equivalent to its characteristic function above. For example x|a . A fuzzy set F is a generalisation of a crisp set. It is characterized by the membership function μF(x) that takes on values in the interval [0,1]. A membership function provides a measure of the degree of membership of an element in the universal of X to the fuzzy set. Related to the crisp set example, let A = x is roughly in [a,b], then the membership function will be: 0,1 A fuzzy set F in X may be represented as a set of ordered pairs of a generic element x and its grade of membership function μF(x). | , With fuzzy sets, we can construct a fuzzy logic rule system. D. Fuzzy Logic Rule System In general, a Fuzzy Logic Rule System (FLRS), also called fuzzy control system, has structure as Fig. 2 [13]:
of
• Desire: motivational state of the system that the agent wants to accomplish. • Intention: current course of action that the agent chooses to perform.
Action
Fig. 2. Fuzzy rule system
The most commonly used FLRS is Mamdani FLRS, created by Professor Mamdani when building a fuzzy rule system to control a steam engine and boiler [14]. The fuzzy rule system will take one crisp input x and return one crisp output y. The system contains four components: rules, fuzzifier, inference engine, defuzzifier. • The fuzzifier maps the crisp input x to a fuzzy set. • Rules can be provided by experts or extracted from numerical data. The rules are collections of one or many IF-THEN statements.
B. Implementation The model is implemented in AnyLogic (a multi-method simulation modelling tool developed by XJ Technologies [18]) and simulates a soccer penalty where a kicker tries to score a goal against a goalkeeper. In the model, there are two Fuzzy Logic Systems which were implemented using Juzzy, a Javabased toolkit for type-1 and type-2 fuzzy logic systems [19]. An online version of the model with simple control is available at RunTheModel website [20].
• The inference engine handles which rules are activated and how to combine the rule. Through the rules, the inference engine maps input to output sets.
The model is a 2D simulation with bird’s eye view and contains two agents (one kicker, one keeper), a ball, and a goal. Only the width of the goal, which is 7.32m, is simulated. The ball is 11m from the goal and the kicker has a short run of 3.5m before kicking the ball.
• The defuzzifier processes the fuzzy set to obtain crisp output y.
For simplification, a number of assumptions has been made:
III.
SOCCER PENALTY CASE STUDY
In order to test the different methods of decision making, we have decided to use a case study approach. The soccer penalty kick situation has been chosen as a case study because two players can be modelled very well using agents. And their decision making processes are complicated enough for implementing the BDI framework as a platform to explore crisp versus fuzzy control based inference. In addition, because this case study is simple to understand, we can focus more on decision making mechanism. Once we implement and have better understanding, we can go to more complex cases. A. Soccer Penalty In soccer penalty, Greg Wood [15] defined kicking strategies in terms of the relationship between the kicker’s last fixation on the target prior to shooting, and resultant ball destination: • A last fixation on the goalkeeper (who is within the center of the goal) coupled with a shot to any location was classified as the Keeper-Dependent (KD) strategy. • A last fixation close to the goal post (to the left or right of the center) coupled with a shot to the same side was classified as the Keeper-Independent (KI) strategy. • If the last fixation was to one side of the goal and the ball was shot to the other, this was classified as the Opposite-Independent (OI) strategy. In order to investigate the interaction between two players in decision making, we only consider the case of KeeperDependent strategy, in which the kicker looks at the keeper and decides the direction of the shoot as the keeper dives. Additionally, Kuhn [16] suggests that 70% of all shots utilise the KD strategy. But using the KD strategy means that the kicker has to change the kick direction in the last minute of the run-up. Van Der Kamp [17] reports that overall a kicker requires at least 400ms to alter kick direction. The report also concluded that if the kicker redirects the ball late in the run-up, this was at the expense of shot accuracy.
• Straight shots (no curved ball). • Wind does not affect the ball trajectory. • Ignore the stress caused by audience or team score. • Goalkeeper does not try to guess shoot direction, but randomly dives left or right. • If goalkeeper dives in the correct direction, he has a 20% chance of stopping the ball. • The kicker never shoots over the crossbar, and the ball never hits the goal post or crossbar. The model consists of two agents: Goalkeeper and Kicker. We use state charts (Fig. 3) to illustrate all the states agents can attain as well as transitions between states. • The kicker first waits, then starts to run and shoot. The decision making of the kicker will happen in the Wait state and RunUp state. In RunUp state, the kicker also reconsiders the decision using self-transition. • The goalkeeper will be waiting until the kicker gets close to the ball and dive. For simplification, the keeper will dive 400ms before the kicker reaches the ball. The decision making and reconsidering of the goalkeeper happens when waiting to dive.
Fig. 3. Statecharts of the kicker and the goalkeeper agent
C. Decision Making System 1) Decisions making of the kicker Since the goalkeeper will randomly dive, we can focus on analysing the complex decision of the kicker. The kicker has three states: Wait, RunUp, and Shoot. After the “Wait” state, he makes decision on how the shot will be carried out. That decision is made based on three input variables:
• Displacement [0,7.32]: determined by distance between gaze direction and target location. • Accuracy [0,1]: distance between target location and ball placement. • Target Area [0,7.32]: deducted from target location. • Risk [0,1]: the risk of the shot.
• Gaze direction: where the kicker look before the run-up.
Each variable is divided into many categories:
• Target location: where the kicker shoot the ball.
• Anxiety: Low, Medium, High
• Anxiety: anxiety level of the kicker.
• Displacement: Close, Average, Far
2) Decision making progress: • The kicker has a list of alternative decisions which generated based on direction of keeper diving. For example, if the keeper dives right, the target locations in the decision list will be on the left side of the goal. The gaze and target locations of alternative decisions are generated from 0 to 7.32 with the step of 0.1.
• Accuracy: Low, Medium, High
• For each decision, the decision making system takes three input variables and calculates the risk of that decision. The risk of a decision is evaluated by either crisp rule-based system or fuzzy rule-based system. • Finally, for the kicker to kick the ball, one of possible decisions is chosen using Roulette Wheel Selection: The risk value is used to associate a probability of selection with each decision. If ri is the risk of decision i in the list of n decisions, its probability is being selected . With ri=[0,1], the decision with lower is ∑ risk will have larger probability and be more likely to be selected. The advantage of this method is that even a decision with lower risk is less likely to be selected, there still is a chance that they might be.
Generate decision list
• Target Area: Area 1, Area 2, ... Area 8, Area 9 • Risk: Low, Medium, High For the crisp system, a category is represented by a crisp set. For instance, in crisp system, Anxiety value is Low if it is between 0 and 0.25 (Fig. 5). On the other hand, the fuzzy system uses a membership function to represent a category. For example, in fuzzy logic system, the Medium category of Anxiety variable is a triangular membership function with three parameters {0.1, 0.5, 0.9} (Fig. 6). The categories of variables are designed so that they are as similar as possible in both two systems. The bound of a category in crisp system is the middle point of “the last parameter of one membership function” and “the first parameter of the next membership function”. For example, in the crisp system, Medium category of Anxiety variable has the lower bound of 0.25, which is the middle point of: • First parameter of Medium membership function: 0.1 • Last parameter of Low membership function: 0.4
Evaluate each risk Crisp system
or
Fuzzy system
Roulette wheel selection One final decision Fig. 4. Decision making progress
3) Modelling of variables Both the crisp system and the fuzzy logic system use five variables: • Anxiety [0,1]: anxiety level of kicker.
Fig. 5. Variables in the crisp system
TABLE II. Target area
Fig. 6. Variables in the fuzzy system
4) Risk evaluation To evaluate the risk for each decision, the decision making system receives three inputs: Anxiety, Gaze Direction, and Target Location. Then the Risk is calculated using two subsystems: • Sub-system 1: Calculate the Accuracy using the Displacement and the Anxiety level. o
Inputs: Displacement, Anxiety level
o
Output: Accuracy
o
Inputs: Target Area, Accuracy
o
Output: Risk
Displacement Close Close Close Average Average Average Far Far Far
RULE LIST OF SUB-SYSTEM 1
Anxiety Low Medium High Low Medium High Low Medium High
Accuracy Low Medium High Low Medium High Low Medium High Low Medium High Low Medium High Low Medium High Low Medium High Low Medium High Low Medium High
Risk High High Medium High Medium Low High Medium Low High Medium Medium High High Medium High Medium Medium High Medium Low High Medium Low High High Medium
Overall risk (1=highest) 1
3
3
2
1
2
3
3
1
Rule Table I and II are used for calculation the Accuracy and Risk of each system. One example of the first rule from Table I: IF displacement IS Close AND anxiety IS Low THEN Accuracy is High. Further detail on calculating using a rule is explained in next sections for both systems. The rule tables have four columns: • The first two columns are categories of inputs.
• Sub-system 2: Take the two inputs of Target Area and Accuracy (output of sub-system 1), and calculate the Risk of the decision.
TABLE I.
Area1 Area1 Area1 Area2 Area2 Area2 Area3 Area3 Area3 Area4 Area4 Area4 Area5 Area5 Area5 Area6 Area6 Area6 Area7 Area7 Area7 Area8 Area8 Area8 Area9 Area9 Area9
RULE LIST OF SUB-SYSTEM 2
Accuracy High High Medium Medium Medium Low High Medium Low
Overall accuracy (1=highest) 1
2
3
• The third column is the category of output. • The fourth column summaries the average value of the outputs, based on first variable. For instance, in table I the Accuracy is higher for Close Displacement than Average Displacement. In rule table I, the closer the displacement is, the higher the accuracy is. In rule table II, areas near the goal post and in the middle are the most dangerous (area 1, 5, 9); areas close to the goalkeeper (area 4, 6) are less dangerous; areas 2, 3, 7, 8 are the safest. 5) Crisp rule-based systems The crisp system evaluates the risk as follows: • Sub-system C1 takes Displacement and Anxiety level as inputs and determines which categories they belong to. Use the rule table I to find out the category of Accuracy (output). Return the mean value in the range of that category as output of C1. • Sub-system C2 takes the Target Area and Accuracy (output of C1) as inputs and determines which categories they belong to. Use the rule table II to find out the category of Risk. Return a mean value in the
range of that category as output of C22. That output will be the risk of a decision.
IV.
EXPERIMENTTATION AND RESULTS
Because of the randomisaation when selecting the final decision using Roulette Wheel Selection, the simulation model had been run for 10,000 timees for 10 steps of anxiety level varied from 0 to 1. For every shot, the final decision with all related information is saved to a file for later analysis. A. Experiment 1: Successful shhots related to Anxiety The first experiment invesstigates how the percentage of successful shots of both system ms varies according to the anxiety variable (Fig. 9). With the fuzzy system, thee successful shots remain stable between 0 and 0.6, and start to drop slightly from anxiety level 0.7, and reach the lowest poinnt at anxiety 0.8. With the crisp system, the successful shots sttay constant between 0 and 0.7, and then fall considerably at annxiety level 0.8.
6) Fuzzy rule-based systems To evaluate the risk, the fuzzy logic systeem uses two fuzzy inference sub-systems (Fig. 8). Each fuzzy sub-system will calculate the output using Mamdani method: • Take two crisp inputs • Fuzzification of crisp inputs: find out the value of membership function of each input. • Rule evaluation: check the rule table to see which rules are fired. • Aggregation of rule outputs: aggregatte the fuzzy output into one fuzzy set. o Gravity” of the • Defuzzification: return the “Center of aggregated fuzzy set as output.
The similar drop happens with w the fuzzy system, but since the High membership functioon of Anxiety variable starting from 0.6, the successful shotss start to drop from that point. Moreover, because the degreee of High anxiety is increasing gradually from 0.6, the numbeer of successful shots decreases accordingly (not a sudden drop d when comparing to crisp system). So the first conclusion is thhat the result of crisp system can have a sudden change when variable is changing from one category/range to another. On the other hand, the result of the fuzzy system will be affectedd by how fast the degree of a membership function changes.
%goal
Fig. 7. Crisp system
For the crisp system, the suudden drop at anxiety 0.8 can be explained by the anxiety level categories and rule tables. The anxiety level is categorized as “High” if it is greater than 0.75. And with high level of anxiety,, the rule table shows a low level of accuracy which leads to more m misses, even if the kicker chose to shoot in the safer zonee.
89 87 85 83 81 79
Crisp Fuzzy 0
0.2 0.4 0.6 0 0.8
1
Anxiety Fig. 9. Percentage of goal
B. Experiment 2: Kicker’s targget location Now, consider how the kickker targets the ball location. Fig. 10 shows the distribution of target t locations over the 7.32m width of the goal. The distributions are highlyy related to the rule tables: • For both systems, there is less focus on the middle and two areas near the goaal posts (area 1, 5, 9: the most Fig. 8. Fuzzy system
Number of times
• When using the crisp rule-based system, there are two peaks at location 2.2 and 5.1.
Accuracy
dangerous area in Rule Table II), and more focus on the safer areas (area 2, 3, 7, 8).
4000 3000
1 0.8 0.6 0.4 0.2 0
Crisp Fuzzy 0
2000 1000
Crisp
0
Fuzzy 0
1
2
3
4
5
6
Risk Fig. 10. Distribution of ball target location
• Displacement is Close, which results in the highest overall accuracy • Area is 3 or 7, which results in the lowest overall risk The locations with Close Displacement (highest overall accuracy) are less than 1.5m from the middle point 3.66, which are from 2.16 to 5.16. For all the locations between 2.16 and 5.16, we can find out which points belong to the areas with lowest overall risk (area 2, 3, 7, 8): •
Between 2.16 and 2.29 (area 3)
•
Between 5.03 and 5.16 (area 7)
And because in this experiment, the target locations in the decision list are generated from 0 to 7.32 with the step of 0.1, there are only two locations (2.2 and 5.1) which satisfied the above conditions. In order to explain how the drops of risk value happen, we need to look at the risk value in more details. In Fig. 13, at target location 2.2, the fuzzy system always has risk output of 0.5, while crisp system gives risk 0.125 (Low category) for anxiety from 0 to 0.7 and risk 0.5 (Medium category) for anxiety from 0.8. Since the low risks cover most of the anxiety ranges, the average risk is small.
3
4
5
6
7
Fig. 11. Accuracy
Target location
1 0.8 0.6 0.4 0.2 0
Crisp Fuzzy 0
1
2
3
4
5
6
7
Target location Fig. 12. Risk
Risk
The two drop of risk variables in the crisp system can be explained by the two rule tables and their relation with the Displacement variable. For both location 2.2 and 5.1:
2
Target location
7
To explain the peaks (which means the kicker chooses to target these locations more), let’s have a closer look at the accuracy and the risk of different target location of KD strategy (Fig. 11 and Fig. 12). In Fig. 11, the accuracy of both systems is similar. But in Fig. 12, at location 2.2 and 5.1, the risk of the crisp system spectacularly falls to the lowest points. And because of such dramatic low risks, target location 2.2 and 5.1 have been chosen more frequently by Roulette Wheel Selection.
1
1 0.8 0.6 0.4 0.2 0
Crisp Fuzzy 0
0.2 0.4 0.6 0.8
1
Anxiety Fig. 13. Risk at target position 2.2
When using the crisp system, it can be said that, the reasons for the dramatic drop is there are a few locations which can lead to outputs from the Low category. Actually, the similar sudden change happens with areas near the goal post, they have a very high risk, a big jump comparing to the average risk. This is because those areas resulted in risks of High category. The sudden change does not occur in the fuzzy system because of the membership functions. In this case, a target location can belong to two membership functions (two areas), which results in the aggregated fuzzy set of risk outputs being wider. Therefore, the value of defuzzification (centre of gravity) will be closer to the middle point, comparing to mean value of crisp system’s output. Also it creates a smoother transition when a variable is changing between different areas. In conclusion, the crisp rule-based system is not suitable for a fuzzy environment because of the sudden leaps or drops of crisp results between different ranges of variable. This
sometimes leads to unwanted behaviours, which a modeller needs to spend time to recalibrate. Meanwhile, the fuzzy ruledbased system is better at handling fuzziness with smooth transition in output and more consistent decisions. In order to achieve the similar behaviours with the fuzzy system, the crisp system needs to be redesigned with more categories of variables and more rules, resulted in a more complex crisp system. V.
[3]
[4] [5] [6] [7]
CONCLUSION AND FUTURE WORK
This paper has investigated the topic of two different rulebased systems in Agent-Based Simulation: a fuzzy system and a crisp system. This was achieved by comparing the performance and behaviour of BDI agents in a soccer penalty case study. The implementation has demonstrated the openness of BDI framework in embedding other models within its components. The experimentation results show that the results of fuzzy system are related to how fast the degree of membership function changes. In addition, the fuzzy system can be used to better handle uncertainty of a fuzzy environment with smooth transition of output which results in more consistent decisions. On the other hand, the classical crisp rulebased system can result in unwanted preferred actions because of the sudden leaps or drops between different ranges of variables. The behaviour in the simple case study indicates shifting reasoning model in BDI systems from crisp to fuzzy rule based system can provide the path to a superior approach for the simulation of human behaviour. For future work, we can investigate these decision making methods on additional case studies, for instance, electricity used in a shared house. It is also interesting to compare other approaches with BDI such as Case-Based Reasoning (CBR), Markov Decision Process (MDP). Another direction for future work is to embed different methods into BDI frameworks at different components. There could be many ways to embed a method into BDI, or we can combine several methods together to see whether it is effective doing so.
[8]
[9]
[10]
[11]
[12]
[13] [14]
[15]
[16] [17]
REFERENCES [1] [2]
Bellman, Richard E., and Lotfi Asker Zadeh. "Decision-making in a fuzzy environment." Management science 17, no. 4 (1970): B-141. Rao, Anand S., and Michael P. Georgeff. "BDI agents: From theory to practice." In Proceedings of the first international conference on multiagent systems (ICMAS-95), pp. 312-319. 1995.
[18] [19]
[20]
Aamodt, Agnar, and Enric Plaza. "Case-based reasoning: Foundational issues, methodological variations, and system approaches." AI communications 7, no. 1 (1994): 39-59. Boutilier, Craig. "Toward a logic for qualitative decision theory." Proc. of KR 94 (1994): 75-86. Bellman, Richard. "A Markovian decision process." No. P-1066. RAND CORP SANTA MONICA CA, 1957. Bratman, Michael E. "Intention, plans, and practical reason." (1999). Li, Yifan, Petr Musilek, and Loren Wyard-Scott. "Fuzzy logic in agentbased game design." In Fuzzy Information, 2004. Processing NAFIPS'04. IEEE Annual Meeting of the, vol. 2, pp. 734-739. IEEE, 2004. Polat, Seçkin, and Cafer Erhan Bozda . "Comparison of fuzzy and crisp systems via system dynamics simulation." European Journal of Operational Research 138, no. 1 (2002): 178-190. Bonabeau, Eric. "Agent-based modeling: Methods and techniques for simulating human systems." Proceedings of the National Academy of Sciences of the United States of America 99, no. Suppl 3 (2002): 72807287. Siebers, Peer-Olaf, and Uwe Aickelin. "A first approach on modelling staff proactiveness in retail simulation models." Journal of Artificial Societies and Social Simulation, 14(2) 2. (2011). Wooldridge, Michael, and Nicholas R. Jennings. "Intelligent agents: Theory and practice." Knowledge engineering review 10, no. 2 (1995): 115-152. Georgeff, Michael, Barney Pell, Martha Pollack, Milind Tambe, and Michael Wooldridge. "The belief-desire-intention model of agency." In Intelligent Agents V: Agents Theories, Architectures, and Languages, pp. 1-10. Springer Berlin Heidelberg, 1999. Mendel, Jerry M. "Uncertain rule-based fuzzy logic system: introduction and new directions." (2001). Mamdani, Ebrahim H., and Sedrak Assilian. "An experiment in linguistic synthesis with a fuzzy logic controller." International journal of man-machine studies 7, no. 1 (1975): 1-13. Wood, Greg, and Mark R. Wilson. "Gaze behaviour and shooting strategies in football penalty kicks: implications of a'keeper-dependent approach."International Journal of Sport Psychology 41, no. 3 (2010): 293-312. Kuhn, W. "Penalty-kick strategies for shooters and goalkeepers." Science and football (1988): 489-492. Van Der Kamp, John. "A field simulation study of the effectiveness of penalty kick strategies in soccer: Late alterations of kick direction increase errors and reduce accuracy." Journal of sports sciences 24, no. 05 (2006): 467-477. Multimethod Simulation Software and Solutions. “AnyLogic”. Accessed May 30, 2013. http://www.anylogic.com/ C. Wagner, "Juzzy – A Java based Toolkit for Type-2 Fuzzy Logic", Proceedings of the IEEE Symposium Series on Computational Intelligence, Singapore, April 2013. Vu, Tuong. “Football Penalty Shootout”. Accessed August 15, 2013. http://www.runthemodel.com/models/1267/
