2005 ACM Symposium on Applied Computing
An Empirical Evaluation of Communication Effectiveness in Autonomous Reactive Multiagent Systems David Hurt
Paul Tarau
University of North Texas P.O. Box 311366 Denton, Texas 76203 00-1 (940) 565-2767
University of North Texas P.O. Box 311366 Denton, Texas 76203 00-1 (940) 565-2806
[email protected] [email protected] Honeybees have been observed to communicate regarding the location of food sources [5]. The “dance” uses the sun and the Earth's gravity as references for orientation for finding the location of food, as shown in Figure 1.
ABSTRACT This paper describes an experiment designed to measure the effect of collaborative communication on task performance of a multiagent system. A simulation of a multiagent environment modeled after a bee colony examined the effects of collaboration through communication for various numbers of agents and environment sizes. Results show that collaboration enables a smaller number of agents to perform as well as a significantly larger number of agents without coordination. In particular, results indicate that the biologically inspired communication model of the bee is a particularly effective method of agent communication and collaboration.
This method of using fixed points for orientation is in use for several robotic systems as described in [4].
Categories & Subject Descriptors: I.2.11 Distributed Artificial Intelligence – Multiagent Systems
General Terms: Performance, Experimentation, Keywords: Multi-Agent Collaboration 1. INTRODUCTION Many models of communication and coordination in multiagent systems are based on biological models. In particular insects like ants and bees often serve as an inspiration for coordinating multiple independent agents [4]. Complex behaviors such as foraging, nest building, and task specialization emerge from simple reactive behavior on the part of individual agents.
Figure 1. The honey bee waggle dance uses the Sun and the Earth's gravity to communicate orientation Similarly, the gatherer agents have the ability to communicate the location of food sources to other gatherer agents within range of the communication. The stimulus-response behavior of the gatherers is programmed to cause a gatherer who receives this communication to go to the location in the message and start gathering food. This allows gatherer agents to collaborate in bringing food back to the hive once it is found.
Earlier work has suggested that communication and collaboration between individual agents in a multiagent system can significantly improve the performance of the entire system [6]. Many insects, such as ants and bees, exhibit this collaborative behavior. Ants follow pheromone trails, an implicit means of communication, to find sources of food. Honey bees employ explicit communication to inform other bees in their colony regarding the location of food sources.
An experiment was devised to validate the hypothesis that communication would improve the ability to locate and harvest resources. Additionally, another parameter (communication location) was used to determine whether communication at the site of resource location (site communication), or communication at a central site (hive communication) would provide the most improvement in the ability to locate and harvest resources. The experimental variables considered the three dimensions of agent communication ability, number of agents, and environment size.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. SAC’05, March 13-17, 2005, Santa Fe, New Mexico, USA. Copyright 2005 ACM 1-58113-964-0/05/0003…$5.00.
2. RELATED WORK Previous work has shown coordination in multiagent systems to be an effective means of improving performance for completing tasks. Balch and Arkin [1] show effective use of communication can induce substantial improvement in the performance of
74
multiagent systems. Goldman and Rosenchein [6] show that cooperative behavior on the part of individual agents can lead to an increase in overall system performance, even if that cooperative behavior detracts from individual performance.
location requires the capability to process symbolic information [7]. This communication has a finite radius and in practice resembles wireless radio communication more than the honey bee waggle dance model.
Balch and Arkin [1] showed that in some instances, communication between agents offers no substantial improvement in overall system performance. This is an indication that some other methods may improve multiagent system behavior.
The simulation used two bee-inspired communication methods. One method, hive communication, requires the gatherer to return to the hive before communicating the location of the food to other gatherers. The other method, site communication, requires the gatherer to communicate (broadcast) the location of the food as soon as the gatherer detects the food source. This broadcast near the location of the food is derived from work in cooperative navigation described in [11]. Both methods resemble the HelpBased Cooperation Protocol described in [8], although the agents do not wait for help before resuming the task of gathering food.
Other research has demonstrated usefulness of similar biologically inspired communication models. Dorigo [3] demonstrates an antinspired model. In this model agents communicate indirectly through stigmergy, or changing the environment. Specifically, Dorigo uses simulated pheromone trails, similar to ants, to allow agents to communicate optimal paths to follow through an environment. Other similar research [10] also uses pheromone trails for path planning in the presence of obstacles.
Figure 2 shows an example of the simulation environment. In this case, the environment is a 10x10 grid of cells. One cell, in this case, the cell at row 5, column 8, is the hive and will never contain food. The other cells contain various amounts of food resources.
This simulation differs from ant-inspired models in two primary ways. First, ant-based models depend on indirect communication through the environment while this simulation uses direct communication between agents. Second, agents in this simulation communication goal-oriented information, including the endpoint of a path, while ant-based models can communicate direction to an end-point, but not necessarily distance.
The gatherer agents are mobile in the sense that they can travel to multiple locations in the simulation grid. Gatherer agents search locations in the grid for food. If the agents find food, then they harvest one unit of food and return to the hive. Then the agent returns to the location of the food to harvest another unit of food and bring it back to the hive. This cycle continues until no more food remains at the location. Once the gatherer returns to the source of the food and finds it completely harvested, then the gatherer resumes searching for other sources of food. As mentioned above, gatherers can communicate sources of food to other gatherers. If additional gatherers receive the communication, then they will also start the harvest-return cycle for that location.
3. SIMULATION ENVIRONMENT The simulation models autonomous, cooperating agents. The scenario simulated is a group of agents (bees) in an environment filled with food resources (attractors). The environment is a finite space (representing a field) and is modelled as a two-dimensional grid of cells. One of the cells will be the location of the hive. The other cells have food resources, modelled as atomic units. A cell may have no food resources, or it may have several units. The environment contains a total of one hundred (100) food units distributed throughout the cells in the grid.
5
9 4
The hive contains a variable number of gatherer agents. Each gatherer agent represents a forager bee. Gatherer agents have mobility and can leave the hive and travel to other locations in the simulation grid space. Gatherer agents search for food and bring it back to the hive once finding the food. In some circumstances, gatherer agents can collaborate to cooperatively gather food from a known food source. The collective goal of all the agents is to locate all the food and bring all the food back to the hive as quickly as possible. This is the forage task identified in Balch and Arkin [1]. Initially, the gatherer agents have no knowledge regarding the location of the food. Gatherer agents must locate the food and then bring it back to the hive. Each gatherer can carry no more than one unit of food at a time. Therefore, gatherers may return to a single location multiple times as long as more food is contained at a location.
7
5 3
5 8
1
3 4
6
HIVE
2 11
3 2
7
1
3
10
1
Figure 2. Simulation Grid
The gatherer agents possess autonomy and collaboration, but lack any real ability to learn, since behavior is purely a result of the stimulus-response behavior model programmed into the gatherers. This corresponds to eusocial behavior as defined in [2].
Gatherer agents are reactive in nature and employ a stimulusresponse behavior model. The stimulus is constrained to a small set of events: •
In the same manner as honey bees can use a waggle dance to communicate the location of food sources to other bees in the hive, the gatherer agents can broadcast the location of a food source to other gatherers, who may then assist in the harvesting of food at the indicated location. Note that the transmission of food
•
75
ENCOUNTER events occur when a gatherer detects food and moves toward it. ATTACH events occur when the gatherer reaches the food and grabs it, starting a return to the hive
•
Agents using hive communication broadcast food location in response to a DEPOSIT event.
DEPOSIT events occur when the gatherer reaches the hive, deposits the food, and starts a return to the last known location of food. • ENCOUNTER-EMPTY events occur when a gatherer reaches a location expecting to find food and finds the food already harvested • RE-ORIENT events occur when a gatherer is in the WANDER state and receives a communication with the location of a food source. It then moves toward the food source. The response to an event is highly dependent upon the internal state of the agent. However, no global environment state or history of past actions is kept by the individual gatherer agents. The behavior of the gatherers is determined by a finite state automata (FSA) derived from the forage activity model in Balch and Arkin [1]. The FSA is significantly modified from the original version, and is shown in Figure 3 below.
4. EXPERIMENT DESCRIPTION The experiment consisted of running discrete event simulations with multiply varying parameters. In each simulation run, the standard program structure (simulation framework, simulation events, gatherer agent) was the same. Special data capture events caused the key state values such food harvested and communication events occurring to be recorded at periodic time intervals. A special end-simulation event ensures that the simulation terminates. Data points for the key state are saved to a file for later analysis. To determine the effects of the different communication methods, the simulations were run for each communication method. The three communication methods tested were 1.
RE-ORIENT
WANDER
ENCOUNTER(1)
ENCOUNTEREMPTY (1) Agents using site communication broadcast food location when they find the food and the ENCOUNTER event occurs. (2) Agents using hive communication broadcast food location when they return to the hive and deposit their food and the DEPOSIT event occurs.
ACQUIRE
ATTACH DEPOSIT (2)
No communication (baseline)
2.
Communication at the site of food (site communication),
3.
Communication at the site of the hive (hive communication).
To determine the effects of grid size, a variety of grid sizes were used (10x10 up to 100x100). Each grid size was repeated for each communication method. A set of pre-defined environments for each grid size were used to ensure that all simulations ran with the same input data. The amount of food in each grid is kept constant at 100 units, so larger grid sizes lead to a decrease in the food density. To determine the effects of the number of gatherer agents, a variety of gatherer counts were used (5, 10, 15, and 20). Each gatherer count was repeated for each communication method and grid size.
DELIVER
To minimize the effects of the random number generator, each simulation run (grid size, communication method, gatherer count, grid set) was repeated 10 times, with the data points averaged to reduce the effects of statistical noise.
Figure 3. Gatherer Simulation FSA WANDER is the initial state for the gatherer agent. In the WANDER state, the gatherer performs a random walk starting at the hive until (1) the gatherer locates a food source (ENCOUNTER event) or (2) the gatherer receives information about the location of a food source from another gatherer (REORIENT event). In either case, the gatherer transitions to the ACQUIRE state and starts moving toward the food. This random walk behavior is similar to other bee foraging simulations [9]. Agents using site communication broadcast food location in response to the ENCOUNTER event.
5. RESULTS The experiment measured food gathered vs. time for various simulation parameters. A common plot of this value appears in Figure 4. The curves are all non-decreasing, so the integral of the curve might be a useful data point for comparison. Better performance is indicated by higher area under the curve, indicating an ability to find food faster. The integral of the food gathered vs. time graph reduces each graph to a single scalar value which can be used for higher-level analysis.
In the ACQUIRE state, the gatherer moves toward the food until the food is located. Once the food is located, the gatherer harvests one unit of food (ATTACH event) and starts moving toward the hive, transitioning to the DELIVER state. If the gatherer arrives at the target location, but finds no food (due to other gatherers harvesting the food first, for example), then the gatherer transitions to the WANDER state and resumes the search for other food sources. This is the response to the ENCOUNTER-EMPTY event. The DELIVER state is used to bring food back to the hive. The gatherer keeps moving in the direction of the hive until arriving at the hive. When the gatherer reaches the hive, it deposits the food it has harvested in the hive, transitions back to the ACQUIRE state, and begins moving in the direction of the target food source.
Figure 4. Typical food vs. time graph
76
Figure 5 shows food gathering performance over various grid sizes for the various gatherer counts for the hive communication method. Although not shown, the trend for performance vs. gatherer size holds for all communication methods. The graph shows several expected trends. First, food gathering performance decreases with increasing grid sizes, indicating a longer time needed to find food in a larger area with more sparse food resources. Second, food gathering performance increases for increasing numbers of gatherers, indicating less time needed to find food with more gatherers searching.
600000 550000
Food Gathering Performance
500000 450000 400000 350000 300000
200000
550000
Food Gathering Performance
5 Agents, No communication 5 Agents, Hive communication 10 agents, No communication 10 agents, Hive communication 20 agents, No communication
250000
600000
150000
500000
10
20
30
40
50
450000
70
80
90
100
Figure 7. Gatherer Performance Showing Effects of Communication
400000
Figure 8 shows that for all grid sizes and all gatherer counts, hive communication provides a performance benefit compared to the baseline method of no communication. The site communication method also shows improvement, but to a smaller degree. Figure 9 shows that for all grid sizes and all gatherer counts, hive communication provides a larger performance benefit over the baseline method than does the site communication method
350000 300000 5 Gatherers 10 Gatherers 15 Gatherers 20 Gatherers
250000 200000 10
20
30
40
50
60
70
80
90
100
Grid Size
Figure 5. Simulator Performance for All Gatherer Counts for Hive Communication vs. Grid Size
Improvement Due to Hive Communication
Improvement
Figure 6 shows the relative performance of the communication methods for 20 gatherers. The relative positions of the curves hold across all grid sizes. The graph indicates that agents using hive communication perform better than agents using site communication, and that both perform better than the baseline method of agents using no communication. 600000
40.00% 35.00% 30.00% 25.00% 20.00% 15.00% 10.00% 5.00% 0.00% 0
20
40
550000 Food Gathering Performance
60
Grid Size
60
80
100
120
Grid Size 5 Gatherers
500000
10 Gatherers
15 Gatherers
20 Gatherers
Figure 8. Hive Communication Advantage over No Communication (Baseline)
450000
400000
Hive Communication Improvement Over Site Communication
350000 Baseline Site Communication Hive Communication 10
20
30
500.00% 400.00% 40
50
60
70
80
90
100
Improvement
300000
Grid Size
Figure 6. Simulator Performance for All Communication Methods for Twenty Gatherers vs. Grid Size
300.00% 200.00% 100.00% 0.00% -100.00% 0
In an observation of particular interest, Figure 7 shows a surprising effect that in some cases, a smaller number of cooperating agents can perform as well as a group of agents twice as large without communication. In particular, Figure 7 shows for a 100x100 grid, that 10 gatherers with hive communication find food roughly as fast as 20 gatherers without communication. This same pattern holds for 5 gatherers with hive communication and 10 gatherers without communication.
20
40
60
80
100
120
Grid Size
.
5 Gatherers
10 Gatherers
15 Gatherers
20 Gatherers
Figure 9. Hive Communication Advantage over Site Communication
77
6. SUMMARY AND CONCLUSION
8. REFERENCES
Based on the data collected, several observations can be deduced.
[1] Balch, Tucker and Arkin, Ronald C. Communication in
1.
For any number of gatherers and communication method, increasing the grid size (and decreasing food density) decreases the performance in finding food.
2.
For any grid size and communication method, increasing the number of gatherers increases the performance in finding food. Again, not in a linear fashion (The principle of diminishing returns)
3.
4.
Reactive Multiagent Robotic Systems. Autonomous Robots, 1 (1994), 1-25.
[2] Cao, Y. Uny, Fukunaga, Alex S., Kahng, Andrew B. and Meng, Frank, Cooperative Mobile Robotics: Antecedents and Directions, IEEE 08186 -7108-4/95, 1995.
[3] Dorigo, Marco, The Ant System: Optimization by a Colony of Cooperating Agents. IEEE Transactions on Systems, Man, and Cybernetics-Part B, 26, 1 (1996), 1-13.
For any grid size and number of gatherers, adding the ability to communicate and cooperate between gatherers increases the performance in finding food compared to gatherers who cannot cooperate.
[4] Franz, M.O. and Mallot, H.A. Biomemetic Robot Navigation, Robotics and Autonomous Systems 30 (2000), 133-153.
For almost all grid sizes and numbers of gatherers, the method of communicating food location near the hive increases food gathering performance over communicating that same information near the site of food discovery. The number of communication events is significantly higher when communication takes place near the hive, indicating a higher density of potentially receptive agents to help harvest the food near the hive. This reflects the observed behavior of the honey bee in nature.
5.
For small to medium search areas, adding collaboration ability is as effective as adding more agents.
6.
For large search areas, adding more agents tends to be more effective for improving food gathering performance than adding collaboration, probably due to increasing difficulty of collaboration as gatherers become spread further apart in search for food.
[5] Frisch, Karl von. The dance language and orientation of bees. Harvard University Press, Harvard, MA 1993.
[6] Goldman, C. V., and Rosenschein, J. S., Emergent Coordination through the Use of Cooperative StateChanging Rules, Proceedings of the Twelfth National Conference on Artificial Intelligence, 1994, 408-413.
[7] Jung, David and Zelinsky, Alexander. Grounded Symbolic Communication Between Heterogeneous Cooperating Robots. Autonomous Robots, 3, 8 (2000) 269-292.
[8] Lin, F. and Hsu, J.Y. Cooperation Protocols in Multi-agent Robotic systems. Autonomous Robots, 4 (1997) 175-198.
[9] Pérez-Uribe and Hirsbrunner, Beat. Learning and Foraging in Robot-bees. citeseer.ist.psu.edu/397182.html
[10] Parunak, H.V., Brueckner, S., & Sauter, J. Synthetic Pheromone Mechanisms for Coordination of Unmanned Vehicles. Proceedings of the First International Conference on Autonomous Agents and Multi-Agent Systems, 2002, 449450.
Our experiments show that biologically-inspired agent communication mechanisms provide effective means to optimize overall performance of goal-oriented multi-agent systems. While some of our assumptions are domain-specific (study of a bee colony and finite food supply) they emphasize the importance of exploring various communication models through simulations. Future work will focus on potential applications related to various resource discovery problems and simulation of more complex agent societies.
[11] Vaughan, R. T., Sty, K., Sukhatme, G. S., & Matari'c, M. J. Whistling in the dark: Cooperative trail following in uncertain localization space. Proceedings of the Fourth International Conference on Autonomous Agents, 2000, 187194.
7. ACKNOWLEDGEMENTS Thanks to the track’s chair and reviewers for their feedback on the earlier draft of this paper.
78