Stability Analysis for Replicator Dynamics of ... - Semantic Scholar
53rd IEEE Conference on Decision and Control December 15-17, 2014. Los Angeles, California, USA
Stability Analysis For Replicator Dynamics of Evolutionary Snowdrift Games Pouria Ramazi and Ming Cao Abstract— Stability analysis is presented in this paper to study the evolution of large populations of well mixed individuals playing three typical reactive strategies - always cooperate, tit-for-tat and suspicious tit-for-tat. After parameterizing the corresponding payoff matrices, we use replicator dynamics, a powerful tool from evolutionary game theory, to investigate how population dynamics evolve over time. We show the corresponding equilibria as well as their stability properties change as the payoff for mutual cooperation changes. Both theoretical analysis and simulation study are provided, which complements and further develops some existing results in theoretical biology and sociology.
I. I NTRODUCTION In recent years, more and more researchers have been working on studying dynamics evolving in large-scale networks [1], [2], [3], [4], [5], [6], which is a central theme in the fast developing network science. Decision-making dynamics in large populations have attracted attention from economists, biologists, sociologists, computer scientists and engineers alike due to their wide applications, but some challenging questions remain open and require the buildup of some general mathematical framework [7], [8]. One of such questions is why individuals decide to cooperate with others within a large interacting collective when to defect may better serve their self-interests [9]. While there is no convincing standard answer to this challenging question yet, some powerful theoretical tools have been developed. Evolutionary game theory has proven to be useful for modeling and analyzing the emergence and maintenance of cooperation in natural and social settings as reported by biologists, economists and sociologists [10]. Mainly based on simulation results, researchers have found that network topologies, penalties on betraying behaviors, social norms as well as other components in game setups can all affect the fitness of cooperators in face of defectors. However, fewer mathematical proofs have been constructed to support such claims. This is partly due to the stochastic, nonlinear, time-varying, large-scale nature of the evolutionary games on networks. Besides the two issues of how to model and how to analyze the decision-making dynamics in large populations, there is another challenging problem of how to control such dynamics in interacting populations. It is the aim of this paper to model and analyze the evolution of a population of individuals playing typical reactive strategies. This The work was supported in part by the European Research Council (ERCStG-307207) and the EU INTERREG program under the auspices of the SMARTBOT project. P. Ramazi and M. Cao are with Faculty of Mathematics and Natural Sciences, ITM, University of Groningen, The Netherlands,
{p.ramazi, m.cao}@rug.nl 978-1-4673-6088-3/14/$31.00 ©2014 IEEE
contributes to the understanding of why selfish individuals cooperate, and more importantly, how to control the portions of individuals playing different strategies in the long run. We focus on a special class of anti-coordination evolutionary games, repeated snowdrift games, to carry out some rigorous stability analysis for the evolutionary dynamics for the competition of three typical decision-making strategies. The first strategy we consider is the always-cooperate (ALLC) strategy. It is one of the simplest possible reactive strategies under which the player always cooperates, regardless of the opponent’s move. Although the ALLC strategy does not seem to be smart in the sense that the player never learns from her opponent even after being defected, it has been identified by biologists and sociologists to be instrumental for the overall population to be more cooperative [11]. The second strategy we investigate is the simple yet successful tit-for-tat (TFT) strategy where the player cooperates if her opponent cooperated and defects if her opponent defected, and so she simply mimics her opponent’s strategy in the last round. Moreover, in the first round, T F T players cooperate, which can be interpreted as they are looking forward to mutual cooperation. In general, this reactive strategy facilitates those opponents who are also pursuing mutual cooperation, but at the same time, stands against defective opponents such as the ones who always defect. This is perhaps the most well known winning strategy in repeated games. The last strategy to be studied is the suspicious-tit-for-tat (STFT) which is the same as a T F T strategy except that ST F T players defect in the first round and thus they take less risk comparing to T F T players. Such players may loose the chance to enjoy mutual cooperation because of their suspicion. For example, even when two ST F T players are matched, they end up in mutual defection. Some previous studies such as [12] have made pairwise comparison of these reactive strategies. It is the aim of this paper however, to show how a large population of a mixture of all of these three types of players evolves over time. A parameterized payoff matrix and an arbitrary number of repetitions of the base game are considered and the asymptotic stability of the replicator dynamics is the focus. Similar ideas have been discussed in [13], where the notion of evolutionary stable set has been investigated for the case when the base game is repeated for 5 times and a payoff matrix parameterized by one variable is considered. After setting up the parameterized payoff matrix, we utilize the replicator dynamics to model the evolution of the population ratios of the three types. A classification of the phase portraits of general system dynamics similar to what we consider here
4515
has been provided in [14] (later corrected in [15]) where the stability of the corresponding linearized systems about the equilibrium points has been studied. We however, take a different approach by leveraging notions such as domain of attraction and asymptotic stability from dynamical systems theory and weakly dominated strategies from classical evolutionary game theory, and examine in detail how the final composition of population shares of different players may be determined. So the main results of the paper may lead to further study on how to model and analyze different strategic interactions for decision-making processes in large populations. The rest of the paper is organized as follows. We formulate the mathematical framework of repeated snow-drift games with the three reactive strategies ALLC, T F T and ST F T under the replicator dynamics in Section II. Then in Section III, we give the main results, and compare these three reactive strategies when the population starts with a nonzero share of each of these strategies. Some simulation results are provided in Section IV. In the end we make concluding remarks. II. F RAMEWORK A. Base game A finite symmetric two-player game in a normal form with the payoff matrix C D C R S (1) D T P is considered to be a base game G. The first pure strategy, denoted by C is to “cooperate”, and the second, denoted by D, to “defect”. The well known snowdrift game is studied as the base game in this paper where the following condition holds on the payoffs T > R > S > P.
(2)
B. Repeated game A repeated game Gm with reactive strategies is constructed from the base game G by repeating it for m rounds, and limiting players’ choice of strategies in a round to be based on the opponent’s choice in the previous round. Define a reactive strategy s as a triple (p, q, r), where p is the probability to cooperate in the first round, q is the probability to cooperate if the opponent has cooperated in the previous round and r is the probability to cooperate if the opponent has defected in the previous round. Three reactive strategies are considered in this paper: • the always cooperate (ALLC) strategy (1, 1, 1) under which the player always cooperates; • the tit-for-tat (TFT) strategy (1, 1, 0) under which the player cooperates in the first round, and then does the same as what her opponent did in the previous round; • the suspicious tit-for-tat (STFT) strategy (0, 1, 0) under which the player defects in the first round, and then does the same as what her opponent did in the previous round.
When two players play Gm , their associated payoffs are determined according to the reactive strategies they play. Each player has three reactive strategies to choose; hence, a 3 × 3 payoff matrix A can be used to indicate different payoff possibilities of each player. Based on the base-game payoff matrix (1), A can be written as ALLC
A = [aij ] = TFT
ST F T
mR
mR
S + (m − 1)R
TFT
mR
mR
m dm 2 eS + b 2 cT
T + (m − 1)R
m dm 2 eT + b 2 cS
ST F T
mP
and the derivation of the payoffs is explained in the following. In an m-round match between the T F T and ALLC, both cooperate in the first round, and hence, T F T cooperates in the remaining rounds like ALLC. In other words, a mutual cooperation occurs in every round. The same happens when ALLC and T F T play against each other. This explains the upper-left 2 × 2 sub-matrix of A. On the other hand, when ALLC and ST F T F are matched, ST F T suspiciously defects in the first round while ALLC continuously cooperates. However, in the remaining rounds, ST F T gains trust and keeps cooperating resulting in a mutual cooperation. This clarifies a13 and a31 . Moreover, being suspicious and hence defecting in the first round, ST F T ’s always defect each other in a match, and this explains why a33 = mP . Finally, in a match between T F T and ST F T , T F T starts with trust while ST F T with suspicion resulting in ST F T beating T F T in the first round. However, in the second round, T F T looses trust and beats ST F T who has just gained some. This will be switched again in the next round and so on, which corresponds to a23 and a32 . So indeed the repeated game Gm can be viewed as a normal, symmetric two-player game with the payoff matrix A and with the pure-strategy set K = {ALLC, T F T, ST F T }. The rest of the settings are based on this specific game; however, they can easily be extended to a normal, symmetric two-player game with any number of pure strategies. C. Strategy space and dominance relationship Define the unit vectors 1 0 0 sC = 0 , sT = 1 , sST = 0 0 0 1 and the set S = {sC , sT , sST }. Each pure strategy is characterized by the unit vector whose elements are all zero except for that whose index is that of the row of the pure strategy in A. Hence, sC , sT and sST correspond to ALLC, T F T and ST F T respectively. We sometimes simply denote a pure strategy by its corresponding unit vector. For example, we often refer to sC as the pure strategy ALLC. In the strategy-vector space, the convex hull spanned by all unit vectors sC , sT and sST is a simplex, and is denoted
4516
ALLC
by ∆. The unit vectors are called the vertices of the simplex. In general, any mixed strategy can be denoted by a vector y ∈ ∆, which can be obtained by a convex combination of the vertices of ∆. The convex hull of a nonempty subset H of S is a face of ∆ and is denoted by ∆(H). For example, ∆({sC , sT }) indicates a face spanned by the unit vectors sC and sT , which is a mixed strategy of the two pure strategies ALLC and T F T . For ease of notation, we remove the braces when H is presented by its members. e.g., in the previous example we write ∆(sC , sT ) instead. A face is said to be a boundary face or simply a boundary, if it is spanned by some proper subset of S. Furthermore, an interior point of a face ∆(H) is a vector h ∈ ∆(H) whose elements are all non-zero. The interior of a set H is denoted by int(H) and is the set of all interior points of H. For a payoff matrix π, when the first and second players play x and y respectively, the associated payoff to the first player is defined by
Denote the initial value of xi by x0i and that of x by x0 . Under the dynamics (3), let ξ(t, x0 ) ∈ ∆ be the solution mapping at time t starting from the initial state x0 ∈ ∆. In other words, ξ(t, x0 ) represents the population state x at time t when the system has started at x0 ∈ ∆. Denote the ith component of ξ(t, x0 ) by ξi (t, x0 ). E. Equilibrium points and sets For ease of notation in future analysis, the following points are introduced, each of which represents a stationary point of the replicator dynamics (3) under some circumstances to be explained later: T (m−1)(P −R)+P −S R−T 0 x1 = mP , −(m−2)R−S−T mP −(m−2)R−S−T T m m −(d m mR−(d m 2 eS+b 2 cT ) 2 eT +b 2 cS ) x2 = 0 mPm(R+P . −T −S) m(R+P −T −S) The next three points are based on a given initial state x0 , so we denote them by functions g i (·), i = 1, 2, 3: (m + 3)R − mP − T x0C 0 (m + 2)R − mP (xC + x0T ) (m + 3)R − mP − T x0T , 0 (m + 2)R − mP (xC + x0T ) T −R
T
u(x, y) = x πy where u(·, ·) is called the utility function. In a face ∆(H), a strategy x is said to strictly dominate another strategy y if u(x, z) > u(y, z) for all z ∈ ∆(H), and correspondingly y is said to be a strictly dominated strategy. Similarly, in ∆(H), a strategy x weakly dominates y if u(x, z) ≥ u(y, z) for all z ∈ ∆(H) and u(x, r) > u(y, r) for some r ∈ ∆(H). D. Replicator dynamics The population evolution of individuals playing different strategies is studied by using a powerful model, replicator dynamics, from evolutionary game theory. Consider a large but finite number of individuals, each assigned with some pure strategy in the set K. Let xC (t), xT (t) and xST (t) denote the population shares of individuals playing pure strategies sC , sT and sST respectively at time t. Also define the set B = {‘C 0 , ‘T 0 , ‘ST 0 }. Then, define the associated population state vector as X x(t) = xi (t)si .
(m + 2)R − mP n(T + S − 2P )x0C 0 0 0 (T − + (S − R)xT + n(xC + xT )(T + S − 2P ) n(T + S − 2P )x0T 2 0 g (x ) = 0 + (S − R)x0 + n(x0 + x0 )(T + S − 2P ) , (T − R)x C T C T (T − R)x0C + (S − R)x0T (T − R)x0C + (S − R)x0T + n(x0C + x0T )(T + S − 2P )
R)x0C
g 3 (x0 ) =
x0C 0 xC + x0T
x0T 0 xC + x0T
T 0
.
We will also need the set Φm of the equilibrium points defined for odd or even m as o n )+R−T , Φ2n+1 = x | x = (θ, 1 − θ, 0)T , 0 ≤ θ ≤ n(2R−S−T n(2R−S−T ) n o n(2R−S−T ) Φ2n = x | x = (θ, 1 − θ, 0)T , 0 ≤ θ ≤ T −R+n(T +S−2R) and also the following set from the simplex
i∈B
Hence, xi denotes the ith component of x. We eliminate the notation t in the rest of the paper, unless it is necessary to avoid possible confusion. The replicator dynamics describing the evolution over time of each individual’s population share are given by x˙ i = [u(si , x) − u(x, x)]xi
g 1 (x0 ) =
(3)
for all i ∈ B. According to the dynamics (3), the reproduction rate of each individual is proportional to the difference of her payoff, u(si , x), when playing against the average population, and the population average payoff u(x, x). Loosely speaking, the more an individual acquires when playing against her opponents, comparing to the average payoff of the individuals, the more new offspring she produces, and hence, the greater her population ratio becomes.
m D = {x | x ∈ ∆, (T −R)xC +(d m 2 eT +b 2 cS−mR)xT ≤ 0}.
III. TRIPLE - WISE COMPARISON OF THE STRATEGIES In this section, we provide the stability analysis for the case when the initial population state belongs to the interior of the simplex, namely when each of the three strategies starts with a nonzero population share. Naturally the stability results depend on the number m that determines for how many times the agents play the base game repeatedly since m structurally changes the payoff matrix A. We summarize the results in two theorems for odd and even ms, the proofs of which are not presented due to lack of space. Well known stability results in both evolutionary game theory and system dynamics are used for the proofs. Now we first present the theorem corresponding to an odd m.
4517
Theorem 3.1: Assume (2) holds. Let x0 ∈ int(∆), and m = 2n + 1, n ≥ 0. Then, under the replicator dynamics (3) it holds that T +S • if R < , 2 lim ξ(x0 , t) = x2 ; (4)
•
and the corresponding phase portrait consists of straight lines passing through sST ; nT + (n − 1)S , if R > 2n − 1 ∃x∗ ∈ Φ2n ∪ {x1 } : lim ξ(x0 , t) = x∗ t→∞
t→∞
•
if R =
and
T +S , 2 0
1
0
lim ξ(x , t) = g (x ),
t→∞
•
(5)
and the corresponding phase portrait consists of straight lines passing through the origin; T +S (n + 1)T + nS if
(6)
(n + 1)T + nS , 2n + 1 ∃x∗ ∈ Φ2n+1 ∪ {x1 } : lim ξ(x0 , t) = x∗ t→∞
(7)
and x0 ∈ int(∆ − D) ⇒ lim ξ(x0 , t) = x1 . (8) t→∞ Except for the last case where R is lower bounded, the theorem precisely determines where the solution trajectories end up according to the initial state. For example, when R is smaller than the average of T and S, starting from an interior point of the simplex, the final state will be x2 which belongs to ∆(sT , sST ). Hence, for a small enough mutual cooperation payoff, no matter how the population shares of different pure strategies are initially distributed in the interior of the simplex, all of the agents will eventually, play either T F T or ST F T , which implies that agents playing ALLC will not survive in such a case. We now present the theorem for the case of an even m. Theorem 3.2: Assume (2) holds. Let x0 ∈ int(∆), and m = 2n, n ≥ 1. Then, under the replicator dynamics (3) it holds that T +S • if R < , 2 lim ξ(x0 , t) = x2 ; (9)
x0 ∈ int(∆ − D) ⇒ lim ξ(x0 , t) = x1 . (14) t→∞ At first glance, Theorem 3.1 claims that for an even m there are more possible cases to happen comparing to an odd m. However, one of the cases, (10), claims a simple yet interesting outcome of the replicator dynamics. It simply implies that when the mutual cooperation payoff R equals the average of T and S, no matter how small the initial population share of T F T players is, as long as it is nonzero, eventually all of ALLC and ST F T players will be wiped out of the population and every individual will eventually become a T F T player. Remark 2.1: The results in Theorems 3.1 and 3.2 are stronger than typical stability results and they identify the stationary points that the solution trajectories converge to, when they start from any x0 ∈ int(∆) (except for (7) and (13) where an estimation of the domain of attraction is given, and for the case of (11) where a set of equilibria is shown to be globally asymptotically stable in int(∆)). Remark 2.2: In both Theorems 3.1 and 3.2, it is guaranteed that the population state converges to a point in the simplex, so no chaotic or periodic behavior can happen in general. Remark 2.3: Theorems 3.1 and 3.2 can be applied to tune the payoffs T, R, S and P in order to change the population shares of the three reactive strategies in the long run. One may combine the common cases of Theorems 3.1 and 3.2 to obtain the following corollary. Corollary 2.1: Assume (2) holds. Let x0 ∈ int(∆). Then, under the replicator dynamics (3) it holds that T +S , • if R < 2 lim ξ(x0 , t) = x2 ;
t→∞
t→∞
•
if R =
•
T +S , 2 lim ξ(x , t) = sT ;
t→∞
(15)
n d m−1 eT + b m−1 cS d m eT + b m cS o 2 2 2 , 2 , m−1 m t→∞
(10)
(16)
and
T +S nT + (n − 1)S if max
∃x∗ ∈ Φm ∪ {x1 } : lim ξ(x0 , t) = x∗
0
•
(13)
nT + (n − 1)S , 2n − 1 2 0 g (x ) 0 lim ξ(x , t) = t→∞ g 3 (x0 )
(11)
if R =
R−S x0C > 0 xT T −R , R−S x0C ≤ x0T T −R
(12)
x0 ∈ int(∆ − D) ⇒ lim ξ(x0 , t) = x1 . t→∞ The corollary provides a general result for all m which is quite useful. Based on equation (15), if R is less than the average of T and S, no matter how many times the snow-drift game is repeated and no matter how the population shares of the three pure strategies are initialized as long as they are nonzero, the population state converges to x2 where no T F T player exists. Moreover, according to (16), for a large enough R, the final population state converges to either a point in Φm , where no ST F T player exists, or to x1 , where no T F T
4518
player exists. In both cases, ALLC players survive. Hence, as a general result, one can conclude that under the replicator dynamics (3) and when none of the population shares of the three pure strategies are zero, ALLC players do not survive in the long run for a mutual cooperation payoff R smaller than the average of T and S, while they for sure survive for a large R. As a further result, the corollary also claims that if the population state starts from a point in ∆ − D, its trajectory converges to x1 . Hence, ∆ − D is an estimation of the domain of attraction of x1 . IV. S IMULATION RESULTS In order to visualize the previous results, the phase portraits of the replicator dynamics (3) corresponding to a repeated game Gm with an even m are presented for different values of mutual cooperation payoff R. We consider the base game as a snowdrift game with the following payoff matrix R 3 . (17) 15 1 Let the base game be repeated for 4 times, i.e., m = 4. Then the payoff matrix A of the repeated game can be calculated accordingly. We investigate the replicator dynamics (3) for five different values of R to match the five cases in Theorem 3.1. In each case, the corresponding phase portrait, when the system starts from an interior point of the simplex, is drawn to illustrate the solution trajectories of the dynamics, and the stability of different equilibrium points of the system. The game dynamics simulation software Dynamo [16] is used to plot the figures. The five cases are as follows. • For R = 4 which is less than the average of T and S, the phase portrait is shown in Fig. 1a. As (9) implies, all solution trajectories converge to x2 in this case where no ALLC player exists. 2 • By increasing R, the equilibrium point x moves towards sT , and when R = 9 equals the average of T and S, x2 matches sT . Hence, eventually the population completely becomes T F T players in this case as (10) states. So the ALLC players still do not get the chance to survive in the long run. The corresponding phase portrait is shown in Fig. 1b. • After increasing R to 10, ALLC players start showing up in the long run. For this case, the condition of (11) in Theorem 3.2 is satisfied, and as (11) implies, each solution trajectory converges to a point in Φm where a mixture of T F T and ALLC players exists. However, in most of the cases, the final state is closer to sT than sC which implies that in the long run, usually the population consists of more T F T players than ALLC players. The corresponding phase portrait is shown in Fig. 1c. • One may expect to get some solution trajectories to converge to a point close to sC by increasing R. However, for R = 11 which satisfies the condition of equation (12) in Theorem 3.2, the set of stationary points Φm breaks in to two parts which are in the form of a segment: One on the face ∆(sT , sC ) as before,
and the other from the closest stationary point to sC on ∆(sT , sC ), to the equilibrium point on the face ∆(sC , sST ). Hence, sC does not get the chance to become a stationary point which implies that the final population state never becomes ALLC players when all types of the players are available in the beginning. On the other hand, from now on, by increasing R, ST F T individuals again get the chance to appear in the long run, as well as the other two types of individuals. The corresponding results are shown in Fig. 1d. As can be seen, firstly, all of the solution trajectories are straight lines passing through the sST point. Secondly for each x0 such trajectory, if the initial state has a low ratio of xC0 , T the trajectory converges to g 3 (x0 ) while if the ratio is high, the trajectory converges to the face ∆(sT , sC ). This has been explicitly determined by (12). • Finally, by further increasing R to values greater than nT +(n−1)S , all of the second part of the equilibrium 2n−1 set in the last case disappears except for the stable equilibrium point x1 on the face ∆(sC , sST ). The corresponding phase portrait for R = 12 is shown in Fig. 1e. Each solution trajectory converges to either x1 or a point in Φm which is what (13) implies, and if the initial state is above the line connecting sST to the farthest point from sT in ∆(sT , sC ), the corresponding solution trajectory converges to x1 which is what (14) implies. No matter whether the solution trajectories converge to x2 or Φm , the final population state always includes ALLC players. Moreover, as R increases further, x1 gets closer to sC . Hence, for large Rs the final population state includes ALLC individuals, and also the greater R becomes, the higher the ratio of the ALLC individuals becomes. To conclude, T F T players always have the chance to survive in the long run, and in one case, i.e., R equals the average of T and S, they become the only survivors. For ST F T players, under small R, they also may exist in the final state, but when R increases to become greater than or equal to the average of T and S, they do not appear in the long run. However, when R becomes greater than or equal to nT +(n−1)S , ST F T players again get the chance to survive 2n−1 in the long run, but this time as R increases, their chance decreases. Finally, for ALLC individuals, there almost exists a direct relation between increments in R and their chance to survive in the long run. So as a rule of thumb, the greater the R, the more the ALLCs. Similar results hold for an odd m, so this rule of thumb is applicable to any m. V. C ONCLUDING REMARKS We have investigated the final population composition of the three reactive strategies ALLC, T F T and ST F T under the replicator dynamics when the snow-drift game is considered as the base game. We show how the final population state is affected by the changes in the mutual cooperation payoff R in two theorems, one when the base game is repeated for odd times and the other when it is
4519
(a) R = 4
(c) R = 10
(b) R = 9
(d) R = 11
(e) R = 12
4
Fig. 1: Flow patterns for the repeated game G under the replicator dynamics (3) and base-game payoff matrix (17). Color temperatures are used to show motion speeds where red corresponds to the fastest and blue the slowest motion. The circles denote equilibria. Note that in case there exists a set of equilibria, the simulation only finds and plots some of them.
repeated for even times. In most of the cases, the theorems explicitly determine that starting from an interior state, to which point the solution trajectory converges. This, from long-term stability point of view, is as useful as providing an analytical solution to the dynamics. The results of the paper can be used to tune the payoff values of a snow-drift game in order to reach a desired final population state of the reactive strategies under the replicator dynamics. For future study we aim to investigate the final population composition of other reactive strategies under similar settings of this paper. R EFERENCES [1] M. O. Jackson, Social and Economic Networks. Princeton University Press, 2010. [2] M. E. Newman, “The structure and function of complex networks,” SIAM Review, vol. 45, no. 2, pp. 167–256, 2003. [3] A.-L. Barab´asi and R. Albert, “Emergence of scaling in random networks,” Science, vol. 286, no. 5439, pp. 509–512, 1999. [4] C. J. Anderson, S. Wasserman, and B. Crouch, “A p∗ primer: logit models for social networks,” Social Networks, vol. 21, no. 1, pp. 37– 66, 1999. [5] B. Charlesworth, P. Sniegowski, and W. Stephan, “The evolutionary dynamics of repetitive DNA in eukaryotes,” Nature, pp. 215–220, 1994. [6] R. Koopmans, “Movements and media: Selection processes and evolutionary dynamics in the public sphere,” Theory and Society, vol. 33, no. 3-4, pp. 367–391, 2004. [7] D. Araujo, K. Davids, and R. Hristovski, “The ecological dynamics of decision making in sport,” Psychology of Sport and Exercise, vol. 7, no. 6, pp. 653–676, 2006. [8] S. Battiston, E. Bonabeau, and G. Weisbuch, “Decision making dynamics in corporate boards,” Physica A, vol. 322, pp. 567–582, 2003.
[9] E. Fehr and S. G¨achter, “Altruistic punishment in humans,” Nature, vol. 415, no. 6868, pp. 137–140, 2002. [10] R. B. Myerson, Game Theory: Analysis of Conflict. Harvard University Press, 2013. [11] C. Hilbe, M. A. Nowak, and K. Sigmund, “Evolution of extortion in Iterated Prisoners Dilemma games,” Proceedings of the National Academy of Sciences, vol. 110, no. 17, pp. 6913–6918, 2013. [12] M. A. Nowak, Evolutionary Dynamics: Exploring the Equations of Life. Harvard University Press, 2006. [13] C. Wang, B. Wu, M. Cao, and G. Xie, “Modified snowdrift games for multi-robot water polo matches,” in Proc. of the 24th Chinese Control and Decision Conference (CCDC), pp. 164–169, 2012. [14] I. M. Bomze, “Lotka-volterra equation and replicator dynamics: a twodimensional classification,” Biological Cybernetics, vol. 48, no. 3, pp. 201–211, 1983. [15] ——, “Lotka-volterra equation and replicator dynamics: new issues in classification,” Biological Cybernetics, vol. 72, no. 5, pp. 447–453, 1995. [16] E. D. W. H. Sandholm and F. Franchetti. (2012) Dynamo: Diagrams for evolutionary game dynamics. [Online]. Available: http://www.ssc.wisc.edu/ whs/dynamo
4520
Stability Analysis For Replicator Dynamics of Evolutionary Snowdrift Games Pouria Ramazi and Ming Cao Abstract— Stability analysis is presented in this paper to study the evolution of large populations of well mixed individuals playing three typical reactive strategies - always cooperate, tit-for-tat and suspicious tit-for-tat. After parameterizing the corresponding payoff matrices, we use replicator dynamics, a powerful tool from evolutionary game theory, to investigate how population dynamics evolve over time. We show the corresponding equilibria as well as their stability properties change as the payoff for mutual cooperation changes. Both theoretical analysis and simulation study are provided, which complements and further develops some existing results in theoretical biology and sociology.
I. I NTRODUCTION In recent years, more and more researchers have been working on studying dynamics evolving in large-scale networks [1], [2], [3], [4], [5], [6], which is a central theme in the fast developing network science. Decision-making dynamics in large populations have attracted attention from economists, biologists, sociologists, computer scientists and engineers alike due to their wide applications, but some challenging questions remain open and require the buildup of some general mathematical framework [7], [8]. One of such questions is why individuals decide to cooperate with others within a large interacting collective when to defect may better serve their self-interests [9]. While there is no convincing standard answer to this challenging question yet, some powerful theoretical tools have been developed. Evolutionary game theory has proven to be useful for modeling and analyzing the emergence and maintenance of cooperation in natural and social settings as reported by biologists, economists and sociologists [10]. Mainly based on simulation results, researchers have found that network topologies, penalties on betraying behaviors, social norms as well as other components in game setups can all affect the fitness of cooperators in face of defectors. However, fewer mathematical proofs have been constructed to support such claims. This is partly due to the stochastic, nonlinear, time-varying, large-scale nature of the evolutionary games on networks. Besides the two issues of how to model and how to analyze the decision-making dynamics in large populations, there is another challenging problem of how to control such dynamics in interacting populations. It is the aim of this paper to model and analyze the evolution of a population of individuals playing typical reactive strategies. This The work was supported in part by the European Research Council (ERCStG-307207) and the EU INTERREG program under the auspices of the SMARTBOT project. P. Ramazi and M. Cao are with Faculty of Mathematics and Natural Sciences, ITM, University of Groningen, The Netherlands,
{p.ramazi, m.cao}@rug.nl 978-1-4673-6088-3/14/$31.00 ©2014 IEEE
contributes to the understanding of why selfish individuals cooperate, and more importantly, how to control the portions of individuals playing different strategies in the long run. We focus on a special class of anti-coordination evolutionary games, repeated snowdrift games, to carry out some rigorous stability analysis for the evolutionary dynamics for the competition of three typical decision-making strategies. The first strategy we consider is the always-cooperate (ALLC) strategy. It is one of the simplest possible reactive strategies under which the player always cooperates, regardless of the opponent’s move. Although the ALLC strategy does not seem to be smart in the sense that the player never learns from her opponent even after being defected, it has been identified by biologists and sociologists to be instrumental for the overall population to be more cooperative [11]. The second strategy we investigate is the simple yet successful tit-for-tat (TFT) strategy where the player cooperates if her opponent cooperated and defects if her opponent defected, and so she simply mimics her opponent’s strategy in the last round. Moreover, in the first round, T F T players cooperate, which can be interpreted as they are looking forward to mutual cooperation. In general, this reactive strategy facilitates those opponents who are also pursuing mutual cooperation, but at the same time, stands against defective opponents such as the ones who always defect. This is perhaps the most well known winning strategy in repeated games. The last strategy to be studied is the suspicious-tit-for-tat (STFT) which is the same as a T F T strategy except that ST F T players defect in the first round and thus they take less risk comparing to T F T players. Such players may loose the chance to enjoy mutual cooperation because of their suspicion. For example, even when two ST F T players are matched, they end up in mutual defection. Some previous studies such as [12] have made pairwise comparison of these reactive strategies. It is the aim of this paper however, to show how a large population of a mixture of all of these three types of players evolves over time. A parameterized payoff matrix and an arbitrary number of repetitions of the base game are considered and the asymptotic stability of the replicator dynamics is the focus. Similar ideas have been discussed in [13], where the notion of evolutionary stable set has been investigated for the case when the base game is repeated for 5 times and a payoff matrix parameterized by one variable is considered. After setting up the parameterized payoff matrix, we utilize the replicator dynamics to model the evolution of the population ratios of the three types. A classification of the phase portraits of general system dynamics similar to what we consider here
4515
has been provided in [14] (later corrected in [15]) where the stability of the corresponding linearized systems about the equilibrium points has been studied. We however, take a different approach by leveraging notions such as domain of attraction and asymptotic stability from dynamical systems theory and weakly dominated strategies from classical evolutionary game theory, and examine in detail how the final composition of population shares of different players may be determined. So the main results of the paper may lead to further study on how to model and analyze different strategic interactions for decision-making processes in large populations. The rest of the paper is organized as follows. We formulate the mathematical framework of repeated snow-drift games with the three reactive strategies ALLC, T F T and ST F T under the replicator dynamics in Section II. Then in Section III, we give the main results, and compare these three reactive strategies when the population starts with a nonzero share of each of these strategies. Some simulation results are provided in Section IV. In the end we make concluding remarks. II. F RAMEWORK A. Base game A finite symmetric two-player game in a normal form with the payoff matrix C D C R S (1) D T P is considered to be a base game G. The first pure strategy, denoted by C is to “cooperate”, and the second, denoted by D, to “defect”. The well known snowdrift game is studied as the base game in this paper where the following condition holds on the payoffs T > R > S > P.
(2)
B. Repeated game A repeated game Gm with reactive strategies is constructed from the base game G by repeating it for m rounds, and limiting players’ choice of strategies in a round to be based on the opponent’s choice in the previous round. Define a reactive strategy s as a triple (p, q, r), where p is the probability to cooperate in the first round, q is the probability to cooperate if the opponent has cooperated in the previous round and r is the probability to cooperate if the opponent has defected in the previous round. Three reactive strategies are considered in this paper: • the always cooperate (ALLC) strategy (1, 1, 1) under which the player always cooperates; • the tit-for-tat (TFT) strategy (1, 1, 0) under which the player cooperates in the first round, and then does the same as what her opponent did in the previous round; • the suspicious tit-for-tat (STFT) strategy (0, 1, 0) under which the player defects in the first round, and then does the same as what her opponent did in the previous round.
When two players play Gm , their associated payoffs are determined according to the reactive strategies they play. Each player has three reactive strategies to choose; hence, a 3 × 3 payoff matrix A can be used to indicate different payoff possibilities of each player. Based on the base-game payoff matrix (1), A can be written as ALLC
A = [aij ] = TFT
ST F T
mR
mR
S + (m − 1)R
TFT
mR
mR
m dm 2 eS + b 2 cT
T + (m − 1)R
m dm 2 eT + b 2 cS
ST F T
mP
and the derivation of the payoffs is explained in the following. In an m-round match between the T F T and ALLC, both cooperate in the first round, and hence, T F T cooperates in the remaining rounds like ALLC. In other words, a mutual cooperation occurs in every round. The same happens when ALLC and T F T play against each other. This explains the upper-left 2 × 2 sub-matrix of A. On the other hand, when ALLC and ST F T F are matched, ST F T suspiciously defects in the first round while ALLC continuously cooperates. However, in the remaining rounds, ST F T gains trust and keeps cooperating resulting in a mutual cooperation. This clarifies a13 and a31 . Moreover, being suspicious and hence defecting in the first round, ST F T ’s always defect each other in a match, and this explains why a33 = mP . Finally, in a match between T F T and ST F T , T F T starts with trust while ST F T with suspicion resulting in ST F T beating T F T in the first round. However, in the second round, T F T looses trust and beats ST F T who has just gained some. This will be switched again in the next round and so on, which corresponds to a23 and a32 . So indeed the repeated game Gm can be viewed as a normal, symmetric two-player game with the payoff matrix A and with the pure-strategy set K = {ALLC, T F T, ST F T }. The rest of the settings are based on this specific game; however, they can easily be extended to a normal, symmetric two-player game with any number of pure strategies. C. Strategy space and dominance relationship Define the unit vectors 1 0 0 sC = 0 , sT = 1 , sST = 0 0 0 1 and the set S = {sC , sT , sST }. Each pure strategy is characterized by the unit vector whose elements are all zero except for that whose index is that of the row of the pure strategy in A. Hence, sC , sT and sST correspond to ALLC, T F T and ST F T respectively. We sometimes simply denote a pure strategy by its corresponding unit vector. For example, we often refer to sC as the pure strategy ALLC. In the strategy-vector space, the convex hull spanned by all unit vectors sC , sT and sST is a simplex, and is denoted
4516
ALLC
by ∆. The unit vectors are called the vertices of the simplex. In general, any mixed strategy can be denoted by a vector y ∈ ∆, which can be obtained by a convex combination of the vertices of ∆. The convex hull of a nonempty subset H of S is a face of ∆ and is denoted by ∆(H). For example, ∆({sC , sT }) indicates a face spanned by the unit vectors sC and sT , which is a mixed strategy of the two pure strategies ALLC and T F T . For ease of notation, we remove the braces when H is presented by its members. e.g., in the previous example we write ∆(sC , sT ) instead. A face is said to be a boundary face or simply a boundary, if it is spanned by some proper subset of S. Furthermore, an interior point of a face ∆(H) is a vector h ∈ ∆(H) whose elements are all non-zero. The interior of a set H is denoted by int(H) and is the set of all interior points of H. For a payoff matrix π, when the first and second players play x and y respectively, the associated payoff to the first player is defined by
Denote the initial value of xi by x0i and that of x by x0 . Under the dynamics (3), let ξ(t, x0 ) ∈ ∆ be the solution mapping at time t starting from the initial state x0 ∈ ∆. In other words, ξ(t, x0 ) represents the population state x at time t when the system has started at x0 ∈ ∆. Denote the ith component of ξ(t, x0 ) by ξi (t, x0 ). E. Equilibrium points and sets For ease of notation in future analysis, the following points are introduced, each of which represents a stationary point of the replicator dynamics (3) under some circumstances to be explained later: T (m−1)(P −R)+P −S R−T 0 x1 = mP , −(m−2)R−S−T mP −(m−2)R−S−T T m m −(d m mR−(d m 2 eS+b 2 cT ) 2 eT +b 2 cS ) x2 = 0 mPm(R+P . −T −S) m(R+P −T −S) The next three points are based on a given initial state x0 , so we denote them by functions g i (·), i = 1, 2, 3: (m + 3)R − mP − T x0C 0 (m + 2)R − mP (xC + x0T ) (m + 3)R − mP − T x0T , 0 (m + 2)R − mP (xC + x0T ) T −R
T
u(x, y) = x πy where u(·, ·) is called the utility function. In a face ∆(H), a strategy x is said to strictly dominate another strategy y if u(x, z) > u(y, z) for all z ∈ ∆(H), and correspondingly y is said to be a strictly dominated strategy. Similarly, in ∆(H), a strategy x weakly dominates y if u(x, z) ≥ u(y, z) for all z ∈ ∆(H) and u(x, r) > u(y, r) for some r ∈ ∆(H). D. Replicator dynamics The population evolution of individuals playing different strategies is studied by using a powerful model, replicator dynamics, from evolutionary game theory. Consider a large but finite number of individuals, each assigned with some pure strategy in the set K. Let xC (t), xT (t) and xST (t) denote the population shares of individuals playing pure strategies sC , sT and sST respectively at time t. Also define the set B = {‘C 0 , ‘T 0 , ‘ST 0 }. Then, define the associated population state vector as X x(t) = xi (t)si .
(m + 2)R − mP n(T + S − 2P )x0C 0 0 0 (T − + (S − R)xT + n(xC + xT )(T + S − 2P ) n(T + S − 2P )x0T 2 0 g (x ) = 0 + (S − R)x0 + n(x0 + x0 )(T + S − 2P ) , (T − R)x C T C T (T − R)x0C + (S − R)x0T (T − R)x0C + (S − R)x0T + n(x0C + x0T )(T + S − 2P )
R)x0C
g 3 (x0 ) =
x0C 0 xC + x0T
x0T 0 xC + x0T
T 0
.
We will also need the set Φm of the equilibrium points defined for odd or even m as o n )+R−T , Φ2n+1 = x | x = (θ, 1 − θ, 0)T , 0 ≤ θ ≤ n(2R−S−T n(2R−S−T ) n o n(2R−S−T ) Φ2n = x | x = (θ, 1 − θ, 0)T , 0 ≤ θ ≤ T −R+n(T +S−2R) and also the following set from the simplex
i∈B
Hence, xi denotes the ith component of x. We eliminate the notation t in the rest of the paper, unless it is necessary to avoid possible confusion. The replicator dynamics describing the evolution over time of each individual’s population share are given by x˙ i = [u(si , x) − u(x, x)]xi
g 1 (x0 ) =
(3)
for all i ∈ B. According to the dynamics (3), the reproduction rate of each individual is proportional to the difference of her payoff, u(si , x), when playing against the average population, and the population average payoff u(x, x). Loosely speaking, the more an individual acquires when playing against her opponents, comparing to the average payoff of the individuals, the more new offspring she produces, and hence, the greater her population ratio becomes.
m D = {x | x ∈ ∆, (T −R)xC +(d m 2 eT +b 2 cS−mR)xT ≤ 0}.
III. TRIPLE - WISE COMPARISON OF THE STRATEGIES In this section, we provide the stability analysis for the case when the initial population state belongs to the interior of the simplex, namely when each of the three strategies starts with a nonzero population share. Naturally the stability results depend on the number m that determines for how many times the agents play the base game repeatedly since m structurally changes the payoff matrix A. We summarize the results in two theorems for odd and even ms, the proofs of which are not presented due to lack of space. Well known stability results in both evolutionary game theory and system dynamics are used for the proofs. Now we first present the theorem corresponding to an odd m.
4517
Theorem 3.1: Assume (2) holds. Let x0 ∈ int(∆), and m = 2n + 1, n ≥ 0. Then, under the replicator dynamics (3) it holds that T +S • if R < , 2 lim ξ(x0 , t) = x2 ; (4)
•
and the corresponding phase portrait consists of straight lines passing through sST ; nT + (n − 1)S , if R > 2n − 1 ∃x∗ ∈ Φ2n ∪ {x1 } : lim ξ(x0 , t) = x∗ t→∞
t→∞
•
if R =
and
T +S , 2 0
1
0
lim ξ(x , t) = g (x ),
t→∞
•
(5)
and the corresponding phase portrait consists of straight lines passing through the origin; T +S (n + 1)T + nS if
(6)
(n + 1)T + nS , 2n + 1 ∃x∗ ∈ Φ2n+1 ∪ {x1 } : lim ξ(x0 , t) = x∗ t→∞
(7)
and x0 ∈ int(∆ − D) ⇒ lim ξ(x0 , t) = x1 . (8) t→∞ Except for the last case where R is lower bounded, the theorem precisely determines where the solution trajectories end up according to the initial state. For example, when R is smaller than the average of T and S, starting from an interior point of the simplex, the final state will be x2 which belongs to ∆(sT , sST ). Hence, for a small enough mutual cooperation payoff, no matter how the population shares of different pure strategies are initially distributed in the interior of the simplex, all of the agents will eventually, play either T F T or ST F T , which implies that agents playing ALLC will not survive in such a case. We now present the theorem for the case of an even m. Theorem 3.2: Assume (2) holds. Let x0 ∈ int(∆), and m = 2n, n ≥ 1. Then, under the replicator dynamics (3) it holds that T +S • if R < , 2 lim ξ(x0 , t) = x2 ; (9)
x0 ∈ int(∆ − D) ⇒ lim ξ(x0 , t) = x1 . (14) t→∞ At first glance, Theorem 3.1 claims that for an even m there are more possible cases to happen comparing to an odd m. However, one of the cases, (10), claims a simple yet interesting outcome of the replicator dynamics. It simply implies that when the mutual cooperation payoff R equals the average of T and S, no matter how small the initial population share of T F T players is, as long as it is nonzero, eventually all of ALLC and ST F T players will be wiped out of the population and every individual will eventually become a T F T player. Remark 2.1: The results in Theorems 3.1 and 3.2 are stronger than typical stability results and they identify the stationary points that the solution trajectories converge to, when they start from any x0 ∈ int(∆) (except for (7) and (13) where an estimation of the domain of attraction is given, and for the case of (11) where a set of equilibria is shown to be globally asymptotically stable in int(∆)). Remark 2.2: In both Theorems 3.1 and 3.2, it is guaranteed that the population state converges to a point in the simplex, so no chaotic or periodic behavior can happen in general. Remark 2.3: Theorems 3.1 and 3.2 can be applied to tune the payoffs T, R, S and P in order to change the population shares of the three reactive strategies in the long run. One may combine the common cases of Theorems 3.1 and 3.2 to obtain the following corollary. Corollary 2.1: Assume (2) holds. Let x0 ∈ int(∆). Then, under the replicator dynamics (3) it holds that T +S , • if R < 2 lim ξ(x0 , t) = x2 ;
t→∞
t→∞
•
if R =
•
T +S , 2 lim ξ(x , t) = sT ;
t→∞
(15)
n d m−1 eT + b m−1 cS d m eT + b m cS o 2 2 2 , 2 , m−1 m t→∞
(10)
(16)
and
T +S nT + (n − 1)S if max
∃x∗ ∈ Φm ∪ {x1 } : lim ξ(x0 , t) = x∗
0
•
(13)
nT + (n − 1)S , 2n − 1 2 0 g (x ) 0 lim ξ(x , t) = t→∞ g 3 (x0 )
(11)
if R =
R−S x0C > 0 xT T −R , R−S x0C ≤ x0T T −R
(12)
x0 ∈ int(∆ − D) ⇒ lim ξ(x0 , t) = x1 . t→∞ The corollary provides a general result for all m which is quite useful. Based on equation (15), if R is less than the average of T and S, no matter how many times the snow-drift game is repeated and no matter how the population shares of the three pure strategies are initialized as long as they are nonzero, the population state converges to x2 where no T F T player exists. Moreover, according to (16), for a large enough R, the final population state converges to either a point in Φm , where no ST F T player exists, or to x1 , where no T F T
4518
player exists. In both cases, ALLC players survive. Hence, as a general result, one can conclude that under the replicator dynamics (3) and when none of the population shares of the three pure strategies are zero, ALLC players do not survive in the long run for a mutual cooperation payoff R smaller than the average of T and S, while they for sure survive for a large R. As a further result, the corollary also claims that if the population state starts from a point in ∆ − D, its trajectory converges to x1 . Hence, ∆ − D is an estimation of the domain of attraction of x1 . IV. S IMULATION RESULTS In order to visualize the previous results, the phase portraits of the replicator dynamics (3) corresponding to a repeated game Gm with an even m are presented for different values of mutual cooperation payoff R. We consider the base game as a snowdrift game with the following payoff matrix R 3 . (17) 15 1 Let the base game be repeated for 4 times, i.e., m = 4. Then the payoff matrix A of the repeated game can be calculated accordingly. We investigate the replicator dynamics (3) for five different values of R to match the five cases in Theorem 3.1. In each case, the corresponding phase portrait, when the system starts from an interior point of the simplex, is drawn to illustrate the solution trajectories of the dynamics, and the stability of different equilibrium points of the system. The game dynamics simulation software Dynamo [16] is used to plot the figures. The five cases are as follows. • For R = 4 which is less than the average of T and S, the phase portrait is shown in Fig. 1a. As (9) implies, all solution trajectories converge to x2 in this case where no ALLC player exists. 2 • By increasing R, the equilibrium point x moves towards sT , and when R = 9 equals the average of T and S, x2 matches sT . Hence, eventually the population completely becomes T F T players in this case as (10) states. So the ALLC players still do not get the chance to survive in the long run. The corresponding phase portrait is shown in Fig. 1b. • After increasing R to 10, ALLC players start showing up in the long run. For this case, the condition of (11) in Theorem 3.2 is satisfied, and as (11) implies, each solution trajectory converges to a point in Φm where a mixture of T F T and ALLC players exists. However, in most of the cases, the final state is closer to sT than sC which implies that in the long run, usually the population consists of more T F T players than ALLC players. The corresponding phase portrait is shown in Fig. 1c. • One may expect to get some solution trajectories to converge to a point close to sC by increasing R. However, for R = 11 which satisfies the condition of equation (12) in Theorem 3.2, the set of stationary points Φm breaks in to two parts which are in the form of a segment: One on the face ∆(sT , sC ) as before,
and the other from the closest stationary point to sC on ∆(sT , sC ), to the equilibrium point on the face ∆(sC , sST ). Hence, sC does not get the chance to become a stationary point which implies that the final population state never becomes ALLC players when all types of the players are available in the beginning. On the other hand, from now on, by increasing R, ST F T individuals again get the chance to appear in the long run, as well as the other two types of individuals. The corresponding results are shown in Fig. 1d. As can be seen, firstly, all of the solution trajectories are straight lines passing through the sST point. Secondly for each x0 such trajectory, if the initial state has a low ratio of xC0 , T the trajectory converges to g 3 (x0 ) while if the ratio is high, the trajectory converges to the face ∆(sT , sC ). This has been explicitly determined by (12). • Finally, by further increasing R to values greater than nT +(n−1)S , all of the second part of the equilibrium 2n−1 set in the last case disappears except for the stable equilibrium point x1 on the face ∆(sC , sST ). The corresponding phase portrait for R = 12 is shown in Fig. 1e. Each solution trajectory converges to either x1 or a point in Φm which is what (13) implies, and if the initial state is above the line connecting sST to the farthest point from sT in ∆(sT , sC ), the corresponding solution trajectory converges to x1 which is what (14) implies. No matter whether the solution trajectories converge to x2 or Φm , the final population state always includes ALLC players. Moreover, as R increases further, x1 gets closer to sC . Hence, for large Rs the final population state includes ALLC individuals, and also the greater R becomes, the higher the ratio of the ALLC individuals becomes. To conclude, T F T players always have the chance to survive in the long run, and in one case, i.e., R equals the average of T and S, they become the only survivors. For ST F T players, under small R, they also may exist in the final state, but when R increases to become greater than or equal to the average of T and S, they do not appear in the long run. However, when R becomes greater than or equal to nT +(n−1)S , ST F T players again get the chance to survive 2n−1 in the long run, but this time as R increases, their chance decreases. Finally, for ALLC individuals, there almost exists a direct relation between increments in R and their chance to survive in the long run. So as a rule of thumb, the greater the R, the more the ALLCs. Similar results hold for an odd m, so this rule of thumb is applicable to any m. V. C ONCLUDING REMARKS We have investigated the final population composition of the three reactive strategies ALLC, T F T and ST F T under the replicator dynamics when the snow-drift game is considered as the base game. We show how the final population state is affected by the changes in the mutual cooperation payoff R in two theorems, one when the base game is repeated for odd times and the other when it is
4519
(a) R = 4
(c) R = 10
(b) R = 9
(d) R = 11
(e) R = 12
4
Fig. 1: Flow patterns for the repeated game G under the replicator dynamics (3) and base-game payoff matrix (17). Color temperatures are used to show motion speeds where red corresponds to the fastest and blue the slowest motion. The circles denote equilibria. Note that in case there exists a set of equilibria, the simulation only finds and plots some of them.
repeated for even times. In most of the cases, the theorems explicitly determine that starting from an interior state, to which point the solution trajectory converges. This, from long-term stability point of view, is as useful as providing an analytical solution to the dynamics. The results of the paper can be used to tune the payoff values of a snow-drift game in order to reach a desired final population state of the reactive strategies under the replicator dynamics. For future study we aim to investigate the final population composition of other reactive strategies under similar settings of this paper. R EFERENCES [1] M. O. Jackson, Social and Economic Networks. Princeton University Press, 2010. [2] M. E. Newman, “The structure and function of complex networks,” SIAM Review, vol. 45, no. 2, pp. 167–256, 2003. [3] A.-L. Barab´asi and R. Albert, “Emergence of scaling in random networks,” Science, vol. 286, no. 5439, pp. 509–512, 1999. [4] C. J. Anderson, S. Wasserman, and B. Crouch, “A p∗ primer: logit models for social networks,” Social Networks, vol. 21, no. 1, pp. 37– 66, 1999. [5] B. Charlesworth, P. Sniegowski, and W. Stephan, “The evolutionary dynamics of repetitive DNA in eukaryotes,” Nature, pp. 215–220, 1994. [6] R. Koopmans, “Movements and media: Selection processes and evolutionary dynamics in the public sphere,” Theory and Society, vol. 33, no. 3-4, pp. 367–391, 2004. [7] D. Araujo, K. Davids, and R. Hristovski, “The ecological dynamics of decision making in sport,” Psychology of Sport and Exercise, vol. 7, no. 6, pp. 653–676, 2006. [8] S. Battiston, E. Bonabeau, and G. Weisbuch, “Decision making dynamics in corporate boards,” Physica A, vol. 322, pp. 567–582, 2003.
[9] E. Fehr and S. G¨achter, “Altruistic punishment in humans,” Nature, vol. 415, no. 6868, pp. 137–140, 2002. [10] R. B. Myerson, Game Theory: Analysis of Conflict. Harvard University Press, 2013. [11] C. Hilbe, M. A. Nowak, and K. Sigmund, “Evolution of extortion in Iterated Prisoners Dilemma games,” Proceedings of the National Academy of Sciences, vol. 110, no. 17, pp. 6913–6918, 2013. [12] M. A. Nowak, Evolutionary Dynamics: Exploring the Equations of Life. Harvard University Press, 2006. [13] C. Wang, B. Wu, M. Cao, and G. Xie, “Modified snowdrift games for multi-robot water polo matches,” in Proc. of the 24th Chinese Control and Decision Conference (CCDC), pp. 164–169, 2012. [14] I. M. Bomze, “Lotka-volterra equation and replicator dynamics: a twodimensional classification,” Biological Cybernetics, vol. 48, no. 3, pp. 201–211, 1983. [15] ——, “Lotka-volterra equation and replicator dynamics: new issues in classification,” Biological Cybernetics, vol. 72, no. 5, pp. 447–453, 1995. [16] E. D. W. H. Sandholm and F. Franchetti. (2012) Dynamo: Diagrams for evolutionary game dynamics. [Online]. Available: http://www.ssc.wisc.edu/ whs/dynamo
4520
