Open Sequential Equilibria of Multi-Stage Games ... - Semantic Scholar
Open Sequential Equilibria of Multi-Stage Games with Infinite Sets of Types and Actions by Roger B. Myerson and Philip J. Reny Department of Economics, University of Chicago Notes October 2015 http://home.uchicago.edu/~rmyerson/research/seqm.pdf https://sites.google.com/site/philipjreny/ Abstract: We consider how to extend Kreps and Wilson's 1982 definition of sequential equilibrium to multi-stage games with infinite sets of types and actions. A concept of open sequential equilibrium is defined by taking limits of strategy profiles that can consistently satisfy approximate sequential rationality for all players at arbitrarily large finite collections of observable open events. Existence of open sequential equilibria is shown for a broad class of regular projective games. Examples are considered to illustrate the properties of this solution and the difficulties of alternative approaches to the problem of extending sequential equilibrium to infinite games. 1
Goal: formulate a definition of sequential equilibrium for multi-stage games with infinite type sets and infinite action sets, and prove existence for some broad class of games. Sequential equilibria were defined for finite games by Kreps-Wilson 1982, but rigorously defined extensions to infinite games have been lacking. ("Perfect Bayesian equilibrium" defined for finite games in Fudenberg-Tirole 1991, Harris-Stinchcombe-Zame 2000 explored definitions with nonstandard analysis.) It is natural to try to define sequential equilibria of an infinite game by taking limits of sequential equilibria of finite games that "approximate" it. But no general definition of "good finite approximation" has been found. It is easy to define sequences of finite games that seem to be converging to the infinite game (in some sense) but have limits of equilibria that seem wrong. Instead we look at limits of strategy profiles that are approximately optimal (among all strategies in the game) on finite sets of events that can be observed by players in the game. A strategy profile is an ε-approximate sequential equilibrium on a set of observable events F iff it gives positive probability to each event C in F, and each player i who can observe C has no strategy that could improve i's conditional expected payoff given C by more than ε. An open sequential equilibrium is defined as a limit of a net of (ε,F)-sequential equilibria as ε→0 and F expands to include all finite collections of open sets that players can observe.
2
Multi-stage games Γ = (N,K,A,Θ,T,T,M,τ,p,u) i ∈ N = {players}, a finite set; k ∈ K = {1,...,#K} = {dates of the game}. L = {(i,k): i∈N, k∈K} = {dated players}. We write ik for (i,k). Aik = {possible actions for player i at date k}; history independent. Tik = {possible types for player i at date k} has open sets Tik with countable basis, Hausdorff. Θk = {possible date k states}. σ-algebras (closed under countable ∩ and complements) of measurable subsets are specified for Aik, Tik, and Θk. Tkj has its Borel σ-algebra. Products are given their product σ-algebras. A = ×k≤K×i∈NAik. T = ×k≤K×i∈NTik. Θ = ×k≤KΘk . Θ×A = {possible outcomes of the game}. The subscript, 0, we can construct a function f:[0,1]→[0,1.5] such that: f(y) = 0 ∀y∈[0,δ), f(⋅) takes finitely many values on [δ,1] and, for every x in [δ,1]: x < f(x) < 1.5x, and f(x) has probability 0 under each strategy in 1's given finite set. Then there is a larger game where we add the strategy f for player 1 and give player 2 the ability to recognize a1 in the finite range of f. This finite game has a perfect equilibrium where player 2 would accept any price f(θ). But when 2 would accept f(x) for any x, player 1 could do strictly better by the strategy of choosing a1 = maxx∈[0,1] f(x) for all θ. So we can eliminate such false equilibria by requiring approximate optimality among all strategies in the original game. Thus we define optimality for a player relative to the player's entire set of strategies.
10
Problems of requiring sequential rationality tests with positive probability in all events Example 4: Player 1 chooses a11∈{L,R}. If a11= L, then Nature chooses θ∈[0,1] uniformly, if a11= R, then player 1 chooses a12∈[0,1]. Player 2 then observes t2 = θ if a11= L, observes t2 = a12 if a11= R, and chooses a2∈{L,R}. Payoffs (u1,u2) are as follows (battle of the sexes): a2 = L a11 = L (1,2) a11 = R (0,0)
a2 = R (0,0) (2,1)
All BoS equilibria are reasonable (the choice, θ or a12, from [0,1] is payoff irrelevant). But if we required that all events that can have positive probability under some strategies must eventually receive positive probability along a sequence (or net), for "consistency," then the only possible equilibrium payoff would be (2,1). For any x∈[0,1], the event {t2 = x} can have positive probability, but only if positive probability is given to the strategy (a11= R, a12= x), because {θ = x} has probability 0. So in any scenario where P({t2=x}) > 0, player 2 should choose a2= R when t2=x. But then player 1 can obtain a payoff of 2 with the strategy (a11= R, a12= x). To allow other equilibria, we avoid sequential rationality tests on individual points. With a11=L, all open subsets of T2=[0,1] (with the usual topology) have positive probability, and a2=L is sequentially rational. Other topologies? If {0.5} were open in T2, (R,R) would be only open seq'l eqm outcome! (If we reversed (u1,u2), (R,R) would be forced only at countably-many x where {x} is open.) 11
Problems from allowing perturbations of nature Example 5: Nature chooses θ=(ω1,ω2) independent and uniform [−1,3]. Player 1 observes ω1, chooses a1 ∈{−1, 1}. Then player 2 observes a1, chooses a2 ∈ {−1,1}. Payoffs are u1(ω1,ω2,a1,a2) = a1a2, u2(ω1,ω2,a1,a2) = ω2a2. So player 2 wants a2 to match the sign of ω2 and player 1 wants a1 to match a2. No player has any information about ω2, so player 2 should think E(ω2)=1 > 0, and so 2 should (almost) always choose a2=1, and so player 1 should choose a1=1. But if we perturbed nature to put a small positive probability on {(ω1,ω2) = (−1,−1)}, then we could get a sequential equilibrium with s1(ω1) = −1 ∀ω1 > −1, s1(−1) = 1, s2(a1) = −a1. When player 2 will always mismatch a1, player 1 expects to get u1 = −1 whatever he does, and so 1 is willing to apply the strategy of choosing a1=−1 except when ω1=−1; but then the perturbation of nature makes {a1=1} an unlikely signal that 2 should switch to a2 = −1.
12
Open sequential equilibria may not be subgame perfect if payoffs are discontinuous Example 6: First player 1 chooses a1∈[0,1]. Then player 2 observes t2=a1 and chooses a2∈[0,1]. Payoffs are u1(a1,a2) = u2(a1,a2) = a2 if (a1,a2) ≠ (0.5,0.5), but u1(0.5,0.5) = u2(0.5,0.5) = 2. The unique subgame-perfect equilibrium has s2(a1) = 1 if a1≠0.5, s2(0.5) = 0.5, and a1 = 0.5, with the result that payoffs are u1=u2=2. This is an open sequential equilibrium. But there is another open sequential equilibrium in which player 1 chooses a1 randomly according to a uniform distribution on [0,1], and player 2 always chooses a2=1, applying the strategy s2(a1) = 1 ∀a1∈[0,1], and so payoffs are u1=u2=1. When a1 has a uniform distribution on [0,1], the observation that a1 is in any open neighborhood around 0.5 would still imply a probability 0 of the event a1=0.5, and so player 2 could not increase her conditionally expected utility by deviating from s2(a1)=1. When player 2 would always choose a2=1, player 1 has no reason not to randomize. This failure of subgame perfection occurs because sequential rationality is not being applied at the exact event of {a1=0.5}, where 2's payoff function is discontinuous. With open sequential rationality, player 2's behavior at {a1=0.5} is being justified by the possibility that a1 was not exactly 0.5 but just very close to it, where she would prefer a2=1. (What if changed from usual topology on T2 = [0,1] by adding {0.5} as a discrete open set?) Such payoff discontinuities could also arise from discontinuous responses in later periods. But subgame-perfect open sequential equilibria exist for a broad class of games. 13
Discontinuous responses may admit a possibility of other equilibria Example 7 (Harris-Stinchcombe-Zame 2000): Nature chooses (κ,θ)∈{–1,1}×[0,1]. The coordinates are independent and uniform. Then player 1 observes t1 = θ and chooses a1∈[0,1]. Then player 2 observes t2 = κ|a1 – θ| and chooses a2∈{–1,0,1}. Payoffs are u1(κ,θ,a1,a2) = –|a2|, u2(κ,θ,a1,a2) = –(a2 – κ)2 . Thus, player 2 should choose a2 to equal her expected value of κ, and player 1 wants to hide information about κ from 2. In any neighborhood of any t2≠0, player 2 knows κ=1 if t2>0, and she knows κ=–1 if t2 0, s2(t2) = –1 if t2 < 0. There is a sequential equilibrium in which player 1 hides information about κ with the strategy s1(θ) = θ, and player 2 does s2(0) = 0, but s2(t2) = –1 if t20. This equilibrium is reasonable, but 2's behavior is discontinuous at 0. We admit another equilibrium with s1(θ) = 1 ∀θ; s2(t2) = 1 if t2 > 0, s2(t2) = –1 if t2 ≤ 0. Player 2's behavior at 0 can be justified by considering neighborhoods (–ε, ε2) around 0. Alternative models: (7') Change from usual topology on T2 = [−1,1] by adding {0} as a discrete open set. (7'') θ∈Θ = [−2,−1]∪[1,2], t1 = |θ|, t2 = (sgn(θ))|(a1−|θ|)|, u2 = −(a2−sgn(θ))2. 14
A Bayesian game where sequential rationality for all types is not possible Example 8 (Hellman 2014): N={1,2}, K=1. θ = (θ0,θ1,θ2) ∈ Θ ={1,2}×[0,1]×[0,1]. θ0 is equally likely to be 1 or 2; it names the player who is "on". Types are t1=θ1, t2=θ2. When θ0=i, ti is Uniform [0,1], other -i has type t-i = 2ti if ti0 and any finite collection F of open sets of types for 1 and 2. Pick an integer m≥1 such that P(t10, there exists an (ε,F)-sequential equilibrium (ŝε) such that ŝε2 = s2, |ŝε1(D|θ) − s1(D|θ)| < ε ∀θ∈Θ ∀D⊆A1, and |P(B|C,ŝε) − a1∈C β(B|a1) P(da1|C,ŝε)| < ε ∀B∈F∩T1 ∀C∈F∩T2. Proof . Given ε, we can construct a partition Q of Θ such that each set in F∩T1 is a union of partition elements and, ∀(a1,a2), the continuous function u2(a1,a2,θ) does not change more than ε/3 as θ varies over any set in Q. Pick a finite set Z ⊂ A1 that includes one point in each set C in F∩T2 such that P(C|s)=0. For sufficiently small δ>0, we can pick ŝε1 so that ∀B∈Q such that p(B)>0, ∀θ∈B, ∀â1∈Z, ŝε1(â1|θ) = δ β(B|â1)/p(B), and ŝε1(D|θ) = s1(D|θ) [1 − δ z∈Z β(B|z)/p(B)] ∀D⊆A1\Z. 19
References David Kreps and Robert Wilson (1982): "Sequential Equilibria," Econometrica 50:863-894. Drew Fudenberg and Jean Tirole (1991): "Perfect Bayesian and Sequential Equilibrium," Journal of Economic Theory 53:236-260. Christopher J. Harris, Maxwell B. Stinchcombe, and William R. Zame (2000): "The Finitistic Theory of Infinite Games," UTexas.edu working paper. http://www.laits.utexas.edu/~maxwell/finsee4.pdf Christopher J. Harris, Philip J. Reny, and Arthur J. Robson (1995): "The existence of subgame-perfect equilibrium in continuous games with almost perfect information," Econometrica 63(3):507-544. Ziv Hellman (2014): "A game with no Bayesian approximate equilibria," Journal of Economic Theory 153:138-151. Ziv Hellman and Yehuda Levy, "Bayesian games with a continuum of states," Hebrew University working paper (2013). John L. Kelley, General Topology (Springer-Verlag, 1955). Paul Milgrom and Robert Weber (1985): "Distributional Strategies for Games with Incomplete Information," Mathematics of Operations Research 10:619-32. Robert Samuel Simon (2003): "Games of incomplete information, ergodic theory, and the measurability of equilibria," Israel Journal of Mathematics 138:73-92. 20
Example 8 (Hellman 2014) picture u-i=0 in state i State 1
State 2 ui in state i if ti>0.5 a-i=R a-i=L ai=L 0.7 0 ai=R 0 0.3 (ti = (1+t-i)/2)
1
t2
0.5
ui in state i if ti
Goal: formulate a definition of sequential equilibrium for multi-stage games with infinite type sets and infinite action sets, and prove existence for some broad class of games. Sequential equilibria were defined for finite games by Kreps-Wilson 1982, but rigorously defined extensions to infinite games have been lacking. ("Perfect Bayesian equilibrium" defined for finite games in Fudenberg-Tirole 1991, Harris-Stinchcombe-Zame 2000 explored definitions with nonstandard analysis.) It is natural to try to define sequential equilibria of an infinite game by taking limits of sequential equilibria of finite games that "approximate" it. But no general definition of "good finite approximation" has been found. It is easy to define sequences of finite games that seem to be converging to the infinite game (in some sense) but have limits of equilibria that seem wrong. Instead we look at limits of strategy profiles that are approximately optimal (among all strategies in the game) on finite sets of events that can be observed by players in the game. A strategy profile is an ε-approximate sequential equilibrium on a set of observable events F iff it gives positive probability to each event C in F, and each player i who can observe C has no strategy that could improve i's conditional expected payoff given C by more than ε. An open sequential equilibrium is defined as a limit of a net of (ε,F)-sequential equilibria as ε→0 and F expands to include all finite collections of open sets that players can observe.
2
Multi-stage games Γ = (N,K,A,Θ,T,T,M,τ,p,u) i ∈ N = {players}, a finite set; k ∈ K = {1,...,#K} = {dates of the game}. L = {(i,k): i∈N, k∈K} = {dated players}. We write ik for (i,k). Aik = {possible actions for player i at date k}; history independent. Tik = {possible types for player i at date k} has open sets Tik with countable basis, Hausdorff. Θk = {possible date k states}. σ-algebras (closed under countable ∩ and complements) of measurable subsets are specified for Aik, Tik, and Θk. Tkj has its Borel σ-algebra. Products are given their product σ-algebras. A = ×k≤K×i∈NAik. T = ×k≤K×i∈NTik. Θ = ×k≤KΘk . Θ×A = {possible outcomes of the game}. The subscript, 0, we can construct a function f:[0,1]→[0,1.5] such that: f(y) = 0 ∀y∈[0,δ), f(⋅) takes finitely many values on [δ,1] and, for every x in [δ,1]: x < f(x) < 1.5x, and f(x) has probability 0 under each strategy in 1's given finite set. Then there is a larger game where we add the strategy f for player 1 and give player 2 the ability to recognize a1 in the finite range of f. This finite game has a perfect equilibrium where player 2 would accept any price f(θ). But when 2 would accept f(x) for any x, player 1 could do strictly better by the strategy of choosing a1 = maxx∈[0,1] f(x) for all θ. So we can eliminate such false equilibria by requiring approximate optimality among all strategies in the original game. Thus we define optimality for a player relative to the player's entire set of strategies.
10
Problems of requiring sequential rationality tests with positive probability in all events Example 4: Player 1 chooses a11∈{L,R}. If a11= L, then Nature chooses θ∈[0,1] uniformly, if a11= R, then player 1 chooses a12∈[0,1]. Player 2 then observes t2 = θ if a11= L, observes t2 = a12 if a11= R, and chooses a2∈{L,R}. Payoffs (u1,u2) are as follows (battle of the sexes): a2 = L a11 = L (1,2) a11 = R (0,0)
a2 = R (0,0) (2,1)
All BoS equilibria are reasonable (the choice, θ or a12, from [0,1] is payoff irrelevant). But if we required that all events that can have positive probability under some strategies must eventually receive positive probability along a sequence (or net), for "consistency," then the only possible equilibrium payoff would be (2,1). For any x∈[0,1], the event {t2 = x} can have positive probability, but only if positive probability is given to the strategy (a11= R, a12= x), because {θ = x} has probability 0. So in any scenario where P({t2=x}) > 0, player 2 should choose a2= R when t2=x. But then player 1 can obtain a payoff of 2 with the strategy (a11= R, a12= x). To allow other equilibria, we avoid sequential rationality tests on individual points. With a11=L, all open subsets of T2=[0,1] (with the usual topology) have positive probability, and a2=L is sequentially rational. Other topologies? If {0.5} were open in T2, (R,R) would be only open seq'l eqm outcome! (If we reversed (u1,u2), (R,R) would be forced only at countably-many x where {x} is open.) 11
Problems from allowing perturbations of nature Example 5: Nature chooses θ=(ω1,ω2) independent and uniform [−1,3]. Player 1 observes ω1, chooses a1 ∈{−1, 1}. Then player 2 observes a1, chooses a2 ∈ {−1,1}. Payoffs are u1(ω1,ω2,a1,a2) = a1a2, u2(ω1,ω2,a1,a2) = ω2a2. So player 2 wants a2 to match the sign of ω2 and player 1 wants a1 to match a2. No player has any information about ω2, so player 2 should think E(ω2)=1 > 0, and so 2 should (almost) always choose a2=1, and so player 1 should choose a1=1. But if we perturbed nature to put a small positive probability on {(ω1,ω2) = (−1,−1)}, then we could get a sequential equilibrium with s1(ω1) = −1 ∀ω1 > −1, s1(−1) = 1, s2(a1) = −a1. When player 2 will always mismatch a1, player 1 expects to get u1 = −1 whatever he does, and so 1 is willing to apply the strategy of choosing a1=−1 except when ω1=−1; but then the perturbation of nature makes {a1=1} an unlikely signal that 2 should switch to a2 = −1.
12
Open sequential equilibria may not be subgame perfect if payoffs are discontinuous Example 6: First player 1 chooses a1∈[0,1]. Then player 2 observes t2=a1 and chooses a2∈[0,1]. Payoffs are u1(a1,a2) = u2(a1,a2) = a2 if (a1,a2) ≠ (0.5,0.5), but u1(0.5,0.5) = u2(0.5,0.5) = 2. The unique subgame-perfect equilibrium has s2(a1) = 1 if a1≠0.5, s2(0.5) = 0.5, and a1 = 0.5, with the result that payoffs are u1=u2=2. This is an open sequential equilibrium. But there is another open sequential equilibrium in which player 1 chooses a1 randomly according to a uniform distribution on [0,1], and player 2 always chooses a2=1, applying the strategy s2(a1) = 1 ∀a1∈[0,1], and so payoffs are u1=u2=1. When a1 has a uniform distribution on [0,1], the observation that a1 is in any open neighborhood around 0.5 would still imply a probability 0 of the event a1=0.5, and so player 2 could not increase her conditionally expected utility by deviating from s2(a1)=1. When player 2 would always choose a2=1, player 1 has no reason not to randomize. This failure of subgame perfection occurs because sequential rationality is not being applied at the exact event of {a1=0.5}, where 2's payoff function is discontinuous. With open sequential rationality, player 2's behavior at {a1=0.5} is being justified by the possibility that a1 was not exactly 0.5 but just very close to it, where she would prefer a2=1. (What if changed from usual topology on T2 = [0,1] by adding {0.5} as a discrete open set?) Such payoff discontinuities could also arise from discontinuous responses in later periods. But subgame-perfect open sequential equilibria exist for a broad class of games. 13
Discontinuous responses may admit a possibility of other equilibria Example 7 (Harris-Stinchcombe-Zame 2000): Nature chooses (κ,θ)∈{–1,1}×[0,1]. The coordinates are independent and uniform. Then player 1 observes t1 = θ and chooses a1∈[0,1]. Then player 2 observes t2 = κ|a1 – θ| and chooses a2∈{–1,0,1}. Payoffs are u1(κ,θ,a1,a2) = –|a2|, u2(κ,θ,a1,a2) = –(a2 – κ)2 . Thus, player 2 should choose a2 to equal her expected value of κ, and player 1 wants to hide information about κ from 2. In any neighborhood of any t2≠0, player 2 knows κ=1 if t2>0, and she knows κ=–1 if t2 0, s2(t2) = –1 if t2 < 0. There is a sequential equilibrium in which player 1 hides information about κ with the strategy s1(θ) = θ, and player 2 does s2(0) = 0, but s2(t2) = –1 if t20. This equilibrium is reasonable, but 2's behavior is discontinuous at 0. We admit another equilibrium with s1(θ) = 1 ∀θ; s2(t2) = 1 if t2 > 0, s2(t2) = –1 if t2 ≤ 0. Player 2's behavior at 0 can be justified by considering neighborhoods (–ε, ε2) around 0. Alternative models: (7') Change from usual topology on T2 = [−1,1] by adding {0} as a discrete open set. (7'') θ∈Θ = [−2,−1]∪[1,2], t1 = |θ|, t2 = (sgn(θ))|(a1−|θ|)|, u2 = −(a2−sgn(θ))2. 14
A Bayesian game where sequential rationality for all types is not possible Example 8 (Hellman 2014): N={1,2}, K=1. θ = (θ0,θ1,θ2) ∈ Θ ={1,2}×[0,1]×[0,1]. θ0 is equally likely to be 1 or 2; it names the player who is "on". Types are t1=θ1, t2=θ2. When θ0=i, ti is Uniform [0,1], other -i has type t-i = 2ti if ti0 and any finite collection F of open sets of types for 1 and 2. Pick an integer m≥1 such that P(t10, there exists an (ε,F)-sequential equilibrium (ŝε) such that ŝε2 = s2, |ŝε1(D|θ) − s1(D|θ)| < ε ∀θ∈Θ ∀D⊆A1, and |P(B|C,ŝε) − a1∈C β(B|a1) P(da1|C,ŝε)| < ε ∀B∈F∩T1 ∀C∈F∩T2. Proof . Given ε, we can construct a partition Q of Θ such that each set in F∩T1 is a union of partition elements and, ∀(a1,a2), the continuous function u2(a1,a2,θ) does not change more than ε/3 as θ varies over any set in Q. Pick a finite set Z ⊂ A1 that includes one point in each set C in F∩T2 such that P(C|s)=0. For sufficiently small δ>0, we can pick ŝε1 so that ∀B∈Q such that p(B)>0, ∀θ∈B, ∀â1∈Z, ŝε1(â1|θ) = δ β(B|â1)/p(B), and ŝε1(D|θ) = s1(D|θ) [1 − δ z∈Z β(B|z)/p(B)] ∀D⊆A1\Z. 19
References David Kreps and Robert Wilson (1982): "Sequential Equilibria," Econometrica 50:863-894. Drew Fudenberg and Jean Tirole (1991): "Perfect Bayesian and Sequential Equilibrium," Journal of Economic Theory 53:236-260. Christopher J. Harris, Maxwell B. Stinchcombe, and William R. Zame (2000): "The Finitistic Theory of Infinite Games," UTexas.edu working paper. http://www.laits.utexas.edu/~maxwell/finsee4.pdf Christopher J. Harris, Philip J. Reny, and Arthur J. Robson (1995): "The existence of subgame-perfect equilibrium in continuous games with almost perfect information," Econometrica 63(3):507-544. Ziv Hellman (2014): "A game with no Bayesian approximate equilibria," Journal of Economic Theory 153:138-151. Ziv Hellman and Yehuda Levy, "Bayesian games with a continuum of states," Hebrew University working paper (2013). John L. Kelley, General Topology (Springer-Verlag, 1955). Paul Milgrom and Robert Weber (1985): "Distributional Strategies for Games with Incomplete Information," Mathematics of Operations Research 10:619-32. Robert Samuel Simon (2003): "Games of incomplete information, ergodic theory, and the measurability of equilibria," Israel Journal of Mathematics 138:73-92. 20
Example 8 (Hellman 2014) picture u-i=0 in state i State 1
State 2 ui in state i if ti>0.5 a-i=R a-i=L ai=L 0.7 0 ai=R 0 0.3 (ti = (1+t-i)/2)
1
t2
0.5
ui in state i if ti
