Extensive Games with Imperfect Information

• • • Extensive Games with Imperfect Info In this section, we model situations in which players move sequentially but may or may not be informed about previous player’s actions. With perfect information, we needed to specify players, terminal histories, when players had the move, preferences and payoffs.

With imperfect information, we need to add to that list:

a specification about each player’s information about the history at every point at which she moves.

Extensive Games with Imperfect Info Definition: an

information set

nodes satisfying: for a player is a collection of decision – The player has the move at every node in the information set, and – When the play of the game reaches a node in the information set, the player with the move does not know which node in the information set has (or has not) been reached

Extensive Games with Imperfect Info Example 314.2 in Osborne: Battle of the Sexes I U2 CP II II U2 CP U2 CP 2,1 0,0 0,0 1,2 Nash equilibria?

Subgame Perfect Nash Equilibria ?

Extensive Games with Imperfect Info • Other Examples 315.1 in Osborne: Card Game (Like problem set question)

Extensive Games with Imperfect Info Definition: Strategy in an Extensive game of Imperfect information:

A (pure) strategy of player i in an extensive game is a function that assigns to each of i’s information sets, I i , an action in A(I i ) (ie, the set of actions available to player i at information set I i ).

Definition: Mixed Strategy in an Extensive game of Imperfect information: A mixed strategy of a player in an extensive game is a

probability distribution over the player’s pure strategies.

Extensive Games with Imperfect Info Definition: Nash Equilibrium in an Extensive game of Imperfect information: The mixed strategy profile,

(mixed strategy) Nash equilibrium if, for each player i and every mixed strategy

a

i

a

*, in an extensive game is a of player i, player i’s expected payoff to

a

* is at least as large as her expected payoff to (

a

i ,

a

* -i ) according to a payoff function whose expected value represent player i’s preferences over lotteries.

Extensive Games with Imperfect Info 320.2 in Osborne (Commitment and Observability) II X Y X (3,2) (1,1) I Y (4,3) (2,4)

Extensive Games with Imperfect Info 320.2 in Osborne (Commitment and Observability) II X Y X (3,2) (1,1) I Y (4,3) (2,4) NE of Simultaneous game: (Y,Y)

Extensive Games with Imperfect Info 320.2 in Osborne (Commitment and Observability) II X Y X (3,2) I Y (4,3) NE of Simultaneous game: (Y,Y) (1,1) (2,4) X I Y X II Y X II Y 3,2 1,1 4,3 2,4 SPNE outcome of extensive game (I moves first, II second) of perfect information: (X,X)

Extensive Games with Imperfect Info 320.2 in Osborne (Commitment and Observability) -- Imperfect Information Game: How many information sets?

How many subgames?

3,2 X Y 1,1 II 4,3 X Y 2,4 1 e X Chance e Y X X 3,2 •

Nash Equilibrium for

e

< 1/4 ?

Y 1,1 I II Y 4,3 X e X Chance 1 e Y Y 2,4

•Player I moves first, but his choice is imperfectly observed by player II.

•Strategies: player I: (X, Y) player II: ( (X,X),(X,Y),(Y,X),(Y,Y) ). For player II, a strategy (

a,b

) means play

a

if the signal is X and play

b

if the signal is Y.

3,2 X Y 1,1 II 4,3 X Y 2,4 1 e X Chance e Y X I Y e X Chance 1 e Y II X 3,2 •

Nash Equilibrium for

e

< 1/4 ?

Y 1,1 4,3 X Y 2,4

Strategic Form: I\II X Y (X,X) 3,2 4,3 (X,Y) 3-2e,2-e* 2+2e,4-e (Y,X) 1+2e,1+e 4-2e,3+e (Y,Y) 1,1 2,4 * Payoff calculated as: ( 3(1-e)+1(e), 2(1-e)+1(e) ) = (3-2e,2-e) 3,2 X Y 1,1 1 e X Chance e Y X X 3,2 •

Nash Equilibrium for

e

< 1/4 ?

Y 1,1 II I II 4,3 X Y 2,4 Y 4,3 X e X Chance 1 e Y Y 2,4

Strategic Form: I\II X Y (X,X) 3,2 4,3 *3-2e>2+2e  -4e>-1  e < 1/4 **4-2e>1+2e  -4e>-3  e < 3/4 (X,Y) 3-2e,2-e* 2+2e,4-e (Y,X) 1+2e,1+e 4-2e,3+e** (Y,Y) 1,1 2,4 3,2 Y 1,1 4,3 2,4 X II X Y 1 e X Chance e Y X I Y e X Chance 1 e Y II X 3,2 •

Nash Equilibrium for

e

< 1/4 ?

Y 1,1 4,3 X Y 2,4 Equilibrium = (Y,YY)

Extensive Games with Imperfect Info 320.2 in Osborne (Commitment and Observability) II X Y X (3,2) (1,1) I Y (4,3) NE of Simultaneous game: (Y,Y)  outcome = (2,4) (2,4) X I Y X II Y X II Y 3,2 1,1 4,3 2,4 SPNE of extensive game (I moves first, II second) of perfect information: (X,X)  outcome = (3,2) (Bayesian) NE outcome of sequential imperfect information game (Y,YY)  outcome (2,4) Player 1 loses his first-mover advantage!

Extensive Games with Imperfect Info Osborne: 317.1 / 322.1 / 323.1: Entry Game Motivates a need for “beliefs” of players.

Extensive Games with Imperfect Info • In simultaneous (strategic) games of perfect information, our equilibrium concept was the Nash equilibrium (NE). When we moved to sequential games, we introduced a slightly stronger equilibrium called subgame perfect Nash equilibium (SPNE), primarily to rule out non-credible threats. In static games of imperfect information, we needed a further “refinement” which we called Bayesian Nash Equilibrium (BNE). Finally, in extensive games of imperfect information, we make a further refinement and introduce Perfect Bayesian Nash Equilibrium, or (PBE) for short.

Extensive Games with Imperfect Info A note on terminology: Perfect Bayesian Equilibrium or PBE Weak Sequential Equilibrium or WSE (Osborne’s terminology) PBE = WSE

Perfect Bayesian Equilibrium*

• Gibbons Game, Page 176 L I M R (1,3) II II L’ (2,1) L’ R’ (0,0) (0,2) R’ (0,1)

Perfect Bayesian Equilibrium*

• Gibbons Game, Page 176 – Strategic Form L L’ (2,1) II R’ (0,0) • So 2 NE in pure strategies: (L,L’) and (R,R’) • Subgame Perfect NE? Since the game has no subgames besides the whole game, NE=SPNE.

• But player II playing R’ is based on a non credible threat since L’ is always optimal if player II gets to move. I M (0,2) R (1,3) (0,1) (1,3) So we rule out (R,R’) with the following 2 refinements.

Perfect Bayesian Equilibrium*

• Requirement 1: At each information set, the player with the move must have a

belief

about which node in the information set has been reached by the play of the game. For a nonsingleton information set, a belief is a probability distribution over the nodes in the information set; for a singleton information set, the player’s belief puts probability one on the single decision node.

Perfect Bayesian Equilibrium*

2.

Requirement 2: Given their beliefs, the players’ strategies must be

sequentially rational

. That is, at each information set the action taken by the player with the move (and the player’s subsequent strategy) must be optimal given the player’s belief at that information set and the other players’ subsequent strategies (where a “subsequent strategy” is a complete plan of action covering every contingency that might arise after the given information set has been reached)

Perfect Bayesian Equilibrium*

• Gibbons Game, Page 176 L (2,1) L’ II [p] R’ (0,0) I R M L’ II [1-p] R’ (1,3) (0,2) (0,1) • (p,1-p) are the beliefs of player II. p is the probability that player II puts on the history that player I has played L and (1-p) is the probability that player II puts on the history that player I has played M.

Perfect Bayesian Equilibrium*

• • • • Gibbons Game, Page 176 L L’ II [p] R’ I M R L’ II [1-p] R’ (1,3) (2,1) (0,0) (0,2) (0,1) So, given player II’s beliefs, the expected payoff to player II from playing R’ is 0*p + 1*(1-p) = 1-p. The expected payoff to player II from playing L’ is 1*p + 2*(1-p) = 2-p. Since 2-p > 1-p, for ALL p, then R’ can never be played in equilibrium.

So (R,R’) does not satisfy the requirement #2.

Perfect Bayesian Equilibrium*

Requirements 1 and 2 insist that the players have beliefs and act optimally given these beliefs, but not that these beliefs be reasonable. We need further requirements.

Definition: for a given equilibrium in a given extensive game, an information set is

on the equilibrium path

equilibrium strategies, and is equilibrium strategies.

if it will be reached with positive probability if the game is played according to the

off the equilibrium path

if it is certain not to be reached if the game is played according to the

Perfect Bayesian Equilibrium*

• • Requirement 3: At information sets on the equilibrium path, beliefs are determined by Bayes’ rule and the players’ equilibrium strategies.

Requirement 4 :At information sets off the equilibrium path, beliefs are determined by Bayes’ rule and the players’ equilibrium strategies where possible.

Perfect Bayesian Equilibrium*

• • • Gibbons Game, Page 176 L L’ II [p] R’ I M R L’ II [1-p] R’ (1,3) (2,1) (0,0) (0,2) (0,1) So in the SPNE where players play (L,L’), player 2’s belief must be p = 1. “Given player 1’s equilibrium strategy (L), player 2 knows which node in the information set has been reached.” As a second example, suppose in a mixed strategy equilibrium, player I played L with probability q1, M with probability q2, and R with probability 1-q1-q2. Then, this would force player II to set p = q1/(q1+q2), by Bayes rule.

Perfect Bayesian Equilibrium*

Definition: A through 4.

Perfect Bayesian Equilibrium

consists of strategies and beliefs satisfying 1 • • Crucial new features of a PBE: beliefs are elevated to the level of importance of strategies in our definition of an equilibrium. An equilibrium consists of strategies and beliefs for each player at each information set at which the player has the move.

Players must have reasonable beliefs both on and off the equilibrium path.

• • •

Signaling Games

Signaling Games: a dynamic game of imperfect information involving a sender and a receiver, S and R. Nature draws a type for the S from a set of possible types. S observe his type and chooses a message to send to R. R observes the message (not the type) and then choose an action. Payoffs are then realized.

•

Signaling Games

Sender may be a worker who signals to a potential employer (Reciever) with an education choice based on his type, possibly his productivity level. See Spence (1973): Job Market Signaling.

•

Signaling Games

Consider the following signaling game: a1 a1 m1 Sender t1 m2 a2 a2 z Receiver Nature Receiver 1-z a1 a1 m1 m2 Sender t2 a2 a2 If both types of senders send the same message in an equilibrium, we call this equilibrium “Pooling” If a sender sends a different message depending on his type, we called this equilibrium “Separating.”

•

Signaling Games

Denote (p,1-p) and (q,1 q) denote R’s beliefs at each of his (two) information sets.

a1 [p] a2 m1 Sender t1 m2 [q] a1 a2 z Receiver Nature Receiver a1 [1-p] a2 m1 1-z Sender t2 m2 [1-q] a1 a2 This game has four possible (pure strategy) equilibria: 1) Pooling on m1, 2) pooling on m2, 3) Separating with t1 playing m1 and t2 playing m2, 4) Separating with t1 playing m2 and t2 playing m1.

•

Signaling Games

Lets add payoffs and solve an example. 1,3 u [p] 4,0 d Receiver 2,4 u [1-p] d 0,1 Solve for all PBE of this game ...

L L Sender t1 0.5

Nature 0.5

Sender t2 R R 2,1 [q] u d 0,0 Receiver [1-q] u 1,0 d 1,2

Signaling Games

Need to consider the following candidate equilibria: 1) Pooling on (L,L); 2) Pooling on (R,R); 3) 4) Separating on (L,R); Separating on (R,L). For each type of candidate equilibrium we need strategies of each player and beliefs of any player that is uncertain at any point in the game. In this game, the receiver is uncertain at two of his information sets (the one following the L signal and the one following the R signal). So overall, we need: • A strategy for the sender, (X,Y)  type 2).

(what he does if he’s type 1, what he does if he’s • • • A strategy for the receiver, (X,Y)  does if he sees the R signal).

(what he does if he sees the L signal, what he A belief for the receiver at the left side information set, (p,1-p) the L signal came from a type 1 sender, his belief that the L signal came from a type 2 sender).

 (his belief that A belief for the receiver at the right side information set, (q,1-q) the R signal came from a type 1 sender, his belief that the R signal came from a type 2 sender).

 (his belief that

Signaling Games

Pooling on (L,L). • This means that the sender plays L if he’s type 1 or type 2. • On the left, the receiver doesn’t get any additional information about the sender’s type when he sees L being played (because both types are playing L). So the receivers belief should be p = ½, the prior probability distribution of types.

• What does the receiver do if he sees the L signal? We see he always plays u on the left for any beliefs (including ½, ½).

• What does the receiver do if he sees the R signal? We see he would like to play u on the top, d on the bottom, but he cannot distinguish top from bottom. He can find beliefs (q,1-q) to make him want to do either. • Now lets see if the sender really wants to player (L,L). Playing (L,L) results in the receiver playing u, so a type 1 sender gets a payoff of 1 and a type 2 sender gets a payoff of 2. – If a type 1 sender deviates to R, his payoff is either 2 (if u) or 0 (if d). So in order for this to be a PBE, a type 1 sender requires that the receiver have beliefs such that he wants to play d if the receiver sees the R signal. Otherwise, the type 1 sender would want to deviate from L (and get 1) to R (and get 2). So a type 1 sender requires E rec [d|R] >= E rec [u|R]  0q+2(1-q) >= 1q+0(1-q)  q <= 2/3.

– If a type 2 sender deviates to R, his payoff is either 1 (if u) or 1 (if d). These are both less than what he gets by playing L (2), so we say that a type 2 sender puts no requirement on the beliefs of the receiver following the R signal.

So we have a PBE at {(L,L), (u,d), (p,1-p), (q,1 q) | p = ½, q <= 2/3 }

Signaling Games

• • • • • Pooling on (R,R). This means that the sender plays R if he’s type 1 or type 2. On the right, the receiver doesn’t get any additional information about the sender’s type when he sees R being played (because both types are playing R). So the receivers belief should be q = ½, the prior probability distribution of types.

What does the receiver do if he sees the R signal? With q = ½, E rec [u|R] = 1(1/2) + 0(1/2) = ½, while E rec [d|R] = 0(1/2) + 2(1/2) = 1. So the receiver should play d if he sees R.

What does the receiver do if he sees the L signal? We see he always plays u on the left, no matter his beliefs (it’s a dominant strategy). Now lets see if the sender really wants to player (R,R). Playing (R,R) results in the receiver playing d, so a type 1 sender gets a payoff of 0 and a type 2 sender gets a payoff of 1. – If a type 1 sender deviates to L, his payoff is 1, higher than he gets by playing R, so he would want to deviate and (R,R) cannot be part of a PBE. We could stop here … – If a type 2 sender deviates to L, his payoff is 2, again higher than what he gets by playing R, so he would also want to deviate.

So there does not exist a PBE involving the sender pooling on R.

Signaling Games

• • • • Separating on (L,R). This means that a type 1 sender plays L and a type 2 sender player R.

Given our definition of a PBE, beliefs are determined by equilibrium strategies. Since both information sets of the receiver are ON the equilibrium path, the only beliefs that are consistent with (L,R) is p = 1 and q = 0. I.e., if only type 1 senders play L, then if a receiver sees L, he should believe that it came from a type 1 sender with probability 1. If only type 2 senders play R, then if a receiver sees R, he should believe that it came from a type 2 sender with probability 1. So, the receiver’s optimal strategy is (u,d). – – Now lets see if the sender really wants to player (L,R). Playing (L,R) results in the receiver playing (u,d), so a type 1 sender gets a payoff of 1 and a type 2 sender gets a payoff of 1.

Should a type 1 sender deviate? By deviating to R, his payoff becomes 0, so he doesn’t want to deviate.

Should a type 2 sender deviate? By deviating to L, his payoff becomes 2, so he does want to deviate. So there does not exist a PBE involving the sender separating on (L,R).

Signaling Games

• • • • Separating on (R,L). This means that a type 1 sender plays R and a type 2 sender player L.

Given our definition of a PBE, beliefs are determined by equilibrium strategies. Since both information sets of the receiver are ON the equilibrium path, the only beliefs that are consistent with (R,L) is p = 0 and q = 1. I.e., if only type 1 senders play R, then if a receiver sees R, he should believe that it came from a type 1 sender with probability 1. If only type 2 senders play L, then if a receiver sees L, he should believe that it came from a type 2 sender with probability 1. So, the receiver’s optimal strategy is (u,u). – – Now lets see if the sender really wants to player (R,L). Playing (R,L) results in the receiver playing (u,u), so a type 1 sender gets a payoff of 2 and a type 2 sender gets a payoff of 2.

Should a type 1 sender deviate? By deviating to L, his payoff becomes 1, so he doesn’t want to deviate.

Should a type 2 sender deviate? By deviating to R, his payoff becomes 1, so he doesn’t want to deviate. So there exists a PBE at {(R,L), (u,u), (p,1-p), (q,1-q) | p = 0, q = 1 }

In general,

Signaling Games

1) Pooling on (L,L)  p= the prior 2) Pooling on (R,R)  q= the prior 3) Separating on (L,R)  p=1, q=0 4) Separating on (R,L)  q=1, p=0

• • • • •

The Market for Lemons

Akerlof models used car market. Buyers value good cars at b

G

and lemons at b

L

.

Sellers value good cars at s

G

and lemons at s

L

.

Sellers know the true quality of the car and offer a price, P. Buyers only know a certain proportion of cars, q, are good and a proportion, 1-q, are lemons.

• • • • •

The Market for Lemons

– – – – – – – Safe assumption: b

G >b L

and s

G > s L

.

Assume that under perfect information, trade would take place for all cars:

b L b G > s L > s G

Under imperfect information, buyers buy if E[V] = q*b

G +(1-q)*b L



P

In a separating equilibrium, owners of good cars do not sell their cars and owners of lemons sell their cars. This happens if:

b L = P < s G

b G • this is easily satisfied (see homework 5) How about a pooling equilibrium where all cars are sold at the price P. Buyers Require: P  Sellers Require: P  So: s

G q*b G +(1-q)*b L s G



q*b G +(1-q)*b L

• this is hard to satisfy (see homework 5) especially if q is small.

For reasonable valuations and probabilities, a pooling equilibrium will not exist.

If q is small, q*b G +(1-q)*b L might be here s G b L s L

• • • • • • •

Card Game

2 Players. Each player antes $1 into a pot.

Player 1 picks a card from a deck that is either High or Low with equal probability. Player 2 does not observe the card.

Player 1 can then See or Raise. If he Sees and the card is High, player 1 wins the pot for a payoff of (+1,-1). If he Sees and the card is Low, player 2 wins the pot for a payoff of (-1,+1).

If player 1 Raises, then player 2 can either Pass or Meet. Again player 2 does not know if the card is High or Low.

If player 2 Passes, player 1 wins the pot for a total payoff of (1,-1) no matter if the card is High or Low.

If player 2 Meets, then if the card is High, player 1 wins the pot with a total payoff of (2,-2) and if the card is Low, player 2 wins the pot with a total payoff of (-2,2).

What is the Nash Equilibrium (not PBE) of this game?

• Extensive Form Game:

Card Game

Nature High Low See 1 +1,-1 1 Raise See -1,+1 Raise 2 Pass Meet Pass Meet +1,-1 +2,-2 +1,-1 -2,+2 •Player 1 has two information sets. Player 2 has one.

•Strategies of player 2: (Pass or Meet) •Strategies of player 1: (Raise,Raise),(Raise,See),(See,Raise),(See,See)  player 1 does if the card is High and what player 1 does if the card is Low.

i.e., what

Card Game

Strategic Form:

Player 1\2

R,R R,S S,R S,S

Pass

1,-1 0,0 1,-1 0,0

Meet

0,0 1/2,-1/2 -1/2,1/2 0,0

Card Game

Strategic Form:

Player 1\2

R,R R,S S,R S,S

Pass

1,-1 0,0 1,-1 0,0

Meet

0,0 1/2,-1/2 -1/2,1/2 0,0 •So no pure strategy NE.

•Mixed? Notice (S,S) is strictly dominated by for player 1 by an equal mixture of (R,R) and (R,S).

Card Game

Strategic Form:

Player 1\2

R,R (p1) R,S (p2) S,R (p3)

Pass (q)

1,-1 0,0 1,-1

Meet (1-q)

0,0 1/2,-1/2 -1/2,1/2

1’s Expected Payoff

q ½(1-q) q-1/2(1-q) •Note that if q=1  0  q=0  player 1 would mix on p1 and p3 with p2= contradiction so q<1.

•Note that if q=0  p1=p3=0  q=1  player 1 would only mix on p2 with contradiction so q>0.

•So player 2 is mixing with positive weight on both pass and meet.

•Also player 1 can’t be playing a pure strategy, because player 2 would want to respond with a pure strategy, but we already know that there are no PSNEs.

Card Game

Strategic Form:

Player 1\2

R,R (p1) R,S (p2) S,R (p3)

Pass (q)

1,-1 0,0 1,-1

Meet (1-q)

0,0 1/2,-1/2 -1/2,1/2

1’s Expected Payoff

q ½(1-q) q-1/2(1-q) • Could player 1 be mixing on all 3 remaining strategies? We require: q=1/2(1-q)  q=1/2 and ½ =? 1/2-1/2(1-1/2)  ½ 1/4. So this cannot be a NE.

• So player 1 must be mixing on 2 strategies. Given the conditional expected payoffs, E[R,R] can never equal E[S,R]. E[R,S] = R[S,R] if q = 1/2 , but E[R,R] = ½ > E[R,S] = E[S,R]. So (S,R) is never played with positive probability. Player 1 must be mixing on (R,R) and (R,S).

Card Game

Strategic Form:

Player 1\2

R,R (p) R,S (1-p)

Pass (q)

1,-1 0,0

Meet (1-q)

0,0 1/2,-1/2

1’s Expected Payoff

q ½(1-q) • E[R,R] = E[R,S]  q=1/2(1-q)  q=1/3.

• E[Pass] = E[Meet]  -p = -1/2(1-p)  ½=3/2p  p=1/3.

• So the unique NE of this game is {(1/3,2/3,0,0),(1/3,2/3)} Player 1 “bluffs” with probability 1/3.

Why PBE Still Isn’t Enough… 2,-1 X II [p] Y L I M [1-p] II R X Y -4,-2 -1,-2 0,-1 1,1

Why PBE Still Isn’t Enough… 2,-1 X II [p] Y L I M [1-p] II R X Y

I\II

L M R -4,-2 -1,-2

X

2,-1 -1,-2 1,1 0,-1

Y

-4,-2 0,-1 1,1 1,1

Why PBE Still Isn’t Enough…

I\II

L M R X II [p] Y L I M [1-p] II R X Y 2,-1

X

2,-1 -1,-2 1,1 -4,-2

Y

-4,-2 0,-1 1,1 -1,-2 0,-1 1,1 2 NE = SPNE @ (L,X) and (R,Y)

Why PBE Still Isn’t Enough… X II [p] Y L I M [1-p] II R X Y 1,1 2,-1 -4,-2 -1,-2 0,-1 2 NE = SPNE @ (L,X) and (R,Y). PBE?

For (L,X), II’s information set is on the equilibrium path, so {(L,X),(p,1-p)|p=1} is a PBE.

For (R,Y), II’s information set is off the equilibrium path, so we require E[Y]>=E[X] or -2p-(1-p) >= -p-2(1-p) or p<= 0.5. So {(R,Y),(p,1-p)|p<=0.5} is a PBE.

Problems?

Why PBE Still Isn’t Enough… X II [p] Y L I M [1-p] II R X Y 1,1 2,-1 -4,-2 -1,-2 0,-1 {(L,X),(p,1-p)|p=1} is a PBE.

{(R,Y),(p,1-p)|p<=0.5} is a PBE.

R strictly dominates M for player 1! So if player 2 gets a chance to move, by

forward induction

, he should expect that player 1 has played L, not M. So the second equilibrium, while still a PBE, could be eliminated with further equilibrium refinements.