News and Notes 3/18 • • • • Two readings in game theory assigned Short lecture today due to 10 AM fire drill HW 2 handed back.
Download ReportTranscript News and Notes 3/18 • • • • Two readings in game theory assigned Short lecture today due to 10 AM fire drill HW 2 handed back.
News and Notes 3/18
• Two readings in game theory assigned • Short lecture today due to 10 AM fire drill • HW 2 handed back today, midterm handed back Tuesday • No MK OHs today
Introduction to Game Theory Networked Life CSE 112 Spring 2004 Prof. Michael Kearns
Game Theory
• A mathematical theory designed to model: – how
rational individuals
should behave – when individual outcomes are determined by
collective
– strategic behavior behavior • Rational usually means selfish --- but not always • Rich history, flourished during the Cold War • Traditionally viewed as a subject of economics • Subsequently applied by many fields – evolutionary biology, social psychology • Perhaps the branch of pure math most widely examined outside of the “hard” sciences
Prisoner’s Dilemma
cooperate defect cooperate -1 , -1 defect -0.25
, -10 -10 , -0.25
-8 , -8 • Cooperate = deny the crime; defect = confess guilt of both • Claim that ( defect , defect ) is an
equilibrium:
– if I am definitely going to defect , you – so you will also defect – same logic applies to me choose between -10 and -8 • Note
unilateral
nature of equilibrium: – I fix a behavior or strategy for you, then choose my best response • Claim: no other pair of strategies is an equilibrium • But we would have been so much better off
cooperating…
• Looking ahead: what do people actually do?
Penny Matching
heads tails heads tails 1 , 0 0 , 1 0 , 1 1 , 0 • What are the equilibrium strategies now?
• There are none!
– if I play heads then you will of course play tails – but that makes me want to play tails – which in turn makes you want to play too heads – etc. etc. etc.
• But what if we can each (privately)
flip coins?
– the strategy pair ( 1/2 , 1/2 ) is an equilibrium • Such randomized strategies are called
mixed strategies
The World According to Nash
• If > 2 actions, mixed strategy is a
distribution
– e.g. 1/3 rock, 1/3 paper, 1/3 scissors on them • Might also have > 2 players • A general mixed strategy is a vector P = (P[1], P[2],… P[n]): – P[i] is a distribution over the actions for player i – assume
everyone
knows all the
distributions
P[j] – but the “coin flips” used to
select
from P[i] known
only
to i • P is an equilibrium if: – for every i, P[i] is a best response to all the other P[j] • Nash 1950: every game has a mixed strategy equilibrium – no matter how many rows and columns there are – in fact, no matter how many players there are • Thus known as a
Nash equilibrium
• A major reason for Nash’s Nobel Prize in economics
Facts about Nash Equilibria
• While there is always at least • Equilibrium is a
static
notion
one
– does not suggest how players might , there might be
learn many
– zero-sum games: all equilibria give the same payoffs to each player – non zero-sum: different equilibria may give different payoffs!
to play equilibrium – does not suggest how we might
choose
among multiple equilibria • Nash equilibrium is a
strictly competitive
notion – players cannot have “pre-play communication” – bargains, side payments, threats, collusions, etc. not allowed • Computing Nash equilibria for large games is difficult
Hawks and Doves
hawk dove hawk dove (V-C)/2 , (V-C)/2 0 , V V , 0 V/2 , V/2 • Two parties confront over a resource of value V • May simply display aggression, or actually have a fight • Cost of losing a fight: C > V • Assume parties are equally likely to win or lose • There are three Nash equilibria: – ( hawk , dove ), ( dove , hawk ) and ( V/C hawk , V/C hawk ) • Alternative interpretation for C >> V: – the
Kansas Cornfield Intersection
game (a.k.a.
Chicken
) – hawk = speed through intersection, dove = yield
Board Games and Game Theory
• What does game theory say about richer games?
– tic-tac-toe, checkers, backgammon, go,… – these are all games of
complete information
with
state
– incomplete information: poker • Imagine an absurdly large “game matrix” for chess: – each row/column represents a complete strategy for playing – strategy = a mapping from
every possible board configuration
next move for the player – number of rows or columns is huge --- but finite!
• Thus, a Nash equilibrium for chess exists!
– it’s just completely infeasible to
compute
it – note: can often “push” randomization “inside” the strategy to the
Repeated Games
• Nash equilibrium analyzes “one-shot” games – we meet for the first time, play once, and separate forever • Natural extension:
repeated
games – we play the same game (e.g. Prisoner’s Dilemma) many times in a row – like a board game, where the “state” is the history of play so far – strategy = a mapping from the history so far to your next move • So repeated games also have a Nash equilibrium – may be different from the one-shot equilibrium!
– depends on the game and details of the setting • We are approaching
learning
in games – natural to adapt your behavior (strategy) based on play so far
Repeated Prisoner’s Dilemma
• If we play for R rounds, and both know R: – ( always defect, always defect ) still the only Nash equilibrium – argue by backwards induction • If uncertainty about R is introduced (e.g. random stopping): – cooperation – a form of and
tit-for-tat
can become equilibria • If computational restrictions are placed on our strategies: – as long as we’re too feeble to
bounded rationality count
, cooperative equilibria arise – formally: < log(R) states in a finite automaton
The Folk Theorem
• Take any one-shot, two-player game • Suppose that (u,v) are the (expected) payoffs under some mixed strategy pair (P[1],P[2]) for the two players – (P[1], P[2])
not
necessarily a Nash equilibrium – but (u,v) gives better payoffs than the – example: sec. level is ( -8 , -8
security levels
• security level: what a player can get no matter what the other does ) in Prisoner’s Dilemma; ( -1 , -1 ) is better • Then there is always a Nash equilibrium for the
infinite
repeated game giving payoffs (u,v) – makes use of the concept of
threats
• Partial resolution of the difficulties of Nash equilibria…
Correlated Equilibrium
• In a Nash equilibrium (P[1],P[2]): – player 2 “knows” the – equilibrium relies on
distribution
• Suppose now we also allow • Then two strategies are in P[1] – but doesn’t know the “random bits” player 1 uses to select from P[1]
private randomization public (shared)
shared bits = 110100110, then play hawk” randomization – so strategy might say things like “if private bits = 100110 and
correlated equilibrium
if: – knowing only your
strategy
and the
shared
best response, and vice-versa bits, my strategy is a • Nash is the special case of no shared bits
Hawks and Doves Revisited
hawk dove hawk dove (V-C)/2 , (V-C)/2 0 , V V , 0 V/2 , V/2 • There are three Nash equilibria: – ( hawk , dove ), ( dove , hawk ) and ( V/C hawk , V/C hawk ) • Alternative interpretation for C >> V: – the
Kansas Cornfield Intersection
game (a.k.a.
Chicken
) – hawk = speed through intersection, dove = yield • Correlated equilibrium: the
traffic signal
– if the shared bit is green to me, I am playing hawk – if the shared bit is red to me, I will play dove – you play the symmetric strategy – splits waiting time between us --- a
different
outcome than Nash
Correlated Equilibrium Facts
• Always exists – all Nash equilibria are correlated equilibria – all probability distributions over Nash equilibria are C.E.
– and some more things are C.E. as well – a
broader
concept than Nash • Technical advantages of correlated equilibria: – often easier to
compute
than Nash • Conceptual advantages: – correlated behavior is a fact of the real world – model a
limited
form of cooperation – more general cooperation becomes extremely complex and messy • Breaking news (late 90s – now): – CE is the natural convergence notion for “rational” learning in games!
A More Complex Setting: Bargaining
• Convex set S of possible payoffs • Players must bargain to settle on a solution (x,y) in S • What “should” the solution be?
• Want a
general
answer • A
function
F(S) mapping S to a solution (x,y) in S • Nash’s axioms for F: – choose on red boundary (Pareto) – scale invariance – symmetry in the role of x and y – “independence of irrelevant alternatives”: • if green solution was contained in smaller red set, must also be red solution possible outcomes x = payoff to player 1
Nash’s Bargaining Solution
• There’s that
only one
choice of F that satisfies all these axioms • And the winner is: – choose (x,y) on the boundary of S
maximizes
xy • Example: rich-poor bargaining Cash 1 (rich) 0 25 50 75 100 Cash 2 (poor) 100 75 50 25 0 Utility 1 (rich) 0.00
0.25
0.50
0.75
1.00
Utility 2 (poor) U1 x U2 outcomes 1.00
0.98
0.000
0.245
0.90
0.450
0.73
0.00
0.548
0.000
Social Choice Theory
• Suppose we must collectively choose between alternatives – e.g. Bush, Kerry, Nader, Sharpton,… • Under current voting scheme, “gamesmanship” encouraged – e.g. prefer Nader to Gore, but Gore is more “electable” – not a “truth revealing” mechanism • An idealized voting scheme – we each submit a
complete ordering
on the candidates – e.g. Sharpton > Bush > Nader > Kerry – then
combine
the orderings to choose a
global
ordering (outcome) – we would like the outcome to be “fair” and “reasonable” • What do “fair” and “reasonable” mean?
• Again take an
axiomatic
approach
Social Choice Axioms
• Let’s call F the mapping from preferences to outcome • Suppose for some preferences, x > y in the global outcome – then if we move x up in all preferences, F still has x > y • Suppose we look at some subset S of alternatives – e.g. S = {Kerry, Sharpton} – suppose we modify preferences only • Non-dictatorship:
outside
– otherwise F is ignoring the preferences!
– no single individual determines output of F of S – F’s ranking of S should remain unchanged (irrelevant alternatives) • Always some way of making F output x > y, for any x,y
Arrow’s Impossibility Theorem
• There is
no mapping
F satisfying all four axioms!
• Long history of alternate axioms, (im)possibility research • A mathematical demonstration of the difficulty of selecting collective outcomes from individual preferences
Next Up
• Have so far examined simple games between two players • Strategic interaction on the smallest “network”: – two vertices with a single link between them – much richer interaction than just info transmission, messages, etc.
• Classical game theory generalizes to many players – e.g. Nash equilibria always exist in multi-player matrix games – but this fails to capture/exploit/examine
structured
interaction • We need specific models for networked games: – games on networks: local interaction – shared information: economies, financial markets – voting systems – evolutionary games