Transcript Algorithmic Mechanism Design

FOURTH PART
Algorithmic Issues in
Strategic Distributed Systems
Suggested readings


Algorithmic Game Theory, Edited by Noam
Nisan, Tim Roughgarden, Eva Tardos, and Vijay
V. Vazirani, Cambridge University Press.
Blog by Noam Nisan
http://agtb.wordpress.com/
Two Research Traditions

Theory of Algorithms: computational issues





What can be feasibly computed?
How long does it take to compute a solution?
Which is the quality of a computed solution?
Centralized or distributed computational models
Game Theory: interaction between self-interested
individuals


What is the outcome of the interaction?
Which social goals are compatible with selfishness?
Different Assumptions

Theory of Algorithms (in distributed systems):



Processors are obedient, faulty (i.e., crash),
adversarial (i.e., Byzantine), or they compete without
being strategic (e.g., concurrent systems)
Large systems, limited computational resources
Game Theory:


Players are strategic (selfish)
Small systems, unlimited computational resources
The Internet World

Users often selfish



Have their own individual goals
Own network components
Internet scale


Massive systems
Limited communication/computational
resources
 Both strategic and computational issues!
Fundamental question

How the computational aspects of a strategic
distributed system should be addressed?
Algorithmic
Theory of
Game
=
+
Game Theory Algorithms
Theory
Basics of Game Theory

A game consists of:





A set of players (or agents)
A specification of the information available to each
player
A set of rules of encounter: Who should act when, and
what are the possible actions (strategies)
A specification of payoffs for each possible outcome
(combination of strategies) of the game
Game Theory attempts to predict the final
outcomes (or solutions) of the game by taking
into account the individual behavior of the
players
Solution concept
How do we establish that an outcome is a solution? Among
the possible outcomes of a game, those enjoying the
following property play a fundamental role:

Equilibrium solution: strategy combination in which
players are not willing to change their state. But this
is quite informal: what does it rationally mean that a
player does not want to change his state? In the
Homo Economicus model, this makes sense when he
has selected a strategy that maximizes his individual
wealth, knowing that other players are also doing the
same.
Roadmap


We will focus on two prominent types of equilibria: Nash
Equilibria (NE) and Dominant Strategy Equilibria (DSE)
Computational Aspects of Nash Equilibria




Can a NE be feasibly computed, once it exists?
What about the “quality” of a NE?
Case study: Network Flow Games (i.e., selfish routing in
Internet), Network Design Games, Network Creation Games
(Algorithmic) Mechanism Design



Which social goals can be (efficiently) implemented in a strategic
distributed system?
Strategy-proof mechanisms in DSE: Vickrey-Clarke-Groves
(VCG)-mechanisms and one-parameter mechanisms
Case study: Shortest Path, Minimum Spanning Tree, Singlesource Shortest-path Tree, Single-Minded Combinatorial
Auctions
FIRST PART:
(Nash)
Equilibria
(Some) Types of games






Cooperative/Non-cooperative
Symmetric/Asymmetric (for 2-player
games)
Zero sum/Non-zero sum
Simultaneous/Sequential
Perfect information/Imperfect information
One-shot/Repeated
Games in Normal-Form
We start by considering simultaneous, perfectinformation and non-cooperative games. These games are
usually represented explicitly by listing all possible
strategies and corresponding payoffs of all players (this
is the so-called normal–form); more formally, we have:



A set of N rational players
For each player i, a strategy set Si
A payoff matrix: for each strategy combination (s1,
s2, …, sN), where siSi, a corresponding payoff
vector (p1, p2, …, pN)
 |S1||S2| … |SN| payoff matrix
A famous game:
the Prisoner’s Dilemma
Non-cooperative, symmetric, non-zero sum, simultaneous,
perfect information, one-shot, 2-player game
Prisoner II
Prisoner I
Don’t
Implicate
Implicate
Don’t
Implicate
1, 1
6, 0
Implicate
0, 6
5, 5
Strategy
Set
Strategy
Set
Payoffs (for
this game,
these are
years in jail, so
they should be
seen as a cost
that a player
wants to
minimize)
Prisoner I’s decision
Prisoner II
Prisoner I Don’t Implicate
Implicate

Prisoner I’s decision:



Don’t Implicate
Implicate
1, 1
6, 0
0, 6
5, 5
If II chooses Don’t Implicate then it is best to Implicate
If II chooses Implicate then it is best to Implicate
It is best to Implicate for I, regardless of what II does:
Dominant Strategy
Prisoner II’s decision
Prisoner II
Don’t Implicate
Implicate
1, 1
6, 0
0, 6
5, 5
Prisoner I Don’t Implicate
Implicate

Prisoner II’s decision:



If I chooses Don’t Implicate then it is best to Implicate
If I chooses Implicate then it is best to Implicate
It is best to Implicate for II, regardless of what I does:
Dominant Strategy
Hence…
Prisoner II
Prisoner I




Don’t
Implicate
Implicate
Don’t
Implicate
1, 1
6, 0
Implicate
0, 6
5, 5
It is best for both to implicate regardless of what the other one does
Implicate is a Dominant Strategy for both
(Implicate, Implicate) becomes the Dominant Strategy Equilibrium
Note: If they might collude, then it’s beneficial for both to Not
Implicate, but it’s not an equilibrium as both have incentive to deviate
Dominant Strategy
Equilibrium




Dominant Strategy Equilibrium: is a strategy
combination s*= (s1*, s2*, …, sN*), such that si* is a
dominant strategy for each i, namely, for any possible
alternative strategy profile s= (s1, s2, …, si , …, sN):

if pi is a utility, then pi (s1, s2,…, si*,…, sN) ≥ pi (s1, s2,…, si,…, sN)

if pi is a cost, then pi (s1, s2, …, si*, …, sN) ≤ pi (s1, s2, …, si, …, sN)
Dominant Strategy is the best response to any
strategy of other players
If a game has a DSE, then players will immediately
converge to it
Of course, not all games (only very few in the
practice!) have a dominant strategy equilibrium
A more relaxed solution concept:
Nash Equilibrium [1951]

Nash Equilibrium: is a strategy combination
s*= (s1*, s2*, …, sN*) such that for each i, si* is a
best response to (s1*, …,si-1*,si+1*,…, sN*), namely,
for any possible alternative strategy si of player i

if pi is a utility, then pi (s1*, s2*,…, si*,…, sN*) ≥ pi (s1*, s2*,…, si,…, sN*)

if pi is a cost, then pi (s1*, s2*, …, si*, …, sN*) ≤ pi (s1*, s2*, …, si, …, sN*)
Nash Equilibrium




In a NE no agent can unilaterally deviate from
his strategy given others’ strategies as fixed
Each agent has to take into consideration the
strategies of the other agents
If a game has one or more NE, players need
not to converge to it
Dominant Strategy Equilibrium  Nash
Equilibrium (but the converse is not true)
Nash Equilibrium: The Battle of
the Sexes (coordination game)
Woman
Man


Stadium
Cinema
Stadium
2, 1
0, 0
Cinema
0, 0
1, 2
(Stadium, Stadium) is a NE: Best responses to each other
(Cinema, Cinema) is a NE: Best responses to each other
 but they are not Dominant Strategy Equilibria … are
we really sure they will eventually go out
together????
A crucial issue in game theory:
the existence of a NE


Unfortunately, for pure strategies games
(as those seen so far, in which each
player, for each possible situation of the
game, selects his action
deterministically), it is easy to see that
we cannot have a general result of
existence
In other words, there may be no, one, or
many NE, depending on the game
A conflictual game: Head or Tail
Player II
Player I
Head
Tail
Head
1,-1
-1,1
Tail
-1,1
1,-1
Player I (row) prefers to do what Player II
does, while Player II prefer to do the
opposite of what Player I does!
 In any configuration, one of the players
prefers to change his strategy, and so on and
so forth…thus, there are no NE!

On the existence of a NE



However, when a player can select his strategy
randomly by using a probability distribution over his
set of possible pure strategies (mixed strategy), then
the following general result holds:
Theorem (Nash, 1951): Any game with a finite set of
players and a finite set of strategies has a NE of mixed
strategies (i.e., there exists a profile of probability
distributions for the players such that the expected
payoff of each player cannot be improved by changing
unilaterally the selected probability distribution).
Head or Tail game: if each player sets
p(Head)=p(Tail)=1/2, then the expected payoff of each
player is 0, and this is a NE, since no player can improve
on this by choosing unilaterally a different
randomization!
Fundamental computational issues
concerned with NE
1. Finding (efficiently) a mixed/pure (if any) NE
2. Establishing the quality of a NE, as compared to a
cooperative system, namely a system in which agents
can collaborate (recall the Prisoner’s Dilemma)
3. In a repeated game, establishing whether and in how
many steps the system will eventually converge to a
NE (recall the Battle of the Sexes)
4. Verifying that a strategy profile is a NE,
approximating a NE, NE in resource (e.g., time, space,
message size) constrained settings, breaking a NE by
colluding, etc...
(interested in a Thesis, or even in a PhD?)
Finding a NE in pure strategies
By definition, it is easy to see that an entry (p1,…,pN) of the
payoff matrix is a NE if and only if pi is the maximum ith
element of the row (p1,…,pi-1, {p(s):sSi} ,pi+1,…,pN), for each
i=1,…,N.
 Notice that, with N players, an explicit (i.e., in normal-form)
representation of the payoff functions is exponential in N
 brute-force (i.e., enumerative) search for pure NE is
then exponential in the number of players (even if it is still
polynomial in the input size, but the normal-form
representation needs not be a minimal-space representation
of the input!)
 Alternative cheaper methods are sought: for many games of
interest, a NE can be found in poly-time w.r.t. to the number
of players (e.g., using the powerful potential method)

On the quality of a NE


How inefficient is a NE in comparison to an idealized
situation in which the players would collaborate selflessly
(in other words, the distributed system become
cooperative), with the common goal of maximizing the
overall social welfare, i.e., a social-choice function C
which depends on the payoff of all the players (e.g., C is
the sum of all the payoffs)?
Example: in the Prisoner’s Dilemma (PD) game, the DSE
(and NE) incurs a total of 10 years in jail for the players.
However, if the prisoners would cooperate by not
implicating reciprocally, then they would stay a total of
only 2 years in jail!
A worst-case perspective:
the Price of Anarchy (PoA)

Definition (Koutsopias & Papadimitriou, 1999): Given a
game G and a social-choice function C, let S be the set of
all NE. If the payoff represents a cost (resp., a utility)
for a player, let OPT be the outcome of G minimizing
(resp., maximizing) C. Then, the Price of Anarchy (PoA) of
G w.r.t. C is
C ( s) 
C ( s) 
 resp., inf

PoAG(C) = sup
sS C (OPT )
sS C (OPT )


Example: in the PD game, PoAPD(C)=10/2=5
A case study for the existence and quality
of a NE: selfish routing on Internet




Internet components are made up of
heterogeneous nodes and links, and the network
architecture is open-based and dynamic
Internet users behave selfishly: they generate
traffic, and their only goal is to download/upload
data as fast as possible!
But the more a link is used, the more is slower, and
there is no central authority “optimizing” the data
flow…
So, why does Internet eventually work is such a
jungle???
The Internet routing game
Internet can be modelled by using game theory: it is
a (congestion) game in which
players
strategies
users
paths over which users
can route their traffic
Non-atomic Selfish Routing:
• There is a large number of (selfish) users;
• All the traffic of a user is routed over a single
path simultaneously;
• Every user controls an infinitesimal fraction of
the traffic.
Mathematical model
(multicommodity flow network)
• A directed graph G = (V,E) and a set of N players
• A set of commodities, i.e., source–sink pairs (si,ti), for i=1,..,k
(each of the N≥k players is associated with a commodity)
• Let Ni be the amount of players associated with (si,ti), for
each i=1,..,k; then, the rate of traffic between si and ti is
ri=Ni/N, with 0≤ ri ≤1 and i=1,…,k ri = 1
• A set Πi of paths in G between si and ti for each i=1,..,k, and
the corresponding set of all paths Π=Ui=1,…,k Πi
• Strategy for a player: a path joining its commodity
• Strategy profile: a flow vector f specifying the rate of
traffic fP routed on each path PΠ (notice that 0≤ fP ≤1, and
that for every i=1,..,k we have PΠi fP =ri)
Mathematical model (2)
• For each eE, the amount of flow absorbed by e w.r.t. f is
fe=P Π : eP fP
• For each edge e, a real-value latency function le(x) of its
absorbed flow x (this is a monotonically non-decreasing
function which expresses how e gets congested when a
fraction 0≤x≤1 of the total flow f uses e)
• Cost of a player: the latency of its used path P Π:
c(P)=eP le(fe)
• Cost (or average latency) of a flow f (social-choice
function): C(f)=PΠ fP·c(P)=PΠ fP·eP le(fe)=eE fe·le(fe)
Observation: Notice that the game is not given in normal
form!
Flows and NE
Definition: A flow f* is a Nash flow if no
player can improve its cost (i.e., the cost of its
used path) by changing unilaterally its path.
QUESTION: Given an instance
(G,r=(r1,…,rk),l=(le1,…, lem)) of the non-atomic
selfish routing game, does it admit one or more
Nash flows? And in the positive case, what is
the PoA of the game?
Example: Pigou’s game [1920]
Total amount of flow: 1
Latency depends on
the congestion (x is
the fraction of flow
using the edge)
le1(x)=x
s
t
le2(x)=1
Latency is
fixed
 What is the (only) NE of this game? Trivial: all the
fraction of flow tends to travel on the upper edge  the
cost of the flow is C(f) = 1·le1(1) +0·le2(0) = 1·1 +0·1 = 1
 What is the PoA of this NE? The optimal solution is the
minimum of C(x)=x·x +(1-x)·1  C(x)=x2-x+1  C’(x)=2x-1 
OPT for C’(x)=0, i.e., x=1/2C(OPT)=1/2·1/2+(1-1/2)·1=0.75
 PoA(C) = 1/0.75 = 4/3
Existence of a Nash flow


Theorem (Beckmann et al., 1956): If for each edge e
the function x·le(x) is convex (i.e., its graphic lies below the
line segment joining any two points of the graphic) and
continuously differentiable (i.e., its derivative exists at
each point in its domain and is continuous), then the Nash
flow of (G,r,l) exists and is unique, and is equal to the
optimal min-cost flow of the following instance:
x
(G,r, λ(x)=[∫0 l(t)dt]/x).
Remark: The optimal min-cost flow can be computed in
polynomial time through convex programming methods.
Flows and Price of Anarchy

Theorem 1: In a network with linear latency functions,

Theorem 2: In a network with degree-p polynomials
the cost of a Nash flow is at most 4/3 times that of the
min-cost flow  every instance of the non-atomic selfish
routing satisfying this constraint has PoA ≤ 4/3.
latency functions, the cost of a Nash flow is O(p/log p)
times that of the min-cost flow.
(Roughgarden & Tardos, JACM’02)
A bad example for non-linear latencies
Assume p>>1
xp
s
1
1
0
1-
t
 close to 0
A Nash flow (of cost 1) is arbitrarily more
expensive than the optimal flow (of cost
close to 0)
Improving the PoA:
the Braess’s paradox
Does it help adding edges to improve the PoA?
NO! Let’s have a look at the Braess Paradox
(1968)
v
x
Cost of each path= x+1=1/2+1 = 1.5
1
1/2
s
t
1
1/2
w
x
Cost of the flow= 2·(1.5·1/2)=1.5
(notice this a NE and it is also an
optimal flow)
The Braess’s paradox (2)
To reduce the cost of the flow, we try to add a nolatency road between v and w. Intuitively, this should
not worse things!
x
v
1
0
s
1
w
t
x
The Braess’s paradox (3)
However, each user is tempted to change its route now,
since the path s→v→w→t has less cost (indeed, x≤1)
If only a single user changes its
route, then its cost decreases from
1.5 to approximately 1, i.e.:
v
x
1
c(s→v→w→t) = x+0+x ≈ 0.5 + 0.5 = 1
0
s
1
t
x
w
But the problem is that all the
users will decide to change!
The Braess’s paradox (4)



So, the cost of the flow f that now entirely uses the
path s→v→w→t is:
C(f) = 1·1+1·0+1·1=2>1.5
Even worse, this is a NE (the cost of the path
s→v→w→t is 2, and the cost of the two paths not
using (v,w) is also 2)!
The optimal min-cost flow is equal to that we had
before adding the new road and so, the PoA is
PoA 
2
4

1.5
3
Notice it is 4/3, as in the
Pigou’s example, and it is equal
to the upper bound we gave
for linear latency functions
Convergence towards a NE
(in pure strategies games)



Ok, we know that selfish routing is not so bad
at its NE, but are we really sure this point of
equilibrium will be eventually reached?
Convergence Time: number of moves made by
the players to reach a NE from an initial
arbitrary state
Question: Is the convergence time
(polynomially) bounded in the number of
players?
Convergence towards the Nash flow
 Positive result: If players obey to a best response
dynamics (i.e., each player at each step greedily selects a
strategy which maximizes its personal utility) then the
non-atomic selfish routing game will converge to a NE.
Moreover, for many instances (i.e., for prominent graph
topologies and/or commodity specifications), the
convergence time is polynomial.
 Negative result: However, there exist instances of the
non-atomic selfish routing game for which the
convergence time is exponential (under some mild
assumptions).