Transcript Game Theory and Networking Tutorial

Network Routing Problem
S2
B
A
S1
R1
D
R2
S3
C
E
R3
 Input:
 network topology, link metrics, and traffic matrix
 Output:
 set of routes to carry traffic
Network Routing: Classical
Approach
 Routing as optimization problem
 e.g., minimum total delay in network
 focus on global network performance (social optimal)
 performance of individual user not important
 Centralized or distributed algorithms
 e.g., link state or distance vector
 Passive users

users are oblivious to routing decisions
Network Routing: Game-Theoretic
Approach
 Routing as game between users
 users determine route
 decision based solely on individual performance
(selfish routing)
 strongly dependent on other users’ decisions
 Non-cooperative game (non-zero sum)
 users compete for network resources
 Equilibrium point of operation
 Nash equilibrium point (NEP)
Selfish Network Routing
 Advantages
no need of centralized control or global agreement
on routing algorithm
 individual user’s performance considered
 greater adaptability

• changes in user demands or changes in network conditions
 Disadvantages
multiple equilibria (eq. selection problem)
 convergence to equilibrium
 no network-wide optimality at equilibrium

• cost of “selfish routing”

user’s must have detailed knowledge of network
Applications of Game Theory to
Network Routing
 Competitive routing in multiuser communication networks
A. Orda, R. Rom and N. Shimkin
IEEE/ACM Transactions on Networking, 1 (5) 1993
 How bad is selfish routing?
T. Roughgarden and E. Tardos
Journal of the ACM, 49 (2) 2002
 Selfish routing with atomic players
T. Roughgarden
ACM/SIAM Symp. on Discrete Algorithms (SODA) 2005
Simple Model: Network of
Parallel Links
R1
S1
S2
S3
A
B
R2
S4
R3
R4
 set of users share a set of parallel links
 each user has fixed demand (data rate)
 users decide how to split demand across links
 minimize individual cost
 link has a load dependent cost (e.g., delay)
Network of Parallel Links
L  {1,, L}
 set of users: I  {1,, I}
 set of parallel links:
 each user has a fixed demand (data rate):
r
 user splits its demand across links
i
flow of user i on link l: f l
 flow configuration of user i:

 system flow configuration:
f  ( f ,, f )
i
i
1
i
L
f  ( f ,, f )
1
 feasible configurations
 satisfy nonnegative and demand constraints
I
i
User’s Cost Function
 Cost function of user i: C ( f )
 cost depends on flow configuration of all users
i
 Assumptions on cost function
i
 sum of user-link cost function: C ( f ) 
Cli ( f l )
lL
 can be infinite
i
f
 convex in l
i
 when finite, continuously differentiable in Cl
 at least one user with infinite cost can change its flow
configuration to have finite cost

• aggregate demand must be less than aggregate link capacity
Very mild conditions on cost function
The Game
 Users individually decide their flow configuration
 goal is to minimize its own cost
 Nash Equilibrium Point (NEP)
system flow configuration such that no user can reduce
their cost by changing its flow allocation
~
~1
~I
 f  ( f ,..., f )  F is a NEP if for all i, the following
holds:
~
~
~

C i ( f )  min {C i ( f 1 ,..., f i ,..., f I )}
f i F i
No user can reduce their cost by
rerouting their own flow
The Issues
 Existence of NEP
 is at least one NEP guaranteed to always exist
 Uniqueness of NEP
 under which conditions (if any) do we have a single
NEP
 Convergence to (and stability of) NEP
 play dynamics that lead to a NEP
 System properties at the NEP

e.g., how does users divide allocate their flows
Existence of NEP
 N-person convex game [Rosen65]
 joint strategy set is convex, closed and bounded
 each player’s payoff function is convex in their own
strategy
 existence of NEP proven by Katutani fixed point
theorem
 Can also show using Kuhn-Tucker conditions
 necessary and sufficient for system flow
configuration to be a NEP
Uniqueness of NEP
 Uniqueness of NEP only under a type of cost
functions (type-A functions)
cost function has two parameters: user’s i and
aggregate of all others
 monotonically increasing in each parameter
 still very general (e.g., M/M/1 delay function)

 Proof by contradiction using Kuhn-Tucker
conditions
System has a single operating point
System Properties at NEP
 Assume all users share same type-A cost
function

but users can have different demands
 Monotonicity of link usage
 user with higher demand uses more of each and
every link used
 a user with higher demand uses more links
 Higher capacity links receive more users
 does not hold in general, only under yet another
type of cost function (which still captures M/M/1)
Intuitive but assuring properties
Dynamical System
 Simple case study
 two-users sharing two parallel links
S1
A
S2
B
R1
R2
 Dynamical model: Elementary Stepwise System
 Users take turns in updating their flow
configuration
• measure load on links, adjust its flow to minimize cost

i
fl (n)
flow of user i on link l at step n
Convergence to NEP
 Let
f denote unique NEP of game
*
 Initialize system with any feasible flow
configuration: f(0)
 Convergence to NEP guaranteed
lim f (n) f
*
n 
 Framework used in proof
in general
 limited
not aplicable
to two link, two user structure
General Topology
S2
B
A
S1
R1
D
R2
S3
C
E
R3
 Users decide how to split their demands over
possible paths

users know network topology (directed graph)
Existence and Uniqueness of NEP
 Existence of NEP
 same
argument as before (N-person convex game)
 No unique NEP for type-A cost functions
 shown by counterexample
 Uniqueness shown only under very strict
conditions for cost function

not very interesting networking scenarios
Analysis of general network in this modeling
framework is much harder
The “Price of Anarchy”
 Equilibria of non-cooperative games usually
inefficient
e.g., prisoner’s dilemma
 Pareto optimal usually not a NEP

 Quantify inefficiency in terms of a global
objective

“price of anarchy” (coordination versus competition)
Price of Anarchy =
of a Game

objective function value at NEP
optimal objective function value
if multiple NEP exists, take sup (or inf) over NEP set
Cost of Selfish Routing
 How does total cost compare?
 flow allocation at a NEP
 optimal flow allocation
 Total cost of flow configuration:
C ( f )   cl ( f l ) f l
lL
where cl (.) is load dependent link cost function
 e.g., link delay

Example (1/4)
r1 = 0.5
r2 = 0.5
cl1 ( x)  x
S1
S2
A
B
R2
cl2 ( x)  1
 flow configuration cost
C ( f )  cl1 ( fl1 ) f l1  cl2 ( f l2 ) fl2  fl1  fl2
2
 optimal flow allocation
f *  ( fl1* , fl2* )  (0.5,0.5)
C( f )  0.75  3 / 4
*
 can be realized with
f 1  (0.25,0.25) , f 2  (0.25,0.25)
R1
Example (2/4)
r1 = 0.5
r2 = 0.5
cl1 ( x)  x
S1
A
S2
B
cl2 ( x)  1
R1
R2
 But this is not NEP…
 Cost of a flow configuration to user i
Ci ( f )  cl1 ( fl1 ) fl1i  cl2 ( fl2 ) fl2i  fl1 fl1i  fl2i
C ( f )  0.5 * 0.25  0.25  0.375
i
*
 By rerouting traffic user 1 (or 2) can reduce its
cost:
f '  (0.3,0.2)
C1 ( f ' )  0.55* 0.3  0.2  0.365
1
lower
cost!
Example (3/4)
r1 = 0.5
r2 = 0.5
cl1 ( x)  x
S1
A
S2
B
cl2 ( x)  1
 NEP given by
R1
R2
f  (0.5,0) , f  (0.5,0)
1

2
link 1 is a dominant strategy (link 2 never used)
 Cost to user i at NEP
higher
cost!
C ( f )  0.5
 Total cost of NEP configuration
i
C( f )  1
higher
cost
Example (4/4)
r1 = 0.5
r2 = 0.5
cl1 ( x)  x
S1
S2
A
B
cl2 ( x)  1
R1
R2
 Optimal cost:
C( f * )  3 / 4
 NEP cost: C ( f )  1
1
4
 Price of Anarchy: C ( f )


*
C( f ) 3 / 4 3
Thm:[Roughgarden/Tardos00] POA of selfish
routing w/affine cost functions is at most 4/3

for any network topology and traffic matrix!
Another example (non-linear cost)…
r1 = 0.5
r2 = 0.5
cl1 ( x)  x
S1
S2
d
A
B
cl2 ( x)  1
R1
R2
 NEP: both users only use link 1
 cost is 1
 Optimal: 1-ε for link 1 and ε for link 2
ε depends on d, but is small for large d
 cost ≈ 0

 Price of anarchy can be arbitrarily large

goes to infinity as d goes to infinity
So how bad is selfish routing?
 It depends...
 cost functions, network topology, traffic matrix,
user demands, etc.
 In reality, not so bad
 achieves close to optimal cost in Internet-like
environments (simulation study)
 Another positive (and nice) result:
Thm:[Roughgarden/Tardos00] selfish routing is no
worst than the optimal routing of twice as much
traffic

for any cost function, network topology and traffic
matrix!
Title
Congestion Control Problem
S2
B
A
S1
R1
D
R2
S3
C
E
R3
 Input:
 network topology, routes, link characteristics,
traffic matrix
 Output:
 set of data rates to be used
Congestion Control: Classical
Approach
 Congestion control as optimization problem
 match user’s demand to network capacity and
achieve some fairness among users
 focus on global network performance (social optimal)
 performance of individual user not important
 Centralized or distributed algorithms
 e.g., TCP, max-min fairness
 Passive users

users are oblivious to congestion decisions
Congestion Control: GameTheoretic Approach
 Congestion control as game between users
 users determine their own data rates
 decision based solely on individual performance
 Non-cooperative game (non-zero sum)

users compete for network resources
 Equilibrium point of operation
 Nash equilibrium point (NEP)
Key Assumption: A higher sending rate do not
necessarily yields better performance for user
Routing Games vs Congestion
Control Games
 Routing games
 users determine
network routes
 multi-path routing
and traffic splitting
is possible
 users’ data rates
are given and must
be routed
 Congestion games
users determine
their data rate
 network routes are
given (single path)

Applications of Game Theory to
Congestion Control
 Making greed work in networks: a game-theoretic
analysis of switch service disciplines
S. Shenker
IEEE/ACM Transactions on Networking, 3 (6) 1995
 An evolutionary game-theoretic approach to congestion
control
D. Menasché, D. Figueiredo, E. de Souza e Silva
Performance Evaluation, 62 (1-4) 2005
Simple Model: Single Bottleneck
Link
S1
S2
S3
S4
R1
R2
R3
R4
 set of users share a bottleneck link
 users decide their data rates
 maximize individual performance
 user’s performance depends on link load
 e.g., quality of service provided by link
Single Bottleneck Link
 Users determine sending rate: ri
 Link modeled as M/M/1 queue
unit capacity
 packet scheduling policy

 Scheduling policy induces average queue
length for each user

ci : avg. queue length of user i
 User’s utility function
U i (ri , ci )
strictly increasing in ri
 strictly decreasing in c i
 convex and derivable everywhere

Scheduling Policy
 Determined by system operator
 Allocation function
P
i
C (r )

scheduling policy P induces an avg. queue length for
each user given all user’s data rate
 FIFO example
FIFO
i
C
(r ) 
ri
1   j rj
 Must satisfy some constraints
 aggregate average queue size same as M/M/1
 Allocation function can be realized by
different service disciplines
Fair Share Allocation
 Allocate service capacity fairly among user’s
demand

user’s requesting less obtain higher priority
 Implemented through a priority queueing
algorithm
Priority Level

User
A
B
C
D
1
r1
-
-
-
2
3
4
r1
r1
r1
r2 - r1
r2 - r1
r2 - r1
-
-
r3 - r2
r3 - r2
r4 - r3
Assume:
r1 < … < rN
fraction of
traffic gets
lower
priority
MAC: Set of Monotonic Allocation
Functions
 Consider a set of possible allocation functions

Ci
0 :
ri
ri
increases,
ci increases

Ci
0 :
rj
rj
increases,
ci does not decrease

Ci
 0 at
rj
o
r  Ci
rj
0

for all r
with rk  rok
 Includes all typical service disciplines
 FIFO, LIFO, PS, fair share allocation
The Problem Investigated
 Relationship between NEP and service
disciplines (MAC functions)
 Which service disciplines yield good NEP?
 Properties of NEP of a given MAC
efficiency
 fairness
 convergence to equilibrium
 user protection

System designer can select service
discipline that yields good equilibrium
Efficiency of NEP
 Efficiency in terms of Pareto optimal

no global objective function of system outcome
 Pareto optimal outcome:
 no other outcome is preferred by all users
Thm:[Shenker95] There is no allocation function
in MAC such that every NEP is Pareto optimal
 Under some additional constraints fair share
is always efficient
constrained users’ utility function
 symmetric rate vector

Uniqueness of NEP
 Allocation functions can induce multiple NEP
 undesirable since users cannot coordinate
Thm:[Shenker95]
 Fair share mechanism always has a unique NEP
 Fair share is the only allocation function that
always yields a unique NEP
Convergence to Equilibrium
 Dynamics through a generalized hill climbing
algorithm
users eliminate strategies that always perform
worst
 system converges to a reduced set of strategies

 Different from best-response dynamics
Thm:[Shenker95]
 With the fair share mechanism, all generalized
hill climbing algorithm converges to the NEP

Convergence is also fast (superlinear) and stable
Title
Application to Multimedia Traffic
R1
R2
R3
R4
S1
r1
r2
r3
 Users share common bottleneck link
 User’s choose data rate to be sent by source
 only few data rates available
 Utility given by “perceived” quality
Investigate dynamics and convergence
using evolutionary game theory

Why Evolutionary Game Theory
 Model how users
change their strategy
 Users are not perfect:
stochastic dynamics,
myopic, etc
 Which NEP will be
achieved (if more than
one exists)
 Efficiency of selected
NEP
Evolutionary
Game Theory
Entities of Model and Interactions
yields perceived
quality to
Users
(strategy set)
performance
metric feeds
QoS Model
(E-model or other)
choice of strategies
causes impact on
Link model
(M/M/1/k or other)
Two-layer Markovian Model
users’
actions
link
perf.
layer 1
0, 3
1, 2
2, 1
3, 0
layer 2
QoS of
each user
QoS of
QoS of
QoS of
each user each user each user
States and Users Utility
(i )
l : number of users selecting strategy l in state
n
M
: number of data rates available to users
(i )
1
(i )
l
(i )
m
(i )
M
n ,, n ,, n , n
U (l, si )
: state
si
: utility function of strategy l in state
 No constraints on users’ utility function
 should be defined for every state
si
si
Transition Matrix
(i )
1
(i )
l
(i )
m
(i )
M
n ,, n ,, n , n

n1( j ) ,, nl( j ) 1,, nm( j ) 1, nM( j )
nl(i ) (U (m, s j )  U (l , si )) if U (l , s j )  U (l , si )
   (i )
otherwise
 nl 
 Transitions determined by QoS in each state
 rate of change proportional to gain
 transitions can reduce QoS (users make errors)
 Markov chain is ergodic
Main Problem Investigated
Assume:
t  ,   0
 System in steady state
 Users make no mistakes
States that have
non-negligible
steady state
probability
States that
correspond to NEP
What is the relationship?
Proposition 1
If a state has nonnegligible steady
state probability
 under the condition
that this state is
contained in a quasiabsorbing set
This state is
also a NEP
Proposition 2
If a state is
a NEP
 proof via simple
counter-example
This state also has
non-negligible
steady state
probability
Summary of Results
 States with non-negligible SS probability
are NEP
 correspond
to “stable” states
 Some NEP are not stable
 system
dynamics cannot converge on them
 Still possible to have multiple stable NEP
 not
clear where system will converge
 state with highest probability?
Title
Title