Transcript שקופית 1 - Tel Aviv University

Graphical Models
Michael Kearns
Michael L. Littman
Satinder Signh
Presenter: Shay Cohen
So far we have seen…



Players payoffs and the games are
represented in tabular form
n agents with 2 actions: n matrices of
exponential size: 2 n
Needed: More compact representations and
algorithms for manipulating them
Graphical models (not formal)



n-player game is given by undirected
graph with n vertices and n matrices
Payoff is determined only by the
neighbors
“local games” composing “global game”
Examples



Games with geographical aspects
involved (salespersons)
Topology of computer networks with a
limited set of neighbors
… and so on
Reminder…



n-player two-action game: n matrices of
size 2 n
M i ( x ) specifies the payoff for pure
strategy x


Nash-Equilibrium: M i ( p)  M i ( p[i : p' ])
(for all i and for all p’)
 -Nash-Equilibrium:


M i ( p)    M i ( p[i : p' ])
Graphical Games




Graphical game: (G,M)
G is undirected graph on n vertices
M is a set of n matrices representing
the payoff of player i with its neighbors
Size of M i is 2| NG ( i )| when
NG (i)  { j | (i, j )  G} {i}
Algorithm TreeNash
Works in two passes: the downstream
pass and the upstream pass
 Downstream: passes indicator tables
(with witnesses) from the leafs to the
root
 Upstream: selects witnesses from root
to the leafs
(see the attached appendix)

TreeNash – more details

Downstream: A parent U will send to a
child V a binary-valued table T(v,u) s.t.:
T(v,u)=1  there is NE for
U
U
(G , MV v ) in which U=u (v,u – mixed
strategies)

Upstream: A child V will be V=v s.t. for
all its parents U i: T (v, ui )  1
Downstream in general

W – child, V – current node, U – parents
T ( w, v)  1  u1 ,...,uk :
1. T (v, ui )  1
2. V  v is b.r. toU i  ui ,W  w
(b.r. – best response)
How? - Downstream
T(w,u)=1 u b.r. to w
U
V
T(w,v)=1 v b.r. to w
T(z,w)=1 for some (u,v):
1. T(w,u)=1, T(w,v)=1
W
2. W=w b.r. to U=u,V=v,Z=z
T(z)=1for some w:
Z
(b.r. – best response)
1. T(z,w)=1
2. Z=z b.r. to W=w
How? – Upstream
U
Choose U=u, V=v s.t.
T(w,u)=1 and T(w,u)=1
V
W
Z
Choose Z=z, W=w s.t.
T(z,w)=1
TreeNash



Theorem: TreeNash computes a Nash
equilibrium for the tree game (G,M)
Non-deterministic choices: select all of
them, and all NE will be found
But the tables are continuous… How do
we compute them?
Approximate TreeNash



Tables will be of finite size:   grid
All computations of best responses are
computations of -best responses in
the grid
1
Each table has 2 entries, therefore

running time is
(1/  )
2 k
(k parents)
Approximate TreeNash (2)

Lemma: Let p be a NE for (G,M) and
let q be the nearest (in L1 metric)
mixed strategy on the   grid .
2
Then provided   2 /(k log (k / 2))
k 2
q is a 2 (k log(k )) -NE for (G,M)
Approximate TreeNash (3)

Theorem: For any >0, let
k 2
2
  min( /(2 (k log(k ))),2 /(k log (k / 2)))
Then ApproximateTreeNash computes an
-NE for the tree game (G,M).


Exact TreeNash


Tables will be made of finite unions of
rectangles
Each table T(v,u) will be represented by
a v-list: 0  v1  ...  vm  1
For each interval [vl , vl 1 ] there is a
subset of [0,1] of disjoint intervals:
I l .... I l where T(v,u)=1
1
t
Exact TreeNash (2)


Assume T (v, ui ) share a common v-list
(by merging)
Downstream: How do we find T(w,v)
using them, and keep such
representation of rectangles?
Exact TreeNash (3)

Fix a v-interval and set of intervals
appropriate to the v-interval
for each

parent: v  [vl , vl 1 ]  I , u  I 1j,l  I 2j ,l  ... I kj ,l
1
2
ji {1,...t}


T(w,v)=1 is of the form WxI - why?
What would be the region W for which
some v in the interval is b.r. to u,w?
k
Exact TreeNash (4)


Denote M V (v, u , w) expected payoff of V



(u , w)  M V (0, u , w)  M V (1, u , w)
I1 ,...,I k  [0,1]

Lemma: If


W  {w  [0,1] |  u  I1  ... I k .(u , w)  0}
then W is either empty, a continuous
interval in [0,1] or union of two
intervals.
Exact TreeNash (5)



Can be shown that the leafs can be
represented using at most 3 rectangles
Therefore, the representation can be
kept and is exponential in the number
of vertices
Witnesses can be found easily, because
representation is finite
ExactTreeNash

Theorem: ExactTreeNash computes a
Nash equilibrium for the tree game
(G,M). The algorithm runs in
exponential time in the number of
vertices of G
Polynomial algorithm


Use downstream pass and upstream
pass as well
Pass breakpoints policies (W child of V):
w0  0  w1  ...  wt  1 and{vi }
t
i 1
Interpretation (“b.p. for V”):
wi 1  w  wi  v playsvi
w  wi  v can play any v  [vi , vi 1 ]
How? - Downstream

Denote:



v(u , w)  M V (1, u , w)  M V (0, u , w)
0  v0  ...  vs  1 - ordered set of
breakpoints of V’s parents
Wl - Set of values that W can play that
allow V to play any strategy, given V  vl
W b - Set of values that W can play, and
V’s parents play according to V=b, then
V=b is a best response - b {0,1}
How? - Downstream




Lemma: Wl is either empty, a single
interval or the union of two intervals
Lemma: [0,1]   Wl  W 0  W 1
l
Construct the policy for V by covering
[0,1] with them – will produce at most
set of 2+l breakpoints.
How do we start with the leafs?
How? - Upstream


Add a dummy root with constant payoff
and no influence on the real root
Once we select a value for the child, the
value for the parents are determined
according to the policies
Running time


Sorting and computing new breakpoint
k
policy: O(t logt  2 )
(t – number of breakpoints)
Number of breakpoints is bounded by
2n, therefore total running time:
O(n logn  n 2 )
2
2 k
Summary


First framework gave us:
1. Finding approximation for NE in
graphical games which are trees in
polynomial time
2. Finding NE for trees in exponential
time (ALL of the NEs representation)
Second algorithm: finding NE in
polynomial time