Simple Stochastic Games Mean Payoff Games Parity Games Uri Zwick Tel Aviv University Zero sum games –3 –5 –2 Mixed strategies Max-min theorem …
Download ReportTranscript Simple Stochastic Games Mean Payoff Games Parity Games Uri Zwick Tel Aviv University Zero sum games –3 –5 –2 Mixed strategies Max-min theorem …
Simple Stochastic Games Mean Payoff Games Parity Games
Uri Zwick Tel Aviv University
Zero sum games
1 0 1 2 –5 7 –3 2 –2 Mixed strategies Max-min theorem
Stochastic games
[Shapley (1953)] 1 0 1 2 –5 7 –3 2 –2 3 2 –7 –4 –3 – 1 4 – 1 7 Mixed positional (memoryless) optimal strategies
Simple Stochastic games (SSGs)
2 –5 7 2 –4 – 1 4 – 1 7 Every game has only one row or column Pure positional (memoryless) optimal strategies
Simple Stochastic games (SSGs) Graphic representation M m R MAX min
RAND
The players construct an (infinite) path
e
0 ,
e
1 ,… Terminating version Non-terminating version Discounted version Fixed duration games easily solved using dynamic programming
Simple
Stochastic games (SSGs) Graphic representation – example min m Start vertex M M MAX R RAND
Simple Stochastic game (SSGs) Reachability version [Condon (1992)] M MAX m min R
RAND
M M 0-sink No weights All prob. are ½ 1-sink
Objective:
Max / Min the prob. of getting to the 1-sink Technical assumption: Game halts with prob. 1
Simple
Stochastic games (SSGs) Basic properties Every vertex in the game has a value
v
Both players have positional optimal strategies Positional strategy for MAX: choice of an outgoing edge from each MAX vertex Decision version: Is value
v
“Solving” binary SSGs
The values
v i
of the vertices of a game are the unique solution of the following equations: The values are rational numbers requiring only a linear number of bits Corollary: Decision version in NP co-NP
Markov Decision Processes (MDPs) M MAX m min R
RAND
Theorem: [Derman (1970)] Values and optimal strategies of a MDP can be found by solving an LP
NP
co-NP – Another proof
Deciding whether the value of a game is at least (at most)
v
is in NP co-NP To show that value guess an optimal strategy
v
, for MAX Find an optimal counter-strategy by solving the resulting MDP.
for min
Is the problem in P ?
Mean Payoff Games (MPGs) [Ehrenfeucht, Mycielski (1979)] M MAX m min Non-terminating version Discounted version MPGs Reachability SSGs Pseudo polynomial algorithm R
RAND
(PZ’96) (PZ’96)
Mean Payoff Games (MPGs) [Ehrenfeucht, Mycielski (1979)] Value – average of the cycle
Parity Games (PGs)
Priorities 3 EVEN 8 ODD EVEN wins if largest priority seen
infinitely often
in even Equivalent to many interesting problems in automata and verification: Non-emptyness of -tree automata modal -calculus model checking
Parity Games (PGs) Mean Payoff Games (MPGs)
[Stirling (1993)] [Puri (1995)] 3 EVEN 8 ODD Chang priority
k
to payoff (
n
)
k
Move payoff to outgoing edges
Simple
Stochastic games (SSGs) Additional properties An SSG is said to be binary if the outdegree of every non-sink vertex is 2 A switch is a change of a strategy at a single vertex A switch is profitable for MAX if it increases the value of the game (sum of values of all vertices) A strategy is optimal iff no switch is profitable
A
randomized
subexponential algorithm for binary SSGs [Ludwig (1995) ] [Kalai (1992) Matousek-Sharir-Welzl (1992) ] Start with an arbitrary strategy for MAX Choose a random vertex
i
V
MAX Find the optimal strategy ’ for MAX in the game in which the only outgoing edge from i is (
i
, (
i
)) If switching ’ at
i
then ’ is not profitable, is optimal Otherwise, let ( ’)
i
and repeat
A
randomized
subexponential algorithm for binary SSGs [Ludwig (1995) ] [Kalai (1992) Matousek-Sharir-Welzl (1992) ] MAX vertices All correct !
Would never be switched !
There is a hidden order of MAX vertices under which the optimal strategy returned by the first recursive call correctly fixes strategy of MAX at vertices 1,2,…,
i
the
Exponential algorithm for PGs [McNaughton (1993)] [Zielonka (1998)] Vertices of highest priority (even) First Second recursive recursive call call In the worst case, both recursive calls are on games of size
n
1 Vertices from which EVEN can force the game to enter A
Deterministic subexponential alg for PGs Jurdzinski, Paterson, Z (2006) Second recursive call
Idea:
Look for small dominions!
Dominions of size
s
can be found in O(
n s
) time
Dominion
A (small) set from which one of the players can without the play ever leaving this set
Open problems
● Polynomial algorithms?
● Faster subexponential algorithms for parity games? ● Deterministic subexponential algorithms for MPGs and SSGs?
● Faster pseudo-polynomial algorithms for MPGs?