Transcript Conditional Probabilities over Probabilistic and

Conditional Probabilities over
Probabilistic and Nondeterministic
Systems
M. E. Andrés
and
P. van Rossum
Radboud Universiteit Nijmegen, The Netherlands.
Overview


Motivation
Background





Markov Decision Processes and Schedulers
Conditional Probabilities
pCTL
Our Logic (cpCTL)
Model Checking issues


Fully probabilistic case
Probabilistic and Nondeterministic case
 Comparison (pCTL vs cpCTL)
 cpCTL Complications



Model Checker
Counterexamples
Future work
TACAS - April 1st
Budapest, Hungary
2
Miguel E. Andres
Radboud University
Overview


Motivation
Background





Markov Decision Processes and Schedulers
Conditional Probabilities
pCTL
Our Logic (cpCTL)
Model Checking issues


Fully probabilistic case
Probabilistic and Nondeterministic case
 Comparison (pCTL vs cpCTL)
 cpCTL Complications



Model Checker
Counterexamples
Future work
TACAS - April 1st
Budapest, Hungary
3
Miguel E. Andres
Radboud University
Motivation

Model Checking
j= '
Model

Temporal Logics
LTL – CTL
(+ prob)
pCTL
(+ nondet)
pCTL
NEW (+ cond prob) cpCTL
TACAS - April 1st
Budapest, Hungary
§DeadL
P[§DeadL] · 0:1
P+ [§DeadL] · 0:1
P+ [§DeadLj¤SingU] · 0:1
4
Miguel E. Andres
Radboud University
Motivation

Conditional Probabilities
Risk assessment
Anonymity
Strong Anonymity
Probable innocence
P[dyke breaks| it rains heavily]
Diagnosability
P[A failed|error message E]

What we do
 Define
cpCTL
Deterministic Case
 Model Checker for cpCTL
Nondeterministic Case
 Present a Notion of Counterexamples
TACAS - April 1st
Budapest, Hungary
5
Miguel E. Andres
Radboud University
Overview


Motivation
Background





Markov Decision Processes and Schedulers
pCTL
Conditional Probabilities
Our Logic (cpCTL)
Model Checking issues


Fully probabilistic case
Probabilistic and Nondeterministic case
 Comparison (pCTL vs cpCTL)
 cpCTL Complications



Model Checker
Counterexamples
Future work
TACAS - April 1st
Budapest, Hungary
6
Miguel E. Andres
Radboud University
Background – MDPs
²
 The Model (MDP) ²
MDP =(S,s0 ; L; ¿ ), where:
²
²
 Example
S is the ¯nite state space of the system
s0 2 S is the initial state
L: S ! }(P ) is a labeling function
¿ : S ! }(Distr(S))
Probabilistic and Nondeterministic
Finite Paths
s0 s2
s0 s2 s3
..
.
TACAS - April 1st
Budapest, Hungary
7
Paths
s0 s2 (s3 )!
s0 (s1 )!
..
.
Miguel E. Andres
Radboud University
Background – Schedulers

Schedulers
: Finite Path ! Distr(S)
S2 ! ¼2
²P[s s s ] = 1
²P[s0 s2 s5 ] = 08
0 2 6
S2 ! ¼3
²P[s s s ] = 0
²P[s0 s2 s5 ] = 1
0 2 6
40
1
S2 !
4 ¼
2
3
S !
4 ¼
2
3
Schedulers resolve the Nondeterminism!
TACAS - April 1st
Budapest, Hungary
8
Miguel E. Andres
Radboud University
Background – pCTL
./2 f<; ·; >; ¸g

Syntaxis
© := P j © ^ © j :© j 8ª j 9ª j P
ª := ©U © j §© j ¤©

Semantic
s j= var
,
,
s j= Á ^ Ã
,
s j= :Á
,
s j= 8Á
,
s j= 9Á
s j= P· [Á] ,
a
¾
¾
¾
j= ÁU Ã
j= §Á
j= ¤Á
TACAS - April 1st
Budapest, Hungary
./a
[ª]
a 2 [0; 1]
Path
var 2 L(S)
s j= Á and s j= Ã
s 6j= Á
¾ j= Á for all f. paths ¾ starting from s
¾ j= Á for any f. path ¾ starting from s
max´ P [Á] , P+ [Á] · a
s;´
s
, Á holds until at some point à holds
, ¾ j= trueU Á
, ¾ j= :§:Á
9
State
Miguel E. Andres
Radboud University
Background – computing satisfaction

Example
3
1
+
= 0; 775
4 40
3 1 1¡
1
+ (
®) + ® = 0; 875
4 4 2
4
6j=
TACAS - April 1st
Budapest, Hungary
10
Miguel E. Andres
Radboud University
Background – Conditional Probabilities

Standard Conditional Probabilities
²
²
²
( ; F; P) is a probability space
A; B 2 F are two events
P(B) > 0
²
²
²
 Conditional Probabilities over MDPs
( s ; Bs ; P´ ) is the probability space
P´ (¢1 \ ¢2 )
j
P´ (¢1 ¢2 ) =
¢1 ; ¢2 2 Bs are two sets of paths
P´ (¢2 )
P´ (¢2 ) > 0
\ B)
P(A
P(A j B) =
P(B)
 Max and Min Conditional Probabilities
P+ (¢1 j ¢2 ) = sup P´ (¢1 j ¢2 )
P¡ (¢1 j ¢2 ) = inf
´ 2Sch>0
´ 2Sch>0
¢2
¢2
TACAS - April 1st
Budapest, Hungary
P´ (¢1 j ¢2 )
11
Miguel E. Andres
Radboud University
Overview


Motivation
Background





Markov Decision Processes and Schedulers
pCTL
Conditional Probabilities
Our Logic (cpCTL)
Model Checking issues


Fully probabilistic case
Probabilistic and Nondeterministic case
 Comparison (pCTL vs cpCTL)
 cpCTL Complications



Model Checker
Counterexamples
Future work
TACAS - April 1st
Budapest, Hungary
12
Miguel E. Andres
Radboud University
Our Logic – cpCTL
pCTL
cpCTL
P
j
© :=
© ^ © j :© j 8ª j 9ª j P

./a
ª := ©U © j §© j ¤©

[ª]
Interpretation
\ B]
P[A
P[AjB] =
P[B]
s j= P· [ÁjÃ]
a
TACAS - April 1st
Budapest, Hungary
P+ [ÁjÃ]
s
^^Ã]
PP [Á
[Á
Ã]··
s;´
s
max
aa
PP [Ã]
[Ã]
´ 2Sch
>0
13
s;´
s
Miguel E. Andres
Radboud University
cpCTL - Example
S0 j= P·
²P
0;99
[§B j¤P² ]
s0 ;´¼
P
2
[§ B j¤ P ] =
s0 ;´¼
P[s0 s1 ]+P[s0 s2 s3 ]
P[s0 s1 ]+P[s0 s2 s3 ]+P[s0 s2 s4 ]
3
max(1 ¡
TACAS - April 1st
Budapest, Hungary
[§ B ^ ¤ P ] ·
s0 ;´ P[s s ]
0;30
99
=
0 1
¤
P s ]+P[s
[ P] s s ]
P[s
31
P
j¤ P ] =
[§Bmax
´
14
=1¡
s00 ;´
1
0 2 6
2® ; 30 )
7 31
· 0; 99
Miguel E. Andres
Radboud University
2®
7
Overview


Motivation
Background





Markov Decision Processes and Schedulers
pCTL
Conditional Probabilities
Our Logic (cpCTL)
Model Checking issues


Fully probabilistic case
Probabilistic and Nondeterministic case
 Comparison (pCTL vs cpCTL)
 cpCTL Complications



Model Checker
Counterexamples
Future work
TACAS - April 1st
Budapest, Hungary
15
Miguel E. Andres
Radboud University
Model Checking Issues

Fully probabilistic case
Can be reduced to a pCTL* problem,
^ using
P [Á Ã]
j
P+ [Á Ã] = max s;´
s
P [Ã]
´
s;´

Probabilistic and Nondeterministic case
pCTL
cpCTL
History Independent
Schedulers
Semi History Independent
Schedulers
Bellman Equations
NO Bellman Equations
+ [Á ^ Ã]
P
Observation
jÃ] 6= s Deterministic Schedulers
Deterministic
Schedulers
P+ [Á
s
P+
[Ã]
s
TACAS - April 1st
Budapest, Hungary
16
Miguel E. Andres
Radboud University
Model Checking Issues – Nondeterministic case

cpCTL case
 Deterministic
Schedulers (Not trivial)
 Semi History Independent Schedulers
 No Bellman equations
Theorem: Deterministic Schedulers0
There exists Deterministic schedulers ´ and ´ such that
P [ÁjÃ] = P+ [ÁjÃ] and P [ÁjÃ] = P¡ [ÁjÃ]
´0
´
Coming…
TACAS - April 1st
Budapest, Hungary
17
Miguel E. Andres
Radboud University
Model Checking Issues – Nondeterministic case

Semi History Independent Schedulers
 Why?
If P+ [§B j§P ] = P
s0
then ´ satis¯es
8
< ¼3
¼
´(¾) =
: 5
¼1
s0 ;´
[ § B j§ P ]
if ¾ = s0
if ¾ = s0 s3
if ¾ = s0 s3 s0
 Definition
Stopping
condition
´ is '-semi History
Independent
if
² ´ takes always the same decision before the system reaches '
² ´ takes always the same decision after the system reaches '
Theorem: sHI
There exists deterministic
and Schedulers
sHI schedulers ´ and ´0 such that
P [ÁjÃ] = P+ [ÁjÃ] and P 0 [ÁjÃ] = P¡ [ÁjÃ]
´
TACAS - April 1st
Budapest, Hungary
´
18
Miguel E. Andres
Radboud University
Model Checking Issues – Nondeterministic case
 Local
1
(
Bellman equation
¡ ®) ¢ P+ [§P ] + ® ¢ P+ [§P ] + 1 ¢ P+ [§P ]
s3
s4
s5
2
2
P + [§ P ] =
s2
¼2
¼3
1 ¢
9 ¢
+
§
P [ P] +
P+ [§P]
s
s7
10
10
6
8
P+ [§P ]
s2
= max
¡ ¢
[§P ] + ® ¢ P+ [§P ] +
< ( 12 ®) P+
s
s
3
:
1
10
Bellman
Equations
TACAS - April 1st
Budapest, Hungary
¢ P+ [§P] +
s6
9
10
4
¼2¿ (s)
¢ P+ [§P ]
s5
Maximum over all outgoing
distributions ¼ of s
¢ P+ [§P]
0 s7
P+ [Á] = max @
s
1
2
X
t2succ(s)
19
1
¼(t) ¢ P+ [Á]A
t
Recursive
Computation
Miguel E. Andres
Radboud University
Model Checking Issues – Nondeterministic case

Why Not Bellman equations?
Bellman equation on cpCTL case…
0
P+ [ÁjÃ] = max @
s0
¼ 2 ¿ (s)
P+ [§B j¤P ] · 0; 99
X
t2 succ(s)
1
¼(t) ¢ P+ [ÁjÃ]A
t
s0
max(1 ¡
If ® ¸
7
62
2® ; 30 )
7 31
then
P+ [§B j¤P ] = P
s0
…but
· 0; 99
s0 ;´¼
3
[§B j¤P ]
P+ [§B j¤P ] = P
s2
TACAS - April 1st
Budapest, Hungary
s2 ;´¼
2
[§B j¤P ] = 1 ¡ 2 ¢ ®
20
Miguel E. Andres
Radboud University
Overview


Motivation
Background





Markov Decision Processes and Schedulers
pCTL
Conditional Probabilities
Our Logic (cpCTL)
Model Checking issues


Fully probabilistic case
Probabilistic and Nondeterministic case
 Comparison (pCTL vs cpCTL)
 cpCTL Complications



Model Checker
Counterexamples
Future work
TACAS - April 1st
Budapest, Hungary
21
Miguel E. Andres
Radboud University
Model Checker - Idea

µ
Ps;´
Idea
j
+
P [Á Ã] = max
s
´
P
µ
¶
^
[Á Ã]
s;´
[Ã]
¶
^
^
P [Áand sHI
Ã] Theorem}
P [Á Ã]
{By deterministic
j
¢
¢
¢
s;´
P+ [Á Ã] = max
;
; s;´k
1
s
P [Ã]
P [Ã]
s;´
s;´
1
k
where f´1 ; ´2 ; : : : ; ´k g is the set of all deterministic and sHI schedulers
©
f (s;
What
^ Ã]; P
Á; Ã)we
= actually
(P
[Ácompute
[Ã]); ¢ ¢ ¢ ; (P
s;´1
P+ [ÁjÃ] = max
s
TACAS - April 1st
Budapest, Hungary
³n a
b
s;´1
s;´k
[Á ^ Ã]; P
s;´k
[Ã])
o
´
j (a; b) 2 f (s; Á; Ã) ^ b 6= 0 [ f0g
22
Miguel E. Andres
Radboud University
ª
Model Checker - Example


Optimizations

Reusing information

Ussing pCTL algorithms after reaching the stopping condition
Example
¡
¢
U
j
U
Case P+ [Á1 Á2 Ã1 Ã2 ]
s
U Ã ]; P+[Ã U Ã ])g
f (s; Á1 U Á2 ; Ã1 U Ã2 ) = f(P+
[Ã
1
2
1
2
s
s
U
U
f
U
g
f (s; Á1 Á2 ; Ã1 Ã2 ) = (P+
[Á Á2 ]; 1)
s ¡1
f (s; Á1 U Á2 ; Ã1 U Ã2 ) = f(0; Ps [Ã1 U Ã2 ])g
f (s; Á1 U Á2 ; Ã1 U Ã2 ) = f(0; 0)g
f (s; Á1 U³Á2 ; Ã1 U Ã2 ) =
´
S
L
¼(t) ¯ f (t; Á U Á ; Ã U Ã )
¼2¿(s)
t2succ(s)
TACAS - April 1st
Budapest, Hungary
1
2
1
23
2
if
if
if
if
s j=
s j=
s j=
s j=
Á2
:Á ^ Ã
:Á2 ^ :2Á ^ :Ã
1
2
2
^
:
^
:
Á
Á
à ^ :Ã
1
2
1
if s j= Á1 ^ :Á2 ^ Ã1 ^ :Ã2
Miguel E. Andres
Radboud University
2
Overview


Motivation
Background





Markov Decision Processes and Schedulers
pCTL
Conditional Probabilities
Our Logic (cpCTL)
Model Checking issues


Fully probabilistic case
Probabilistic and Nondeterministic case
 Comparison (pCTL vs cpCTL)
 cpCTL Complications



Model Checker
Counterexamples
Future work
TACAS - April 1st
Budapest, Hungary
24
Miguel E. Andres
Radboud University
Counterexamples

Counterexamples
j
^Ã] ·
'
=
j
j
,
P
[Á
s = P· [Á Ã]
for all ´ s;´
a
a

P
Model
s;´
[Ã]
Why?
Lemma
P
[Á^Ã]
´
P [Ã]
P (¢1 )
´
1¡P (¢2 )
>a
´
>a
´
where ¢1 µ ¢Á^à , f! 2 j ! j= Á ^ Ãg
and ¢2 µ ¢:Ã , f! 2 j ! j= :Ãg
A Counterexamples
cpCTL
j
counterexample for Pfor
·a [Á Ã] is a pair (¢1 ; ¢2 ) of measurable sets
P (¢1 ) , for some
of paths satisfying ¢1 µ ¢Á^à , ¢2 µ ¢:à , and a < ¡
´
1 P (¢2 )
´
scheduler ´.
TACAS - April 1st
Budapest, Hungary
25
Miguel E. Andres
Radboud University
Overview


Motivation
Backgorund





Markov Decision Processes and Schedulers
pCTL
Conditional Probabilities
Our Logic (cpCTL)
Model Checking issues


Fully probabilistic case
Probabilistic and Nondeterministic case
 Comparison (pCTL vs cpCTL)
 cpCTL Complications



Model Checker
Counterexamples
Future work
TACAS - April 1st
Budapest, Hungary
26
Miguel E. Andres
Radboud University
Future Work


Implement our Algorithms in a probabilistic
model checker.
Investigate features of cpCTL (expressivness –
bisimulation issues).

Improve complexity.

Extend cpCTL to cpCTL*.

More research about counterexamples in cpCTL
and cpCTL*.
TACAS - April 1st
Budapest, Hungary
27
Miguel E. Andres
Radboud University
Thanks for your attention!
TACAS - April 1st
Budapest, Hungary
28
Miguel E. Andres
Radboud University
Why Deterministic Schedulers?
s0
®
s1
Á Ã
1¡®
P [ÁjÃ] =
s0
®P [Á^Ã]+(1¡®)P [Á^Ã]
s1
s2
¡
®P [Ã]+(1 ®)P [Ã]
s1
s2
s2
Á Ã
Lema: Let v1 ; v2 2 [0; 1) and w1 ; w2 2 (0; 1). Then the function
®)v2 is monotonous.
f : R ! R de¯ned by f (®) , ®v1 +(1¡
¡
®w1 +(1 ®)w2
TACAS - April 1st
Budapest, Hungary
29
Miguel E. Andres
Radboud University