Transcript Exploiting Transition Locality in Automatic Verification

Cached Murphi
Enrico Tronci
Dipartimento di Informatica
Università di Roma “La Sapienza”, Via Salaria 113, 00198 Roma
[email protected] http://www.dsi.uniroma1.it/~tronci
Meeting Mefisto, March 31-April 2, 2003, Salerno, Italy
Overview
Murphi is an Explicit Model Checker for Low Level (i.e. close to
the final implementation) Analysis of Protocols and Software-like
systems.
Murphi only checks for Invariants (safety properties).
We will sketch:
•Murphi Input Language
•Cached Murphi Performances
2
History
•Murphi has been realized Alan Hu, David Dill, Ulrich Stern, and many
others from University of Stanford, USA.
•Murphi: http://sprout.stanford.edu/dill/murphi.html
•Cached Murphi has been obtained from Murphi by changing Murphi
engine so as to use a cache based BFS. Cmurphi 4.2 uses a disk based
BFS.
•Cached Murphi is a joint effort of the University of L’Aquila and at the
University of Rome “La Sapienza”.
•Cached Murphi: http://www.dsi.uniroma1.it/~tronci
3
A Simple System
0
1
0
1
0
0
2
3
2
1
1
1
2
5
1
2
1
2
2
0
2
x(t) + d(t)
when x(t) <= 3
x(t) – d(t)
when x(t) > 3
0
4
0
x(t + 1) =
d(t) = 0, 1, 2.
x(0) = 0
4
Murphi Code
x(t + 1) = if x(t) <= 3 then x(t) + d(t) else x(t) – d(t) ; d(t) = 0, 1, 2 ; x(0) = 0;
Spec: x(t) < 5 (FAIL).
Spec: x(t) <= 5 (PASS).
CONST
-- constant declarations
MAX_STATE_VALUE : 5;
TYPE -- type declarations
state_type : 0 .. 10; -- integers from 0 to 10
disturbance_type : 0 .. 2;
VAR -- (global) variable declarations
x : state_type; -- variable of type state_type
-- next state function
function next(x: state_type;
d : disturbance_type): state_type;
begin if (x <= 3) then return (x + d);
else return (x - d); endif
end;
startstate "startstate" -- define initial state
x := 0;
end;
-- nondeterministic disturbances
-- trigger system transitions
ruleset d : disturbance_type do
-- define transition rule
rule "time step" true ==>
begin x := next(x, d); end;
end;
-- define property to be verified
invariant "x less than 5"
(x < MAX_STATE_VALUE);
5
Murphi Error Trace
Startstate startstate fired.
x:0
---------Rule time step, d:1 fired.
x:1
---------Rule time step, d:2 fired.
x:3
---------Rule time step, d:2 fired.
The last state of the trace
(in full) is:
x:5
----------
6
Model Checking as
State Space Exploration
For safety properties (no bad state is reachable) the model checking
problem becomes the reachability problem on the transition graph of the
system to be analyzed.
Given a Finite State System S = (S, I, Next), where:
S : Finite set of states;
I : set of initial states;
Next : function mapping a state to the set of its successors;
Visit all states that S can reach from I.
Murphi defines S = (S, I, Next) using Murphi input language.
7
Explicit State Space Exploration via
BFS
Queue Q;
Hash_Table T;
bfs(init_states, next) {
forall s in init_states enqueue(Q, s); /* load Q with initial states */
forall s in init_states insert(T, s);
/* mark init states as visited */
while (Q is not empty) /* visit */
{ s = dequeue(Q);
forall s’ in next(s)
if (s’ is not in T) { insert(T, s’); enqueue(Q, s’); }
}
}
8
Obstructions
•Queue Q may become very large.
•Hash Table T may become very large.
Counteractions
Use compressed representation for states. That is store state signatures
in T rather than states. Even if a state takes around one hundred of
bytes a state signature takes 5 bytes. Thus, using hashing with open
addressing we can store 200,000,000 states using about 1GB of RAM
(we are not considering the RAM needed for the queue here).
Using secondary memory is thus very appealing. Unfortunately disk
memory is much slower than RAM. Thus suitable visit algorithms are
needed.
9
BFS of some Protocols
Protocol
Bytes
Reach
Rules
Max Q
Diam
n_peterson.m
16 163298 1143086
3775
145
adash.m
144
10466 137708
734
37
adashbug.m
144
3742
39619
646
14
eadash.m
376 133491 1786047
9050
47
ldash.m
144 254986 2647358
14988
64
arbiter.m
8
1103
2365
301
12
cache3.m
12
577
2440
102
16
cache3multi.m
28
13738
65357
1229
29
newcache3.m
52
4357
20201
462
27
sym.cache3.m
28
31433 264758
2877
32
down.m
4
10957
52315
1313
19
kerb.m
80 109282 172111
20523
19
list6.m
24
23410
99874
1095
53
list6too.m
20
1077
11622
64
36
newlist6.m
24
13044
53595
631
53
mcslock1.m
12
23644
94576
928
69
mcslock2.m
12 540219 1620657
15655
111
ns-old.m
24
1121
2578
424
11
ns.m
24
980
2314
382
11
sci.m
56
18193
60455
1175
62
mu -b
mu -b
mu -b
mu -b -c mu -b -c mu -b -c
M
g
T
M
g
T
3204
0,0233
269,13
813
0,0233
273,32
1516
0,075
60,57
55
0,075
62,98
544
0,2
18,19
21 0,18125
17,6
49571 0,06875 4102,33
688 0,06875 4114,71
36930
0,075 4002,97
1307
0,062
3950,5
15
0,4
0,1
7
0,3
0,1
10
0,4
0,15
4
0,4
0,15
435
0,1
34,16
73
0,1
35,11
240
0,1125
8,01
24
0,15
7,93
994
0,1039
36,22
167 0,10625
36,03
92
0,15
1,35
60
0,15
1,22
9046
0,191
291,49
615 0,190625
301,86
645
0,05
13,73
119
0,05
14,9
26
0,2
15,55
6
0,1
16,83
360
0,05
17,23
67
0,05
18,34
373 0,04023
16,17
120 0,04375
16,76
8503
0,0293
234,68
2693 0,02969
237,48
33
0,4
0,69
8
0,4
0,69
29
0,4
0,59
7
0,4
0,62
1071
0,066
27,29
94
0,0687
28,17
10
Under the Hood of Cached Murphi
• Our disk based verification algorithm exploits transition locality to
decrease disk read accesses thus reducing time overhead due to disk
usage.
• Our disk based algorithm has been implementation within the Murphi
verifier
and
is
available
in
http://www.dsi.uniroma1.it/~tronci/cached.murphi.html
• Our experimental results show that even using 1/10 of the RAM
needed to complete verification our disk based algorithm is on
average only 3 times slower than RAM Murphi with enough RAM to
complete the verification task at hand.
• Using just 300M of RAM with our disk based Murphi we were able
to complete verification of a protocol with about 109 reachable states.
This would require more than 5 gigabytes of RAM using RAM
Murphi.
11
Locality
K-transition iff
Transition k-local iff
|level(s’) – level(s)| <= k
level(s’) – level(s) = K
-4
1
-2
0
1
1
0
-1
0 -1
1
1
-1
0
1
1
2
3
4
12
Locality
Our experimental results show that:
For all protocol like systems, for most states, most
transitions (typically more than 75%) are 1-local.
13
Notation
Let d(s, k) be the fraction of transitions from state s that are k-transitions.
Thus d(s, k) is the probability of getting a k-transition when picking at
random a transition from state s.
Consider the experiment of selecting at random a state s and then
returning d(s, k). In this way we get a random variable that we denote
with d(k).
The expected value of d(k) is the average value of d(s, k) on all reachable
states.
s
14
Experimental Evidence of Locality
Protocol
n_peterson.m
adash.m
adashbug.m *
eadash.m
ldash.m
arbiter.m *
cache3.m
cache3multi.m
newcache3.m
sym.cache3.m
down.m *
kerb.m
list6.m
list6too.m
newlist6.m
mcslock1.m
mcslock2.m
ns-old.m
ns.m
sci.m
E{d(-1)}
0
0,038178
0,03766
0,050598
0,010726
0,008489
0,040121
0,038984
0,036034
0,041675
0
0
0,018367
0
1,53E-05
0,012886
0,005465
0,546833
0,585714
0,215646
dev(-1)
0
0,122
0,124403
0,096015
0,071917
0,057543
0,178884
0,102299
0,099187
0,10478
0
0
0,071081
0
0,001751
0,067574
0,045815
0,497802
0,492598
0,238654
E{d(0)}
0
0,005581
0,002289
0,015647
0,076359
0,010737
0,004766
0,002991
0,009121
0,00757
0,077639
0,18417
0,024625
0
0
0
0,000564
0
0
0,010819
dev(0)
0
0,043607
0,029503
0,056255
0,138464
0,053714
0,055844
0,026349
0,053342
0,044577
0,142608
0,385163
0,081523
0
0
0
0,015111
0
0
0,061719
E{d(1)}
0,958174
0,723393
0,793586
0,765236
0,624139
0,92784
0,565482
0,831139
0,736229
0,876884
0,922361
0,441651
0,844018
0,988378
0,999586
0,881379
0,921489
0,101695
0,096939
0,642466
dev(1)
0,087772
0,292406
0,270018
0,178076
0,191624
0,168859
0,381654
0,176004
0,227796
0,17786
0,142608
0,494666
0,189053
0,06214
0,00909
0,162002
0,156949
0,302247
0,295874
0,265885
Sum Avg
0,958174
0,767152
0,833535
0,831481
0,711224
0,947066
0,610369
0,873114
0,781384
0,926129
1
0,625821
0,88701
0,988378
0,9996
0,894265
0,927518
0,648528
0,682653
0,868931
15
Exploiting Locality
We were able to exploit locality to reduce disk read
accesses in the disk based BFS Explicit State Space
Exploration algorithm presented in:
U. Stern, D. Dill, Using Magnetic disk instead of main
memory in the Murphi verifier, CAV 98
Our modified disk based BFS algorithm typically
speeds up verification by a factor of 10
(Formal Methods in Computer Aided Design 2002,
Portland, Oregon, USA, LNCS)
16
Experimental Results with
RAM Murphi
mu –b –c
g = MaxQ/Reach
Protocol
By
Di
Reach States
Rules
Max Q
Min Mem
g
T (sec)
ns
96
12
2,455,257
8,477,970
1,388,415 145,564,125 0.57 1,211
n_peterson 20
241 2,871,372
25,842,348
46,657
15,290,000
0.02 764
newlist6 7
32
91
3,619,556
21,612,905
140,382
22,590,004
0.04 1,641
ldash
144 72
8,939,558
112,808,653
509,751
118,101,934
0.06 12,352
sci
60
94
9,299,127
30,037,227
347,299
67,333,575
0.04 2,852
sci 31151
64
95
75, 081, 011
254,261,319
2,927,550 562,768,255 0.04 35,904
sci 31171
68
143 126,784,943
447,583,731
4,720,612 964,926,331 0.04 99,904
kerb
148 15
9,859,187
4,730,277 738,152,956 0.62 2,830
mcslock1
16
111 12,783,541
76,701,246
392,757
newlist6 8
40
110 81,271,421
563,937,480
2,875,471 521,375,945 0.03 31,114
7,614,392
70,201,817
0.03 3,279
17
Local Disk Murphi vs
RAM Murphi
M
1
0.5
0.1
Prot
ns
n_pet
erson
newli
st6 7
mcslo Sci
Sci
Sci
kerb
ck1
31121 31151 31171
ldash
newli
st6 8
States
1.348
1.178
1.366
1.346
1.260
------
------
------
1.566
------
Rules
1.487
1.178
1.365
1.346
1.279
------
------
------
1.528
------
Time
1.734
2.148
1.703
1.915
1.811
------
------
------
2.037
------
States
1.405
1.124
1.335
1.550
1.189
1.169
1.130
1.282
1.668
1.416
Rules
2.011
1.124
1.334
1.550
1.206
1.195
1.152
1.060
1.626
1.412
Time
2.144
2.056
1.765
2.477
1.798
1.828
1.421
1.234
2.226
2.612
States
1.373
1.199
1.384
1.703
1.183
1.143
1.097
1.279
1.702
1.406
Rules
1.645
1.199
1.382
1.703
1.200
1.167
1.115
1.080
1.658
1.405
Time
1.953
2.783
2.791
5.259
2.888
2.553
1.743
1.438
3.770
4.436
M = <RAM used in LD Murphi>/<Min Mem needed for RAM Murphi>,
Dummy = <Dummy LD Murphi>/<Dummy RAM Murphi>,
Dummy = States, Rules, Time
18
Local Disk Murphi vs
RAM Murphi (continued)
Time Statistics
M
1
0.5
0.1
Min
1.703
1.234
1.438
Avg
1.891
1.954
2.961
Max
2.148
2.612
5.259
M = <RAM used in LD Murphi>/<Min Mem needed for RAM Murphi>
19
Disk Murphi vs RAM Murphi
Mem
1
0.5
0.1
Prot
ns
n_peterson
newlist6 7 mcslock1
Sci 31121 ldash
States
1.000
1.000
1.000
1.000
1.000
0.355
Rules
1.000
1.000
1.000
1.000
1.000
0.245
Time
1.259
2.623
1.331
1.821
1.616
> 50.660
States
1.000
1.000
1.000
1.000
0.361
-----
Rules
1.000
1.000
1.000
1.000
0.647
-----
Time
242.131
2.430
1.357
1.691
> 11.863
----
States
0.747
0.527
0.253
0.137
-----
-----
Rules
0.309
0.507
0.203
0.115
-----
-----
Time
> 77.895 > 90.704
>42.817
>11.605
----
----
Mem = <RAM used in LD Murphi>/<Min Mem needed for RAM Murphi>,
Dummy = <Dummy LD Murphi>/<Dummy RAM Murphi>,
Dummy = States, Rules, Time
20
Comparing LDMurphi with DMurphi
Mem
Prot
ns
n_peterson
newlist6 7 mcslock1
Sci 31121 ldash
1
Time
0.726
1.221
0.781
0.950
0.892
> 24
0.5
Time
112.934
1.182
0.768
0.683
>6
> 24
0.1
Time
> 39
> 32
> 15
>2
>6
> 24
Min
Avg
Max
Time Statistics
Mem
1
Time
0.726
> 4.762
> 24
0.5
Time
0.683
> 24.261
112.934
0.1
Time
>2
> 19.667
> 39
Mem =
<RAM used in disk Murphi (LD or D)>/<Min Mem needed for RAM Murphi>
Time = <Time DMurphi>/<Time LDMurphi>
21
A Large Protocol
Prot
Param
Bytes
Reach
Rules
Max states Q
mcslock2
N=4
16
945,950,806
3,783,803,224 30,091,508
Diam
Time
(sec)
MaxMem
(MB)
(LDMurphi)
HT mem (KB) Q Mem (KB)
TotMem (KB)
(RAM Murphi) (RAM Murphi) (RAM Murphi)
153
406,275
300
4,729,754
481,465
5,211,219
22
Hybrid Systems with CMurphi
We added to Cmurphi finite precision real numbers. This allows us to
easily handle discrete time Hybrid Systems.
E.g. using Cmurphi we were able to automatically verify a Turbogas
Control System, a system out of reach for Hytech as well as SMV
(Hybrid Systems Computation and Control (HSCC) 2003, Praga, Czech
Republic, LNCS).
23
Conclusions
• Cached Murphi can be effectively used to analyze protocols, software
like systems and hybrid systems.
• Cached Murphi exploits statistical properties of transition graphs to
save memory and to speed up disk based verification.
• Looking for new statistical properties of transition graphs is a natural
next step for our research.
24