Transcript Exploiting Transition Locality in Automatic Verification

Verifica e Validazione Automatica di
Sistemi Complessi
Enrico Tronci
Dipartimento di Informatica, Università di Roma “La Sapienza”, Via Salaraia 113,
00198 Roma, Italy, [email protected], http://www.dsi.uniroma1.it/~tronci
Workshop on Research and Innovations
NEXT
20 Mat 2005
Automatic Verification: A Money Saver
Testing without automation tends to discover errors towards the end of the design
flow. Error fixing is very expensive at that point and may delay product release.
Methods to discover errors as soon as possible are needed.
Without Automated
40
testing
35
With Automated testing
30
25
Source:
Mercury Interactive,
Siebel Siemens
20
15
10
5
0
1X
Early development
3-6X
10X
15-40X
30-70X
401000X
Number of times more
expensive to fix
Implementation
2
Model Checking Game
Sys
(VHDL, Verilog, C, C++
Java, MathLab, Simulink, …)
BAD
(CTL, CTL*, LTL, …)
Model Checker
(Equivalent to
Exhaustive testing)
FAIL
Counterexample
I.e. sequence of events
(states) leading to an
undesired state.
PASS
I.e. no sequence of
events (states) can
possibly lead to an
undesired state.
3
Mutual Exclusion (Mutex)
n1
S1
t1
S2
n2
S1=n1 & S2=t2
t2
1
c1
T
2
S2=n2 & S1=t1
c2
n1, n2, 1
t1, n2, 1
c1, n2, 1
n1, t2, 1
t1, t2, 1
c1, t2, 1
n1, c2, 1
t1, c2, 1
c1, c2, 1
n1, n2, 2
t1, n2, 2
c1, n2, 2
n1, t2, 2
t1, t2, 2
c1, t2, 2
n1, c2, 2
t1, c2, 2
c1, c2, 2
SPEC
Mutual exclusion:
No starvation S1:
AG (S1 != c1 | S2 != c2)
… true
AG (S1 = t1 --> AF (S1 = c1)) … true
4
Mutex (~ arbitrary initial state)
n1
S1
t1
n2
S2
S1=n1 & S2=t2
t2
1
c1
Mutual exclusion:
No starvation S1:
c2
T
2
S2=n2 & S1=t1
AG (S1 != c1 | S2 != c2)
…
AG (S1 = t1 --> AF (S1 = c1)) …
5
SMV output (mutex)
-- specification AG (S1 != c1 | S2 != c2) is true
-- specification AG (S1 = t1 -> AF S1 = c1) is true
resources used:
user time: 0.02 s,
system time: 0.04 s
BDD nodes allocated: 635
Bytes allocated: 1245184
BDD nodes representing transition relation: 31 + 6
6
Hybrid Systems
Hybrid Systems are systems with discrete as well as
continuous state variables. Typically requirements analysis for
embedded software/hardware leads to study verification of
hybrid systems.
Examples of hybrid systems:
• Industrial Plants
• Automotive systems (cost of software in new cars compares
with that of the mechanics)
• Avionics
• Biological models
•…
7
Gas Turbine System
Disturbances: electric users, param. var, etc
Settings
Fuel Valve
Opening
FG102
Gas Turbine
(Turbogas)
Controller
Vrot, Texh, Pel, Pmc
Vrot: Turbine Rotation speed
Texh: Exhaust smokes Temperature
Pel: Generated Electric Power
Pmc: Compressor Pressure
8
PLAN
• Build discrete time model of ICARO Turbogas Control
System.
• Code system model with Murphi verifier.
This is very similar to simulation code, only more abstract because of
model checking limitations (state explosion).
• Run verification experiments.
9
Experimental Results
MAX_D_U Reachable Rules
States
Fired
(KW/sec)
Diameter
CPU (sec) Result
1000
2,246,328
6,738,984
12904
16988.18
PASS
1750
7,492,389
22,477,167 7423
54012.18
PASS
2500
1,739,719
5,186,047
1533
12548.25
FAIL
5000
36,801
109,015
804
271.77
FAIL
Results on a INTEL Pentium 4, 2GHz Linux PC with 512 MB RAM.
Murphi options: -b, -c, --cache, -m350
10
Fail trace: MAX_D_U = 2500 KW/sec
10 ms time step (100 Hz sampling frequency)
Electric
user
demand (KW)
Rotation
speed
(percentage
of
max = 22500 rpm)
Allowed range for
rotation speed:
40-120
11
Fail trace: MAX_D_U = 5000 Kw/sec
10 ms time step (100 Hz sampling frequency)
Electric
user
demand (KW)
Rotation
speed
(percentage
of
max = 22500 rpm)
Allowed range for
rotation speed:
40-120
12
Probabilistic Model Checking (1)
Sometimes we can associate a probability with each transition. In such cases
reachability analysis becomes the task of computing the stationary distribution of a
Markov Chain. This can be done using a Probabilistic Model Checker (state space
too big for matrices).
0.4
1
0.3
0
0.7
0.2
0.8
0.6
2
13
Finite Horizon Markov Chain Analysis
… of our turbogas
MAX_D_U
(KW/sec)
Visited
States
Rules
Fired
Horizon
CPU time (s)
Probability of
violating spec
2500
3,018,970
8,971,839
1600
68562
7.373292e-05
3500
2,226,036
6,602,763
1400
50263
1.076644e-04
4500
1,834,684
5,439,327
1300
41403
9.957147e-05
5000
83,189
246,285
900
2212
3.984375e-03
14
Obstructions
State Explosion:
That is the HUGE number of reachable states that large systems have.
Integration in the design flow:
People devoted to verification, validation, specification and testing needed … among
other things
15
Open Source Model Checkers
Here are a few examples of open source model checkers.
SMV, NuSMV (Carnegie Mellon University, IRST) [smv,VHDL / CTL]
SPIN (Bell Labs)
[PROMELA (C like)/ LTL]
Murphi (Stanford, “La Sapienza”, L’Aquila)
[Pascal like/assert() style]
VIS (Berkeley, Stanford, Colorado University)
[BLIF, Verilog/CTL, LTL]
PVS (Stanford)
[PVS/PVS]
TVLA (Tel-Aviv)
[TVLA/TVLA]
Java PathFinder (NASA)
[Java Bytecode/LTL]
BLAST (Berkeley)
[C/assert()]
16
Java Verification (BANDERA)
SAnToS Group at Kansas State University
17
Some Commercial Model Checkers
Here are a few examples of commercial model checkers.
Cadence (Verilog, VHDL)
Synopsis (Verilog, VHDL)
Innologic (Verilog)
Telelogic (inside SDL suite)
Esterel
Coverity (C, C++)
18
In House Model Checkers
Here are a few examples of in house model checkers.
FORTE (INTEL)
[Verilog, VHDL/Temporal Logic]
SLAM (Microsoft)
[C/assert()]
BEBOP (Microsoft)
[C/assert()]
Rule Based (IBM)
[Verilog, VHDL/CTL, LTL]
CANVAS (IBM)
[Java/constraints-guarantees]
Verisoft (Bell Labs)
[C/C]
19
Summing Up
Automatic Verification (reachability analysis) is a very useful tool for design and
analysis of complex systems such as: digital hardware, embedded software and
hybrid systems.
Automatic Verification allows us to:
Decrease the probability of leaving undetected bugs in our design, thus increasing
design quality.
Speed up the testing/simulation process, thus decreasing costs and time-to-market.
Early error detection, thus decreasing design costs.
Support exploration of more complex, hopefully more efficient, solutions by supporting
their debugging.
20
Adoption Paths
• Integrating Automatic Verification in Design Flow
(to reach state of the art)
• Custom Model Checker
(for competitive advantage, to go beyond state of the art)
21