Transcript slides

A Theory of Mutations with
Applications to Vacuity,
Coverage, and Fault Tolerance
Orna Kupferman1
Wenchao Li2
Sanjit A. Seshia2
1 Hebrew
University
2 UC Berkeley
FMCAD 2008
1
FMCAD 2008
2
This system is correct even
under faults (e.g. flips in
latches)
Why? Convince me.
It satisfies its specification
under these faults.
Doesn’t this mean the
specification coverage is low?
Adam
So is my specification not
good enough or is my system
fault-tolerant?
Bob
Need fault-tolerance!
But also need to certify it!
FMCAD 2008
3
Problem

Current mutation-based metrics are
inadequate to reason about specification
coverage for fault-tolerant circuits in
model checking.
FMCAD 2008
4
Preliminaries

Coverage



Introduce ∆ to an
implementation I
and check I’ ² S.
Vacuity

Introduce ∆ to a
specification S and
check I ² S’.
Fault Tolerance

I with fault f still
satisfies S.
All three involve
introducing mutations in
the verification process!
FMCAD 2008
5
Contributions
A theory of mutations:

formally ties together coverage
and vacuity in model checking;

enables reasoning coverage for
fault-tolerant circuits.
FMCAD 2008
6
Agenda

Related Work



A Theory of Mutations




Coverage
Vacuity
Coverage and Vacuity are dual
Aggressiveness amongst mutations
Applications
Conclusion
FMCAD 2008
7
Coverage


Is my specification complete?
Coverage metrics for model checking
[HKHZ 99; KGG 99; CKV 01,03]
path
state
FSM Coverage
FMCAD 2008
8
Coverage



Functional Coverage in BMC [GKD 07]
Detect “forgotten cases” [Claessen 07]
Coverage for fault-tolerant systems
[FPFRT 03, DBBDCMF 05]
 Single stuck-at fault model
FMCAD 2008
9
Vacuity


Is my specification satisfied trivially?
Vacuity detection [KV 99, 03; BBER 01;
AFFGP 03; CG 04; BFGKM 05; BK 08]
G (req → F grant)
G (req → false)
Replace a sub-formulae in
the most challenging way.
Trivially true in a
system where req
is never sent.
FMCAD 2008
10
Agenda

Related Work



A Theory of Mutations




Coverage
Vacuity
Coverage and Vacuity are dual
Aggressiveness amongst mutations
Applications
Conclusion
FMCAD 2008
11
Examples of Mutations


Can mutate inputs, outputs, or latches
1001
Stuck-at
1000

1000

Restricting a signal to a value
1000

Modifies
behaviors

1001
Removes
behaviors
Freeing (abstracting) a signal
old
1000
1001
new
100X
100X
FMCAD 2008
Adds
behaviors
12
A Theory of Mutations

Properties:




Invertability: (Cμ)ν = C
Monotonicity: I ² S → Iμ ² Sμ
Duality
Interesting Mutations:



Conditional stuck-at
Conditional add/remove transitions
Permuting events
FMCAD 2008
13
Duality
Iμ ² S ↔ I ² Sν
low coverage
vacuity
,where ν and μ are dual mutations.
FMCAD 2008
14
z
Circuit with input = {z}, control signals = {x,
y}, output = {x}, described by the state
representation on the right.
xy
x
S simulates I’ and S’ simulates I
0,1
I
0,1
S
01
0
add
behavior
remove
behavior
0,1
I’
0,1
01
0
01
0
00
0
0
1
0
1
11
1
10
1
0,1
11
1
0,1
S’
0
01
0
FMCAD 2008
1
00
0
15
Aggressiveness


Mutation  is more aggressive
than  if applying  makes it
harder for the design to satisfy
its specification.
I ² S → I  ² S
 ≥imp 
or
 ≥spec 
I ² S → I ² S
FMCAD 2008
16
Some Aggressive Orders






Free(x) ≥ k-SEU(x)
Free(x) ≥ Stuck_at_0(x)
Free(x) ≥ Flip(x)
Delay_k+1 ≥ Delay_k
k-SEU(x) ≥ m-SEU(x) ≥ for k ≥ m
More interesting ones can be found in the
paper.
FMCAD 2008
17
Coverage for Fault-tolerance

For a fault-tolerant system I and a
set of mutations {j} such that


Ij ² S for all 1≤j≤k.
The fault-tolerant system loosely
satisfies S if there is a mutation 
such that

j ≤imp  for all 1≤j≤k;

I ² S.
FMCAD 2008
18
Agenda

Related Work



A Theory of Mutations




Coverage
Vacuity
Coverage and Vacuity are dual
Aggressiveness amongst mutations
Applications
Conclusion
FMCAD 2008
19
Applications



Useful vacuity information can be
obtained for free from coverage
checks.
Analyze coverage for fault-tolerant
systems.
Improving specifications


Catch bugs
Strengthen environmental assumptions
FMCAD 2008
20
Vacuity from Coverage




S: G (sp[2..0] = 3’b110 → X (sp[2..0] =
3’b111)
In our experiment, applying the “Flip(x)”
mutation to sp[0] still satisfies S.
S’: G (sp[2..0] = 3’b110 → X (sp[2..0] =
3’b110)
S & S’ → G ¬(sp[2..0] = 3’b110)
FMCAD 2008
21
Certifying Fault-Tolerance
1-SEU
System
behaviors
System
behaviors
System
behaviors
2-SEU
Original
low-coverage spec.
FMCAD 2008
High-coverage spec.
certifies system’s
target resilience
22
Experiments
VIS benchmarks, results obtained with
Cadence SMV model checker
FMCAD 2008
23
Improving Specifications

Chip Multiprocessor Router [Peh 01]
Simplied model
S: G (ξ → X ¬(grant = 2’b11)
S’: G (ξ → X (grant = 2’b10)

However, the process still requires some
user assistance.
FMCAD 2008
24
Conclusion

A theory of mutations that




Unifies coverage and vacuity
Can be used to certify the correctness
of fault-tolerant circuits
A new technique to tighten
specifications
The ideas here can be applied to
other verification techniques.
FMCAD 2008
25
Q&A
Thank you!
FMCAD 2008
26
References
FMCAD 2008
27