Transcript A View from the Engine Room: Computational Support for Symbolic Model Checking Randal E.

A View from the Engine Room:
Computational Support for
Symbolic Model Checking
Randal E. Bryant
Carnegie Mellon University
http://www.cs.cmu.edu/~bryant
Outline
Boolean Reasoning as Engine for Model Checking

BDDs & SAT
An Evaluation of SAT


Current capabilities & limitations
Making further progress
Beyond SAT

–2–
Enhancing DPLL to do more than find single solution
25MC
The Origins of Symbolic Model
Checking



–3–
1987 notes by Ken
McMillan
Backward traversal
of Petri net state
space
Realized that
reachability could
be performed via
symbolic Boolean
manipulation
25MC
Role of Boolean Manipulation in MC
Contributions of BDDs to Model Checking

Separate problem from implementation
 BDDs provide clean API to model checker

Performed well for many examples
The Emergence of SAT


Initially for bounded model checking [Biere, et al., ’96]
More recently for full model checking
 SAT enumeration [McMillan ’02]
 Interpolation-based abstraction-refinement [McMillan ’03]
Important Point

–4–
Advances in Boolean manipulation drive progress in model
checking
25MC
–5–
in
TI
(2
00
5)
118
Sa
tE
l it
eG
(2
00
4)
147
Si
eg
e
04
)
(2
00
2)
(2
00
3-
er
kM
(2
00
1)
(2
00
0)
1,000
zC
ha
ff
B
ra
sp
zC
ha
ff
G
Run-time (sec.)
Recent Progress in SAT Solving
3600
3,000
2,000
766
81
46
0
25MC
Conventional Wisdom on SAT
BDDs vs. DPLL

DPLL better than BDDs for straight SAT
 Especially problems with large numbers of variables
Best Research Strategy is to Keep Refining DPLL

Certainly has lead to big improvements!
Claim

–6–
This wisdom is overly simplistic
25MC
Comparing Parity Trees



–7–
Compare linear chain of XORs
to randomly trees
Known hard problem for
resolution-based SAT solvers
16 n-input trees for different
values of n
25MC
Parity: Exhaustive Testing
Exhaustive
900
800
700
CPU secs.
600
500
OK
TIME
400
300
200
100
0
0
8
16
24
32
40
48
56
Bits

–8–
Testing  109 cases is no big deal
25MC
Parity: DPLL (ca. 2002 Limmat)
Limmat
900
800
700
CPU secs.
600
500
OK
TIME
400
300
200
100
0
0
8
16
24
32
40
48
56
Bits

–9–
Known difficult problem for DPLL
25MC
Parity: DPLL (MiniSAT)
MiniSAT
900
800
700
CPU secs.
600
500
OK
TIME
400
300
200
100
0
0
8
16
24
32
40
48
56
Bits

– 10 –
Recent SAT solvers have made remarkable progress
25MC
Parity: BDDs
BDD
900
800
700
CPU secs.
600
500
OK
TIME
400
300
200
100
0
0
8
16
24
32
40
48
56
Bits

– 11 –
Trivial problem for BDDs
25MC
Associativity Testing
int addL
(int x, int y, int z)
{
return (x+y)+z;
}
int mulL
(int x, int y, int z)
{
return (x*y)*z;
}


– 12 –
?
=
?
=
int addR(int x, int y)
(int x, int y, int z)
{
return x+(y+z);
}
int mulR(int x, int y)
(int x, int y, int z)
{
return x*(y*z);
}
Typical of arithmetic verification problems
Evaluate for different argument word sizes
25MC
Associativity of Addition
Add Associativity
900
800
700
600
500
MiniSAT
400
Exhaustive
BDD
300
200
100
0
0
8
16
24
32
Word Size

– 13 –

Easy for BDDs
Recent DPLL handle readily
25MC
Associativity of Multiplication
Mult Associativity
900
800
700
600
500
MiniSAT
BDD
400
300
200
100
0
0
2
4
6
8
10
12
Word Size

– 14 –
BDDs better than DPLL
25MC
Associativity of Multiplication
Mult Associativity
900
800
700
600
500
MiniSAT
400
Exhaustive
BDD
300
200
100
0
0
2
4
6
8
10
12
Word Size

– 15 –
Both worse than exhaustive
25MC
Progress in SAT Research
Evolution of DPLL


Incremental advances yielding more than incremental
improvements
Encourages continued incrementing
Downside



Gene pool of SAT solvers diminishing
All use DPLL, nonchronological backtracking, 2-literal
watching …
New approaches must overcome high performance standard
Claim

– 16 –
We need to be looking beyond incremental changes
25MC
Breaking Free
Raise the Bar on Benchmarks


Identify challenge benchmarks
Examples
 Arithmetic problems
 Breaking cryptosystems or secure hashes
 Combinatorial optimization

Parameterize to allow scaling analysis
Acknowledge Value of Niche Solvers

– 17 –
Don’t worry about problems that current solvers handle well
25MC
BDD/DPLL Hybrids
Very Different Approaches


DPLL: Search for one solution from top down
BDDs: Encode all solutions from bottom up
Significant Recent Effort



BDD preprocessing for SAT solver [Jin & Somenzi, ’04]
DPLL on ZDD-represented clause sets [Aloul, et al., ’01]
Satisfy conjunction of BDDs [Damiano & Kukula, ’03, Franco
et al., ’04]
Evaluation


– 18 –

Incomplete
Can help when one approach (BDD / DPLL) much better than
other
But what about problems that neither does well?
25MC
Beyond SAT
Dealing With Quantifiers


DPLL as QBF solver has had limited success
Strength for BDDs
 Especially with deep, alternating quantifier nesting
 E.g., model checking
Unsatisfiability

Impressive progress on generating proofs and unsat cores
 Using scaffolding from DPLL

Many applications
 E.g., refinement steps in model checking

– 19 –
No counterpart with BDDs
25MC
Challenge Problem: Quantifier
Elimination
X
Y
...
...
G = X F
F

Y
...
G
Core Problem For Model Checking


Bit-level: Relational product
Predicate abstraction
 Flanagan & Qadeer, ’02, Lahiri, Bryant, Cook, ’03
Methods

BDDs: quantifier elimination
 Use early quantification

DPLL: SAT enumeration
 Plaisted, ’00, Gupta, et al., ’00, McMillan ’02, Clarke et al., ’03
– 20 –
25MC
Quantifier Elimination Example
Example from Predicate Abstraction




Lahiri, Bryant, Cook, ’03
G = X F
Current state variables X
Next state variables Y
 x1, x2, x3 , x4, x5, x6
[
(x1  x2  x3  x4  x5  x6)
 (x1  x2  x3  x4  x5  x6) ]
 (x2  y2)  (y2  y1)  (x4  x6  y1)
 x3  y4  x4 y3
 x5  y6  x6  y5
– 21 –
Current
State
Transition
Constraints
25MC
Set Enumeration


Run SAT checker over formula
Generate blocking clause for each newly generated element
(x1  x2  x3  x4  x5  x6)
 (x1  x2  x3  x4  x5  x6) ]
 (x2  y2)  (y2  y1)  (x4  x6  y1)
[
 x3  y4  x4 y3
 x5  y6  x6  y5
– 22 –
 (y1  y2  y3  y4  y5  y6)
x1 x2 x3 x4 x5 x6 y1 y2 y3 y4 y5
y6
1
0
1
0
1
0
0
0
0
1
0
1
1
0
1
0
1
0
0
1
0
1
0
1
1
0
1
0
1
0
1
0
0
1
0
1
1
0
0
1
0
1
1
0
1
0
1
0
25MC
Compressing Set Representation
y1 y2 y3 y4 y5
y6
0
1
0
0
1
0
0
1
0
1
0
1
1
0
0
1
0
1
1
0
1
0
1
0



– 23 –
BDD
Rep.
y1 y2 y3 y4 y5
y6
0
*
0
1
0
1
*
0
0
1
0
1
1
0
1
0
1
0
Disjunct set elements to form BDD
Extract prime implicants from BDD
Experience: 10X reduction in number of terms
25MC
SAT Enumeration Observations
Performance


Better than BDDs when |X| >> |Y|
Only have to enumerate for unique assignments to Y
Improvements


Attempt to enlarge solution as enumerate [McMillan ’02]
Build into DPLL search loop
 Lahiri, Nieuwenhuis, Oliveras, ’06
 Handle successful cases similarly to failures

Make solver stop before it assigns values to all variables
 Implemented?
Observation

– 24 –
Enumerative methods seem inelegant
25MC
Conclusions
25MC = 20OBDD

Boolean methods have driven much of the progress in
model checking
 BDDs & SAT
SAT Progress

Impressive, but still room for improvement
Beyond SAT


– 25 –
Quantifiers
Unsatisfiability
25MC
Comments?