Transcript Modular Verification of Concurrent Assembly Code

Modular Verification of Concurrent Assembly Code
with Dynamic Thread Creation and Termination
Xinyu Feng
Yale University
Joint work with Zhong Shao
Motivation

Proof-carrying code (PCC)

In principle: verify any property on any code





Real binaries & no loss of efficiency
Embedded OS, device drivers…
All safety & liveness properties…
Formal, machine-checkable proofs
In reality: only works for sequential code
Can concurrent code ever be supported by
the PCC framework ?
2005-9-16
NJPLS@Stevens
Challenges

Challenges for Proof-carrying concur. code

A general framework for concurrent assembly
code verification


Specification/proof generation


Lack of structures (e.g. cobegin/coend blocks)
Spec inference, proof assistant, theorem prover
Concurrent assembly code verification

No directly applicable logic


2005-9-16
Traditional Hoare-logic: only sequential code
Type Systems: no Concurrent Typed Assembly
Language (TAL)
NJPLS@Stevens
Previous work

Rely-Guarantee (R-G) Method




Shared memory concurrency
Thread modular verification
Only for higher-level code: cobegin/coend
CCAP[Yu&Shao, ICFP’04]



The first PCC framework supporting concurrent
assembly code
R-G method
Only support static threads

2005-9-16
P1 || … || Pn
NJPLS@Stevens
Concurrency Programming

cobegin/coend




S::=…| cobegin P1 || P2 codend | …
Higher-level, well-structured
Only support properly nested concurrent code
fork/join



2005-9-16
S::=…| tid := fork f(a) | join tid | …
More flexible: improperly nested code
OSes/Java/…
NJPLS@Stevens
Our Contributions

A new PCC framework: CMAP


Verification of general properties
Dynamic thread creation/termination



Modular verification
Realistic features


2005-9-16
Generalize the Rely-Guarantee method
Multiple instantiations of thread code
Thread argument passing, thread-local data
NJPLS@Stevens
Outline of This Talk





Background: the Rely-Guarantee
Method
Challenges for Dynamic Thread
Creation/Termination
Our Approach
The CMAP Framework
Conclusion and Future Work
2005-9-16
NJPLS@Stevens
The Rely-Guarantee Method
Thread 1 (A1,G1)
Shared
Memory
Thread 2 (A2,G2)
S1
A1: S2 – S3, S4 – S5,…
G1: S1 – S2, S3 – S4,…
2005-9-16
G1  A2
G2  A1
S2
S3
S4
S5
A2: S1 – S2, S3 – S4,…
G2: S2 – S3, S4 – S5,…
NJPLS@Stevens
The Rely-Guarantee Method


Thread + Thread Environment
Rely and Guarantee


A, G: State  State  Prop
Thread Modularity

Non-Interference (interface compatibility):


Safety of each thread

2005-9-16
i,j. ij  Gi  Aj
Ti: (Ai, Gi)
NJPLS@Stevens
GCD Example [Yu&Shao’04]
Thread1:
Thread2:
while(a<>b){
while(a<>b){
if(a > b)
if(b > a)
a := a-b;
b := b-a;
}
2005-9-16
}
NJPLS@Stevens
Outline of This Talk





Background: the Rely-Guarantee
Method
Challenges for Dynamic Thread
Creation/Termination
Our Approach
The CMAP Framework
Conclusion and Future Work
2005-9-16
NJPLS@Stevens
Concurrency Programming

cobegin/coend




S::=…| cobegin P1 || P2 codend | …
Higher-level, well-structured
Only support properly nested concurrent code
fork/join



2005-9-16
S::=…| tid := fork f(a) | join tid | …
More flexible: improperly nested code
OSes/Java/…
NJPLS@Stevens
Static and Dynamic Threads
f(a)
.
.
.
“Static Threads”
fork f(a1)
fork f(a2)
fork f(an)
…
2005-9-16
NJPLS@Stevens
“Dynamic Threads”
Challenges

First attempt

Check NI between all static threads


Ti: (Ai, Gi)
i,j. ij  Gi  Aj
 Too
2005-9-16
rigid to handle changing env.
NJPLS@Stevens
Challenges: Changing Env. I

T1
T3
T2

A
B
A-B: initialize data d


B-C: collaborate with T3 to
process d


C
D

no other threads will change d
A : d = d’
T3 may change d
Still do not allow other threads
change d
C-D: T3 terminates

No other threads can change d
Use pc to mark stages?
2005-9-16
NJPLS@Stevens
Challenges: Changing Env. I
Global data: int data[100]
main:
T1
A
int i:=0;
B
…
while (i<100){
data[i]:=f(i);
fork child(i);
i++;
}
2005-9-16
T3
NJPLS@Stevens
C
D
T2
Challenges: Changing Env. II

T1
T2
T3
T2 and T3 have no overlap in
their lifetime


non-interference between all
threads?
Only check those that overlap?

2005-9-16
How to specify the overlapping?
NJPLS@Stevens
Challenges: multiple instantiations
f(a)
.
.
.
(Aa, Ga)
fork f(a1)
fork f(a2)
(Aa1, Ga1)
2005-9-16
(Aa2, Ga2)
fork f(an)
(Aan, Gan)
NJPLS@Stevens
GaiAaj
GaAa?
Challenges: Modularity
(A1, G1)
(A2, G2)
T1:
.
.
.
jmp f
T2:
.
.
.
jmp f
use everywhere?
f:
.
.
.
exit
2005-9-16
Certify once,
NJPLS@Stevens
Outline of This Talk





Background: the Rely-Guarantee
Method
Challenges for Dynamic Thread
Creation/Termination
Our Approach
The CMAP Framework
Conclusion and Future Work
2005-9-16
NJPLS@Stevens
Our Approach (1)

Problems for checking NI of static threads




Changing environment
Multiple instantiations
Modularity issues
CMAP: “lazy checking”

At each step, all live (dynamic) threads do not
interfere
2005-9-16
NJPLS@Stevens
Our Approach (2)
t0
(A0, G0)
WF(Q, ):
…
tn
Q
…
(An, Gn)

each ti satisfies (Ai, Gi)
ti , t j  Q.ti  t j  Gi  Aj 
How to track the changing thread queue?
2005-9-16
NJPLS@Stevens
Our Approach (3)

Borrow ideas from typechecking data heaps (as in TAL):
Initial condition: 0 . WF(Q0, 0)
Q

WF

2005-9-16
Q'
WF

'
NJPLS@Stevens
 ::= add | sub | jd f |…
| exit | fork | yield
Our Approach (4)

Thread Termination: exit
t
Q
exit
ti
WF
(A,G)
2005-9-16

Q\{ti}
Q
WF!
(Ai,Gi) \{(Ai, Gi)}
NJPLS@Stevens

Our Approach (5)

Thread Creation: fork f(a)
t
Q
fork
Q
t
WF
(A,G)
WF
?

(1) t' does not interfere with Q
(2) t does not interfere with the new env.
2005-9-16
NJPLS@Stevens
t’
Our Approach (6)

G
t
i Ai
Q
i Gi
 A

fork
t
WF
(A,G)
 (G
G' 
i Ai)A''
2005-9-16
WF?
(A’,G’)
? {A’’,G’’}

G''  
G i Ai
Q{t’}
i AGi
 A''
(i GAi)G'' 
 A'
NJPLS@Stevens
Our Approach (7)

Queue Extension
WF(Q{t}, {(A, G)})
fork f(a)
A  A’’, G’’  G
WF(Q{t',t}, {(A’’, G’’), (AG’’, GA’’)})
2005-9-16
NJPLS@Stevens
Our Approach (8)

Queue Update
WF(Q{t}, {(A, G)})
AA’, G’G; t: (A’, G’)
WF(Q{t}, {(A’, G’)})
2005-9-16
NJPLS@Stevens
Our Approach (9)
(A1, G1)
(A2, G2)
T1:
.
.
.
jmp f
T2:
.
.
.
jmp f
(A, G)
f:
.
.
.
exit
2005-9-16
NJPLS@Stevens
Certify once,
use everywhere?
AiA, GGi
Our Approach (10)

Check static threads Lazy Check




Changing Env.  Changing (A, G)
Multiple instantiation  Not care
Modularity  Certify only once
General Enough


2005-9-16
Language (higher-level/assembly)
Thread Model (preemptive/non-preemptive)
NJPLS@Stevens
Example – Unbounded Thread
Creation
Global data: int data[100]
main:
int i:=0;
void child(x:int){
while (i<100){
data[x] = g(x, data[x])
data[i]:=f(i);
}
fork child(i);
i++;
}
2005-9-16
NJPLS@Stevens
Example – Unbounded Thread
Creation

Specification of Child:



2005-9-16
Ax:
Gx:
Non-interference between children:
NJPLS@Stevens
Example – Unbounded Thread
Creation

How to specify the main thread?
main:
Do we need a G such that:
int i:=0;
while (i<100){
But main cannot satisfy
such a G!
data[i]:=0;
fork(child, i);
i++
}
2005-9-16
NJPLS@Stevens
main:
int i:=0;
while (i<100){
(A, G)
data[i]:=0;
fork(child, i);
i++
(A, G)
(A’, G’)
(A, G)
}
2005-9-16
NJPLS@Stevens
Outline of This Talk





Background: the Rely-Guarantee
Method
Challenges for Dynamic Thread
Creation/Termination
Our Approach
The CMAP Framework
Conclusion and Future Work
2005-9-16
NJPLS@Stevens
The CMAP Framework


The abstract machine
The verification logic




Example programs




Specification language
Inference rules
Soundness proof
Unbounded dynamic thread creation
Readers/Writers problem
Lock-free program
All implemented in Coq!
2005-9-16
NJPLS@Stevens
The CMAP Framework - Machine
R
I1
f1:
(data heap) H
I2
f2:
h1:
I1
h2:
I2
…
(thrd entries) T
I
I
0
1
2
…
R
I
r1 r2 r3 … rn
(register file) R
(state) S::=(H,R)
add
…
fork h
yield
exit
(instr. seq.) I
(program) P::=(C,T,S,Q,I)
2005-9-16
…
(dyn. queue) Q
…
(code heap) C
R
NJPLS@Stevens
The CMAP Framework
The paper on CMAP (Feng&Shao ICFP’05):
http://flint.cs.yale.edu/publications/cmap.html
2005-9-16
NJPLS@Stevens
Conclusion

Problems for unbounded dynamic thread creation




Changing environment (fork/exit)
Multiple instantiation of thread code
No previously known modular verification method
Our approach

INV: active threads in the system do not interfere




Combine the type-based proof technique with R-G method
Unify thread’s assumption/guarantee with env.’s
guarantee/assumption
Thread modularity + code/proof reuse
The CMAP framework and its Coq implementation
2005-9-16
NJPLS@Stevens
Future Work

Certified Thread Libraries



fork, yield, exit
join, lock, monitors
Where is the threads ?
User-level thread + thread lib.
Surface language



2005-9-16
Higher-level specifications
Partially infer A and G
Certifying compilation to CMAP
NJPLS@Stevens
Thank you!
2005-9-16
NJPLS@Stevens