Transcript Extended Static Checking for Java

Extended Static Checking for Java
or
Light-weight formal methods:
from objects to components
K. Rustan M. Leino
Microsoft Research, Redmond, WA
Joint work with Cormac Flanagan, Mark
Lillibridge, Greg Nelson, James B. Saxe,
Raymie Stata
Compaq SRC
6 Nov 2002
FMCO 2002, Leiden, The Netherlands
Objects and components

Some technical problems:




Meanings of specifications
Ingredients of formalisms
How to build tools
Other problems:


Cost-effectiveness
Getting programmers to use the above
Formal methods
coverage
program
verification
extended
static
checking
decidability ceiling
light-weight
formal methods
type
checking
effort
Note: Illustration not to scale
User’s view
Annotated
Java program
ESC/Java
Warning
messages
public class Bag {
private /*@non_null*/ int[] a;
private int n;
//@ invariant 0 <= n && n <= a.length;
public Bag(/*@non_null*/ int[] initialElements) {
n = initialElements.length;
a = new int[n];
System.arraycopy(initialElements, 0, a, 0, n);
}
public void add(int x) {
if (n == a.length) {
int[] b = new int[2*(a.length+1)];
System.arraycopy(a, 0, b, 0, n);
a = b;
}
a[n] = x;
n++;
}
Bag.java:18: Array index possibly
too large
Bag.java:45: Precondition possibly
violated
ESC/Java distinguishing features





Annotation language captures design
decisions
Powered by automatic theorem prover
Not decidable
Not sound or complete
Performs modular checking
Method-modular checking


Check that each method satisfies its
specification, assuming that all called
routines satisfy theirs
Reason about implementation when it is
written, not when it is used or extended
Demo
Design tradeoffs




Missed errors
Spurious warnings
Annotation overhead
Performance
Tool architecture
Annotated Java program
Translator
Verification condition
Automatic theorem prover
Counterexample context
Post processor
Warning messages
Valid
Resource
exhausted
Tool architecture, detail
Translator
Annotated Java program
Sugared command
Primitive command
Passive command
Verification condition
Automatic theorem prover
Counterexample context
Post processor
Warning messages
Annotated Java
program
Tool architecture, detail
Annotated Java program
Primitive
command
Passive
command
Verification
condition
Automatic
theorem prover
Counterexample
context
Post processor
Warning
messages
Translator
Translator
Sugared
command
Sugared command
Primitive command
Passive command
Verification condition
Automatic theorem prover
Counterexample context
Post processor
Warning messages
Annotation language
Annotated Java
program
Translator
Sugared
command
Primitive
command



Passive
command


Automatic
theorem prover
Post processor
Warning
messages


non_null
Method annotations

Verification
condition
Counterexample
context
Simple
requires E;
modifies w;
ensures P;
exsures (T x) Q;
Object invariants

invariant E;
Annotation language
Annotated Java
program
Translator
Sugared
command
Primitive
command



Passive
command


Automatic
theorem prover
Post processor
Warning
messages


non_null
Method annotations

Verification
condition
Counterexample
context
Simple
requires E;
modifies w;
ensures P;
exsures (T x) Q;
Object invariants

invariant E;
Annotation language
Annotated Java
program
Translator
Sugared
command
Primitive
command
Passive
command
Verification
condition
Automatic
theorem prover
Counterexample
context
Post processor
Warning
messages

Specification expressions

side-effect free Java expressions


\result, \old(E)



ensures \result == \old(x);
==>
(\forall T x; P), (\exists T x; P)


no ++, no method calls
(\forall int j; 0 <= j && j < n ==> a[j] > 0);
\typeof(E), \type(T), <:

requires \typeof(x) == \typeof(this);
Annotation language
Annotated Java
program
Translator
Sugared
command
Primitive
command
Passive
command
Verification
condition
Automatic
theorem prover
Counterexample
context
Post processor
Warning
messages

Specification expressions

side-effect free Java expressions


\result, \old(E)



ensures \result == \old(x);
==>
(\forall T x; P), (\exists T x; P)


no ++, no method calls
(\forall int j; 0 <= j && j < n ==> a[j] > 0);
\typeof(E), \type(T), <:

requires \typeof(x) == \typeof(this);
Annotation language
Annotated Java
program
Translator
Sugared
command
Primitive
command
Passive
command
Verification
condition
Automatic
theorem prover
Counterexample
context
Post processor
Warning
messages

Concurrency

monitored_by lock


\lockset[lock]



/*@ monitored_by this */ long x;
requires \lockset[this];
lock0 < lock1
\max(\lockset)

requires \max(\lockset) < this;
Annotation language
Annotated Java
program
Translator
Sugared
command
Primitive
command
Passive
command
Verification
condition
Automatic
theorem prover
Counterexample
context
Post processor
Warning
messages

Concurrency

monitored_by lock


\lockset[lock]



/*@ monitored_by this */ long x;
requires \lockset[this];
lock0 < lock1
\max(\lockset)

requires \max(\lockset) < this;
Annotation language
Annotated Java
program
Translator
Sugared
command
Primitive
command

Ghost variables

ghost public int objectState;
 ghost public \TYPE elementType;

Passive
command
Verification
condition
Automatic
theorem prover
Counterexample
context
Post processor
Warning
messages
ghost public T x;

set x = E;
set objectState = Open;
 set elementType = \type(T);

Annotation language
Annotated Java
program
Translator
Sugared
command
Primitive
command

Ghost variables

ghost public int objectState;
 ghost public \TYPE elementType;

Passive
command
Verification
condition
Automatic
theorem prover
Counterexample
context
Post processor
Warning
messages
ghost public T x;

set x = E;
set objectState = Open;
 set elementType = \type(T);

Annotation language
Annotated Java
program
Translator
Sugared
command
Primitive
command

Miscellaneous


Passive
command
Verification
condition
assert E;
assume E;


nowarn

Automatic
theorem prover
Counterexample
context
Post processor
Warning
messages

assume x >= 0; // because x == y*y
x = a[j]; //@ nowarn
axiom E;

axiom (\forall int x; x >> 2 >= 0);
Sugared commands
Annotated Java
program
Translator
Sugared
command
Primitive
command
Passive
command
Verification
condition
Automatic
theorem prover
Counterexample
context
Post processor
Warning
messages

S,T ::=
|
|
|
|
|
|
|
|
|
assert E
assume E
x= E
raise
S;T
S!T
S [] T
loop {inv E} S  T end
call x = t.m(E)
…
Sugared commands
Annotated Java
program
Translator
Sugared
command

assert t != null;
tmp = select(f, t);
assert tmp != null;
x = select(g, tmp)
Primitive
command
Passive
command
Verification
condition
Automatic
theorem prover
Counterexample
context
Post processor
Warning
messages
x = t.f.g;

if (x < 0) { x = -x; }
/*@ assert x >= 0; */
( assume x < 0; x = -x
[] assume !(x < 0)
);
assert x >= 0
Sugared commands
Annotated Java
program
Translator
Sugared
command

assert lblneg(“[email protected]”, t != null);
tmp = select(f, t));
assert lblneg(“[email protected]”, tmp != null);
x = select(g, tmp)
Primitive
command
Passive
command
Verification
condition
Automatic
theorem prover
Counterexample
context
Post processor
Warning
messages
x = t.f.g;

if (x < 0) { x = -x; }
/*@ assert x >= 0; */
( assume x < 0;
assume lblpos(“Then^280:7”, true); x = -x
[] assume !(x < 0);
assume lblpos(“Else^280:7”, true)
);
assert x >= 0
Primitive commands
Annotated Java
program
Translator
Sugared
command
Primitive
command
Passive
command
Verification
condition
Automatic
theorem prover
Counterexample
context
Post processor
Warning
messages

S,T ::=
|
|
|
|
|
|
|
|
|
assert E
assume E
x= E
raise
S;T
S!T
S [] T
loop {inv E} S  T end
call x = t.m(E)
…
Primitive commands
Translator
Annotated Java
program
Sugared
command

Primitive
command

Passive
command
Verification
condition
Automatic
theorem prover
Counterexample
context
Post processor
Warning
messages
//@ requires Pre; modifies w; ensures Post;
void m(U u);
call x = t.m(E)
var u in
u = E;
assert Pre;
var w0 in
w0 = w;
havoc w;
assume Post
end
end
Passive commands
Annotated Java
program
Translator
Sugared
command
Primitive
command
Passive
command
Verification
condition
Automatic
theorem prover
Counterexample
context
Post processor
Warning
messages

S,T ::=
|
|
|
|
|
|
assert E
assume E
x= E
raise
S;T
S!T
S [] T
Passive commands
Annotated Java
program
Translator
Sugared
command
Primitive
command
Passive
command
Verification
condition
Automatic
theorem prover
Counterexample
context
Post processor
Warning
messages

if (x < 0) { x= -x; }
/*@ assert x >= 0; */
( assume x0 < 0; assume x1 == -x0;
assume x2 == x1
[] assume !(x0 < 0);
assume x2 == x0
);
assert x2 >= 0
Weakest preconditions
Annotated Java
program
Translator
Sugared
command
Primitive
command
Passive
command

wp(assert E, Q) = E  Q

wp(assume E, Q) = E  Q
wp(S;T, Q) = wp(S, wp(T,Q))

wp(S [] T, Q) = wp(S, Q)  wp(T, Q)

wp(S, Q) = wp(S, true)  wlp(S, Q)

wlp(S, Q) = wlp(S, false)  Q

Verification
condition
Automatic
theorem prover
Counterexample
context
Post processor
Warning
messages
Verification condition
Annotated Java
program
Translator
Sugared
command

Universal background predicate

Primitive
command

Passive
command
(FORALL (t) (<: t t))
Type-specific background predicate

(<: T_T |T_java.lang.Object|)
Verification
condition
Automatic
theorem prover
Counterexample
context
Post processor
Warning
messages

Verification condition:
BPUniv  BPT  VCmethod
Annotated Java
program
Translator
Sugared
command
Primitive
command
Passive
command
Verification
condition
Automatic
theorem prover
Counterexample
context
Post processor
Warning
messages
(AND
(<: T_T |T_java.lang.Object|)
(EQ T_T (asChild T_T |T_java.lang.Object|))
(DISTINCT arrayType |T_boolean| |T_char| |T_byte| |T_short| |T_int|
|T_long| |T_float| |T_double| |T_.TYPE|
T_T |T_java.lang.Object|)))
(EXPLIES

(LBLNEG |vc.T.abs.2.2|
(IMPLIES
(AND
(EQ |elems@pre| elems)
(EQ elems (asElems elems))
(< (eClosedTime elems) alloc)
(EQ LS (asLockSet LS))
(EQ |alloc@pre| alloc))
(NOT
(AND
(EQ |@true| (is |x:2.21| T_int))
(OR
(AND
(OR
(AND
(< |x:2.21| 0)
(LBLPOS |trace.Then^0,3.15| (EQ |@true| |@true|))
(EQ |x:3.17| (- 0 |x:2.21|))
Verification condition
class T {
static int abs(int x) {
if (x < 0) { x = -x; }
//@ assert x >= 0;
}
}
Annotated Java
program
Translator
Sugared
command
Theorem prover: “Simplify”

Nelson-Oppen cooperating decision
procedures
Primitive
command

Passive
command

Verification
condition
Automatic
theorem prover
Counterexample
context



Key features:




Post processor
Warning
messages
conguence closure
linear arithmetic
partial orders
quantifiers

automatic: no user interaction
refutation based: searches for counterexamples
heuristics tuned for program checking
labels
time limit
Counterexamples and warnings
Annotated Java
program
Translator
Sugared
command

Primitive
command
Passive
command
Verification
condition
Automatic
theorem prover
Counterexample
context
Post processor
Warning
messages

Counterexample:
labels: (|[email protected]| |vc.Bag.add.20.2| |trace.Then^0,21.23|)
context:
(AND
(NEQ |tmp1!a:23.23| null)
(NEQ this null)
(EQ |alloc@pre| alloc)
(EQ |tmp4!n:26.6| 0)
…
(<= alloc (vAllocTime |tmp3!a:26.4|))
)
Bag: add(int) ...
-----------------------------------------------------------------------Bag.java:26: Warning: Array index possibly too large (IndexTooBig)
a[n] = x;
^
Execution trace information:
Executed then branch in "Bag.java", line 21, col 23.
------------------------------------------------------------------------
ESC/Java research platform

Annotation inference


Other checking tools



Calvin [Qadeer et al.], Stale-value concurrency
checker [Burrows & Leino]
Formal verification without driving theorem
prover
Simplify theorem prover


Daikon [Ernst], Houdini [Flanagan & Leino]
SLAM [Ball, Rajamani, et al.], oolong [Leino et al.], …
Teaching

Kansas State University [Dwyer & Hatcliff]
Family of languages and tools




ESC/Java
Java Modeling Language (JML) [Leavens
et al.]
LOOP [Jacobs et al.]
JML + JUnit [Cheon & Leavens]
Further research:
Checking components

Component-modular checking



Fewer specifications
More inference
Specifying and checking side effects

Modifies clauses (frame problem), alias
confinement
Conclusions




ESC/Java: tool and platform for research
JML family of specification languages and tools
Ready for use by disciplined programming teams
and in teaching
Future challenges:



component-modular checking
more specification methodologies
Download ESC/Java:
research.compaq.com/SRC/esc
Sources now available, too!