Transcript The titre - Uppsala University

Test generation
using
model-checking
Thierry Jéron
Irisa/Inria Rennes, France
http://www.irisa.fr/vertecs/
Adapted from works with
C. Jard, B. Jeannet, J. C. Fernandez,
H. Marchand, P. Morel, V. Rusu,
V. Tschaen, E. Zinovieva et al.
Model-checking / Conformance Testing
S: Specification
I: Black box (unknown except its interface)
P: Safety Property
e.g. given by observer A: P
S: Specification
S ² P iff S does not exhibit
(finite) bad behaviors wrt P
i.e. L(S) Å L(A: P, Violate) = ;
Exaustive check on
S £ A: P
I conf. / S iff I does not exhibit
(finite) bad external behaviors wrt S
Rmk: safety property on ext. behavior
I conf. S , STr (I) Å STr(CT(S), Fail) = ;
Violate
Fail
Finite partially controllable
experiment on
Exec(I || CT(S))
Similar problems, different means
2
Some links between Model-checking for Conformance Testing
• Test selection using model-checking :
– S deter., controllable, P reachability: TC ' counter-exple of S ² : P
[Engels et al. 97, Gargantini et al.99]
– Extension to coverage [Hong et al.02, Blom et al.04]
– Non-controllable case is a bit more complex (this talk)
• Checking properties on the implementation
– Black-box checking [Peled]: learn I by experiment, model-check I ² P
– Model-checking and testing safety properties: build tests from S and
P that check whether I ioco S, I ² P and S ² P.
3
Outline
• Conformance Testing with ioco
• Test selection using model-checking
• Selection by a test purpose for (finite) ioLTS
• Selection by a test purpose for (infinite)
ioSTS
• Selection by a (negative) safety observer
• Conclusion
4
ioco Testing theory [Tretmans 96]
• Specification: known ioLTS S
• Implementation: unknown ioLTS I
• Conformance
– Visible behaviors : suspension traces STr(.)
– ioco: partial inclusion of STr(I) in STr(S)
• Test cases : ioLTS TC to build
– Execution I || TC (synchronisation on common actions) ! verdict.
– Properties: TS= {TC} is
Sound:
verdict(I||TC) = fail ) : (I ioco S)
Exhaustive: : (I ioco S) ) 9 TC 2 TS, verdict(I||TC)= fail
• Test case selection problem:
Renounce to exhaustiveness in practice,
select a finite TS likely to discover non-conformances
6
The ioLTS model
Quiescence is observable
!
?A
! Y
! X
t
?B
!
t
! Z
M = (Q,q0,,!)
 = ? [ ! [t
?A
! Y
! X

!
?A
t
?B
!
det
t
! Z
(M) = (Q,q0, ,!)
•  = ? [ ! [t
• ! = ! [ !
• ! = ! [ {q©! |
q 2 quiescent(M)}
Suspension automaton
!X
!
!
?B
!Z
!Y
?B
!
det((S))=(2Q,t*(q0),vis,!vis)
 vis = ? [ !
Deterministic automaton
STr(S) , Tr((S))
Visible behaviors
7
Conformance relation
I ioco S , 8  2 STr(S),
Out((I) after ) µ Out((S) after )
I2 ioco S
: I1 ioco S
det((S))
!
!
!Y
!
?B
!Z
?A
?A
?A
!X
!
!
!
!X
?B
!
!
?B
!X
!
!Y
!X
?B
!
!Z
!
?A
!
?B
!Z
!Y
?B
Unspecified
Input
Allowed
!
Unspecified
Output/quiescence
Not allowed
8
Non-conformance Observer
Complete det((S)) by unspecified outputs in
! non-conformance observer A«
Tr(A: ioco S ,Fail) = STr(S) .
!
: ioco S »
\ STr(S)
!
det((S))
?A
Fail
!X
!
I ioco S
,
STr(I) Å Tr(A: ioco S ,Fail) = ;
!
?B
I ioco S , I || CT(S) 9 Fail
?B
!Z
!
mirror
?
Rmk: If I was known, could be model-checked
The canonical tester of S for ioco is
CT(S) , mirror (A: I ioco S)
!Y
!
!A
Fail
?X
?
?
!B
?Z
?Y
!B
?
9
Non-deterministic test generation [Tretmans 96]
A test case TC is (the mirror of) any finite
« controllable » unfolding of CT(S)
A: I ioco S
!
?A
?A
Fail
!X
!
!
?B
!Y
Fail
!Y
!X
!
?B
Pass
!Z
!
!
!Z
Pass
Pass
!
?A
TS = {TC that can be built this way} is
• sound
• exhaustive
TS infinite ) selection needed
Fail
!Y
!X
?B
!
Pass
Pass
!Z
Pass
10
Test selection with (positive) Test Purpose
Test purpose = description of some interesting visible behaviours of S
to be tested
Test purpose model :
deterministic and complete ioLTS
+ Accepting states
TP = (Qtp, q0tp, vis,!tp, Accept)
Tr(TP) = vis* ( ( complete)
Tr (TP, Accept) µ vis*
*
TP
*
*
!x
!z
Accept
Test selection goal : build a sound test cases that is both:
– A positive observer of STr(S) Å Tr(TP,Accept) ! Pass verdict
– A negative observer of STr(S) . ! \ STr(S) ! Fail verdict
11
Principles of test selection
S
(S)
det((S))
STr(S)
TP
A: ioco S
SP = A: ioco S £ TP
CTG
TC
Tr(A:ioco S) =pref(STr(S). !)
Tr(A:ioco S, Fail) = STr(S). ! \ STr(S)
Tr(TP) = vis*
Tr(TP,Accept)
Tr(SP) = Tr(A:ioco S) Å Tr(TP)
Tr(SP,Fail) = Tr(A :ioco S, Fail) Å Tr(TP)
Tr(SP,Accept,: Fail) = STr(S) Å Tr(TP,Accept)
Focus on Accept
i..e restrict to states
coreachable to Accept
Tr(CTG, Pass) = Tr(SP,Accept,: Fail)
Tr(CTG, Fail) µ Tr(SP,Fail)
Controllable Test Case
12
Synchronous product
A:ioco S
3 (Accept, Fail)
!
!oth
?C
?A
!X
!oth
!
Fail
!
?B
?C
!Y
*
*
TP
*
!
!Y
!Y
?B
SP = A:ioco
S
x TP
!Y
!
?B
!
?C
?B
!x
Accept
!
!Z
?B
!
!Z
Accept
!z
!Y
!X
!
!Z
?C
?A
!
!
!Y
= : Fail £ AcceptTP
Tr(SP) = pref(STr(S). !)
! Fail verdict
Tr(SP,FailSP) = STr(S). ! \ STr(S)
Tr(SP,AcceptSP) = STr(S) Å Tr(TP,Accept) ! Pass verdict
13
Selection of the Complete Test Graph CTG
Goal: remove traces that
cannot lead to Accept
!oth
Init
1. Computation of
coreach(Accept)
!
!Y
i.e. states on traces to Accept
pref(Tr(SP,Accept))
=
Tr(SP, coreach(Accept))
?C
?A
!Y
!X
coreach(Accept)
!
!
?B
!
?B
?C
!Z
!
!
!Z
?B
!
!Y
Accept
14
Building the Complete Test Graph (2)
2. Verdict assignment
!oth
Pass verdict ! Accepted traces:
Tr(SP,Accept) = STr(S) Å Tr(TP,Accept)
Inconc verdict ! Refused traces
correct traces after which
Accept is unreachable
Optimality:
Cut controlable ones and detect early :
[pref(Tr(SP,Accept)).! \ Tr(SP, Fail)]
\ pref(Tr(SP,Accept))]
Fail verdict ! Non-conformant traces
pref(Tr(SP,Accept)).! Å Tr(SP,Fail)
) preserves soundness
Init
!
?A
!Y
!X
!
Inconc
?
?B
coreach(Accept)
?B
! Z
Pass
reach(Init)
15
Building the Complete Test Graph (3)
3.
Mirror image
? oth
Init
! Tester point of view
?
CTG(S,TP)
!A
?Y
Properties
• soundness:
only non-conformance is detected
• (limit) exhaustiveness:
every non-conformance may be
detected
: (I ioco S) )
9 TP, verdict(CTG(S,TP) || I) = fail
?X
?
Inconc
?
!B
coreach(Accept)
!B
?Z
Pass
NB: controllable test cases : no choice between output and ouput/input
16
Implementation in TGV
S
S
(S)
S £ TP
det((S))
(S £ TP)
TP
A: ioco S
A: ioco S £ TP
det((S £TP))
CTG
TC
Internal actions in TP
)improves selection
Specification
TP
CTG
TC
Lazzy construction of
intermediate IOLTS
) limits state space explosion
UML
SDL
Lotos
IF
Umlaut
Object
Géode
Caesar
open
IF
open
Test
Purpose
bcg, IF
Simulation API of S
TGV
Test case (bcg, aut, TTCN, )
• Integrated in several tool chains
(CADP, Objectgéode, Agedis, IF)
• Experimented on numerous case studies
18
Test selection for
ioSTS
• Motivations:
– Use higher level specification model with control and data
– Avoid state space explosion due to enumeration of data values
p¸x
? a(p)
y := y + p
• Specification model
S: ioSTS = i/o automata extended with data
ioSTS semantics [[S]] = infinite states ioLTS
• Implementation model: unknown (infinite state) ioLTS
• Conformance testing theory: ioco
• Test selection from S and Test Purpose TP : ioSTS observer
– Syntactical constructions of
A: ioco S, A: ioco S £ TP , CTG(S,TP)
guided by coreachability analysis
– coreachability undecidable ) over-approximated analysis
– Produces uninstanciated test programs
19
ioSTS model
S = (VS, S, , TS)
TP = (VS [ VTP, TP, , TTP)
Observer of actions and variables of S
End
p=y-x ∧ p ¸ 2
!ok(p)
!end
true
?a(p)
x:=p
?start
Init
Rx
Ry
?a(p)
y:=p
p=2∧x≥3
!ok(p)
Accept:=true
true
Wait
Cmp
Acc
⌝(p=2∧x≥3)
!ok(p)
!nok(p)
p=y-x ∧ p > 2
!nok(p)
NB: Locations = values of an additional variable l
(but useful in figures)
[a(p) : G(vS,p); vS:=A(vS,p)] 2 TS
*
*
Sink
*
[a(p) : G(vS,vTP,p); vTP := A(vS, vTP,p)] 2 TTP
20
Simplifying asumptions on ioSTS (for this talk)
1. ioSTS are supposed to be deterministic
Determinisation of ioSTS into ioSTS is not always possible ) restrictions needed
• No internal loop
Idea: after a finite delay, the effect of
actions must be known: diagnosability
• Finite lookahead
! Determinisation procedure for this sub-class : postpone assignments
« t-closure »
G1
t
G1
A1
G1 Æ G2 ± A1
G2
A2 ± A1
a(p)
A2
« Subset construction »
a(p)
G2
a(pa)
a(pa)
A1
A2
G1 \ G2  ;
G1[ G2
a(pa)
va := pa
Gb
b(pb)
Gc
G1 Æ Gb ± A1
G2 Æ Gc ± A2
Ab
Ac
Ab± A1
Ac ± A2
c(pc)
b(pb)
c(pc)
2. Quiescence is not treated here
augment model with universal quantification in guards
21
ioLTS semantics of
[[S]] =
S = (VS, S, , TS)
End
p=y-x ∧ p ¸ 2
!ok(p)
!end
true
?a(p)
x:=p
?start
Init
Rx
ioSTS
Ry
?a(p)
y:=p
S = (Q,Q0, ,!)
Q = D (VS)
… (Init,vx,vy) …
Q0 = D (VS) Å S
Cmp
 =  £ D (PS)
?start
… (Rx, vx,vy) …
?a(-1) ?a(0)
?a(3)
p=y-x ∧ p > 2
!nok(p)
t = [a(p) : G(vS,p); v’S:=A(vS,p)]
is firable in state v iff 9 , G(v,)=true
v – t ! v’=A(v,)
!end
?a(+1)
… (Ry,3,vy) …
?a(0)
(Cmp,3,0)
!nok(-3)
(Rx,3,0)
?a(6)
(Cmp,3,6)
!ok(3)
(Rx, 3,4)
Runs(S): v0 ! 1(p) v1 ! 2(p) v2 … 2 Q0. (vis,Q)*
Traces(S) : 1(p). 2(p)… 2 vis*
22
(End,-,-)
Principle of syntactical test selection
runs(S)
STr(S)=pvis(runs(S))
ioSTS syntactical operations
S: ioSTS
deterministic
A: ioco S
TP: ioSTS
SP = A: ioco S £ TP
runs(A),
runs(A,Fail)
STr(A)= pref(STr(S).!)
STr(A, Fail) = STr(S).! \ STr(S)
runs(TP)
runs(TP,Accept)
Syntactical Synchronous Product:
runs(SP) = runs(A) Å runs(TP)
runs(SP,Accept) = runs(A) Å runs(TP,Accept)
runs(SP,Fail) = runs(A,Fail)
STr(SP) = STr(A)
STr(SP,Accept)=p (runs(SP, Accept))
vis
CTG: ioSTS
STr(SP,Fail)= STr(A,Fail)
Sub-ioSTS of SP focussed on Accept
Tr(CTG, Pass) = Tr(SP, Accept)
23
Non-conformance observer
1. Add new variable Verd
with initial value none
i.e.  Æ Verd = none
2. 8 ouput !a, 8 t carrying !a :
: (Çi Gi(v,p))
!a(p)
Verd:= Fail
G1(v,p)
!a(p)
v:= A1(v,p)
End
?start
Init
true
Æ Verd=none
Gn(v,p)
!a(p)
v:= An(v,p)
p=y-x ∧ p ¸ 2
!ok(p)
!end
Rx
?a(p)
x:=p
Ry
A: ioco S
?a(p)
y:=p
!oth
Cmp
Fail
STr(A, Fail)
=
STr(S). ! \STr(S)
) soundness
p=y-x ∧ p < 2
!nok(p)
24
Syntactical product
A : ioco S = (VA, A, , TA)
TP = (VS [ VTP, TP, , TTP)
G2(vS,vTP,p)
a(p)
v’TP := A2(vS, vTP,p)
G1(vS,p)
a(p)
v’S := A1(vS,p)
A : ioco S £ TP = (V
A : ioco S £ TP
S
[ VTP, S Å TP, , TS£TP)
G1(vS,p) Æ G2(vS, vTP, p)
a(p)
<v’S; v’TP> := <A1(vS,p);A2(vS, vTP,p)>
25
Syntactical product: example
A : ioco S
End
Verd=none
Idle
SP , A : ioco S£ TP
! oth
!end
?start
Rx
p=y-x Æ p ¸ 2
!ok(p)
?a(p)
x:=p
Rx
Sink
?a(p)
y:=p
Ry
Cmp
End
Wait
p=y-x Æ p < 2
!nok(p)
TP
true
p=y-x Æ p ¸ 2 p=y-x Æ p < 2
Æ⌝(p=2 Æ x≥3)
!nok(p)
!ok(p)
!end
p=y-x Æ p ¸ 2
Æ p=2 Æ x≥3
!ok(p)
Verd=none
*
Wait
p=2 Æ x≥3
!ok(p)
Accept:=true
*
Acc
⌝(p=2 Æ x≥3)
!ok(p)
!nok(p)
?a(p)
?a(p)
Accept:=
?start
x:=p
Idle
Rx
Ry y:=p Cmp
true Rx
Wait
Wait
Wait
Wait
Acc
Sink
*
! oth
Verd:=Fail
¶ (Accept, Fail)
26
Syntactical Test Selection (1)
1. Assignment of Pass verdicts
Pass: Tr(SP,Accept)
A : ioco S £ TP
G(v,p)
a(p)
v:= A(v,p)
!
Observer of Tr(SP, Accept)
G(v,p) Æ Verd = none
a(p)
v:= A(v,p)
Verd :=if AAccept then Pass
else Verd
CTG
27
Syntactical Test Selection (2)
2. Selection and assignment of Inconc verdicts
coreach(Accept) not computable ) compute over-approximations:
coreach ¶ coreach(Accept)
8 assignment A, pre (A) (coreach) ¶ pre(A) (coreach )
Idea:
pre (A) (coreach) = Nec. Cond. to go in coreach
: pre (A) (coreach) = Suf. Cond. to go outside coreach
µ outisde coreach(Accept)
coreach
G(v,p)
a(p)
v’:=A(v,p)
G(v,p) Æ
: pre(A)(coreach)
a(p)
v’:=A(v,p)
G(v,p) Æ
pre(A)(coreach)
a(p)
v’:=A(v,p)
coreach
28
Syntactical test selection (3): guard strengthening
Rule for inputs of S:
keep conditions leading to coreach,
cut other ones (controllable):
G(v,p) Æ
: pre(A)
? a(p)
v’:=A(v,p)
coreach
G(v,p) Æ
pre(A)
? a(p)
v’:=A(v,p)
coreach
Rule for outputs of S
keep all conditions (uncontrollable),
those leading outside coreach
produce Inconc:
G(v,p) Æ
: pre(A)
! a(p)
v’:=A(v,p)
Verd:=Inconc
coreach
G(v,p) Æ
pre(A)
! a(p)
v’:=A(v,p)
coreach
Inconc
29
Test selection: example
1st over-approximation : control
SP = A: ioco S £ TP
⊥
⊥
!end
Verd=
none
Idle
Wait
⊤
?start
Rx
Wait
⊤
Rx
Sink
p=y-x ∧p ¸ 2
Æ⌝(p=2 Æ x≥3)
!ok(p)
End
Wait
?a(p)
x:=p
Ry
Wait
?a(p)
y:=p
⊤
⊤
Abstraction on control:
only the location is taken
into account in coreach
p=y-x Æ p < 2
!nok(p)
p=y-x ∧p ¸ 2
Æ p=2 Æ x≥3
Cmp !ok(p)
Rx
Wait Accept Acc
:= true
! oth
Verd:= Fail
Rx
Sink
p=y-x p ¸ 2
Æ⌝(p=2 Æ x≥3)
!ok(p)
Verd:= Inconc
End
Wait
!end
Verd:= Inconc
Verd=none
CTG1 (S, TP)
Idle
Wait
?start
! oth
Verd:=Fail
Rx
Wait
?a(p)
x:=p
Ry
Wait
?a(p)
y:=p
p=y-x Æ p < 2
!nok(p)
Verd:=Inconc
p=y-x ∧p ¸ 2
Æ p=2 Æ x≥3
Cmp
Rx
!ok(p)
Wait Verd:=Pass Acc
Fail
30
2nd
Test selection: example
approximation computed by NBAC (polyhedra)
SP = A: ioco S x TP
⊥
p=y-x Æ p ¸ 2
Æ ⌝(p=2 Æ x≥3)
!ok(p)
End
Wait
!end
Verd=
none
Idle
Wait
⊤
?start
Rx
Wait
⊤
coreach
Rx
Sink
⊥
?a(p)
x:=p
Ry
Wait
x≥3
?a(p)
y:=p
p=y-x Æ p < 2
!nok(p)
pre(x:=p)(x¸ 3)
=
p¸ 3
p=y-x ∧ p ¸ 2
Æ p=2 Æ x≥3
Cmp !ok(p) Rx
Wait Accept Acc
:=true
y-x=2Æ x≥3
⊤
Rx
Sink
p=y-x Æ p ¸ 2
p=y-x Æ p < 2
Æ ⌝(p=2 Æ x≥3)
!nok(p)
!ok(p)
Verd:= Inconc
Verd:= Inconc
End
Wait
!end
Verd:= Inconc
! oth
Verd:=Fail
Verd=none Idle
?start
Wait
CTG2 (S, TP)
! oth
Verd:=Fail
Rx
Wait
p¸3
?a(p)
x:=p
Ry
Wait
p = x+2
?a(p)
y:=p
p=y-x Æ p ¸ 2
Æ p=2 Æ x≥3
!ok(p)
Cmp Verd:=Pass Rx
Wait
Acc
Fail
31
Simplification: over-approximation reach of reach()
CTG2 (S, TP)
⊤
⊥
p=y-x Æ p ¸ 2
Æ ⌝(p=2 Æ
x≥3)
!ok(p)
Verd:= Inconc
End
Wait
!end
Verd:= Inconc
Verd=
none
Idle
Wait
⊤
?start
Rx
Wait
⊤
Fail
Sink
p¸3
?a(p)
x:=p
Ry
Wait
p = x+2
?a(p)
y:=p
x≥3
! oth
Verd:=Fail
Simplify guards
according to reach
(false ) cut)
p=y-x Æ p > 2
!nok(p)
Verd:= Inconc
NB: semantics is unchanged
p=y-x ∧p ¸ 2
Æ p=2 Æ x≥3
!ok(p)
Cmp Verd:=Pass Rx
Wait
Acc
y-x=2
Æ x≥3
Sink
y-x=2
Æ x≥3
false
false
!ok(p)
!nok(p)
Verd:= Inconc Verd:= Inconc
End
Wait
!end
Verd:= Inconc
Verd=
none
CTG2 (S, TP)
Idle
Wait
?start
Rx
Wait
?oth
Verd:=Fail
p¸3
?a(p)
x:=p
Ry
Wait
p = x+2
?a(p)
y:=p
Cmp
Wait
p=2
!ok(p)
Verd:=Pass
Rx
Acc
Fail
32
Consequences of over-approximation on test cases
For two abstractions 1 and 2
(e.g. 1: control vs 2: polyhedra)
pre1 (A) (coreach1) ¶ pre2 (A) (coreach2)
)
Tr(CTG1) ¶ Tr(CTG2)

reach()
∩
coreach1
In CTG1
• Inconc detected later
• More fail verdicts (all sound)
?
!
reach()
∩
coreach2
Limit cases:
• exact analysis:
best guiding to Accept
• no analysis:
no guiding to Accept
?
?
Inconc
reach()
?oth
Accept
Fail
33
Test execution
Ouputs: [ ! a(p) : G(v,p); v:=A(v,p)] : v is known,
Inputs:
choose  s.t. G(v,),by constraint solving
send !a(), assign v:=A(v, )
[ ? a(p) : G(v,p); v:=A(v,p)]: v is known,
receive ? a()
evaluate G(v,)
if true, assign v:= A(v,) (input complete)
End
Wait
CTG2 (S, TP)
Verd=
none
!end
Verd:= Inconc
Idle
Wait
?start
! oth
Verd:=Fail
Rx
Wait
p¸3
?a(p)
x:=p
Ry
Wait
p = x+2
?a(p)
y:=p
Cmp
Wait
p=2
!ok(p)
Verd:=Pass
Rx
Acc
Fail
!start . !a(4) . !a(6) . ?ok(2) : Pass
!start . ?end : Inconc
!start . !a(5) . !a(7) . ?ok(3) : Fail
!start . !a(6) . !a(8) . ? nok(2) : Fail
34
The STG Tool
Test case selection
•
Symbolic operations
•
Over-approximated analysis by
abstract interpretation (NBAC)
Test case execution
•
Compilation C++/Java
•
Constraint solving (Omega)
Specification Test purpose
dotty
Test case
selection
NBAC
Test case
Compilation
Experimentations :
CEPS
Omega
C++/
Java
Test case
IUT
C++/Java
verdict
35
Model-checking and Testing safety properties
Development process
P properties
S ² P ? y/n/u
I²P?
S specification
I conf S ? n/u
I implementation
36
Model-checking a safety property
Model-checking S ² P ? reduces to
S
End
Idle
!end
?start
Rx
?a(p)
x:=p
A: P
End
Wait
?a(p)
y:=p
Ry
p=y-x Æ p < 2
!nok(p)
*
reachability in
p=y-x Æ p ¸ 2
!ok(p)
*
p = -10
!nok(p)
Wait Violate := true Violate
S £ A: P (undecidable)
!end
Cmp
Idle ?start Rx
Wait
Wait
p=y-x Æ p ¸ 2
!ok(p)
?a(p)
Ry y:=p Cmp
Wait
Wait
p=y-x Æ p < 2
Æ p = -10
p=y-x Æ p < 2
!nok(p)
Æ p  -10
Rx
!nok(p)
Violate
?a(p)
x:=p
S ² : P :?start.?a(10).?a(0).!nok(-10) ! (Rx, Violate)
With any abstraction S ² : P ? is Yes
but S ² : P ; S ² : P
Result of S ² P ? could be Unknown
37
Test selection from a safety observer
A: ioco S
End
Idle
!end
?start
Rx
p=y-x Æ p ¸ 2
!ok(p)
?a(p)
x:=p
! oth
Verd:=Fail
A: P
p = -10
!nok(p)
Wait
Violate := true
!end
Cmp
p=y-x Æ p < 2
!nok(p)
*
End
Wait
?a(p)
y:=p
Ry
*
Violate
: I ioco
A: ioco S £ A: P
!oth
Idle ?start Rx
Wait
Wait
p = -10
!nok(p)
Violate :=true
Verd:=Fail
S
! oth
Verd:=Fail
p=y-x Æ p ¸ 2
!ok(p)
?a(p)
x:=p
Ry
Wait
?a(p)
y:=p
p=y-x Æ p < 2)
Æ p -10
!nok(p)
: (p=y-x Æ p < 2)
Æ p =-10
!nok(p)
Violate :=true
Verd:=Fail
Fail
Violate : I ioco
I²:P
S
Cmp
Wait
p=y-x Æ p < 2
Æ p= -10
!nok(p)
Violate:=true
Violate
S²:P
I²:P
!start.!a(10).
!a(0).?nok(-10)
38
Conclusion
• Unified framework for Ioco-based Test selection
– For (Positive) Test Purpose and (Negative) Safety Observers
– For finite ioLTS and infinite ioSTS
– Using verification: coreachability analysis, over-approximations
• Complementarity of testing and model-checking
Research work needed for, e.g.
• Theories and algorithms for other models of reactive systems
e.g. with time, data, stack, probabilities …and combinations
• Coverage : measures, selection
• Links with structural testing techniques
• ….
39
Bibliography
J. Tretmans. Test Generation with Inputs, Outputs and Repetitive
Quiescence. Software - Concepts and Tools, Vol. 17(3), pp. 103 - 120.
Springer-Verlag, 1996.
C. Jard, T. Jéron, TGV: theory, principles and algorithms, A tool for the
automatic synthesis of conformance test cases for non-deterministic
reactive systems, Software Tools for Technology Transfer (STTT), 6,
October 2004.
B. Jeannet, T. Jéron, V. Rusu, E. Zinovieva, Symbolic Test Selection based
on Approximate Analysis, in TACAS'05, LNCS 3440, Edinburgh
(Scottland), April 2005.
Vlad Rusu, Hervé Marchand, Thierry Jéron,
Automatic Verification and Conformance Testing for Validating Safety
Properties of Reactive Systems,
in Formal Methods 2005 (FM05), July 2005.
Available at http://www.irisa.fr/vertecs/publications.html
40