Automated Model Based Testing From Theory via Tools to
Download
Report
Transcript Automated Model Based Testing From Theory via Tools to
Test Generation
for
Input/Output Transition Systems
Ed Brinksma
Course 2004
Test Cases
!dub
Test case t TTS
TTS - Test Transition System :
!kwart
labels in L { }
tree-structured
?coffee
finite, deterministic
final states pass and fail
either one input !a
or all outputs ?x and
?tea
pass
© Ed Brinksma/Jan Tretmans
TT 2004, Test Derivation
2
fail
!dub
from each state pass, fail :
?tea
?coffee
pass fail
fail
Test Cases
!dub
test case t
!kwart
?coffee
?tea
!dub
!kwart ; Start timer1
?tea
fail
?timer1
fail
?coffee
fail
!dub
?tea
pass
?coffee
fail
!dub ; Start timer1
?tea
pass
?timer2
pass
?coffee
fail
pass fail
© Ed Brinksma/Jan Tretmans
TT 2004, Test Derivation
3
Observations and Verdicts
OBS
( L { } )*
obs
obs : TTS IOTS ( ( L { } )* )
obs (t,i) =
t
t (O) =
=
© Ed Brinksma/Jan Tretmans
{ ( L { } )* |
t i
pass i' or
t i
fail i'
t : ( ( L { } )* ) { fail, pass }
pass
if O . t
fail
otherwise
TT 2004, Test Derivation
pass
4
}
Test Generation
Look for test derivation algorithm der : LTS (TTS)
such that der is sound and exhaustive:
i IOTS .
( t der ( s ) . t ( obs ( t, i ) ) = pass )
i ioco s
i ioco s =def Straces (s) : out (i after ) out (s after )
obs (t,i) = { (L{})* |
t (O) =
=
t i
pass i' or
t i
fail i'
pass if O . t
fail
© Ed Brinksma/Jan Tretmans
pass
otherwise
TT 2004, Test Derivation
5
}
Test Generation
i ioco s =def Straces (s) : out (i after ) out (s after )
s
i
test
!x
!y
!x
?x
!z
pass
out (s after )
out (i after )
= { !x, !y }
= { !x, !z }
© Ed Brinksma/Jan Tretmans
TT 2004, Test Derivation
?y
pass
?z
fail
out (test after ) = LU
6
Test Generation
i ioco s =def Straces (s) : out (i after ) out (s after )
s
i
test
!x
!y
!x
!z
?x
pass
?y
?z
pass fail
pass
out (s after )
out (i after )
out (test after )
= { !x, !y, }
= { !x, !z, }
= LU { }
© Ed Brinksma/Jan Tretmans
TT 2004, Test Derivation
7
Test Generation Algorithm
Algorithm
To generate a test case t(S) from a transition system
specification with S, with S set of states ( initially S = s0 after )
Apply the following steps recursively, non-deterministically:
1
end test case
pass
2
supply input
3
observe output
forbidden outputs
?y
allowed outputs
fail fail
t ( S after !x )
!a
t ( S after ?a )
© Ed Brinksma/Jan Tretmans
?x
allowed outputs or :
!x out ( S )
forbidden outputs or : !y out ( S )
TT 2004, Test Derivation
8
Test Generation Example
s
test
?dub
?dub
!dub
?coffee
!coffee
?tea
?coffee
?tea
fail
© Ed Brinksma/Jan Tretmans
TT 2004, Test Derivation
fail
9
fail
pass
pass
Test run :
Test Execution
t i
pass i'
or
t i
fail i'
t i equal to t || i , with extra rule :
pass
t
then
t i
t'
t
?tea
?coffee
fail
!dub
?tea
if
fail
i
and
t' i
i
?dub
!tea
© Ed Brinksma/Jan Tretmans
Two test runs :
!coffee
t i
?coffee
pass fail
i fails t
dub tea
t i dub coffee
TT 2004, Test Derivation
10
pass i'
fail i'
Test Execution Example
i
test
?dub
?dub
!dub
?dub
?coffee
!coffee
?tea
?dub
?coffee
?tea
Two test runs :
t i
dub
t i dub coffee
© Ed Brinksma/Jan Tretmans
pass i'
pass i'
TT 2004, Test Derivation
fail
fail
i passes t
11
fail
pass
pass
Test Generation Example
Equation solver for y2=x
specification
test
!9
? x (x < 0)
otherwise
? x (x >= 0)
! x
! -x
fail
?3
pass
otherwise
To cope with non-deterministic behaviour,
tests are not linear traces, but trees
© Ed Brinksma/Jan Tretmans
TT 2004, Test Derivation
fail
12
?-3
!4
?2
pass
?-2
pass
Test Execution Example
implementation
test
? x (x < 0)
otherwise
? x (x >= 0)
! -x
! x
!9
fail
?y
pass
otherwise
fail
© Ed Brinksma/Jan Tretmans
?3
TT 2004, Test Derivation
13
?-3
!4
?2
pass
?-2
pass
Validity of Test Generation
For every test t generated with algorithm we have:
Soundness :
t will never fail with correct implementation
i ioco s
implies
i passes t
Exhaustiveness :
each incorrect implementation can be detected
with a generated test t
i ioco s
© Ed Brinksma/Jan Tretmans
implies
t : i fails t
TT 2004, Test Derivation
14
Validity of Test Generation
Theorem:
Soundness :
i ioco s
implies
t der ( s ) . t ( obs ( t, i ) ) = pass
Exhaustiveness :
t der ( s ) . t ( obs ( t, i ) ) = pass
© Ed Brinksma/Jan Tretmans
TT 2004, Test Derivation
15
implies
i ioco s
Soundness and Exhaustiveness
s
test
?dub
?dub
!dub
?coffee
!coffee
?tea
?coffee
?tea
fail
© Ed Brinksma/Jan Tretmans
fail
///
fail
pass
pass
///
fail
pass
Not sound
Still anymore
sound ! !
TT 2004, Test Derivation
16
Formal Testing with Transition
Systems
Test hypothesis :
s LTS
ioco
der : LTS
(TTS)
Ts TTS
IUTIMPS . iIUT IOTS .
tTTS . exec(t,IUT) = obs(t,iIUT)
Soundness and exhaustivess proved:
iIOTS .
( tder(s) . t(obs(t,i)) = pass )
i ioco s
pass
iIUT
IUT IOTS
IMPS
obs
: TTS
exec
:
TESTS
IOTS
IMPS
(traces)
(OBS)
© Ed Brinksma/Jan Tretmans
traces
TT 2004, Test Derivation
17
t:
(traces)
{fail,pass}
fail