Transcript Automated Model Based Testing From Theory via Tools to

Formal Testing with
Input-Output
Transition Systems
Ed Brinksma
Course 2004
Formal Testing
Test hypothesis :
s SPECS
imp
der : SPECS 
(TESTS)
Ts  TESTS
IUTIMPS . iIUT MODS .
tTESTS . exec(t,IUT) = obs(t,iIUT)
Proof soundness and exhaustivess:
iMODS .
( tder(s) . t(obs(t,i)) = pass )
 i imp s
pass
iIUT
IUT MODS
IMPS
obs ::
exec
TESTS 
MODS

IMPS 
(OBS)
© Ed Brinksma/Jan Tretmans
OBS
t:
(OBS)
{fail,pass}
fail
Input-Output
Transition Systems
S0
? dub
S1
S2
! coffee
S3
? kwart
dub, kwart
coffee, tea
from user to machine
initiative with user
machine cannot refuse
from machine to user
initiative with machine
user cannot refuse
input
output
LI  LU = 
LI  LU = L
! tea
S4
LI = { ?dub, ?kwart }
LU = { !coffee, !tea }
© Ed Brinksma/Jan Tretmans
LI
LU
Input-Output
Transition Systems
?dub
?dub
?kwart
Input-Output Transition Systems
?kwart
IOTS (LI ,,LU )  LTS (LI , LU )
?dub
?kwart
!coffee
!tea
?dub
?kwart
LI = { ?dub, ?kwart }
LU = { !coffee, !tea }
© Ed Brinksma/Jan Tretmans
?dub
?kwart
IOTS is LTS with Input-Output
and always enabled inputs:
for all states s,
for all inputs ?a  LI :
S
?a
Input-Output
Transition Systems
?kwart
?kwart
?dub
?kwart
?dub
?kwart
?dub
!tea
!coffee
?dub
?kwart
?dub
?kwart
?kwart
?dub
?kwart
?dub
!tea
?dub
?kwart
© Ed Brinksma/Jan Tretmans
?dub
?kwart
?dub
!coffee
?dub
?kwart
?dub
!coffee
?dub
?kwart
Labelled Transition System
Testing
LTS ( LI  LU )
IOTS (LI , LU )
 LTS
 LTS
 LTS
 SPECS

 MODS

 TESTS

TTS ( LU , LI )
 OBS
 obs


traces
t || i
 der

der : LTS   ( LTS )
 Which imp ?




ioco
(strong, weak, branching, ... ) bisimulation
trace-, testing-, refusal - preorder / equivalence
conf, conf*, aconf,
ioconf, ioco, miocoF
© Ed Brinksma/Jan Tretmans
Formal Correctness
quiescence
Input Output Automata
refusal testing
canonical tester
testing equivalences
© Ed Brinksma/Jan Tretmans
ioco
Preorders
on Transition Systems
i  LTS

implementation
i
specification
s
environment
e
i s

environment
e
 e  E . obs ( e, i )  obs (e, s )

?
© Ed Brinksma/Jan Tretmans

?

?
s  LTS
Preorders on
Input-Output Transition Systems
implementation
i
environment
e
i  IOTS(LI,LU)
imp
imp
specification
s
environment
e
s  LTS(LILU)
 IOTS (LI,LU) x LTS (LILU)
Observing IOTS where system inputs
interact with environment outputs, and v.v.
© Ed Brinksma/Jan Tretmans
Preorders on
Input-Output Transition System
implementation
i
imp
environment
e
environment
e
s  LTS(LILU)
i  IOTS(LI,LU)
i imp s

 e  E . obs ( e, i )  obs (e, s )

IOTS(LU,LI)
© Ed Brinksma/Jan Tretmans
specification
s
Input-Output
Testing Relation
implementation
i
environment
e
i  IOTS(LI,LU)
i
iot s
iot
specification
s
environment
e
s  LTS(LILU)
  e  IOTS(LU,LI) . obs ( e, i )  obs (e, s )
obs ( e, p ) = ( traces (e||i ), Ctraces (e||i ) )
© Ed Brinksma/Jan Tretmans
Input-Output
Refusal Relation
implementation
i
environment
e
i  IOTS(LI,LU)
i
ior s  
ior
specification
s
environment
e
s  LTS(LILU)
e  IOTS(LU,LI {} ) . obs ( e, i )  obs (e, s )
obs ( e, p ) = ( traces (e||i ), Ctraces (e||p) )
© Ed Brinksma/Jan Tretmans
Input-Output
Testing Relation
i,s  LTS :
i
te s  

e  LTS . obs ( e, i )  obs (e, s )
FP ( i )  FP ( s )
FP ( p ) = {  , A 
| A  L,  traces(p),
p afer  refuses A }
i  IOTS(LI,LU) :
i
iot
s 
 e  IOTS(LU,LI) . obs ( e, i )  obs (e, s )
inputs can never be refused by i
outputs can never be refused by e :
i afer  refuses A
© Ed Brinksma/Jan Tretmans
 A =  or A = LU
Input-Output
Testing Relation
i  IOTS(LI,LU) :
i
iot
s 
 e  IOTS(LU,LI) . obs ( e, i )  obs (e, s )

FP ( i )  FP ( s )

{  |  traces(i), i afer  refuses  }
 {  |  traces(s), s afer  refuses  }
and {  |  traces(i), i afer  refuses LU }
 {  |  traces(s), s afer  refuses LU }
 traces(i)  traces(s) and Qtraces(i)  Qtraces(s)
Qtraces : Quiescent traces = traces ending in quiescence
i

i
=
© Ed Brinksma/Jan Tretmans
i
LU
i
=  !x  LU  {} : i
!x

Input-Output
Refusal Relation
i  IOTS(LI,LU) :
i
ior
s 
 e  IOTS(LU,LI {}) . obs ( e, i )  obs (e, s )
 Ftraces( i )  Ftraces ( s )
where:
A
  A{}:
Failure A :
i
Failure trace  :
  ( L ( L ) )* :
Failure traces of i :
Ftraces ( i ) = {   ( L ( L ) )* | i
i

i


i

inputs can never be refused by i
outputs can never be refused by e :
i afer  refuses A
© Ed Brinksma/Jan Tretmans
 A =  or A = LU
}
Input-Output
Refusal Relation
i  IOTS(LI,LU) :
i
ior
 e  IOTS(LU,LI {}) . obs ( e, i )  obs (e, s )
s 
 Ftraces( i )  Ftraces ( s )
 Straces( i )  Straces ( s )
Straces : Suspension traces
= Failure traces restricted to refusals quiescence LU = 
Straces ( i )
=
Ftraces ( i )  ( L  { LU } )*
=
{   ( L  {  } )* | i
© Ed Brinksma/Jan Tretmans

}
Input-Output
Refusal Relation
i  IOTS(LI,LU) :
i
ior
s 
 e  IOTS(LU,LI {}) . obs ( e, i )  obs (e, s )
 Straces( i )  Straces ( s )
   ( L  {  } )*: out ( i after )  out ( s after )
where:
out ( I )
= { !x  LU | i !x
, i I } 
out ( i after ) = { !x  LU {  } | i
© Ed Brinksma/Jan Tretmans
 !x
{|i 
}
i, i  S }
Implementation Relation
ioco
i  IOTS(LI,LU) :
i
ior
s    ( L  {  } )*: out ( i after )  out ( s after )
To allow under-specification :
i ioco s    Straces( s ) : out ( i after )  out ( s after )
© Ed Brinksma/Jan Tretmans
Implementation Relation
ioco
Correctness expressed by implementation relation ioco:
i ioco s =def   Straces (s) : out (i after )  out (s after )
Intuition:
i ioco-conforms to s, iff
• if i produces output x after trace ,
then s can produce x after 
• if i cannot produce any output after trace ,
then s cannot produce any output after  ( quiescence  )
© Ed Brinksma/Jan Tretmans
Implementation Relation
ioco
i ioco s =def   Straces (s) : out (i after )  out (s after )
out ( P )
=
Straces (s)
p

p
p after 
© Ed Brinksma/Jan Tretmans
{ !x  LU | p
 {  | p 
!x
, pP }
p, pP }
=
Ftraces (s)  ( L  { LU } )*
=
{   ( L  {  } )* | s
= p
=
LU
p
{ p’ | p

}
=  !x  LU  {} : p

p’ }
!x
Implementation Relation
ioco
i ioco s =def   Straces (s) : out (i after )  out (s after )
i

?kwart
?dub
!coffee
© Ed Brinksma/Jan Tretmans
= {}
out ( i after ?dub )
= { !coffee }
out ( i after ?dub.?dub )
= { !coffee }
out ( i after ?dub.!coffee) = {  }
?dub
?kwart
?dub
?kwart
out ( i after e )

out ( i after ?kwart )
= {}
out ( i after !coffee )
= 
out ( i after ?dub.!tea )
= 
out ( i after  )
= {}
Implementation Relation
ioco
i ioco s =def   Straces (s) : out (i after )  out (s after )
i
s
?dub
?dub
!coffee
!coffee
?dub
ioco
?dub
out (i after e)
= {}
out (s after e)
= {}
out (i after ?dub)
= { !coffee }
out (s after ?dub)
= { !coffee }
out (i after ?dub.!coffee) = {  }
© Ed Brinksma/Jan Tretmans
out (s after ?dub.!coffee) = {  }
Implementation Relation
ioco
i ioco s =def   Straces (s) : out (i after )  out (s after )
i
s
?dub
?dub
?dub
!tea
!coffee
?dub
out (i after ?dub) = { !coffee }
© Ed Brinksma/Jan Tretmans
!coffee
ioco
out (s after ?dub) = { !coffee, !tea }
Implementation Relation
ioco
i ioco s =def   Straces (s) : out (i after )  out (s after )
i
s
?dub
?dub
?dub
!coffee
!tea
?dub
?dub
out (i after ?dub) = { !coffee, !tea }
© Ed Brinksma/Jan Tretmans
!coffee
ioco

out (s after ?dub) = { !coffee}
Implementation Relation
ioco
i ioco s =def   Straces (s) : out (i after )  out (s after )
i
?dub
s
?dub
?dub
?dub
?dub
!tea
?dub
!coffee
?dub
out (i after ?dub) = { !coffee, !tea }
© Ed Brinksma/Jan Tretmans
!tea
!coffee
ioco
out (s after ?dub) = { !coffee, !tea}
Implementation Relation
ioco
i ioco s =def   Straces (s) : out (i after )  out (s after )
i
?dub
s
?kwart
?dub
?dub
?kwart
!coffee
out (i after ?dub)
!tea
= { !coffee }
out (i after ?kwart) = { !tea }
© Ed Brinksma/Jan Tretmans
!coffee
ioco
out (s after ?dub)
= { !coffee }
out (s after ?kwart) = 
But ?kwart  Straces ( s )
Implementation Relation
ioco
i ioco s =def   Straces (s) : out (i after )  out (s after )
i
?dub
s
?kwart
?dub
?kwart
?dub
?kwart
!coffee
!coffee
!tea
!tea
ioco
out (i after ?dub)
= { !coffee }
out (i after ?kwart) = { !tea }
© Ed Brinksma/Jan Tretmans
out (s after ?dub)
= { !coffee }
out (s after ?kwart) = { !tea }
Implementation Relation
ioco
i ioco s =def   Straces (s) : out (i after )  out (s after )
i
s
?kwart
?dub
?dub
?kwart
out (i after ?kwart) = {  }
© Ed Brinksma/Jan Tretmans
!coffee
!coffee
?dub
?kwart
?dub
?kwart
!tea
ioco
out (s after ?kwart) = { !tea }
Implementation Relation
ioco
i ioco s =def   Straces (s) : out (i after )  out (s after )
i
?dub
?dub
s
?dub
?dub
?dub
!coffee
?dub
out (i after ?dub) = { , !coffee }
© Ed Brinksma/Jan Tretmans
!coffee
ioco
out (s after ?dub) = { !coffee }
Implementation Relation
ioco
i ioco s =def   Straces (s) : out (i after )  out (s after )
i
?dub
?dub
s
?dub
?dub
?dub

!coffee
?dub
out (i after ?dub) = { , !coffee }
© Ed Brinksma/Jan Tretmans
!coffee
ioco
out (s after ?dub) = { , !coffee }
Implementation Relation
ioco
i ioco s =def   Straces (s) : out (i after )  out (s after )
i
?dub
i ioco s
?dub
s ioco i
?dub
!tea
?dub
s
!coffee
?dub
?dub
?dub
?dub
?dub
?dub
?dub
!tea
?dub
?dub
!tea
!coffee
?dub
?dub
out (i after ?dub.?dub) = out (s after ?dub.?dub) = { !tea, !coffee }
out (i after ?dub..?dub) = { !coffee }
© Ed Brinksma/Jan Tretmans
 out (s after ?dub..?dub) = { !tea, !coffee }
Implementation Relation
ioco
?kwart
?dub
?kwart
ioco
?dub
?dub
ioco
!coffee
!tea
?dub
?kwart
ioco
ioco
ioco
?kwart
!tea
© Ed Brinksma/Jan Tretmans
?dub
!coffee
ioco
!coffee
Implementation Relation
ioco
i ioco s =def   Straces (s) : out (i after )  out (s after )
equation solver for y2 =x :
implementation i
specification s
? x (x < 0)
? x (x < 0)
! x
! x
? x (x >= 0)
i ioco s
?y
© Ed Brinksma/Jan Tretmans
s ioco i
? x (x >= 0)
?y
! -x
Genealogy of ioco
Labelled Transition Systems
IOTS
(IOA, IOSM, IOLTS)
Canonical Tester
conf
Testing Equivalences
(Preorders)
Quiescent Trace Preorder
Refusal Equivalence
(Preorder)
Repetitive Quiescent
Trace Preorder
(Suspension Preorder)
ioconf
ioco
© Ed Brinksma/Jan Tretmans
Formal Testing with Transition
Systems
Test hypothesis :
s  LTS
ioco
der : LTS 
(TTS)
Ts  TTS
IUTIMPS . iIUT IOTS .
tTTS . exec(t,IUT) = obs(t,iIUT)
Soundness and exhaustivess proved:
iIOTS .
( tder(s) . t(obs(t,i)) = pass )
 i ioco s
pass
iIUT
IUT IOTS
IMPS
obs
: TTS
exec
:
TESTS
IOTS 
IMPS


(traces)
(OBS)
© Ed Brinksma/Jan Tretmans
traces
t:
(traces)
{fail,pass}
fail