Transcript Testing Telecoms Software with Quviq QuickCheck

Testing Telecoms Software
with
QuickCheck
Thomas Arts
John Hughes
Chalmers/ITU
Joakim Johansson
Ulf Wiger
Ericsson
Session Border Gateway
Monitors
signalling
traffic;
opens and
closes
pinholes
Opens
media pinholes for
calls in
progress
2x1G Ethernet
Media Gateway Controller Linux ”blade”
processor
2x10G Ethernet
Media Proxy
Hardware for
packet
forwarding
Session Border Gateway
Media Gateway Controller
ITU standard
protocol
H.248
(Megaco)
Media Proxy
150KLOC
Erlang
1.5MLOC C
A QuickCheck Property
prop_reverse() ->
?FORALL(Xs,list(int()),
lists:reverse(lists:reverse(Xs)) == Xs).
6> eqc:quickcheck(rev:prop_reverse()).
...........................................
...........................................
..............
OK, passed 100 tests
true
A Wrong Property
prop_reverse_wrong() ->
?FORALL(Xs,list(int()),
lists:reverse(Xs) == Xs).
10> eqc:quickcheck(rev:prop_reverse_wrong()).
.....................Failed! After 22 tests.
[-2,4,0]
Shrinking......(6 times)
[1,0]
false
Testing the Media Proxy Control
Software
Send
random
command
sequences
Check the
responses
QuickCheck
Media Proxy
Generate
random
messages in
a random
command
sequence
But not
completely
random!
Message Example: ASN.1
Description
• A media descriptor contains a list of
streams
MediaDescriptor
{
termStateDescr
streams
{
oneStream
multiStream
} OPTIONAL,
…
}
::= SEQUENCE
TerminationStateDescriptor OPTIONAL,
CHOICE
StreamParms,
SEQUENCE OF StreamDescriptor
…with IWD Restrictions
• A media descriptor contains a list of
streams
MediaDescriptor
{
termStateDescr
streams
{
oneStream
multiStream
} OPTIONAL,
…
}
::= SEQUENCE
TerminationStateDescriptor OPTIONAL,
CHOICE
StreamParms,
SEQUENCE OF StreamDescriptor
QuickCheck Generator for Erlang
Records
mediadescriptor(Streams) when Streams=/=[]->
{mediaDescriptor,
Generators can be
#'MediaDescriptor'{
embedded in any data
streams =
case Streams of
structure
[{Id,Mode}] ->
oneof([{oneStream,streamParms(Mode)},
{multiStream,[stream(Id,Mode)]}]);
_ ->
{multiStream,
[stream(I,M) || {I,M}<-Streams]}
end}}.
Message construction
Logic from the IWD
stream(I,Mode) ->
QuickCheck
#'StreamDescriptor'{
streamID = I, streamParms = streamParms(Mode)}.
Local Control Descriptors
Add Request
Desc.
Desc.
Desc.
Properties,
Media
Package
M
Comment
M
Stream
Local
control
mode
M
Multiple Stream descriptors can be
included.
O
LocalControl will be included in all
cases except when no media
(m-line) is defined in the remote
SDP.
O
The default value of the mode property is
“Inactive”, the property is not
mandatory if the wanted value is
“Inactive”.
StreamParms
::= SEQUENCE
{
localControlDescriptor LocalControlDescriptor OPTIONAL,
localDescriptor
LocalRemoteDescriptor OPTIONAL,
remoteDescriptor LocalRemoteDescriptor OPTIONAL,
…,
statisticsDescriptor StatisticsDescriptor OPTIONAL
}
Two Cases: With and Without
Remote Media
Random
boolean with
high probability
streamParms(Mode) ->
?LET(RemoteMediaDefined, probably(),
if RemoteMediaDefined ->
#'StreamParms'{
localControlDescriptor =
localControl(Mode),
localDescriptor =
localDescriptor(RemoteMediaDefined),
remoteDescriptor =
remoteDescriptor(RemoteMediaDefined)};
true ->
……
end).
Megaco Commands
Context
Context
Megaco Commands
Context
Context
Context
Termination
Add
Returns
Context ID &
Termination ID
Megaco Commands
Context
Uses
context ID
Add
Context
Termination
Termination
Context
Megaco Commands
Context
Context
Context
Termination
Termination
Stream
Stream
Modify
Uses
termination ID
Megaco Commands
Context
Subtract
Context
Termination
Termination
Stream
Stream
Context
Megaco Commands
Context
Context
Context
Termination
Stream
Stream
Subtract
Megaco Commands
Context
Context
Symbolic Test Cases
[{set,1,…,send_add,
[?choose_cxt_id,Streams,Req]},
{assert,…,check_add,[{var,1}],
{set,2,…,send_add,
[{call,…,get_reply_cxt,[{var,1}]},
Streams2,Req2]},
{assert,…,check_add,[{var,2}]}]
• Generated wrt an abstract state machine
tracking contexts, terminations & streams
Encode/Decode Error
• Recall the stream parameters…
StreamParms
::= SEQUENCE
{
localControlDescriptor LocalControlDescriptor OPTIONAL,
localDescriptor
LocalRemoteDescriptor OPTIONAL,
remoteDescriptor LocalRemoteDescriptor OPTIONAL,
…,
statisticsDescriptor StatisticsDescriptor OPTIONAL
}
Add/Modify Error
Add
Modify
Add/Subtract Error
Add
Subtract
• There one day, gone the
next…
Add/Add/Modify Error
Add
Add
Modify
• If the terminations have
different numbers of
streams
Add/Add/Sub/Add… Error
Add
Add
Add
Subtract
Subtract
Add
Subtract
• Extracted
from a test
case 160
commands
long