Transcript Telematik - TTCN-3

Hochschule Osnabrück
University of Applied Sciences
Test-controlled Development
of Communication Services
Ralf Tönjes1, Marten Fischer1, Patrick Wacht2
Contact:
[email protected]
1
University of Applied Science
Osnabrück, Germany
2
University of Applied Science
Frankfurt, Germany
TTCN-3 User Conference 2011
Conference, June 7-9, 2011, Bled, Slovenia
7TTCN-3
– 9User
June
2011 - Bled, Slovenia
1
Prof. Dr.-Ing. Ralf Tönjes
Agenda
Hochschule Osnabrück
University of Applied Sciences
 Motivation
 Architecture
 Service Description
 Service Behaviour Model
 Test Code Generation
 Test Execution
TTCN-3 User Conference, June 7-9, 2011, Bled, Slovenia
2
Prof. Dr.-Ing. Ralf Tönjes
Motivation
Hochschule Osnabrück
University of Applied Sciences
 Testing of communication services is
becoming increasingly important
growing complexity of logic
short time to market desired
Error
 manual testing is time and cost consuming
 test automation required
 current service creation environments (SCE)
do not support test automation
TTCN-3 User Conference, June 7-9, 2011, Bled, Slovenia
3
Prof. Dr.-Ing. Ralf Tönjes
How can Test Automation be Realised?
Hochschule Osnabrück
University of Applied Sciences
 Model-Based Testing
 Model describes how a system should behave in response to an
action (behaviour model)
 Supply the action and see if the system responds as expected
is a partial description of
Model
System
are derived
from
can be run
against
Abstract
tests
Executable
tests
are abstract versions of
TTCN-3 User Conference, June 7-9, 2011, Bled, Slovenia
4
Prof. Dr.-Ing. Ralf Tönjes
Architecture
Hochschule Osnabrück
University of Applied Sciences
Test Creation Evironment
Modelling
Behaviour
Model
Test Case
Generator
ttcn3
TTCN-3
Test Cases
Modular Sub Finite
State Machines
Step 2
Step 3
Test Framework
Service
Description
---------
Step 1
TTCN-3 User Conference, June 7-9, 2011, Bled, Slovenia
Service
Creation
Environment
5
Step 4
Service
SIP
Application
Server
Prof. Dr.-Ing. Ralf Tönjes
Step1: Formal Recording of Service Description
Hochschule Osnabrück
University of Applied Sciences
5 Stages Procedure
 Adopted from telecom software development process OTES
(Objects Through Essential Services)
1. Short textual description of use cases
2. Identification of roles of communication participants
(e.g. browser, SIP-phone, Database)
3. Requirement specification
(including pre- and post-conditions, i.e. states, and
target action, i.e. messages sequence, for users)
4. Enhanced requirements specification
(including technical details, such as value ranges, names of
databases, tables and fields)
5. Identification of the communication interfaces
(5-tuple of source and destination IP-address, port and
protocol)
TTCN-3 User Conference, June 7-9, 2011, Bled, Slovenia
6
Prof. Dr.-Ing. Ralf Tönjes
Example: Click-to-Instant-Message (Click2IM)
Hochschule Osnabrück
University of Applied Sciences
Requirement
Click2InstantMessage_01
Stage 1:
Description
After request of web site, input of destination address and
text required. Inputs will be approved and the text will be
transfered to the softphone having the destination
address. Receipt of test will then be confirmed.
Stage 2:
Roles
• Web Browser
• SIP Softphone
Stage 3:
Preconditions
• Web site accessible
• Destination address filled in
• Text filled in
• Approved inputs
Target
Softphone accessible
Postcondition
• Softphone has received text
• Confirmation is shown on web site
Stage 4:
•….
TTCN-3 User Conference, June 7-9, 2011, Bled, Slovenia
7
Prof. Dr.-Ing. Ralf Tönjes
Step2: Creation of Service Behaviour Model
Hochschule Osnabrück
University of Applied Sciences
TTCN-3 requirements:
1.
message flow description  derived from FSM (test cases)
2.
test environment/SUT description  using connectivity
concept (ports, component, Test Adapter, …)
3.

test data (type definitions, templates)
Behaviour Model Editor with 3 specialised perspectives
Editors
FSM
Editor
TTCN-3 User Conference, June 7-9, 2011, Bled, Slovenia
Connectivity
Editor
8
Test Data
Editor
Prof. Dr.-Ing. Ralf Tönjes
Composition of FSM (Finite State Machine)
Hochschule Osnabrück
University of Applied Sciences
 sFSM (sub Finite State Machine)
 Reusable templates, which represent predefined message flows from standards
 internal transitions are fixed, external transitions used to realise composition
 usually protocol specific (e.g., SIP, TCP, HTTP)
 Composition of sFSM with Transaction User
sFSM-M2
sFSM-M1
TTCN-3 User Conference, June 7-9, 2011, Bled, Slovenia
9
Prof. Dr.-Ing. Ralf Tönjes
The FSM Connector: Transaction User (TU)
Hochschule Osnabrück
University of Applied Sciences
 interfaces of the sub finite state machines have the TU as sender or
receiver
 acts as mediator between possible client and server roles
 does not contain information about the service implementation
TTCN-3 User Conference, June 7-9, 2011, Bled, Slovenia
10
Prof. Dr.-Ing. Ralf Tönjes
Connectivity Concept:
Composition of SUT/Test Environment Description Hochschule Osnabrück
University of Applied Sciences
Element
Description
Compontent
test component; contains the server
and client specific protocol endpoints
(Ports)
Port
Interface between test environment
and SUT (e.g. SIP Port or HTTP Port)
Global Timer
Timer; usable in any state or as event
on transition; ensures a state is left in
or after a specific amount of time
Action
Reactions to an event; set on
transition; either send a message or
manipulate variable/timer
Guard
Additional conditions for transitions;
validates content of a message or
value of a variable
Primitive Variables
Simple variables; used in context with
Guards
TTCN-3 User Conference, June 7-9, 2011, Bled, Slovenia
12
Prof. Dr.-Ing. Ralf Tönjes
Connectivity Concept:
Mapping to TTCN-3
Hochschule Osnabrück
University of Applied Sciences
Connectivity Concept
Global
Timers
Event
Condition
Timer
Component
Component
Definition
Protocol
Timers
Adapter
Configuration
Ports
Port
Definition
M1: Response
M2: Request
Actions
TTCN-3 User Conference, June 7-9, 2011, Bled, Slovenia
TTCN-3
Type
Definition
Templates
13
Prof. Dr.-Ing. Ralf Tönjes
Test Data Editor
Hochschule Osnabrück
University of Applied Sciences
 create and edit message templates
 message structure reflected in tree view
 specific for one port and its protocol
Example
TTCN-3 User Conference, June 7-9, 2011, Bled, Slovenia
14
Prof. Dr.-Ing. Ralf Tönjes
Step 3: Abstract Test Case Generation
Hochschule Osnabrück
University of Applied Sciences
Behaviour Model
Editors
FSM Editor
Connectivity
Editor
(TTCN-3 ports,
timer, …)
Test Data
Editor
Abstract Test Case Generator
FSM
FSM
TTCN-3
&
PathCode
Connectivity
finder
Generator
Parser
TTCN-3
Behaviour Model Analyser
TTCN-3 User Conference, June 7-9, 2011, Bled, Slovenia
15
Prof. Dr.-Ing. Ralf Tönjes
Test Case Generation: FSM Pathfinder
Hochschule Osnabrück
University of Applied Sciences
Path
 set of states and transitions between start- and end-state of
FSM
Pathfinder
 finds all possible paths in FSM between start and end state



FSM considered as deterministic
branching = new test case
employs depth-first-search
TTCN-3 User Conference, June 7-9, 2011, Bled, Slovenia
16
Prof. Dr.-Ing. Ralf Tönjes
Test Case Generation: FSM Parser
Hochschule Osnabrück
University of Applied Sciences
Each path converted into test case by FSM Parser
 state converted into a TTCN-3 function body
 content of function specified by outgoing transition
 FSM written from the service’s point of view message based
actions must be inverted
s1
p1.msg/p1.msg2
s2
function s1_1() runs on c1 {
p1.send(msg);
alt {
p1.receive(msg2) -> …. { s2_1(); }
}
}
TTCN-3 User Conference, June 7-9, 2011, Bled, Slovenia
17
Prof. Dr.-Ing. Ralf Tönjes
Test Case Generation: Connectivity Parser
Hochschule Osnabrück
University of Applied Sciences
 generates component using parameter from Connectivity Editor
Component Definition
Parameter
Name of Component
Ports
Type of Port
Name of Port
own block in editor;
connected with comp.
Timer
Primitive Variables
automatically generated
Message Variables
for each message
template
+ system component (only ports and static name)
TTCN-3 User Conference, June 7-9, 2011, Bled, Slovenia
18
Prof. Dr.-Ing. Ralf Tönjes
Test Case Generation: Connectivity Parser
Hochschule Osnabrück
University of Applied Sciences
 generates port using parameter from Connectivity Editor
 generates configuration file for Test Adapter (parameter provided in Ports)
Test Adapter configuration file
Name of Port
Protocol (TCP/UDP)
Remote IP / Port
Role (Server/Client)
Local IP / Port
Parameter/Port
Port Definition
Parameter
Name
Type
reference in model
send/receivable
Message Types
TTCN-3 User Conference, June 7-9, 2011, Bled, Slovenia
19
Prof. Dr.-Ing. Ralf Tönjes
Test Case Generation: TTCN-3 Code Generator
Hochschule Osnabrück
University of Applied Sciences
 Writes the acquired / generated data as TTCN-3 code using
Velocity*
FSM Parser
Result
Velocity
Templates for
TTCN-3
Generation
Velocity
java
java
java
templ
templ
ate
templ
ate
ate
Generated Files
for Test
Environment
ttcn3
ttcn3
ttcn3
ta
.conf
* Java based Template Engine (http://velocity.apache.org)
TTCN-3 User Conference, June 7-9, 2011, Bled, Slovenia
20
Prof. Dr.-Ing. Ralf Tönjes
Step 4: Test Execution
Hochschule Osnabrück
University of Applied Sciences
 TTCN-3 code will be executed after the code generation has
been completed by the Test Case Generator
 Execution realised within a Test Environment
(e.g. TTWorkbench)
 Test results are stored in a database to measure the
development progress
 Iterative delta testing: only new or modified
test cases executed
 Test Framework for synchronisation of
test developer and service developer
TTCN-3 User Conference, June 7-9, 2011, Bled, Slovenia
21
Prof. Dr.-Ing. Ralf Tönjes
Conclusion
Hochschule Osnabrück
University of Applied Sciences
 Novelty: New approach eases the generation of test cases by
1. composing predefined finite state machines (FSM) for protocol
sequences to a behaviour model (BM) of a service.
2. From the BM a TTCN-3 test suite is automatically derived.
3. The new concept of Transaction User (TU) enables testing of value
added services
 Benefits: Integration of Service Creation and Testing
 Development of functional tests in parallel to service development 
ensuring automated TTCN-3 based testing of services
 Testing is integrated in a service creation environment allowing for
regression tests
 Re-Use by Others: General Applicability
 test generation can be used for any SCE
 generic configurable test adapter based on Java for TTCN-3 (e.g. for
TCP, UDP ,…)
 universal codec for generation of test patterns has been implemented
TTCN-3 User Conference, June 7-9, 2011, Bled, Slovenia
22
Prof. Dr.-Ing. Ralf Tönjes
References
Hochschule Osnabrück
University of Applied Sciences
1. P. Wacht et al.: A New Approach to Design Graphically
Functional Tests for Communication Services, IEEE NTMS,
Paris, 7-10 February 2011.
2. P. Wacht et. al.: ComGeneration: die Dienstebeschreibung als
Basis für automatisiertes Testen, VDE/ITG Fachtagung
Mobilkommunikation, Osnabrück, 18-19 May 2011.
3. A. Lehmann et al.: “TeamCom: A Service Creation Platform for
Next Generation Networks”, IEEE ICIW 2009, Venice/Mestre,
Italy, 24-28 May 2009.
TTCN-3 User Conference, June 7-9, 2011, Bled, Slovenia
23
Prof. Dr.-Ing. Ralf Tönjes