Transcript Using Software Architecture For Code Testing

Using Software Architecture For
Code Testing
Written By:
Henry Muccini Antonio Bertolino
Paola Inverardi
Presented By: yong wang
1
Introduction
• SA is emerging as a promising approach as it is
used to design and analyze complex
distributed system.
• Using suitable abstraction it makes large
application manageable.
• SA support formal modeling of a system with
topologic description and behavioral one in
term of component and connector
2
Introduction
• Crucial part of development process is testing. It
can be used to test that the dynamic behavior of
program by observing the execution of the program
on selected set of test case.
• Specification-based testing checks the
implementation Under Test(IUT) fulfils
specifications, to test the important properties
against which the ITU has to be tested.
• This paper explains how complex systems dynamic
behavior and specifications can be tested using SA.
3
Goal & approach
• For complex system, it is impossible to test all
class.
• The goal of approach is to provide a test
manager with a systematic method to extract
suitable test classes for higher level of testing
and to refine them into concrete test at the code
level.
• It is based on specification of SA dynamics which
is used to identify useful schemes of interaction
between system components and to select test
classes corresponding to relevant architectural
behaviors.
4
approach
• SA dynamics are modeled by Labeled
Transition System (LTS).
• The approach consist of deriving suitable LTS
abstractions, called Abstract LTS or ALTS. ALTS
offer specific views of SA dynamics by
concentrating on relevant features and
abstracting away from uninteresting ones.
• ALTS is a tool to understand what must be
tested. It represents a model of the system
5
Overview Of Approach to SA Testing
• Four Steps
1. Software Architect looks at SA dynamics from
different viewpoints, defines various obsfunctions over SA model each one highlighting a
specific perspective of interest for a test session.
2. Applying each obs-function to LTS, an Abstract
LTS (ALTS) can be generated which is reduced LTS
showing only interested behaviors with respect
to selected views.
6
Overview Of Approach to SA Testing
3. The software architect chooses a set of
important patterns and behavior (path over
ALTS) to be tested.
4. Finally these high level tests are passed to the
software tester, who actually has to run the
test and observe whether the current
implementation “conforms” to its
architectural model. deals with the execution
of these tests at the code level
7
Overview Of Approach to SA Testing
• Steps 1 to 3 presents a rigorous method to
extract architectural tests from an SA
specification
• Step 4 deals with the execution of these tests
at the code level
8
Definition:Labeled Transition System (LTS)
• LTS consist of (S, L, S0, Sf, T)
S = set of states
L = set of distinguished labels (actions)
denoting the LTS alphabet.
S0=initial state.
Sf= Final State.
T = Transition Relation labeled with
elements of L.
9
Case Study: The TRMCS
• The Teleservice and Remote Medical Case
System (TRMCS) is a system that provides
monitoring and assistance to disabled or
elderly people.
• A typical service is to send relevant
information to a local phone center so that
the family, along with those concerned with
medical or technical assistance, can be timely
notified of critical circumstances.
10
Case Study: The TRMCS(arch)
Box=components; arrow = connectors
Arrow label = data elements exchange
through channel
11
Case Study: The TRMCS
• Four components:
1. USER send either “alarm” or “check”
message to the router process. Then he/she
waits for acknowledgement from the router.
In this case two users can concurrently send
alarms and checks.
2. ROUTER waits for signals (check or alarm)
from the user. It forwards the alarm message
to Server and monitors the state of the User
via check message.
3. SERVER dispatches the help requests.
4. TIMER sends a clock signal at each time unit.
12
THE APPROACH
• 4 steps of our approach to SA-based
testing.
• STEP 1 Obs-Functions Definition
• STEP 2 Abstract LTS Derivation
• STEP 3 Test Selection
• STEP 4 Code-Level Testing
13
Step 1: Obs-functions defintion
Problem:
SA---> Labeled Transition System models SA dynamics.
LTS provides a global, monolithic description. Vast
information flattened into graphic. Extracting
relevant for validation is difficult task.
Aim:
provide software architect with a key to decipher the
LTS.
Common practice is to derive a set of simplified
models from global SA model which will provide
different system views.
14
Step 1: Obs-functions defintion
• Ideal
Allow for the formal derivation from the LTS of
reference models for testing, each representing a
relevant pattern of behavior.
Notation:
Test ing criterion: each of the selected patterns of
behavior. It divide LTS action L into two groups:
Relevant Interaction R and Non Relevant Interaction
NR. R U NR=L, R ∩ NR=null.
15
Step 1: Obs-functions defintion
• Using obs-function map the relevant interactions to
domain D and Non relevant interactions to τ
• Obs function can be considered as hiding operator
as it hides non relevant information
Obs: LL’, obs(r ∈R)=d ∈D, obs(n ∈NR)=τ, L’=DU τ
LTS paths
If p = I1I2…..In
: obs(p)=obs(I1I2…..In)=obs(I1)obs(I2)…..obs(In)
We therefore have SA testing criterion an obsfunction that maps the relevant LTS label.
16
STEP 2 Abstract LTS Derivation
• We need a small automaton from the LTS which still
expresses all high level behavior. ALTS.
• The ALTS is obtained by two transformations:
1. By relabeling each transition τ ∈ LTS in the LTS
transition according to obs-function.
2. By minimizing the resulting automaton with
respect to a selected equivalence relation.
Trace-equivalence: reduce much number of τ
transitions and nodes
Bisimulation equivalence: system evolves step by step
17
STEP 2 Abstract LTS Derivation
a)
The abstraction is applied over
this LTS with respect to selected
obs-function. (LTS)
b)
The trace equivalence
minimization function is applied.
(ObsLTS)
c)
The resulting ALTS is shown in Fig
(trace-based ALTS)
d)
This figure represent branch
minimization.
(bisimulation-based ALTS)
18
STEP 3: Test Selection
• Once the ALTS is derived we move to deriving a set
of complete paths appropriately covering the ALTS.
• Now ALTS have less paths the tester might think to
apply extensive coverage criteria on it.
• But ALTS corresponds to many possible LTS paths.
• Therefore less thorough coverage criteria seems
more practical
• Use McCabe’s test technique as it is a good
compromise between arc and path coverage in
the case of ALTS coverage
19
STEP 3: Test Selection
• When considering what to take as the specification
of an “architectural test” we have two options.
1. consider an ALTS complete path as the test
specification. In this case, the test is specified at a
more abstract level and tester instinctively focuses
on testing restricted set of interactions(use)
2. identify those LTS paths of which the selected ALTS
path is an abstraction. As LTS is more detailed than
ALTS in this case tester will have more information
about how to perform tests, but stricter
requirements i.e. tester does not have much
freedom in choosing the code level tests.
20
STEP 4 Code-Level Testing
• In this step tester use architectural paths from step
3 to test the implementation. Two problems are
encountered while doing this
1. TRACEABILITY- concerns “relating the abstract
values of the specification to concrete values of
the implementation”.
2. TEST execution– entails forcing the
Implementation Under Test (IUT) to execute the
specific sequence of events that has been selected.
This is a difficult as non-determinism is introduced.
21
STEP 4 Code-Level Testing
• Step 4.1
Map architectural components and actions into their
low-level implementation. analyze the system
implementation to understand how architectural
actions have been implemented in the code by
sequences of partially ordered method calls.
• Step 4.2
Take into account how an ordered sequence of
actions is implemented by a sequence of low-level
functions. Map actions into sequences of method
calls. Run sequentially and run concurrently.
22
STEP 4 Code-Level Testing
• Step 4.3
Run code and evaluate execution traces with respect
to the expected ones to analyze the source code
conformance with respect to the architectural
behavior.
Nondeterministic testing: repeat the launching of a
program run under some specified input conditions
several times until the sequence is observed.
23
Applying the Approach
• TRMCS components behavior were specified by
using Finite State Process(FSP)language
• The behavior of each component is modeled by
one or more FSP processes, and each process is
described by an LTS which is automatically
generated by Labeled Transition System Analyzer
(LTSA) tools
24
Applying the Approach
• By running the LTSA tool on the TRMCS FSP
specification
• 256 states of LTS
• The LTS labels of interest for analysis are:
– u[i].sendAlarm_To_Router: an alarm message is sent by the
user to the Router
– u[i].receiveAck_From_Router: the Router Acknowledgment
messages received by the user
– r.[i].sendAlarm_To_Server: the Router forwards the Alarm
message to the Server
– sa[0].sendAck_To_Router: the Server sends the Ack to the Router
– r.sendNoFunc_To_Server: the Router sends the NoFunction
message to the Server
25
Applying step 1
• Informal specification:
Test the flow of an Alarm message from the moment a User sends
it to the moment the User receives an acknowledgment.
• Corresponding obs-function: AlarmObs:
D = {Alarm1Dispatch, Alarm2Dispatch, Ack1Dispatch, Ack2Dispatch}
Obs (u.0.sendAlarm_To_Router) = Alarm1Dispatch: User1 issues an Alarm msg
Obs (u.0.receiveAck_From_Router) = Ack1Dispatch: User1 receives an Ack
Obs (u.1.sendAlarm_To_Router) = Alarm2Dispatch: User2 issues an Alarm msg
Obs (u.1.receiveAck_From_Router) = Ack2Dispatch: User2 receives an Ack
For any other rule r, obs (r) =τ
26
Applying step 1
All the interaction that involve this component
FR{a1,a2,no} messages the servers receivers from the Router
TR{ack1,ack2} message sent to the router
ServerRegression Obs-Function
D = {FRa1, FRa2, TRack1, TRack2, FRno}
Obs (r.[0].sendAlarm_To_Server) = FRa1: Alarm1 from Router to Server
Obs (r.[1].sendAlarm_To_Server) = FRa2: Alarm2 from Router to Server
Obs (sa[0].sendAck_To_Router) = TRack1: Ack1 from Server to Router
Obs (sa[1].sendAck_To_Router) = TRack2: Ack2 from Server to Router
Obs (r.sendNotFunc_To_Server) = FRno: NoFunctioning From Router to Server
For any other rule r, obs (r) =τ
27
Applying Step 3
• After the reduction
and minimization
algorithms
– Shaded circle
represents the initial
state
– After the Alarm is
issued, the system
reacted with one of
the possible actions:
• Elaborate the
Alarm and send
back an Ack,
(Alarm1Dispatch)
• Receive another
message from
another user,
(Alarm2Dispatc
h)
28
Applying Step 3
• List of test paths derived according to McCabe’s
coverage criterion follow corresponding test
sequences
29
Reference to the ServerRegression
Observation Associated with ALTS
30
With reference to the ServerRegression
criterion, Fig 4b is obtained
• 4b shows the interactions involving the Server
component
• Receives Alarm1 (FRa1) or Alarm2 (FRa2) from the
Router
• Then, receive other Alarms or send back the Ack (Track)
• It can also receive the NoFunc message from the
Router (FRno)
• Parts 1b, 3b, 7b, and 8b are verified that the NoFunc
message (FRno in fig. 4b) can b received by the Server
31
Applying Step 4.1:
• Test sequence generated from AlarmObs ALTS has
3 test components (User1, User2, and Router)
• Two Architectural events are
– AlarmDispatch and AckDispatch
– Understand how the source code implements these
architectural actions
• AlarmDispatch – represents a system input
– E.g. user pushes the Alarm button of the device
– The device will start by analyzing the actions associating with the
pushing the alarm button
– Then, the Alarm functionality implements
32
• AckDispatch
– The Alarm message is written into a socket
– The Router periodically reads the socket for new
messages
– The Ack function is implemented
– The Router writes into a socket and the user waits
on a defined socket port
33
• Objects and methods implements the Alarm
– Fig. 5a: the User object creates the User Object
• If_1 condition is verified
• Afterward, (if_2), User presses the Alarm button
• User then creates a ClientConnection
• SendNMessages (SNM) objects
• User calls the send() method of the SNM object
• It then tries to create the socket
• If the socket is created (if_3), SNM finally writes the
Alarm message into the socket
34
From the Router side
– Router graphical interface object (xRouter) creates a new
instance of the MasterRouter object
– In turn creates the ServerConnection object
– Tries to create a ServerSocket
– A process is then activated
– Continuously checks the ServerSocket port
– A new message is written in the socket (if_b) the socket is
read and the Alarm is received
35
Applying step 4.2
• Analyze how an architectural test(sequence of
actions) is implemented by the code.
• TS3a:Alarm1Dispatch.Ack1Dispatch.Alarm2Dispat
ch.Ack2Dispatch. Alarm1Dispatch happens before
Ack1Dispatch.
• Code level two option:
1. Code-level sequences implements
AlarmiDispatch and AckiDispatch run
sequentially.
2. Low-level scenarios can be run concurrently.
Method call may interleave.
36
Applying step4.3
• Execute the test sequence and verify the system
implementation behaves as described by our SA test.
• According to TRMCS implementation P, build an
instrumented version P’ to store information about the
execution of SM(synchronization method). Introduce
the “synchronization Method” file. when an SM is
called, a print operation is performed in this file which
capture the following information:<SM ID, User ID, SM
instance, time>
• Running P’ several times with difference input order,
Get file report. Each alarm operation is always
immediately followed by the relative acknowledgment.
37
Some Considerations
A. Difficulty of Application
Step 1. Obs-functions: is an empirical task based on a SA’s
experience
Tasks could be made easier through classification of
observations
By assigning semantics to the messages exchanges
Step 2. Deriving ALTS: ALTS generation can be automated
Step 3. Test Classes Selection: coverage criteria like
McCabe’s test technique could be automated easily
– extracting only paths that expose different
behaviors
38
• Step 4: Code-level Testing:
Testers used to generate test cases
Used to trace high-level info into code-level tests
4 Issues
need to understanding which classes and methods that
implement an architectural functionality
More than one sequence can implement the desired behavior
• This can be handled by requiring the intervention of the
tester who manually evaluates each point of decision
• And, performs a reasoned choice regarding which selection
is most relevant for testing
• Interactions are considered in SA description, not
computations
• Expected behaviors are described only
• But, the implementation has to handle the exceptional
behaviors
39
B. Approach Generality
• Approaches are put into practice as well as
independently
• ADL is selected to describe the SA dynamics
through LTS
• Coverage criteria is selected to extract a
limited number of ALTS paths
• Implementation is used following the OO
paradigm
40
Constraint:
• SA dynamics is described via an LTS
conforming the given definition
• Different ADLs can be used to describe
the SA
• Different ALTS coverage criteria can be
applied
• Different implementation languages and
programming paradigms can be used
41
Related Work
Approach which is presented makes use of
the following concepts.
Testing concurrent and real time systems
Conformance based testing
Tracing information
Architectural testing
42
Related Work Cont
• Most of the work has been done on Testing
Concurrent and Real Time, both are specification
and implementation based with focus on unit
testing.
• Objective was to derive test plans for integration
testing using the LTS , abstraction and event
sequences used in different way.
• Approach to defining ALTS path for high-level test
classes is same as Carver and Tai ( Syn-sequence)
but constraint to specification based testing of
concurrent programs.
43
Related Work Cont.
• ALTS path for SA restricts the event of a
concurrent program when test are selected, the
main difference is in the distance of abstraction
level of the reference model from the
implementation.
• Selection of a test suit based on specification of
implementation under test, and which is
assumed to be given in form of Finite State
Machine (FSM)
• Transition tour, W-method, DS-method, UIO
method and UIO methods were proposed
44
Related Work Cont.
• TAG (tan et al) test generation tool based on FSM
and generated by given specifications but the
draw backs of this tool were due to use of FSM
with limited number of states and impossibility of
ensuring complete coverage.
• Software system using statecharts: natural
language scenarios were designed through
statechart explaining information on test case
generation, thus enabling test cases generation
through path traversal.
45
Related Work Cont.
• Bogdanov presented a method to extract test cases
from state charts to prove the implementation is
behaviorally equivalent to design.
• Das-Boot tool were also presented to generate test suit
for OO (object oriented) systems through statechart.
• Studies have carried out by Fujiwara, Bochmann and
others to generate automatically test suits to test the
conformance of an implemented system.
• Petrenko uses FSM to drive a conformance testing,
where system specification (S) its implementation (I)
are formalized using FSMs to define “Conformance
relations” (equivalence, quasi-equivalence, reduction)
and Wp-method is used to generate test sequences.
46
Related Work Cont.
• Tretmans presented LTS-based conformance testing,
specification (S) using LTSs, implementation (I) using
LTSs or IOTS (Input/output Transition System) and
tests (T) were formalized using LTSs: Given S and
possible implementation I.
IMP (Implementation relations) to correlate S with I
where I conforms to S iff I is “imp”with respect to S.
• Fernandez et al. formalize the specification,
implementations and test purpose by using IOLTS
(input/output LTS). Automatic, on-the-fly
generations of test cases were projected in form of
TGV (test generation and verification) environment.
47
Related Work Cont.
• They also used the SA-derived LTS as reference model
but in different way. Implementation relations (conf,
ioconf, ioco) were also defined and test whether the
implementation is correct with respect to specification,
based on the models described earlier.
• Egyed proposed the “Tracing Information” where
showed the way to detect traceability between
software systems and their models, including some
techniques.
• Work has also been done in bridging the gap between
software architectures and its requirements.
48
Related Work Cont.
• The Architecture Testing has become more
interesting and the authors have defined six criteria
for architecture-based testing, adapting and
specification based approaches.
• They also analyzed the advantage of SA-level testing
for reuse test and test further functional properties.
• Harrold presented the approaches for using software
architecture for effective regression testing.
• She also proposed the architecture-based
integration testing approach, which includes
architecture traceability, simulation and slicing.
49
conclusion
• The research has been carried out on SA
philosophy, methods and tools can be useful
within software development process
• Mainly targeted the using of SA descriptions to
improve conformance testing of a large system.
• This literature was heavily relied upon the
Software Architecture and Specification based
testing.
• Includes tools for design and analysis but very
little had been said about SA- based testing.
50
conclusion
• There are many problems exists due to the
practicality of using SA models and tools despite
the claim that SA must play a role throughout the
software life cycle.
• Relationship between high level architecture tests
cases and the implementation under test may be
very difficult and still there is no complete
solution exists.
• In specification based testing, methods and tools
would be same but there are two main
differences between proposed approach and
existing ones.
51
conclusion
• The SA explanation confined to SA-relevant
behaviors only and not focusing on system functions.
This approach does not test the system completely
as in traditional specification based testing. The
proposed approach concentrates on integration
testing stages.
• The abstraction level of the referenced model and its
relative distance is very high from the
implementation under test in SA based testing
approach.
• While in existing approach it is assumed to be low.
The high abstraction level is a basic assumption of
this study. Where as the traditional approaches have
a close distance between the reference model and
implementation
52
conclusion
• Extract is being presented from the practical
insight gained while performing SA based
testing on the TRMC case study.
• It is believed that in the most cases problems
faced and the solutions found can easily be
generalized to any SA environment, although
the experience reported may be specific to the
case study.
53
future
Work
• It is considered that the approach presented here as a
initial contribution that might not apply directly in
practice, for instance it might not have global
architecture model because of architecture may be
described through complex models.
• Secondly architecture model may be incomplete, which
means that some component behaviors are unknown
or components are not completely specified.
• Studies are currently investigating the possibilities of
generating abstract observation and test cases directly
from partial architecture models.
54
Future Work Cont.
• Secondly how to further stricken the assumptions on
SA based development process, it is showed in
reference that how the testing process can be made
completely systematic by refining general approach to
more specific scenario.
• By adopting style architecture and related framework,
they were able to handle the traceability among SA.
• Adapting the category partition method solved the
code problems.
• And execution over the identified test cases problems
through Argus-1 tool. This approach has been applied
to a case study, which is generating interesting results
55