Transcript GENERALLY ACCESSIBLE Model-Based Testing Forces and Solutions Prof. Walter Kriha, Hochschule der Medien Stuttgart, Computer Science and Media Faculty September 16, 2005

GENERALLY ACCESSIBLE
Model-Based Testing
Forces and Solutions
Prof. Walter Kriha, Hochschule der
Medien Stuttgart,
Computer Science and Media
Faculty
September 16, 2005
SECTION 1
Problem View
Example: Internet Service
When „search“ returns passwords...
User
client
Inter
net
Reverse
Proxy
CSIv
2
Authent
Author.
Identity
Credent.
Server
Server
Server
Vault.
App.
Server
CSIv
2
App.
Server
WS-S
CSIv
2
Registry
Host
WS-S
External
App.
Domain
App.
TTP
Server
Bridge
Server
(TTP)
Other Company
Deploying an application into this environment can take month after month of
laborious testing. But how can you be sure that core security concepts (like
trust zones, end-to-end security, secrecy etc.) are met and maintained by the
software? Automation of tests is a key requirement! Execution of tests must be
fully traced.
2
Challenges
Technical and Social
• multi-tier architectures
• complex infrastructures (J2EE/.NET)
• long deployment chains (Dev.Test/Mit/AIT/PTE/Production)
• Cross-cutting concerns (security, performance)
• Multiple technologies (descriptive, imperative) mixed
• Social factors (roles, skill sets, limitations, change)
It will take advanced tools for developers to perform continuous and
early testing of those complex systems
3
Example: Browser Bugs
An endless story?
Same
Origin
Concept
site A
site B
A
B
Script Engine
Every new feature in a browser (frames, pop-up windows, tabs, bookmarks
etc.) seems to violate this simple principle of protection. Take a look at the
mozilla security buglist. Why is that so?
4
Challenges
Concepts and Implementations
• The Code does not show „same origin“
• Browser models do not contain the concept
• How can we test for such high-level concepts?
Is it possible to capture advanced security concepts in models and use
them for (automatic) testing? Another example that needs much better
testing is the implementation of dynamic Role-based-access-control
(RBAC) systems which rely on static data, rules and environment values
to reach a verdict.
5
SECTION 2
Testing Approaches
Overview
From no concept to a representation of test concepts
I wonder what
this change will
do?
No test concept
textual input,
manual tests,
automated
reporting: What
do they want me
to test?
Another
application
change, need to
fix test scripts
too
Scripts
model and test
language
supported by
tools. The model
itself contains
concepts
through profiles.
Logbook
Model based
The concepts need to be caught in models and used to drive test
engines, simulators or generate code against the production software
7
Model-Based Testing
Architecture
Test
Tooling
Test
Profiles
libraries
Test
Test
Specification
Generator
SUT
behavioral
specification
test models
Test Model
model
Interpreter
MSC
Abstract
Activity
State
Diagram
chart
Control
Data
Flow
Flow
Concrete
This is only one example of many different ways to use MBT. With UML
2.0 activity diagrams are an effective way to represent „unit-of-work“ like
concepts which appear naturally in testing.
8
Is it Better?
Where are the benefits?
• Just building the test model uncovers lots of bugs in the requirements
(model)
• Number and quality of bugs found: A strong tendency towards better
results with MDT
• Manually deriving tests from a model still benefits from the model.
According to Pretschner et.al. testing without a model gives the worst results.
9
Who can do it?
EAI example: How to avoid the textual step
Business creates
Transformation
Rules with
specialized UML
editor
Create Meta-data
and Actions for
Interpreter/Server
Execute
instructions in EAI
server
Server (UML VM)
Tool
A successful example how model driven development enables even endusers to create precise specifications of data transformations. Shouldn't this
be possible with testing as well?
10
Technologies and Tool Support
UML
• In non-technical environments UML is the prevailing modeling language
• Behavioral specifications are expressed as state charts or activity diagrams
• Tools should provide „canned know-how“ for tests of non-functional
requirements
• Model checking and executable UML are possible options for the future. They
bridge the gap between abstract models and implementations by bringing more
concreteness (abstract syntax of actions e.g.) into the models.
For an example of the use of UML for model checking take a look at UMLsec.
11
MDD vs. MBT
Different Benefits
• Different abstraction levels: MDD needs to express more general concepts and
less implementation allow different implementations
• Requirement models in MDD need to be generalized to create flexible systems.
In MBT requirement models need to be made more concrete.
• MBT can use concrete implementation know-how from the SUT
• MBT tools should be able to re-use models from MDD
• MBT tools need to cater to a different class of users
Model-Based Testing is not the same as Model-Driven Development.
12
The Future
Exept and HDM
• Constraint modeling and transformation into tests
• Process modeling, semantics
• Representation of non-functional requirements
Based on a common understanding of concept based learning and
development HDM and Exept will explore the above topics in projects and
courses at HDM.
13
Resources
1.
W.Prenninger, A.Pretschner, Abstractions for Model-Based Testing http://www4.informatik.tumuenchen.de/~prenning/publications/tacos04.pdf (shows that developement and test models
are different. Good introduction to MBT)
2.
Pretschner, Prenninger, Baumgartner, Stauner, One Evaluation of Model-Based Testing and
its Automation. http://www.inf.ethz.ch/personal/pretscha/papers/icse05.pdf (empirical results
on a comparison of MBT vs. manual testing and mixed forms)
3.
M.Friske, H.Schlingloff, Von Use Cases zu Test Cases: Eine systematische Vorgehensweise
http://www.informatik.hu-berlin.de/~hs/Publikationen/2004_MBEES_Schlingloff-Friske_VonUse-Cases-zu-Test-Cases.pdf (shows requirements translation into activity diagrams)
4.
H.Störrle, LMU Munich, Towards a formal Semantics of UML 2.0 Activities
http://www.pst.informatik.uni-muenchen.de/personen/stoerrle/V/AD-11-Limits.pdf (a
denotational semantics for UML Activities based on extended Petri Nets)
5.
Workgroups: GI-FG TAV (Test and Verification of Software)
6.
Workgroups: Dagstuhl Seminar 2004: MBT for reactive systems
http://www.it.uu.se/research/project/motres/subjects.html
7.
Hartmann, Nagin, The AGEDIS Tools for Model Based Testing
8.
Bernhard Merkle, Designermodelle, IX 2005/5. Good Introduction to Model-Driven Architecture
and Development Tools and Processes.
9.
Jan Jürjens, Secure Systems Development with UML, Springer 2005. Uses a UML Profile for
security to validate security protocols. Uses Model Checkers and shows how to test SAP3
permission settings against a model.
10.
T.Stahl, M. Völter, Modellgetriebene Software-Entwicklung. Dpunkt Verlag 2005. Covers metamodelling, Domain Languages, split-level development and generative patterns.
14