Model-Based Testing - Twin Cities Quality Assurance Association
Download
Report
Transcript Model-Based Testing - Twin Cities Quality Assurance Association
Model-based Testing
Effective Test Case Design
And Test Automation
Twin Cities Quality Assurance
Association (TCQAA)
September 12, 2013
Yaxiong Lin
TestOptimal, LLC
Agenda
What is Model-Based Testing (MBT)
MBT in Action (demo)
Benefits of MBT
MBT Process
MBT Use Cases
Testptimal LLC
Agenda
What is Model-Based Testing (MBT)
MBT in Action (demo)
Benefits of MBT
MBT Process
MBT Use Cases
Testptimal LLC
Business Impacts of Software Defects
• U.S. alone: $60 billion per year (NIST 2002)
Source: Crosstalk,
the Journal of Defense Software Engineering
Testptimal LLC
Where Defects are Found
Source Unknown
Testptimal LLC
Costs of Fixing Software Defects
The Real Cost of Software Defects, LKP Consulting Group
http://www.lkpgroup.com/Cost%20of%20Software%20Defects.pdf
Testptimal LLC
Software Testing Challenges
Requirement/design defect impact: 64%
Non-developer testing catches 69% of
defects
Costs 100x to fix defect in production
Gain higher confidence
better test coverage
requirement traceability
Testptimal LLC
What is Model-Based Testing?
“Model-Based Testing is the Automation
of Test Design of Black-box Tests”
Practical Model-base Testing”, Mark Utting and Bruno Legeard
Testptimal LLC
What is Model-Based Testing?
“Model-Based Testing is the Automation
of Test Design of Black-box Tests”
Practical Model-base Testing”, Mark Utting and Bruno Legeard
Model-based testing is application of Model based
design for designing and optionally also executing
artifacts to perform software testing. Models can be
used to represent the desired behavior of a System
Under Test (SUT), or to represent testing strategies
and a test environment.
wikipedia
Testptimal LLC
Why Model?
Visual representation of complex systems
Ask What-If to understand cause-effect
Important role in the technology
advancement and software development
Software testing is a complex task
Testptimal LLC
MBT Modeling Notations
Finite State Machine (FSM)
State Diagram (UML)
Extended Finite State Machine (EFSM)
Message Sequence Chart (MSC)
Sequence Diagram (UML)
Control Flow Graph (CFG)
Activity Diagram (UML)
Event Flow Diagram
UML Testing Profile – UTP
…
Testptimal LLC
Testing Vending Machine
Drink for $1
Accept Quarter and HalfDollar coins
Cancel any time for refund
Testptimal LLC
Vending Machine Test Cases
Quarter + Quarter + Quarter + Quarter => Vend
Quarter + Quarter + HalfDollar => Vend
Quarter + HalfDollar + Quarter => Vend
Quarter + HalfDollar + HalfDollar => Vend
HalfDollar + Quarter + Quarter => Vend
HalfDollar + HalfDollar => Vend
HalfDollar + Quarter + HalfDollar => Vend
Quarter + Cancel => Refund
…
Testptimal LLC
Example Model
Testptimal LLC
Generated Test Sequence
Testptimal LLC
Agenda
What is Model-Based Testing (MBT)
MBT in Action (demo)
Benefits of MBT
MBT Process
MBT Use Cases
Testptimal LLC
Types of MBT
Offline Testing
Online Testing
Testptimal LLC
Demo:
Test Case Generation for Manual Testing
or Offline Testing
Testptimal LLC
Demo:
Login Test Automation – Online Testing
Testptimal LLC
Demo:
WebStore Test Automation – Online
Testing
Testptimal LLC
Testing Web App
Testptimal LLC
WebStore MBT Model
© 2012 Testptimal LLC
Generated Test Sequence
Testptimal LLC
Agenda
What is Model-Based Testing (MBT)
MBT in Action (demo)
Benefits of MBT
MBT Process
MBT Use Cases
Testptimal LLC
Recent MBT User Survey
by Robert Binder, System Verification Associates,2012
MBT reduced escaped bugs by 59%.
MBT reduced testing costs by 17%.
MBT reduced testing duration by 25%.
Testptimal LLC
Benefits of MBT
Catch requirement / design defects
Catch defects earlier
Reduce defects in fields
Improve efficiency (Agility)
handling changes
better communication on defects
Gain higher confidence
better test coverage
requirement traceability
Testptimal LLC
Agenda
What is Model-Based Testing (MBT)
MBT in Action (demo)
Benefits of MBT
MBT Process
MBT Use Cases
Testptimal LLC
MBT Iterative Development Process
Requirement
s
User Stories
Modeling
Analysis
MBT
Models
Execution
Test
Generation
Automation
Testptimal LLC
MBT Process Input/Output
Input
MBT Step
Output
Requirements
User Stories
Modeling
Behavior Models
Test Objectives
Test Generation
Test Strategies
Test Coverage
AUT/Spec
Automation
Automation Scripts
Environment
Configuration
Execution
Test Coverage
Defects, Perform. Stat
Execution Result
Analysis
Model Modification
Testptimal LLC
Model-Based Testing Process
http://journal.thedacs.com/issue/52/145
Department of Defense (DoD) Information Analysis Center (IAC)
Testptimal LLC
Agenda
What is Model-Based Testing (MBT)
MBT in Action (demo)
Benefits of MBT
MBT Process
MBT Use Cases (more demo)
Testptimal LLC
Applications of Model-Based Testing
Conformance Testing
Functional Testing
System Integration Testing
Security Vulnerability Testing
Testptimal LLC
MBT Use Cases (demo)
Finding Temporal Defects
Realistic Load Simulation
Performance
Load/Stress Testing
Session /Scenario Based Testing
Risk Based Testing
Quick bug fix verification
Testptimal LLC
Questions?
MBT Papers:
http://testoptimal.com/MbtReferences.html
Email: [email protected]
Testptimal LLC