Transcript Improving the Design - Multimedia University

Software Testing
Strategies
“How do you organise your software tests?”
TCS2411 Software Engineering
1
Lecture Objectives
To describe the process of planning for
software testing
To understand the different levels of
testing that can be done
To describe the different approaches to
debugging software
TCS2411 Software Engineering
2
Software Testing
Strategy
The approach towards software testing
A set of tests that is planned and
conducted to ensure successful
construction of software
Guideline or “road map” of testing
activities - planning, design, execution,
data collection, evaluation
TCS2411 Software Engineering
3
Examples of Software
Testing Strategies
Testing begins at module level and works
“outward” towards integration of the entire
computer-based system
Different testing techniques at different
points in time
Testing is conducted by developer and an
independent test group
Testing and debugging are different
activities, but debugging is accommodated
TCS2411 Software Engineering
4
A Software Testing
Strategy
System
Engineering
System
Test
Requirements
Validation
Test
TCS2411 Software Engineering
Design
Code
Integration
Test
Unit
Test
5
Software Testing
Organization
Developer : unit testing
Ensure each functions performs according to its
design
Sometimes conduct integration testing
Independent testing group (ITG)
Report to SQA team
Developer and ITG work closely to ensure
thorough test is well conducted
Developer must be present during testing to
correct uncovered errors.
TCS2411 Software Engineering
6
Software Testing Steps
Testing
direction
requirements
High-order tests
Integration test
design
code
requirements
Unit test
design
code
Testing
direction
TCS2411 Software Engineering
7
Unit Testing
Verification of the smallest unit in
software - the module
White box oriented
Can be conducted in parallel for multiple
modules
Unit test cases designed based on source
code - examine likely causes for errors
Test environment includes driver and/or
stubs
TCS2411 Software Engineering
8
Unit Test Environment
Driver
Module to
be tested
Stub
TCS2411 Software Engineering
Stub
…..
9
Unit Testing
Considerations
Interface - tested to ensure that information
properly flows into and out of the unit
Local data structures - examined to ensure
that data stored temporarily maintains its
integrity in an algorithm’s execution
Boundary conditions - tested to ensure that
module operates properly at boundaries
TCS2411 Software Engineering
10
Unit Testing
Considerations
(Continued)
Independent paths - to ensure all
statements in a module executed at least
once
Error-handling paths
clear and correct messages/descriptions
correct processing of errors
not interrupted by system
sufficient information to locate cause of error
TCS2411 Software Engineering
11
Unit Test Environment
•Act as main program
•Accept test data
•Past to the component to
be tested
•Print results
Driver
Module to
be tested
RESULTS
Stub
Interface
Local data structures
Boundary conditions
Independent path
Error Handling path
Stub
•Replace module that are subordinate (called by)
component to be tested
•Use subordinates module interface
•Do minimal data manipulation
•Print verification entry
•Return control to the module
TCS2411 Software Engineering
Notes :
•Driver and stub are
software that must be
written.
•Both are not delivered
with final product
Test cases
12
Integration Testing
Systematic technique for constructing
software while testing to uncover errors
associated with interfacing
Take unit-tested modules and build the
program structure based on the design
Nonincremental (“big bang”) - all modules
combined in advanced  chaos!
Incremental integration - part by part,
e.g. top-down or bottom-up
TCS2411 Software Engineering
13
Top-Down Integration
Beginning with main module, modules are
integrated by moving downward through
the control hierarchy
Subordinate modules are added into the
structure either depth-first or breadth-first
Verifies major control or decision points
early in testing
Problem when lower level processes
needed to test upper levels
TCS2411 Software Engineering
14
Top-Down Integration
Diagram
M1
M2
M5
M3
M6
M4
M7
M8
TCS2411 Software Engineering
15
Top-Down Integration
Process
Beginning at main module, stubs are
replaced one at a time with actual modules
Tests are conducted as each module is
integrated
On the completion of each set of tests,
another stub is replaced with real module
Regression testing (conduct all or some of
previous tests) may be done to ensure no
new errors have been introduced
TCS2411 Software Engineering
16
Bottom-Up Integration
Begins construction and testing with
atomic modules (lowest level in hierarchy)
No need for stubs, all subordinate
modules tested and available
Test drivers required and can be shared
as integration move towards the higher
levels
TCS2411 Software Engineering
17
Bottom-Up Integration
Diagram
Mc
Ma
D1
Mb
D2
D3
Cluster 3
Cluster 1
Cluster 2
TCS2411 Software Engineering
18
Bottom-Up Integration
Process
Low-level modules are combined into
clusters
A driver is written to coordinate test case
input and output
The cluster is tested
Drivers are removed and clusters are
combined moving upward in the program
structure
TCS2411 Software Engineering
19
Regression Testing
New module or major changes in software.
Helps to ensure changes does not introduce
Unintended behavior
Additional errors
Re-execute the subset of all test cases
Most advised : Use automated capture-play back
tools
This tool enable the software engineer to capture test cases
and results for subsequent playback and comparison.
TCS2411 Software Engineering
20
Validation Testing
A series of black box tests that demonstrate
conformity with requirements
Test that software functions as expected by
customer
Test plan and details can be based on
software requirements specification
TCS2411 Software Engineering
21
Alpha Test
Conducted at developer’s site by customer
Software is used in a natural setting with
the developer “looking over the shoulder”
Errors and usage problems recorded,
correction can possibly be done quickly
Test conducted in a controlled
environment
TCS2411 Software Engineering
22
Beta Test
Conducted at one or more customer sites
by end user(s)
Developer generally not present
“Live” application of software in
uncontrolled environment
Customer records all problems and
reports them to developer at regular
intervals
Software modified based on problems
TCS2411 Software Engineering
23
reported before final release to customers
System Testing
A series of tests to fully exercise a
computer-based system
Verify that all system elements have been
properly integrated and perform allocated
functions
Software is one element to be integrated
Software engineer to anticipate potential
interfacing problems
TCS2411 Software Engineering
24
Recovery Testing
Force the software to fail in a variety of
ways
Verify that recovery is properly performed
If recovery is automatic, re-initialisation,
checkpointing mechanisms, data recovery,
and restart are evaluated for correctness
If recovery requires human intervention,
mean time to repair is evaluated
TCS2411 Software Engineering
25
Security Testing
Verify that protection mechanisms built
into the system will protect it from
improper penetration
Tester plays the role(s) of individual who
desires to penetrate the system
System designer to make penetration cost
greater than the value of information
obtained
TCS2411 Software Engineering
26
Stress & Performance
Testing
Stress Testing
Execute system in a manner that demands
resources in abnormal quantity, frequency or
volume
Performance Testing
Test run-time performance of software within
context of an integrated system
TCS2411 Software Engineering
27
Debugging
To find and correct the cause of a
software error
Attempt to match symptom to cause
Bug characteristics:
Symptom and cause may be geographically
remote
Symptom disappear (temporarily) when
another error is corrected
TCS2411 Software Engineering
28
Debugging
(Continued)
Symptom caused by nonerrors
Symptom caused by human errors
Symptom a result of timing problems
Difficult to accurately reproduce input
conditions
Symptom may be intermittent
Symptom due to causes that are distributed
across a number of tasks on different
processors
TCS2411 Software Engineering
29
Debugging Process
Execution
Of cases
Test
Cases
Results
Additional
test
Suspected
causes
Regression
test
Corrections
TCS2411 Software Engineering
Identified
causes
Debugging
30
Debugging Approaches
Brute force
“let the computer find the error”
memory dumps, run-time traces, lots of
WRITE statements  clues to error
Backtracking
trace source code backwards from error
Cause elimination
induction/deduction, “binary partitioning”
isolate potential causes, test each one
TCS2411 Software Engineering
31
Debugging Aids
Debugging Tools
Debugging compiler
Dynamic debugging aids (“tracers”)
Automatic test case generators
Memory dumps
Cross-reference maps
Good design documentation
Clear source code
Other people!
TCS2411 Software Engineering
32
References
“Software Engineering: A Practitioner’s
Approach” 5th Ed. by Roger S. Pressman,
Mc-Graw-Hill, 2001
“Software Engineering” by Ian Sommerville,
Addison-Wesley, 2001
TCS2411 Software Engineering
33