Transcript Improving the Design - Multimedia University

Software Testing
Techniques
“How to find bugs in the software?”
TCS2411 Software Engineering
1
Lecture Objectives
To understand the role of testing in
ensuring software quality
To discuss issues relevant to software
testing
To describe the different techniques used
for testing software
TCS2411 Software Engineering
2
Validation & Verification
Validation
“Are we building the product right?”
Ensure software meets customer’s needs
Verification
“Are we building the right product?”
Ensure software meet specification (errorfree)
TCS2411 Software Engineering
3
Testing Terminology
 Failure
Incorrect or unexpected output
Symptom of at fault
 Fault
Invalid execution state
Symptom of an error
May or may not produce failure
 Error
Defect or anomaly in source code
Commonly referred as bug
May or may not produce fault
TCS2411 Software Engineering
4
What is Software
Testing
Software Testing is a process of executing a
program with the intent of finding an error.
A good test case is one that has a high
probability of finding an as yet undiscovered
error
A successful test is one that uncovers an as
yet undiscovered error
TCS2411 Software Engineering
5
Testing Objective
To design tests that systematically
uncover different classes of errors and to
do so with a minimum amount of time
and effort
TCS2411 Software Engineering
6
Why We Need Software
Testing?
Reveal faults/failures/errors
Locate faults/failures/errors
Show system correctness
Improved confidence that system performs as
specified (verification)
Improved confidence that system performs as
desired (validation)
Indicator of system reliability and system quality
TCS2411 Software Engineering
7
Importance of Testing
Critical Element of Software Quality
Assurance
Well-planned and thorough testing are
important in reducing the high cost of
software failure
Can take up to 40% of development effort
In systems that require high reliability,
testing can be 3 to 5 times of all the other
steps combined
TCS2411 Software Engineering
8
Testing Principles
All test should be traceable to customer
requirements.
Tests should be planned before testing begins.
Testing should begin with individual components
and move towards to integrated cluster of
components.
The most effective testing should be conducted
by an independent third party.
TCS2411 Software Engineering
9
Pre-requisites For Good
Testing
Adequate staff
Adequate support facilities (tools)
Adequate time
TCS2411 Software Engineering
10
Who Should Test The
Software?
Developer (individual units)
Independent test group (ITG)
removes conflict of interest
reports to SQA team
TCS2411 Software Engineering
11
Successful Testing
Uncover errors in the software
Demonstrates that the software functions
appear to be working according to
specification
Data collected as testing is conducted
provide a good indication of software
reliability and software quality
TCS2411 Software Engineering
12
Testing Guidelines
Testing a system’s capabilities is more
important than testing its components
Testing old capabilities is more important
than testing new capabilities
Testing typical situations is more
important than testing boundary value
cases
TCS2411 Software Engineering
13
Test Cases
Test case : unit of testing activity
Test cases have 3 parts :Goal
Aspect of the system being tested
Input and system state
Data provided to the system under stated condition
Expected behavior
The output or action the system should take according to its
requirements
TCS2411 Software Engineering
14
Test Cases (Cont)
Test cases are derived during all phase of
development cycle
Determine the expected result before
running the test case
Record test cases and its results
Allocate
time
TCS2411 Software Engineering
money
people
resources
15
Type Of Test Cases
Test cases are derived for
Valid and expected input
Invalid and unexpected input
Test if the system does less than specified
requirement
Test if the system does more than specified
requirement
TCS2411 Software Engineering
16
White Box Testing
Derived from knowledge of program’s
structure & implementation
Structural testing - analyse code & use
knowledge of the structure of a
component to derive test data
Logical paths are tested by providing test
cases that exercise specific sets of
conditions and/or loops
TCS2411 Software Engineering
17
White Box Testing
(Continued)
Thorough white box testing would lead to
“100 percent correct programs”
Exhaustive testing are impractical - too
many tests!
A limited number of logical paths can be
selected and exercised
Important data structures can be probed
for validity
TCS2411 Software Engineering
18
White Box Test Cases
Guarantee that all independent paths
have been exercised at least once
Exercise all logical decisions on their true
and false sides
Execute all loops at their boundaries and
within their operational bounds
Exercise internal data structures to ensure
their validity
TCS2411 Software Engineering
19
White Box Testing
Techniques
Basis path testing
Flow graph notation
Cyclomatic complexity
Derived test cases
Graph metrics
Control structure testing
Condition testing
Data Flow testing
Loop testing
TCS2411 Software Engineering
20
Flow Graph Notation
1
Edge
Node
1
2,3
2
6
R2
3
6
4
7
R3
4,5
R1
8
Region
7
8
5
9
11
10
9
10
R4
11
TCS2411 Software Engineering
21
Cyclomatic Complexity
Provide quantitative measure for program logical
complexity.
Defined number of independent path
Any path that introduce one ser of processing
statements or new condition
Eg :Path
Path
Path
Path
1
2
3
4
:
:
:
:
1-11
1-2-3-4-5-10-1-11
1-2-3-6-8-9-10-1-11
1-2-3-6-7-9-10-1-11
TCS2411 Software Engineering
22
How Is Cyclomatic
Complexity Computed?
1. Number of regions

The flow graph has 4 regions
2. V(G) = E – N + 2


E : Number of flow graph edges
N : Number of flow graph nodes
 V(G) = 11 edges – 9 nodes + 2 = 4
3. V(G) = P + 1

P : Number of predicate nodes
 V(G) = 3 predicate nodes + 1 = 4
TCS2411 Software Engineering
23
Deriving Test Cases
1. Draw flow graph from design/code as foundation.
…
i=1;
2
total.input = total.valid=0;
1
sum=0;
do while value[i] <> -999 and total.input<100 3
4 increment total.input by 1;
if value[i] >= minimum AND value[i] <= maximum 6
5
then increment total.valid by 1;
sum = sum + value[i]
7
else skip
end if
8
increment i by 1
9 End do
If total.valid > 0 10
11 then average = sum / total valid;
12 else average = -999;
13 End if
…
1
2
3
4
10
12
11
5
13
6
8
7
9
TCS2411 Software Engineering
24
Deriving Test Cases
(cont)
2.
Determine cyclomatic complexity



3.
Determine a basis set of linearly independent graph



4.
V(G) = 6 regions
V(G) = 17 edges – 13 nodes + 2 = 6
V(G) = 5 predicates nodes + 1 = 6
Path 1 = 1-2-10-11-13
Path 2 = 1-2-10-12-13
….
Prepare test cases

Path 1 test case :
 value(k) = valid input where k < i defined below
 value(i) = -999 where 2 <= i <= 100
 Expected result : Correct average based on k value and proper
totals
TCS2411 Software Engineering
25
Discussion on White
Box Testing
Advantages
Find errors on code level
Typically based on a very systematic approach,
covering the complete internal module structure
Constraints
Does not find missing or additional functionality
Does not really check the interface
Difficult for large and complex module
TCS2411 Software Engineering
26
Black Box Testing
Derived from program specification
Functional testing of a component of a
system
Examine behaviour through inputs & the
corresponding outputs
Input is properly accepted, output is
correctly produced
Disregard internal logical structure
TCS2411 Software Engineering
27
Black Box Testing
(Continued)
Attempts to find the following errors:
Incorrect or missing functions
Interface errors
Errors in data structures or external
database access
Performance errors
Initialisation and termination errors
TCS2411 Software Engineering
28
Black Box Testing
Techniques
Graph Based Testing Methods
Equivalence Partitioning
Boundary Value Analysis
Comparison Testing
Orthogonal Array Testing
TCS2411 Software Engineering
29
Equivalence Partitioning
Divide input domain into classes of data
Based on an evaluation of equivalence classes
for an input condition
Guidelines to define equivalence classes
Range input : One valid and two invalid equivalence
Specific value : One valid and two invalid equivalence
A member of a set : One valid and one invalid
equivalence
Boolean : One valid and one invalid equivalence
TCS2411 Software Engineering
30
Example – Data for Automated
Banking Application
The use can access the bank using his personal computer, provide a six digit password, and
follow with a series of typed commands that trigger various banking function. During the
log on sequence the software supplied for the banking application accepts data in the
form:
area code – blank or 3 digit numbers
prefix – 3 digit numbers, nor beginning with 0 or 1
suffix – 4 digit numbers
password – 6 digits alphanumeric strings
commands
– “check”, “deposit”, “bill pay” etc
Input
condition
area code : Input condition : Boolean – area code may or may not present
Input condition : Range – 200 – 999 with specific exception
prefix
: Input condition : Range – specified value > 200 with no 0 digits
suffix
: Input condition : Value – 4 digit length
password : Input condition : Boolean – password may or may not present
Input condition : Value – six character string
command : Input condition : Set – containing command
TCS2411 Software Engineering
31
Boundary Value
Analysis
Complement equivalence partitioning
Test both sides of each boundary
Look at output boundaries for test cases
Test min, min-1, max, max+1, typical
values
Example : 1 <= x <=100
 Valid : 1, 2, 99, 100
Invalid : 0 and 101
TCS2411 Software Engineering
32
Discussion on Black
Box Testing
Advantages
Find missing functionality
Independent from code size and functionality
Find some coding errors
Constraints
No systematic search for low level errors
Specification errors not found
TCS2411 Software Engineering
33
References
“Software Engineering: A Practitioner’s
Approach” 5th Ed. by Roger S. Pressman,
Mc-Graw-Hill, 2001
“Software Engineering” by Ian Sommerville,
Addison-Wesley, 2001
TCS2411 Software Engineering
34