Transcript Ch 4: Input Space Partitioning

Software Testing
Input Space Partition Testing
Input Space Coverage
Four Structures for
Modeling Software
Graphs
Logic
Input Space
Syntax
Applied to
Applied
to
Source
Specs
Source
Specs
Design
Use cases
Applied
to
FSMs
DNF
Source
Models
Integ
Input
2
Benefits of ISP
• Can be equally applied at several levels of
testing
– Unit
– Integration
– System
• Relatively easy to apply with no automation
• Easy to adjust the procedure to get more or
fewer tests
• No implementation knowledge is needed
– just the input space
3
Test Data and Test Cases
• Test data: Inputs which have been
devised to test the system.
• Test cases: Inputs to test the
system and the predicted outputs
from these inputs if the system
operates according to its
specification.
Black-box testing
• Characteristics of Black-box testing:
–Program is treated as a black box.
–Implementation details do not matter.
–Requires an end-user perspective.
–Criteria are not precise.
–Test planning can begin early.
Input Domains
• The input domain to a program
contains all the possible inputs to that
program
• For even small programs, the input
domain is so large that it might as well
be infinite
• Testing is fundamentally about
choosing finite sets of values from the
input domain
6
Input Domains
• Input parameters define the scope of the
input domain
–Parameters to a method
–Data read from a file
–Global variables
–User level inputs
• Domain for each input parameter is
partitioned into regions
• At least one value is chosen from each
region
7
Data Testing
• If you think of a program as a function,
the input of the program is its domain.
• Examples of program data are:
–words typed into MS Word
–numbers entered into Excel
–picture displayed in Photoshop
–the number of shots remaining in an arcade
game
–…
Equivalence
Partitioning
Invalid inputs
• Equivalence
partitioning is the
process of
methodically
reducing the huge
(or infinite) set of
possible test cases
into a small, but
equally effective, set
of test cases.
System
Outputs
Valid inputs
Equivalence partitions
3
4
Less than 4
7
11
10
Between 4 and 10
More than 10
Number of input values
9999
10000
Less than 10000
Input values
50000
100000
99999
Between 10000 and 99999
More than 99999
Partitioning Domains
•
•
•
•
Domain D
Partition scheme q of D
The partition q defines a set of blocks, Bq = b1 , b2 , … bQ
The partition must satisfy two properties :
1. blocks must be pairwise disjoint (no overlap)
2. together the blocks cover the domain D (complete)
b1
b2
b3
bi  bj = ,  i  j, bi, bj  Bq

b=D
b  Bq
11
Using Partitions – Assumptions
• Choose a value from each partition
• Each value is assumed to be equally useful for testing
• Application to testing
– Find characteristics in the inputs : parameters, semantic
descriptions, …
– Partition each characteristic
– Choose tests by combining values from characteristics
• Example Characteristics
– Input X is null
– Order of the input file F (sorted, inverse sorted, arbitrary, …)
– Input device (DVD, CD, VCR, computer, …)
12
Example: Loan application
Customer Name
Account number
Loan amount requested
Term of loan
Monthly repayment
2-64 chars.
6 digits, 1st
non-zero
£500 to £9000
1 to 30 years
Term:
Repayment:
Minimum £10
Interest rate:
Total paid back:
Choosing (or defining) partitions seems easy, but is easy to get wrong
Customer name
Number of characters:
invalid
Valid characters:
Conditions
Customer
name
1
2
valid
A-Z
-’ a-z
space
Valid
Invalid
Partitions
Partitions
2 to 64 chars < 2 chars
valid chars
> 64 chars
invalid chars
64 65
invalid
Any
other
Valid
Invalid
Boundaries
Boundaries
2 chars
1 chars
64 chars
65 chars
0 chars
Loan amount
499
invalid
Conditions
Loan
amount
Valid
Partitions
500 - 9000
500
9000
valid
Invalid
Partitions
< 500
>9000
0
non-numeric
null
9001
invalid
Valid
Invalid
Boundaries
Boundaries
500
499
9000
9001
Design test cases
Test
Case
Description
Expected Outcome
New Tags
Covered
1
Name:
Acc no:
Loan:
Term:
John Smith
123456
2500
3 years
Term:
Repayment:
Interest rate:
Total paid:
3 years
79.86
10%
2874.96
V1, V2,
V3, V4,
V5 .....
2
Name:
Acc no:
Loan:
Term:
AB
100000
500
1 year
Term:
Repayment:
Interest rate:
Total paid:
1 year
44.80
7.5%
537.60
B1, B3,
B5, .....
Search routine specification
procedure Search (Key : INTEGER ; T: array 1..N of INTEGER;
Found : BOOLEAN; L: 1..N) ;
Pre-condition
-- the array has at least one element
1 <= N
Post-condition
-- the element is found and is referenced by L
( Found and T (L) = Key)
or
-- the element is not in the array
( not Found and
not (exists i, 1 >= i >= N, T (i) = Key ))
•
•
•
•
Search routine input partitions
Inputs which conform to the preconditions.
Inputs where a pre-condition does not
hold.
Inputs where the key element is a member
of the array.
Inputs where the key element is not a
member of the array.
Search routine input partitions
Array
Single value
Single value
More than 1 value
More than 1 value
More than 1 value
More than 1 value
E lement
In array
Not in array
First element in array
Last element in array
Middle element in array
Not in array
Search routine test cases
Key (Key )
Input array (T)
17
17
0
17
17
17, 29, 21, 23
45
41, 18, 9, 31, 3 0, 16, 45
23
17, 18, 21, 23, 29, 41, 38
25
21, 23, 29, 33, 38
Output (Found, L)
true, 1
false, ??
true, 1
true, 6
true, 4
false, ??
class BinSearch {
// This is an encapsulation of a binary search function that takes an array of
// ordered objects and a key and returns an object with 2 attributes namely
// index - the value of the array index
// found - a boolean indicating whether or not the key is in the array
// An object is returned because it is not possible in Java to
// reference to a function and so return two values
// the key is -1 if the element is not found
public static void search ( int key, int [] elemArray, Result r )
{
int bottom = 0 ;
int top = elemArray.length - 1 ;
int mid ;
r.found = false ; r.index = -1 ;
while ( bottom <= top )
{
mid = (top + bottom) / 2 ;
if (elemArray [mid] == key)
{
r.index = mid ;
r.found = true ;
return ;
} // if part
else
{
if (elemArray [mid] < key)
bottom = mid + 1 ;
else
top = mid - 1 ;
}
} //while loop
} // search
} //BinSearch
pass
basic
types
by
Binary search (Java)
Binary search equiv. partitions
1
bottom > top
while bottom < = top
2
3
if (elemArray [mid] == key
4
8
5
(if (elemArray [mid]< key
6
9
7
Binary search flow graph
Independent paths
• 1, 2, 3, 8, 9
• 1, 2, 3, 4, 6, 7, 2
• 1, 2, 3, 4, 5, 7, 2
• 1, 2, 3, 4, 6, 7, 2, 8, 9
• Test cases should be derived so that all of
these paths are executed
• A dynamic program analyser may be used
to check that paths have been executed
Two Approaches to Input Domain Modeling
1. Interface-based approach
– Develops characteristics directly from individual
input parameters
– Simplest application
2. Functionality-based approach
– Develops characteristics from a behavioral view of
the program under test
– Harder to develop—requires more design effort
– May result in better tests, or fewer tests that are as
effective
25
1. Interface-Based Approach
• Mechanically consider each parameter
in isolation
• This is an easy modeling technique
and relies mostly on syntax
• Some domain and semantic
information won’t be used
• Ignores relationships among
parameters
26
2. Functionality-Based Approach
• Identify characteristics that correspond to the
•
•
•
•
•
intended functionality
Requires more design effort from tester
Can incorporate domain and semantic
knowledge
Can use relationships among parameters
Modeling can be based on requirements, not
implementation
The same parameter may appear in multiple
characteristics, so it’s harder to translate values
to test cases
27
Identifying Functionalities, Parameters and
Characteristics
• Interface-based : Translate parameters to
characteristics
• Candidates for characteristics :
– Preconditions and postconditions
– Relationships among variables
– Relationship of variables with special values (zero,
null, blank, …)
• Should not use program source – characteristics
should be based on the input domain
– Program source should be used with graph or logic
criteria
28
Interface vs Functionality-Based
public boolean findElement (List list, Object element)
// Effects: if list or element is null throw NullPointerException
//
else return true if element is in the list, false otherwise
Interface-Based Approach
Two parameters : list, element
Characteristics :
list is null (block1 = true, block2 = false)
list is empty (block1 = true, block2 = false)
Functionality-Based Approach
Two parameters : list, element
Characteristics :
number of occurrences of element in list
(0, 1, >1)
element occurs first in list
(true, false)
element occurs last in list
(true, false)
29
Example Handling Constraints
• Sorting an array
– Input : variable length array of arbitrary type
– Outputs : sorted array, largest value, smallest value
Characteristics:
• Length of array
• Type of elements
• Max value
• Min value
• Position of max value
• Position of min value
Partitions:
• Len
{ 0, 1, 2..100, 101..MAXINT }
• Type
{ int, char, string, other }
• Max
{ 0, 1, >1, ‘a’, ‘Z’, ‘b’, …, ‘Y’ }
• Min
{…}
• Max Pos { 1, 2 .. Len-1, Len }
• Min Pos { 1, 2 .. Len-1, Len }
30
Example: student access
A university computer system allows
students an allocation of disc space
depending on their projects.
If they have used all their allotted space,
they are only allowed restricted access, i.e.
to delete files, not to create them. This is
assuming they have logged on with a valid
username and password.
What are the input and output conditions?
List the input and output conditions
• list the ‘input
conditions’ in the
first column of the
table
• list the ‘output
conditions’ under
the input
conditions
Input Conditions
Valid username
Valid password
Account in credit
Output Conditions
Login accepted
Restricted access
Determine input combinations
• add columns to the table for each unique combination of input
conditions.
• each entry in the table may be either ‘T’ for true, ‘F’ for false.
Input Conditions
Valid username T T T T F F F F
Valid password T T F F T T F F
Account in credit T F T F T F T F
Complete the table
• determine the expected output conditions for each
combination of input conditions
Input Conditions
Valid username F T T T
Valid password - F T T
Account in credit - - F T
Output Conditions
Login accepted F F T T
Restricted access - - T F
Determine test case groups
• each column is at least one test case
Input Conditions
Valid username F
Valid password Account in credit Output Conditions
Login accepted F
Restricted access Tags A
T T T
F T T
- F T
F T T
- T F
B C D
Design test cases
• usually one test case for each column but can be none or
several
Test Description
1
2
3
4
5
Username BrbU
Username
usernametoolong
Username BobU
Password abcd
Valid user, no disc
space
Valid user with disc
space
Expected Outcome
Tag
Invalid username
Invalid username
A
A
Invalid password
B
Restricted access
C
Unrestricted access
D