Transcript More on Testing

Testing Data Structures
Tao Xie
Visiting Professor, Peking University
Associate Professor, North Carolina State University
http://people.engr.ncsu.edu/txie/
[email protected]
Objectives

Master practical testing techniques



use similar techniques to test students' own code at
hand
handle programming interview questions related to
testing
Master systematic testing techniques


apply both black-box and white-box testing techniques
effectively use the JUnit framework and code-coverage
tool
Testing Setup
Test inputs
void test1() {
BST t =
new BST ();
t.insert(2);
t.size();
t.remove(2);
t.contain(2);
}
Test
Test
11
Program
public class BST {
void insert(int v)
{ … }
void remove(int v)
{ … }
...
}
Expected
Outputs
Outputs
t.size():
1
t.contain(2):
false
?
=
t.size():
1
t.contain(2):
false
How is it different to test
LinkedList than Anagram?

Testing LinkedList


Test a LinkedList’s get(int X), which returns the
element at the specified position in this list.
Testing Anagram


Test a method which checks whether two words are
anagrams of each other
Test a method which checks to see if a word has any
anagrams in a dictionary of words
How is it different to test
LinkedList than Anagram?

Testing LinkedList


Test a LinkedList’s get(int X), which returns the
element at the specified position in this list.
A data structure has object states
Input = receiver-object state@entry
+ method arguments
Method Execution
Output = receiver-object state@exit
+ method return
How is it different to test
LinkedList than Anagram?

Testing LinkedList


Test a LinkedList’s get(int X), which returns the
element at the specified position in this list.
A data structure has object states

Implicit input for a method besides arguments



How to prepare object states? e.g. a LinkedList with size 5
What object states to prepare?
Implicit output for a method besides return

How to check object states?
Classic Unit Test Construction
public void testLinkedListXXX {
Construct the object state under test (OUT)
Optionally save the state of the OUT
Call the method under test (MUT)
if an exception was generated
Test for unhandled exceptions
else
Assertions on the return, OUT, and arguments
}
LinkedList add Example

How to know we get an expected new LinkedList object
after calling add(5) on a LinkedList object (containing
1 and 5)?
public testLinkedListAdd3 () {
LinkedList s = new LinkedList();
s.add(1);
s.add(5);//s is now prepared as OUT
s.add(5);//MUT
LinedList b = s.clone();
assertTrue(?????);
OR
LinedList b=
new LinkedList();
...
assertTrue(?????);
}
Backup/regenerate OUT
b.add(1);
b.add(5);
Asserting Object States

How to know we get an expected new LinkedList object
after calling add(5) on a LinkedList object (containing 1
and 5)?

Invoke other non-void-return methods (observers) on the new
object, e.g., assertTrue(s.contains(5)),
assertTrue(s.getLast()==5),
assertTrue(s.size()==3).

Invoke toString() on the new object, e.g.,
assertTrue(s.toString().equals(“1,5,5”))

Invoke equals() on the new object, e.g.,
assertTrue(!s.equals(…)). When we call s.add(5) and
s.removeLast(5) on an object state S, we want to check the
new object is equal to S. Need backup/clone S or regenerate S
Testing Techniques

Black-box testing



Equivalence Partitioning Testing
Boundary Value Testing
White-box testing

Statement coverage
test inputs
void test99() {
BST t =
new BST ();
t.insert(2);
t.size();
}
program
public class BST {
void insert(int v)
{ … }
void remove(int v)
{ … }
...
}
Outputs
t.size():
1
t.contain(2):
false
?
=
Expected
Outputs
t.size():
1
t.contain(2):
false
Example
Consider a method findMax that is supposed to find the
max element in a LinkedList:
We test the method on the following inputs and observe
the outputs as shown:
Can we claim that the method is correct ?
From Diane Horton’s handout
Example


It seems these 10 test cases are good enough; but in
fact, they are not well chosen
We can easily construct a method that passes these
then cases but fails in:





A very short list (i.e., of length 1, 2, or 3)
An empty list (i.e., of length 0)
In fact, easy to forget to specify the method’s behavior for
this type of “boundary” case
A list where the max elem is the first or last element.
A list where the max elem is negative
In fact, all 10 tests cover essentially the same
situation

A list of moderate length, all positive integers, the max elem
is somewhere in the middle
From Diane Horton’s handout
Equivalence Partitioning

Input domain is usually too large for exhaustive
testing.

Partition input domain into a finite number of subdomains for the selection of test inputs.

Each sub-domain is known as an equivalence class
and serves as a source of at least one test input.
Input domain
2
1
Too many
test inputs.
Input domain
partitioned into four
sub-domains.
3
4
Four test inputs, one
selected from each subdomain.
How to partition?

Inputs to a program provide clues to partitioning.

Example: given a LinkedList with size 10,
get(int X) returns the element at the
specified position in this list.

Prohibitively large input domain: X can assume a
large number of values.
Which index X shall we test?
How to partition?

Example: given a LinkedList with size 10,
get(int X) returns the element at the
specified position in this list.

We expect LinkedList to




behave the same way for all X<0
behave the same way for all X>9
behave the similar way for all 0<=X<=9
Partition the input domain of P into three subdomains.
How to partition?
One test case:
X=-3
Equivalence class
Equivalence class
X>9
X<0
0<=X<=9
Equivalence class
Another test case:
X=15
Another test case:
X=5
All test inputs in the X<0 sub-domain are considered equivalent.
The assumption is that if one test input in this sub-domain
reveals an error in the program, so will the others.
This is true of the test inputs in the X>9 sub-domain or the
0<=X<=9 sub-domain too.
Then we selected just enough tests to cover each
partition.
Guideline for Partitioning

Input condition specifies a range: create one for the valid
case and two for the invalid cases.



a<=X<=b (valid case)

X<a and X>b (the invalid cases)
Input condition specifies a value: create one for the valid
value and two for incorrect values (below and above the
valid value).


e.g., for a<=X<=b the classes are
This may not be possible for certain data types, e.g., for boolean.
Input condition specifies a member of a set: create one for
the valid value and one for the invalid (not in the set) value.

e.g., contains(Object o)
Boundary Value Testing

Faults tend to be concentrated at edges of input
domain – look for boundary values as test inputs
One test case:
X=-3
Equivalence class
Equivalence class
X>9
X<0
0<=X<=9
Equivalence class
Another test case:
X=15
Another test case:
X=5
X=0 and X=9 are boundaries. Inputs to the program might lie on
the boundary or on either side of the boundary.
•Lie on boundary: 0, 9
•Lie on valid side of the boundary: 1, 8
•Lie on Invalid boundary cases: -1, 10
Testing Arbitrary LinkedList

Example: given a LinkedList with size 10,
get(int X) returns the element at the
specified position in this list.
Input: X
Input condition
0<=X<=9
?
Testing Arbitrary LinkedList

Example: given a LinkedList with size 10,
get(int X) returns the element at the
specified position in this list.
Input: X
Inputs: X, S
Input condition
Input conditions
0<=X<=9

s.size()>=0

0<=X<= s.size()-1
What tests to generate?
Another example

Example: given a LinkedList,
contains(Object e) returns true if this list
contains the specified element.
Inputs: e, S
Input conditions
?
Another example

Example: given a LinkedList,
contains(Object e) returns true if this list
contains the specified element.
LinkedList s of size n
Inputs: e, S
Input conditions
Input condition
• s.size()>=0
• e not in s
e in s (e’s position)
Adapted from Norman Fenton’s slide
How many tests shall be generated?

Example: given a LinkedList,
contains(Object e) returns true if this list
contains the specified element.
LinkedList s of size n
Inputs: e, S
Input conditions
Input condition
• s.size()>=0
• e not in s
e in s (e’s position)
Adapted from Norman Fenton’s slide
How many tests shall be generated?

Example: given a LinkedList,
remove(Object e) returns true if this list
contains the specified element.
LinkedList s of size n
Inputs: e, S
Input conditions
Input condition
• s.size()>=0
• e not in s
e in s (e’s position)
Adapted from Norman Fenton’s slide
findMax example revisited


Consider a method findMax that is supposed to find
the max element in a LinkedList
We can easily construct a method that passes these
then cases but fails in:
1.
2.
3.
4.
A very short list (i.e., of length 1, 2, or 3)
An empty list (i.e., of length 0)
In fact, easy to forget to specify the method’s behavior for this
type of “boundary” case
A list where the max elem is the first or last element.
A list where the max elem is negative
How can we generate these tests using the
techniques we just learned?
• what test conditions?  what tests?
findMax example revisited


Consider a method findMax that is supposed to find
the max element in a LinkedList
We can easily construct a method that passes these
then cases but fails in:
1.
2.
3.
4.
A very short list (i.e., of length 1, 2, or 3)
An empty list (i.e., of length 0)
In fact, easy to forget to specify the method’s behavior for this
type of “boundary” case
A list where the max elem is the first or last element.
A list where the max elem is negative
Input: s
Input conditions: s.size()>=0, 0<=max’s position<s.size()
MIN < max’s value < MAX
White-Box Testing


Determining test cases from a knowledge of the
internal logic of the software
Four main types of white-box testing




Statement Testing
Loop Testing
Path Testing
Branch Testing
White-Box Testing


Statement Testing: Test single statements
Loop Testing:




Path testing:


Cause execution of the loop to be skipped completely.
(Exception: Repeat loops)
Loop to be executed exactly once
Loop to be executed more than once
Make sure all paths in the program are executed
Branch Testing (Conditional Testing): Make sure that
each possible outcome from a condition is tested at
least once
if (i == true)
System.out.println("YES");
else
System.out.println("NO");
Test cases: 1) i = true; 2) i = false
White-Box Testing


Statement Testing: Test single statements
Loop Testing:




Path testing:


Cause execution of the loop to be skipped completely.
(Exception: Repeat loops)
Loop to be executed exactly once
Loop to be executed more than once
Make sure all paths in the program are executed
Branch Testing (Conditional Testing): Make sure that
each possible outcome from a condition is tested at
least once
if (i == true)
System.out.println("YES");
System.out.println("OK");
Test cases: 1) i = true; 2) i = false
Coverage Measurement Tool

Measure statement coverage


Know which statements haven’t been exercised
Then you can try to generate tests to exercise them

A challenging problem though for complex programs
Both black-box and white-box
testing are needed
From Norman Fenton’s slide