Transcript Chapter 1

Software Testing and Quality Assurance:
Class Testing Basics
• Reading Assignment:
– John McGregor and David A. Sykes, A Practical
Guide to Testing Object-Oriented Software,
Addison-Wesley, 2001, ISBN: 0-201-325640.
• Chapter 5: Class Testing Basics
– JUnit Cookbook
• http://junit.sourceforge.net/doc/cookbook/cookbook.htm
Objectives
• To know what to consider when testing a
class.
• To identify test cases for testing a class.
• To know a good way to implement a test
driver for a class.
• Get familiar with a software testing tool
(JUnit).
Topics Covered
• Class testing
• Constructing test cases
• Constructing a test driver
Class Testing
• A class is the fundamental unit of an objectoriented program.
• Class testing comprises the activities
associated with verifying that the
implementation of a class corresponds exactly
with the specification for that class.
• Class testing is roughly analogous to unit
testing in traditional testing processes.
• The main focus is on execution-based testing.
Class Testing (cont...)
• Assumptions:
– Class Under Test (CUT) has a complete
specification expressed in a specification
language like Object Constraint Language
(OCL), a natural language, and/or state
transition diagram
– If more than one form of specification is used
for a class:
• All forms are consistent.
• Information can be taken from any model.
Class Testing: Ways to Test a Class
• Reviews:
– Reviews are subject to human error.
– Reviews require considerably more effort with
respect to regression testing.
• Executing test cases:
– Considerable effort can be required for the
identification of test cases and the development
of test drivers.
Class Testing and Unit Testing
• The purpose of unit testing is to ensure that each unit
meets its specification.
• If the unit meets its specification, then any bugs that
appear when units are integrated are more likely caused
by incorrect interfacing of units.
• Unit testing is done as units are developed.
• Units are tested by code inspection and execution testing
(more emphasis on execution testing).
• Units that have parameters can be unit tested if the driver
can initialize the actual parameters.
• Should the test driver itself be tested?
Class Testing and Unit Testing
(cont...)
• Identify test cases
• Test driver construction: create one or more
instances of a class to run a test case
• Execution
• Result analysis
Class Testing: Dimensions of Class
Testing
• Should we test a class independently as a unit or
as a component of a larger part of the system,
this decision is based on:
– The role of the class in the system; the risk
associated with it.
– The complexity of the class measured in terms of:
• The number of states
• Operations
• Associations with other classes
– The amount of effort associated with developing a
test driver for the class.
Class Testing: Dimensions of Class
Testing (cont...)
• Who — developer
• What — code implements the specification
exactly
• When —
– A test plan should be developed soon after the
specification of the class is ready for coding
– Class testing should be done prior to its use in
other portion of the software
– Regression class testing should be performed
whenever the class implementation is changed
Class Testing: Dimensions of Class
Testing (cont...)
• How — develop a test driver that creates
instances of the class and sets up a suitable
environment around these instances to run a
test case
• How much —
– Test operations and state transitions in all sorts
of combinations
– Exhaustive or selective with combination with
risk analysis.
Constructing Test Cases
• Possible ways for identifying test cases
– Class specification
• Pre- and post-conditions (OCL)
• State transition diagram
– Class implementation
Constructing Test Cases
• Test case construction from Pre- and Post-conditions
– Programming style: defensive vs. contract
– Steps in generating test cases from Pre- and Postconditions
• Identify a list of pre-condition contributions
• Identify a list of post-condition contributions
• Form test case requirements by making all possible
combinations of entries from the contributions lists
• Eliminate any conditions generated that are not meaningful
– Example
• Figures 5.2, 5.3, 5.4, 5.5, and 5.6
Constructing Test Cases: Example
Velocity class as specified in UML model (Figure 5.2)
Velocity
speed: Speed
direction: Direction
Velocity()
Velocity (speed: Speed, direction: Direction)
getSpeed(): Speed
getSpeedX(): Speed
getSpeedY(): Speed
getDirection(): Direction
setSpeed(speed: Speed)
setDirection(direction: Direction)
Reverse()
ReverseX()
ReverseY()
Constructing Test Cases: Example
(cont...)
• OCL specification for the method
setDirection of Velocity class
Velocity::setDirection(dir:Direction)
Pre:
0 <= dir and dir < 360
Post: direction=dir and speed=speed@pre
• Figures 5.4, 5.5, and 5.6
Constructing Test Cases: Example
(cont...)
Constructing Test Cases: Example
(cont...)
Constructing Test Cases: Example
(cont...)
Constructing Test Cases
• Test case construction from state transition diagram:
– Each transition on the diagram represents a requirement for one
or more test cases
– When testing based on state transition diagrams, make sure you
investigate the boundaries and results of each transitions
• Example: Figure 5.8: Total 9 transitions: 6 between states; 1 for
construction, 2 for destruction
Constructing Test Cases: Adequacy
of Test Suites for a Class
• Commonly used measures of adequacy:
– State-based coverage: how many of the
transitions in a state transition diagram are
covered by the test suite.
– Constraint-based coverage: how many pairs
of pre- and postconditions have been covered.
– Code-based coverage: how much of the code
that implements the class is executed across all
test cases in the suite.
Constructing Test Cases
• Boundary conditions:
– Boundary is the input value on which a large change
occurs and must be identified when test cases are
identified.
– Example: sorting an array the boundary values can be
•
•
•
•
An array with zero elements
An array with one element
An array with two elements
An array with many elements
– We need implementation-based testing, test cases
derived from the specifications may not be enough:
• Example: the size of the array is used to choose the sorting
algorithm for sorting an array
Constructing a Test Driver
• A test driver is a program that runs test
cases and collects the results.
• Two approaches to build a test driver
– Embed the testing into the class under test itself
– Implement a separate class
• Keep relationship between CUT and its tester class
through name, like a tester for Velocity class as
VelocityTester.
Constructing a Test Driver: Test
Driver Requirements
• Test driver should have relatively simple
design.
• Test driver must be easy to maintain and
adapt in response to changes.
• Test driver should be reusable to create new
drivers.
Constructing a Test Driver: Test
Driver Requirements (cont...)
• A class model
for requirements
for a tester class:
– Public interface
– Tester is an
abstract class.
Constructing a Test Driver: Test
Driver Requirements (cont...)
• Test case suites:
– Functional (specification-based)— if derived
from specification.
– Structural (implementation-based)— if
identified from code.
– Interaction — if derived from sequence of
events on an object such as pairs of input/output
transitions.
Constructing a Test Driver: Tester
Class Design
• Tester class
example
Constructing a Test Driver: Test
Class Design (cont...)
• Pseudo code for a typical test case method:
Constructing a Test Driver
• The main responsibility of the tester class is
to run test cases and report results.
• Test case methods: one method per test
case or group of closely related test cases:
– Execute a test case by creating the input state,
generating a sequence of events, and checking
the output state.
– Provide traceability to the test plan.
Constructing a Test Driver (cont...)
• Baseline testing: a baseline test suite is a set of test cases
that tests the operations of the CUT that are needed for the
other test cases to verify their outcome.
• This suite includes testing constructors and accessors.
• Two basic approaches to baseline testing: specificationbased and implementation-based:
– Check that all the constructors and accessors are self-consistent.
Create a test case for each constructor and verify that all attributes
are correct by invoking accessors.
– Check that all the constructors and accessors use the variables in
an object correctly.
Constructing a Test Driver (cont...)
• Running test suites:
– The abstract Tester class includes in its
protocol some operations to run all test cases
or selected test suites.
– Each calls a sequence of test case methods.
– Make sure that the baseline test suite is
executed before any of these other suites are
executed.
Constructing a Test Driver (cont...)
• Reporting test results:
– A test case method determines the success of a
test case.
– The test case should report results to the tester
instance.
Key Points
• Ways to test a class: reviews and executing test cases
• The purpose of unit testing is to ensure that each unit
meets its specification.
• Possible ways for identifying test cases
– Pre- and post-conditions (OCL)
– State transition diagram
• Common ways used measures of adequacy: state-based
coverage, constraint-based coverage and code-based
coverage.
• Boundary conditions is the input value on which a large
change occurs.
Key Points (cont...)
• Constructing test drivers:
– Embed the testing into the class under test itself
– Implement a separate class
• Test driver should have relatively simple design, easy to
maintain and reusable to create new drivers.
• Test case suites: functional, structural, and interaction
• Test case methods: one method per test case or group of
closely related test cases:
• Baseline — checking test cases results and accessor and
modifier methods.
JUnit Tutorial
JUnit
http://www.junit.org
Unit Test Automation
• Many advocate that unit test cases be automated and run.
These test cases are then added to the code base (along with
the code) and are re-run whenever new code is added to the
code base.
• If new code breaks an old test case, it is easy to know right
away that the new code broke some already implemented
functionality. Most easily isolating the defect is of prime
importance!
• Once you have written code that implements new
functionality and this code passes its unit test cases, these
test cases are then referred to as ‘regression’ test cases.
• We’ll be using JUnit for this…
What is JUnit?
•
•
•
•
JUnit is a regression testing framework.
Written by Erich Gamma and Kent Beck.
Used by developers to implement unit tests in Java.
JUnit is integrated into a number of IDEs: BlueJ,
JBuilder, and Eclipse.
• The test framework provides tools for:
–
–
–
–
assertions.
running tests.
aggregating tests (suites).
reporting results.
Why use JUnit?
• JUnit tests allow you to write code faster while
increasing quality: less debugging time, more
confidence, and enable refactoring.
• JUnit is simple: compiler tests syntax, JUnit tests
integrity.
• JUnit tests check their own results and provide
immediate feedback.
• JUnit test can be composed into a hierarchy of test
suites.
• Create test assets... Retain their value and can be
run and interpreted by other than original author.
• Effective, open source, integrated.
Terminology
• A unit test is a test of a single class.
• A test case tests the response of a single method to a
particular set of inputs.
• A test suite is a collection of test cases.
• A test runner is software that runs tests and reports
results.
• An integration test is a test of how well classes work
together (JUnit provides some limited support for
integration tests).
• A test fixture sets up the data (both objects and
primitives) that are needed to run tests:
– Example: If you are testing code that updates employee record,
you need an employee record to test it on.
Fixtures
• Handle common objects under test.
• setup() and tearDown() used to initialize
and release common objects.
• The JUnit framework automatically invokes the
setUp() method before each test is run and the
tearDown() method after each test is run.
• Used to insure there are no side effects between
tests.
• Enforce the test independence rule, test execution
order is not guaranteed.
What should be tested and what
should not be tested?
• What should be tested:
– Test things which could break (tests should
succeed quietly: don’t print “Doing foo…done
with foo!”.
– Test boundary conditions
– Test special conditions
– Test exceptions
• What should not be tested:
– Do not test set/get methods.
– Do not test the compiler.
JUnit
JUnit
• TestCase – a class that extends the JUnit TestCase class.
It contains one or more tests represented by testXXX
methods. A test case is used to group together tests that
exercise common behaviors.
• TestSuit - a group of tests. Test suite groups together
related tests.
• TestRunner – A launcher of test suites. JUnit provides a
number of test runners that you can use to execute your
tests.
– There is no test runner interface, only a base test runner that all
test runners extend.
• You only need to write test cases. The other classes work
behind the scenes to bring your tests to life.
JUnit core classes
JUnit core classes
JUnit Mechanics
• Define a subclass of TestCase.
• Override the setUp() & tearDown()
methods.
• Define one or more public testXXX() methods:
– Exercise the object(s) under test.
– Asserts the expected results.
• Define a static suite() factory method:
– Create a TestSuite containing all the tests.
• Optionally define main() to run the TestCase in
batch mode.
Assert
• Assert within a test:
– Call the method being tested and get the actual
result.
– assert what the correct result should be with
one of the provided assert methods.
– These steps can be repeated as many times as
necessary.
JUnit assert methods
• static void assertTrue(boolean test)
static void assertTrue(String message, boolean
test)
– Throws an AssertionFailedError if the test fails (the optional message is
included in the Error)
• static void assertFalse(boolean test)
static void assertFalse(String message, boolean
test)
– Throws an AssertionFailedError if the test fails.
• assertEquals(expected, actual)
assertEquals(String message, expected, actual)
– If the objects are not equal it throws an
AssertionFailedError
– For objects, uses your equals method, if you have defined it properly, as
public boolean equals(Object o)--otherwise it uses ==
JUnit assert methods (cont...)
• assertSame(Object expected,
Object actual)
assertSame(String message,
Object expected, Object actual)
– Asserts that two objects refer to the same object (using ==). If
they are not, it throws an AssertionFailedError.
• assertNotSame(Object expected,
Object actual)
assertNotSame(String message,
Object expected, Object actual)
– Asserts that two objects do not refer to the same object. If they
do, it throws an AssertionFailedError.
JUnit assert methods (cont...)
• assertNull(Object object)
assertNull(String message, Object object)
– Asserts that the object is null.
If it isn’t, it throws an AssertionFailedError.
• assertNotNull(Object object)
assertNotNull(String message, Object object)
– Asserts that the object is not null.
If it isn’t, it throws an AssertionFailedError.
• fail()
fail(String message)
– Causes the test to fail and throw an AssertionFailedError
– Useful as a result of a complex test, when the other assert methods aren’t
quite what you want.
When to use an assert
statement
• Use assertTrue to document a condition
that you “know” to be true.
• Use assertFalse; in code that you
“know” cannot be reached (such as a default
case in a switch statement).
Example: Counter class
• We will create and test a trivial “counter” class:
– The constructor will create a counter and set it to
zero.
– The increment method will add one to the counter
and return the new value.
– The decrement method will subtract one from the
counter and return the new value.
• We write the test methods before we write the
code.
• Don’t be alarmed if, in this simple example, the
JUnit tests are more code than the class itself.
Example: JUnit tests for Counter class
(cont...)
public class CounterTest extends junit.framework.TestCase {
Counter counter1;
public CounterTest() { }
// default constructor
protected void setUp() {
// creates a (simple) test fixture
counter1 = new Counter();
}
protected void tearDown() { } // no resources to release
public void testIncrement() {
assertTrue(counter1.increment() == 1);
assertTrue(counter1.increment() == 2);
}
public void testDecrement() {
assertTrue(counter1.decrement() == -1);
}
}
Note that each test
begins with a brand
new counter
This means you don’t
have to worry about
the order in which the
tests are run
Example: The Counter class itself
(cont...)
public class Counter {
int count = 0;
public int increment()
{
return ++count;
}
public int decrement()
{
return --count;
}
public int getCount() {
return count;
}
}
• Is JUnit testing overkill for this
little class?
• The Extreme Programming view
is: If it isn’t tested, assume it
doesn’t work
• You are not likely to have many
classes this trivial in a real
program, so writing JUnit tests for
those few trivial classes is no big
deal
• Often even XP programmers don’t
bother writing tests for simple
getter methods such as
getCount()
• We only used assertTrue in
this example, but there are
additional assert methods
Test suites
• Obviously you have to test your code to get
it working in the first place:
– You can do ad hoc testing (running whatever
tests occur to you at the moment).
– You can build a test suite (a thorough set of
tests that can be run at any time).
Test suites: advantages and
disadvantages
• Disadvantages of a test suite:
– It’s a lot of extra programming:
• This is true, but use of a good test framework can
help quite a bit.
– You don’t have time to do all that extra work:
• False--Experiments repeatedly show that test suites
reduce debugging time more than the amount spent
building the test suite.
• Advantages of a test suite:
– Reduces total number of bugs in delivered
code.
TestRunners: text
• Text:
– Lightweight, quick quiet.
– Run from command line.
java StringTest
.......
Time: 0.05
Tests run: 7,
Failures: 0,
Errors: 0
TestRunners: swing
• Run with java junit.swingui.TestRunner.
Example: point class
public class Point {
private int x, y;
public Point(int x, int y){ this.x = x; this.y = y;}
public void setX(int x) { this.x = x; }
public void setY(int y) { this.y = y; }
public int getX() { return x; }
public int getY() { return y; }
}
Example (cont...): point test without
using JUnit
public class PointTestNoJunit {
public static void main(String [] args) {
testSetX(new Point(10, 20), 0);
testSetX(new Point(10, 20), 1);
testSetX(new Point(10, 20), 30);
}
private static void testSetX(Point p, int x) {
System.out.print("Before: " + toString(p) + ";");
p.setX(x);
System.out.println(" After setX(" + x + "): " +
toString(p));
}
private static String toString(Point p) {
return "<" + p.getX() + ", " + p.getY() + ">";
}
}
Example (cont...): point test without
using JUnit —output
Before: <10, 20>; After setX(0): <0, 20>
Before: <10, 20>; After setX(1): <1, 20>
Before: <10, 20>; After setX(30): <30, 20>
Before: <10, 20>; After setY(0): <10, 0>
Before: <10, 20>; After setY(1): <10, 1>
Before: <10, 20>; After setY(30): <10, 30>
Example (cont...): point test using
JUnit
import junit.framework.*;
/** A JUnit test class to test the class Point. */
public class PointTest extends TestCase {
/** Tests constructor. */
public void testPoint() {
Point p = new Point(10,20);
assertEquals(10, p.getX());
assertEquals(20, p.getY());
}
/** Tests setX */
public void testSetX() {
Point p = new Point(10, 20);
p.setX(30);
assertEquals(30, p.getX());
assertEquals(20, p.getY());
}
//other test methods, e.g., for setY(), getX(), and getY().
// cont…
Example (cont...): point test suite using
JUnit
/** Returns the test suite for this test class.
*/
public static Test suite() {
return new TestSuite(PointTest.class);
}
/** Run the tests. */
public static void main(String[] args) {
junit.textui.TestRunner.run(suite());
// junit.swingui.TestRunner.run(suite());
}
}
Example (cont...): point test using
JUnit —output
C:/> javac Point.java PointTest.java
C:/> java PointTest
...F.....
Time: 0.016
There was 1 failure:
testSetX1(PointTest)junit.framework.AssertionFailedError:
expected:<30> but was:<1>
at PointTest.testSetX1(PointTest.java:58)
…
at PointTest.main(PointTest.java:17)
FAILURES!!!
Tests run: 8, Failures: 1, Errors: 0
Example (cont...): point test using
JUnit —Test Fixture
public class PointTest extends TestCase {
private Point p; // test fixture variable
public void setUp() { // initializes text fixture variables
p = new Point(10, 10);
}
public void tearDown() { }//clean up text fixture variables
public void testSetX() { // tests SetX
p.setX(20);
assertEquals(20, p.getX());
}
public void testSetY() { // tests SetY
p.setY(30);
assertEquals(30, p.getY());
}
// template and other test methods here…
}
JUnit Testing tips
• Code a little, test a little, code a little, test a little . . .
• Run your tests as often as possible, at least as often as you run the
compiler.
• Begin by writing tests for the areas of the code that you’re the
most worried about . . .write tests that have the highest possible
return on your testing investment.
• When you need to add new functionality to the system, write the
tests first.
• If you find yourself debugging using System.out.println(), write a
test case instead.
• When a bug is reported, write a test case to expose the bug.
• Don’t deliver code that doesn’t pass all the tests.
• Separate production and test code:
– But typically in the same packages.