Transcript A Comparison of Software Testing Using the by Jamie Gordon

A Comparison of Software Testing Using the
Object-Oriented Paradigm and Traditional Testing
by
Jamie Gordon
Bob Roggio
University of North Florida
Jacksonville, Florida
1
Introductory Comments
•Software testing: Critical part of any software development.
•OOT gives rise to critical complex testing issues arise.
•Current research
Partial comparisons between OOT and traditional
Within OOT, some address inheritance and polymorphism
•So much more
2
Objectives of Testing
Test to find faults: minimum of cost and effort
Effective test case planning
leading to discovery errors.
developing test cases
This paper presents
Complexity issues in OO-paradigm and
Issues that may impact test case planning.
3
Test Cases for Traditional Testing
Traditional Testing: usually based on Input Process Output
Works well for the Von-Neuman model of processing: inputs
determine outputs .
Can view testing as an 'ordered pair' <input, expected output>
No concerns are expressed for 'program state.'
Expected outputs expressed in terms of values produced by
inputs.
Importantly, this implies test cases can be determined statically for
dynamic testing.
4
Test Cases for Object-Oriented Testing
Very different in OO testing.
Objects in program have ‘state’ impacted by input parameters and
computations
‘State Changes’ may not at all be evident by the outputs of the
program.
OO provides functionality not provided in traditional: such as
encapsulation, reusability, and more.
5
Testing Levels
•Ease of writing OO programs does not translate into testing!
•Researchers claim testing in OO increases effort for
adequate testing
•Traditional testing: Essentially unit and system testing.
•OOT adds class and cluster testing.
6
Unit Testing
•
•
OOT & Traditional Testing: methods/routines tested independently
Independent units
Don't call other units/methods that use global data
Major Issues in testing methods in OOT:
Methods cannot interact with other classes,
Method
attributes may be changed by methods,
Method may be dependent on its other class's methods.
Other methods called by this method in the class must be deemed correct.
Thus testing individual methods is much more difficult.
(Testing cannot occur in a linear order and must be determined by a different method such as flow
graphs or graph based techniques.)
7
Unit Testing
A case can be made for using drivers and stubs and thus simulating dependent
classes.
A driver is written when a method is dependent upon another for data
A stub is a method that is handed data to process when the module that
processes the data has not yet been written or when the module has not
been adequately tested.
Drivers and Stubs commonly found in testing classes in a hierarchical design.
In OO testing drivers and stubs must be extended to entire class; complicates
testing.
Unit testing in OO – not an easy comparison; not terribly useful either!
8
Class Testing
OO’s version of unit testing.
Class testing: tests methods as they interact with other methods.
•
For any non-trivial task, testing methods in isolation not terribly useful,
Methods are meant to interact.
But other called methods in other classes must need be thoroughly
tested beforehand
or
The dependent classes need to be simulated
9
Cluster Testing
Cluster Testing: No clear correspondence to traditional testing
Cluster Testing: an extension of class testing
Purpose: ensure group/cluster of classes interact correctly.
"Cluster of classes" is a group of classes
Are dependent and cooperate with each other directly.
Cooperating classes must have been thoroughly tested ahead
preferably through class and unit testing. (more ahead)
10
Integration and System Testing
Traditional testing involves testing units together, as a whole
Includes dependent methods and/or dependency on global data.
OO testing: extension of cluster testing.
Clusters are combined ==> the whole system.
Tested with all the dependencies intact.
11
Object Oriented Features that Affect
Testing
This paper will address seven major OO features that significantly
complicate effective OO testing:
1. Encapsulation
2. Inheritance
3. Polymorphism
4. Cohesion (assists in effective OO Testing)
5. Coupling
6. Dynamic Binding
7. Abstraction
12
Encapsulation
Encapsulation:
Used to restrict access to some of an object's
attributes and its methods.
Traditional programming Is Not an issue;
Programs are typically full units whose private/protected
methods are not modified by outside programs.
OO programming: IS an Issue:
Difficult to observe object interactions with variables and methods
not visible outside the class
13
Encapsulation
•
Visibility Issues and State Issues for Encapsulation:
More difficult to be aware of an object's state,
(private attributes and methods can be affected with simple
getters and setters.)
Visibility and State are important for class and cluster testing.
A class's or other class's methods may change an object's
state
State unobservable: If part of that state cannot be observed,
then it will be difficult to design test cases and observe test
results.
14
Encapsulation – more Issues
For Objects strongly encapsulated: must find a way to
ensure private fields are correct if they are modified by
other classes.
Initial state?: Often initial state cannot be determined.
Thus ability to control a test's inputs may be difficult
(This might mean creating new methods to display a class's
state which may or may not go against the class goals as
designed.)
15
Inheritance (1 of 4)
Inheritance is a mechanism for sharing attributes/behaviors from preexisting classes to other subclasses.
Only 'near' comparison in Traditional Testing: method or structure reuse.
Issue: One class a subclass of another, not ==> all of the inherited
methods are correct if they have been verified in the super class
Issue: Superclass well tested does not ==> classes that inherit it will be
correct.
16
Inheritance - Complexity Metric:
Depth of Inheritance Tree (DIT)
DIT = path length of class to root class in inheritance tree.
Fact: Deeper the class in tree, larger number of methods that can be inherited.
Fact: High DIT ==> behavior much more complicated,
Fact: More difficult to predict behaviors,
Fact: Much more difficult to design test cases!
Issue: Cannot test a child class child without its parent class
Behavioral Errors might propagate down the inheritance tree.)
Issue: What about an inherited method changed in the subclass but the
subclass has an untouched, inherited method that uses the changed method?
Overridden method and the untouched method both need to be tested
17
Inheritance - Complexity Metric
Number of Children (NOC) (3 of 4)
NOC: Number of Children a class has.
NOC – Number of immediate subclasses in an
inheritance hierarchy.
Considered a measure of the influence one class may have
over the system as a whole.
Issue: best tested in system and cluster testing
Used to determine how much emphasis should be placed on
testing a specific class.
18
Metrics: DIT and NOC
(4 of 4)
DIT and NOC often used to determine testing overhead
DIT Advice: Researcher advice:
if DIT > 6
design complexity is high and testing overhead can be large.
NOC Advice: If NOC is similarly high,
design of abstract classes is considered diluted
Design of abstract classes is not utilized or
Design of abstract classes are too general
Difficult to assign measurable metrics in inheritance
Inheritance has many forms and
inheritance trees can become terribly complex.
So, Keep inheritance as simple as possible.
19
Polymorphism
(1 of 2)
No real analog to procedural development.
Polymorphism allows attributes to take on many forms or data types.
Operations may
1. return more than one data type or
2. accept more than one data type for parameters.
Polymorphism: crucial to OO programming; makes it versatile and reusable.
But presents great complexity in test design and can be a huge task.
Advice:
A Class or a group of classes should be designed well enough so that the
overhead to design a test is low
Note: not suggesting polymorphism should not be used, but attributes of a
class should be more limited and well-defined in regards to both design and
testing.
20
Polymorphism (2 of 2)
Two kinds of polymorphism: ad hoc polymorphism and universal polymorphism
Ad Hoc polymorphism is sharing names and coercion in expressions
Universal is considered true polymorphism; Is discussed in dynamic binding.
High cost in testing if true polymorphic types are used in design and implementation.
Again, this is discussed in dynamic binding ahead.
Understanding interactions resulting from the polymorphic nature of some objects can
be very daunting (but necessary) when developing test cases.
Testing Polymorphism Advice:
Ad hoc testing is less testing intense because the tests can be derived statically
Universal polymorphism is much more difficult to test because the many forms
an entity can take on during run time; This may be very wide-ranging. .
21
Cohesion
(1 of 3)
Cohesion: a measure by which the methods of a class create a
single, well-defined class.
Traditional paradigm: cohesion: how well module of code hangs together as a
unit.
OO paradigm: talk about cohesiom in terms of its instance variables.
Procedural programs do not have instance variables;
Information is passed via parameters.
Cohesion is concerned with methods with similar parameters and functionality.
OO programs, class is cohesive if
methods contribute to the class as a single unit, and
the class is more reusable, more reliable, and more easily understood.
22
Cohesion Metric: Lack of Cohesion Metric
(LOCM) (2 of 3)
LOCM: the mathematical difference between the
number of methods whose instance variables are
completely dissimilar compared to number of
methods where instance variables are shared.
Researchers: LCOM and lines of code (LOC) are the
most predictive methods over
DIT, NOC, and CBO
(Coupling Between Objects - ahead).
23
Cohesion vs. Coupling in classes (3 of 3)
High cohesion (desirable) & low coupling (desirable) and vice versa.
High cohesion ==> methods similar in variables used and tasks performed.
Data are more easily developed and understood.
Low cohesion ==> more different types of data and methods are found
Low cohesion leads to higher testing costs and error-prone testing
Cohesion Advice: high cohesion is highly desirable!
24
Coupling
(1 of 3)
Coupling: a measure of the dependency between modules.
Strong coupling (undesirable) prevents independent testing and changing
of variables and methods
Strong coupling ==>many or all of the coupled methods need to be
understood as a set rather than each class operating as its own unit.
Strong coupling negatively impacts testing
Strong coupling implies Unit Testing cannot be done independently
usually forcing Cluster Testing.
25
Coupling Metric - Coupling Between Objects
(CBO)
(2 of 3)
CBO is a count of the number of classes to which a class is coupled. ==>
another measure of complexity.
High CBO
poor reliability and higher interoperability between classes.
Note: Some interoperability between classes is generally required as objects need to
communicate in some way.
High CBO
unit testing may be difficult and may require cluster testing.
Coupling Advice:
Good design ==> minimize coupling especially in large
complex systems where strong coupling could force cluster and system testing
absorbing most of the testing resources.
26
(3 of 3)
Other metrics for
measuring coupling
include efferent coupling
and afferent coupling
Efferent coupling: measure
of coupling between
packages and refers to
external packages a package
depends upon.
Afferent coupling measure of
coupling between packages
where classes external to a
package are dependent upon
the classes inside the
package.
27
Dynamic Binding
(1 of 3)
Consider Java: method return type and/or attribute types of
some attributes can be determined at run time vice at
compile time.
Dynamic binding feature : concerns when designing test
cases
Exact data types of attributes cannot be known statically.
Issue: Dynamic binding issues may arise from
universal polymorphism (as opposed to ad hoc poly) in
some languages.
28
Dynamic Binding (2 of 3)
•
Universal Polymorphism has two subtypes:
inclusive polymorphism and parametric polymorphism
In Parametric polymorphism a method/object may be written in a generic manner
through parameters and given a class value when the object is instantiated.
There is very high overhead in testing when we have parametric polymorphism
So in designing test cases with parameter polymorphism, it may only be necessary to
test based on how other classes in the application will instantiate a class.
Serious Issues: This would be a part of cluster or system testing.
Unit testing / class testing is difficult because it may not be known without looking
at the whole system, how a class might be instantiated.
Thus class and unit testing should not be done in the case of dynamic binding
due to its complexity as a unit.
29
Dynamic Binding
(3 of 3)
In Inclusive polymorphism a subclass can be used in place of
a superclass
Inclusive polymorphism may be an easier form of dynamic
binding to test.
This is because it is often known what classes inherit a
superclass.
In this way it can be known what kind of objects may
be bound at runtime.
All combinations of these dynamic bindings must be run in a test
case.
30
Abstraction
(1 of 1)
An Abstract Class:
Cannot be instantiated;
Can serve (oftentimes serve) as an interface for other classes.
If class is an abstract class and not an interface, it provides useful methods and
useful attributes.
Issue: Independent Testing.
Cannot instantiate an abstract class or an interface.
This ==> defined methods cannot be tested independently and
Analysis must thus done from their subclasses.
Issue: Inheritance. IF an abstract class is inherited by more than one class, how
many of these child classes should be tested?
Even if the child classes have not overridden the inherited methods,
All methods would need to be tested because the abstract class by itself cannot
be directly tested.
31
Conclusions
(1 of 3)
Key: OOT is based not only on both the I/O of an object’s
methods, but also how that input and output may influence
the object’s state.
Great features of OO paradigm may:
Create greatly increase in program complexity
(sometimes
in
unintended
and
unseen
ways)
often negatively impacting test effort and time.
Cohesion reduces amount of testing, while encapsulation,
inheritance, polymorphism, coupling, dynamic binding, and
abstraction add complexity and cost in test design.
32
Conclusions
(2 of 3)
Traditional testing: view input and output of a program in a procedural
manner. Test cases tend to be one dimensional. Traditional testing
involves both unit and system testing.
Object-oriented testing: test cases are two dimensional, because
changes in an object’s state must be considered.
Object-oriented testing also requires
Class testing (for how the methods
work together) and
of a single object
Cluster testing (for how coupled objects change each
other’s’ states).
Verification testing (the testing done by the developers) has been truly
changed by the object-oriented paradigm, while Validation testing (that
done by the end-user) has not.
Conclusions
(3 of 3)
There has been no major consensus as to what the best
way to test is or what factors are most important in
testing.
Most assert: testing OO-modules is not as definite as in
traditional programs, where the order of tests follows a
procedural path.
Clearly in OOT, testing rises to a new level of concerns
as we reap the benefits of the OO paradigm.