Transcript Testing based on specifications

Testing based on specifications
Agnė Meilutytė IFM-0/2
2011 04 06
Testing based on specifications
 The specification based tester checks the product's
conformance with every statement in one or more
reference documents. The documents might be formal
specifications, user manuals, advertisements.
Reviewing a specification
We can approach a spec from many different angles:
 What is it saying about the product?
 What claims does it make?
 Do you have enough information to understand the claims?
 Is some information missing?
 Are the claims accurate?
 Would all stakeholders understand the claims in the
same way?
 Should the specified product be changed?
 Will the specified product be testable?
Features of testing based on
specifications
 Refers to the process of testing a program based on what
its specification says its behavior should be.
 In particular, we can develop test cases based on the
specification of the program's behavior, without seeing
an implementation of the program.
 We can develop test cases before the program even
exists!
Testing based on specifications
 Quality
of the system is compliance with the
requirements specification.
 This course does not cover automated extraction of test
ideas from specifications, nor the use of program
comprehension tools. Instead, we assume the existence
of a natural language specification that is analyzed by a
human.
The human does such tasks as:
 identifying a set of reference documents;
 developing
test ideas based on ambiguities or
contradictions in the reference documents;
 developing lists of items or claims to be tested based on
assertions in the reference document, and then
associating test ideas with each;
 creating traceability matrices that list items or assertions
to test, and test cases that tie to those.
,,Black box” testing
 A testing method when a program is
viewed as a ,,black box” disregarding
it’s structure: only the input and
output is know, not the actual code;
 ,,Black box” testing is a method of
software testing that tests
functionality of an application.
the
 Test cases are based on requirements
specifications;
,,Black box” testing
 The plans for testing can be made in the early stages
of the software development;
 ,,Black box” testing usually performed by a separate
person outside the development process, rather than
the developer.
 This method of test can be applied to all levels of
software testing: unit, integration, functional,
system and acceptance.
,,Black Box” testing techniques:
1.
2.
3.
4.
5.
Decision table testing
All-pairs testing
State transition table
Equivalence partitioning
Boundary value analysis
,,Black Box” testing techniques
1. Decision table testing - is the method where test
cases are designed to execute the combinations of
inputs.
2. All-pairs or pairwise testing - is a combinatorial
software testing method that, for each pair of input
parameters to a system, tests all possible discrete
combinations of those parameters.
3. State transition table - is a systematic method of
exploring or analyzing the behavior of an object that
exhibits history dependent behavior.
,,Black Box” testing techniques
4. Equivalence partitioning - is the method, witch
input data divides into data classes and test cases are
obtained from these classes.
5. Boundary value analysis - is the method that
focuses on the boundary input values.
,,Black box” testing scheme
Inputs
I
Input, causing abnormal
behavior.
System
Outputs
O
Outputs that
indicate the
presence of
defects.
Examples illustrate the use of
specification based tests
 The unifying feature of this work is that we start with the
reference document and base our tests and testing
strategy on what we read in the documents.
 Examples:
 MS Paint’s Undo function
 OpenOffice’s Zoom in function
MS Paint
 In this example we do Specification
Based Testing by picking a piece of MS
Paint’s specifications (the user manual)
and making sure that the application
conforms to it. We are going to focus on
the specification of the Undo function.
The specifications claim that the
application can undo up to 3 changes,
but that is not always true. If the user is
editing graphics, there are cases when it
can undo only 2 changes. If the user is
editing text in a text box, it can undo
only 1 change (the last letter).
OpenOffice
 In
this example we perform
Specification Based Testing by
making sure that the specifications
for the Zoom In function in
OpenOffice are satisfied. OpenOffice
1.1.0 is an open source multiplatform
office
suite.
Its
presentation component has a
function Zoom In, which
according to specifications, doubles
the size of the current window. In
reality, it requires the user to select
the part that will be enlarged.
Strengths and blind spots
Strengths
 Critical defense against warranty claims, fraud charges, loss
of credibility with customers.
 Reduces support costs, customer complaints by ensuring that
no false or misleading representations are made to
customers.
Blind spots
 Any issues not in the specification or treated badly in the
specification.
Questions
1. Witch
documents might be used for
specifications based testing?
2. What are the features of testing based on
specifications?
3. Witch level of software testing uses ,,black
box” testing?
4. What the techniques are for ,,black box”
testing?
Thank you for your attention!