Transcript Cleanroom Software Engineering Stephen M. Thebaut, Ph.D. Software Testing and Verification

Cleanroom Software
Engineering
Software Testing and Verification
Lecture 25
Prepared by
Stephen M. Thebaut, Ph.D.
University of Florida
Required Reading and Additional
Reference
• Required Reading:
– Linger, Cleanroom Software Engineering
for Zero-Defect Software, Proceedings,
15th Int. Conf. on Soft. Eng. (1993),
IEEE Computer Society Press, pp. 2-13.
• Additional relevant reference:
– Linger, Trammell, Cleanroom Software
Engineering Reference Model. CMU/SEI96-TR-022, Software Engineering
Institute, 1996.
Cleanroom SE Philosophy
• Cleanroom Software Engineering is a
software development philosophy.
• First introduced in the ‘80s within IBM by
Harlan Mills.
• Based on the notion that defects in
software should be avoided rather than
detected and repaired.
• Software development should not be
viewed as a trial-and-error undertaking.
Cleanroom SE Philosophy (cont’d)
“In traditional software development, errors
were regarded as inevitable. Programmers
were urged to get software into execution
quickly, and techniques for error removal
were widely encouraged. The sooner the
software could be written, the sooner
debugging could begin.”
Cleanroom SE Philosophy (cont’d)
“Today, debugging is understood to be
the most error-prone process in software
development, leading to ‘right in the
small, wrong in the large’ programs...”
Characteristics
• Team-oriented…
“The functional specification is created
by the development team, or by a
separate specification team for large
projects, and the usage specification is
created by the certification team.”
• Object-based box structure specification
and design
• Stepwise refinement
Characteristics (cont’d)
• Uses function-theoretic correctness
verification – components are not
executed or developer-tested!
“Team correctness verification takes the
place of unit testing and debugging, and
software enters system testing directly, with
no execution by the development team...no
private debugging (is) permitted.”
Characteristics (cont’d)
• Statistical usage testing (of integrated
increments) is undertaken for quality
certification (‘‘statistical quality control’’).
“The certification (test) team is responsible
for...certifying the quality of software with
respect to its specification. Certification is
carried out by statistical usage testing that
produces objective assessments of product
quality.”
Characteristics (cont’d)
• Incremental development…
“Management planning and control...is based
on developing and certifying a pipeline of
software increments that accumulate to the
final product.”
• Structured programming
Impact on Development Cycle
“Experienced...teams...can achieve
substantially reduced product development
cycles. The precision of Cleanroom
development eliminates rework and results
in dramatically reduced time for certification
testing compared to traditional methods.
And Cleanroom teams are not hostage to
error correction following product release.”
Box Structure Specification and
Design
• Incorporates black box (external behavior),
state box (retained data), and clear box
(processing) forms.
• “Transition Functions:”
– Black box: (S, SH -> R)
– State box: (S, OS) -> (R, NS)
– Clear box: (S, OS) -> (R, NS) by
procedure (intended function)
• Intended functions are refined into control
structures (programs)
Verification
• Development teams employ mental proofs
of correctness in team reviews…
“Every correctness condition of every
control structure is verified – every team
member must agree that each condition is
correct.”
Quality Certification
• Based on statistical quality control in
manufacturing
• Process (statistical usage testing):
– sample population of user executions based on
expected frequency (stratified random
sampling): operational profile
– measure quality by determining if executions
are correct
– extrapolate to the population of possible
executions (statistical inference)
– if quality is inadequate, identify and correct
flaws in development process
(cont’d)
Quality Certification (cont’d)
• Alternate distributions can be defined for
low-probability, high-consequence
functions.
• Errors tend to be found in failure-rate order
on average (coverage testing is not biased
to find errors in any particular rate order).
Cleanroom Software
Engineering
Software Testing and Verification
Lecture 25
Prepared by
Stephen M. Thebaut, Ph.D.
University of Florida