Towards Destructive Software Testing

Kiumi Akingbehin, PhD Department of Computer and Information Science University of Michigan-Dearborn By Michael J. Dudley

Introduction

Conventional Software Testing Definitions  the activity or process which shows or demonstrates that a program or system performs all intended functions correctly  the activity of establishing the necessary ‘confidence’ that a program or system does what it is supposed to do, based on the specifications and requirements set by the user

Introduction

Conventional Software Testing Definitions  the process of executing a program/system with the intent of finding errors  any activity aimed at evaluating an attribute or capability of a program or system and determining that it meets its required results

Destructive Software Testing (DST)

DST does not conform to any of the previous definitions proposed.

DST can be performed without knowledge of the original requirements of a software product.

 Although some knowledge of the requirements may help in developing a good comprehensive destructive testing strategy.

Basic Principles of DST

Is not a replacement for conventional testing.

Should be performed in addition to conventional testing.

Acknowledges the fact that users of a software product will sometimes not use the software correctly.

 Software Misuse

Software Misuse

Improper or incorrect  Input data will be supplied Letters entered when numbers are expected.

 Commands will be typed Commands that the software was not written to handle.

 GUI sequences will be applied Alternate path that wasn’t expected.

Destructive Testing Adage

Adage  a proverb or wise saying made familiar by long use An popular adage  “garbage in, garbage out” This adage is not good enough for high quality, robust, and reliable software.

A better adage for DST  “garbage in, proper predictable behavior out”.

DST improves the quality of a software product.

Destructive Testing Terminology

Destructive Hardware Testing (DHT)  Hardware systems are destroyed as part of testing Example: Automobile safety testing  The usual practice is to subject the automobile to an actual accident where the vehicle is heavily damaged or destroyed DST was chosen to be in compliance with corresponding concept of DHT

DST Definitions

DST is testing that  assures proper or predictable software behavior when the software is subject to improper usage or improper input  attempts to crash a software product  tries to crack or break a software product  checks the robustness of a software product

Destructive Testing Confusion

The Term Destructive Testing  Should not be confused with the same term used for conventional software testing  Has been used in the past to indicate software that fails conventional testing

Conventional Testing Strategies to DST

Strategies that can be used for DST         Black Box Testing Top-Down Testing Regression Testing Interface Testing Equivalence Partitioning Test Cases Quality Assurance Quality Control        Acceptance Testing Boundary Value Testing Loop Testing Stress Testing Alpha Testing Beta Testing System (High Order) Test

Conventional Testing Strategies to DST

Strategies that can, with modifications, be used for DST      White Box Testing Bottom-Up Testing Basis Path Testing Security Testing Defect Testing      Smoke Testing Performance Testing Unit (Module) Testing Integration Testing Object-Oriented Testing

Conventional Testing Strategies to DST

Strategies that cannot be used for DST    Verification Testing Validation Testing Benchmark Testing For more information on testing strategies  Hetzel, Bill, “The Complete Guide to Software Testing”, John Wiley, 1988.

 Myers, G. J., The Art of Software Testing, John Wiley, New York, 1979.

 Pressman, R. S, Adaptable Process Model: Software Test Specification, www.rspa.com, 2005.

Incorporation of DST into Specifications

Requirements for a software system can be written to promote DST   Such requirements, by definition, are non functional Functional requirements by nature fall into the category of conventional testing that can not be tested using destructive testing Verification Testing Validation Testing Benchmark Testing

Incorporation of Destructive Testing into Specifications

Mandatory destructive testing clauses    The software must be designed in such a way that proper behavior is obtained when the software is subject to improper usage or improper input.

No data shall be lost if the software prematurely terminates as a result of other system failures. The software shall not prematurely or unintentionally terminate as a result of any combination of user keyboard or mouse input.

Incorporation of Destructive Testing into Specifications

Mandatory destructive testing clauses  The software shall never accept or process invalid input data.

 The software shall always produce proper output data regardless of the validity or correctness of input data.

Incorporation of Destructive Testing into Specifications

For Specific software products, it is important to explicitly define  proper software behavior  improper software behavior  improper usage  improper input data  proper output data

Future work

Author is currently working with a team to develop requirements specification for an example case study involving a data conversion program.

Subsequent to the completion of the requirements specification and implementation of the software, test cases will be developed for destructive testing of the software.

Conclusion

The goal of conventional software testing is to ensure a software product correctly performs all the functions specified in the requirements specification.

The goal of destructive testing is to ensure a software product exhibits proper behavior when subject to improper usage or improper input.

Ongoing work includes the development of requirements specification that mandate destructive testing of a case study software product.

Conclusion

Destructive testing is a reflection of the fact that user will sometimes use a software product in an improper manner.

Destructive testing does not replace conventional testing   Supplements conventional testing Performed in addition to conventional testing Destructive testing cannot be detrimental Destructive testing can only be beneficial.

Questions …