Transcript WHY IS SOFTWARE TESTING NEEDED? Presented by Santosh Bethi 22

WHY IS SOFTWARE TESTING NEEDED?
Presented by
-Santosh
Bethi
22nd October, 2003
Real Time Software Attacks
The Funny

The IRS sends out $68 billion tax bills

ATMs in Germany give unlimited amounts of money after the Euro conversion
The Serious

The loss of the Mars Climate Orbiter
– Cause: Improper unit conversion

AT&T’s telephone network breaks down
-- Cause: Untested code path
The Tragic

The Patriot missile failure
– Cause: Cumulative rounding error
Attacks cause Real Problems





Software fails just as mechanical devices fail
Software is often invisible
We often take software for granted
Our reliance on software is increasing
Our vulnerability to software flaws is
increasing
Why Test Software?
“Program testing can be used to show the
presence of bugs but never to show their
absence.”
—Edsger W. Dijkstra, 1972
Testing is a State of Mind
“If our goal is to show the absence of errors, we
will find very few of them.”
“If our goal is to show the presence of errors,
we will discover a large number of them.”
—G. J. Meyers, 1979
Developer’s Mindset
Product
Expertise


A good developer is clever
Emphasis on depth of expertise
Tester’s Mindset
Product
Expertise


A good tester is mischievous
Emphasis on breadth of expertise
Is Testing a Big Problem? YES





Developers don’t test well
Misconception that testing is “boring”
Requirements are not always clear
Must test everything
Exhaustive testing is intractable
Where is Testing Useful?





ATM
Vending Machine
Alarm Clock
Thermostat
Everywhere!
So Why Test?





Reliability
Security
Usability
Usefulness
Sanity
So Why Test?
 Make
the user happier!
How To Test Software
Testing The Design



Ambiguity is the mother of all screwups
Testing starts before the coding
Follow the specification cliché
– Design globally
– Specify locally
Designing a Test Plan




Be complete (thorough, not excessive)
Find the “interesting” tests cases
Do every test at least twice
Update the plans as you go
–
–
Add regression tests
Incorporate user feedback
Designing Tests


Tests are “attacks” against the software
Find vulnerabilities and target them
–
–
Black box
White box
Black Box Tests
Input

Targeted at the apparent simplicity of the software
–
–

Output
Makes assumptions about implementation
Good for testing component interactions
Tests the interfaces and behavior
White Box Tests
Input


Targeted at the underlying complexity of the software
– Intimate knowledge of implementation
– Good for testing individual functions
Tests the implementation and design
Output
Types of Tests

Unit
–

Component
–

Individual classes or
types
Group of related classes
or types
Integration
–
Interaction between
classes
References









[1] http://mars.jpl.nasa.gov/msp98/news/mco991110.html (Nov 1999)
[2]
http://groups.google.com/groups?selm=CMM.0.90.1.745021702.risks%40chiro
n.csl.sri.com
[3] http://www.time.com/time/archive/preview/0,10987,1101900129153701,00.html
[4] http://www.ima.umn.edu/~arnold/disasters/patriot.html (Feb 1991)
[5] http://www.cs.utexas.edu/users/EWD/
[6] http://nas.cl.uh.edu/helm/sum01_swen5431/CurrieTesting.ppt
[7] http://www.cigitallabs.com/resources/definitions/software_testing.html
[8] http://www.webopedia.com/TERM/B/Black_Box_Testing.html
[9] http://www.webopedia.com/TERM/W/White_Box_Testing.html