Transcript System Testing - Vanderbilt University

Testing: Chapter 13
"You make it, I’ll break it"
Purpose:
• Does the product as a whole perform
up to specifications? (validation?)
• Does it perform under normal, upper,
and lower limits wrt the environment?
• Does it meet or exceed all claims
(verification)?
Page 2
7/16/2015
Testing defined
• establishing confidence that a device
does what it is supposed to do
• the process of operating a device with
the intent of finding errors
• detecting specification errors and
deviations from the specification
Page 3
7/16/2015
Testing continued…
• verifying that a system satisfies its specified
requirements or identifying differences
between expected and actual results
• the process of operating a device or
component under specified conditions,
observing or recording the results, and
making an evaluation of some aspect of the
system or component.
Page 4
7/16/2015
Testing …
• Is a positive act of destruction.
• Is needed for good product
development
• Is necessary for consumer satisfaction…
Page 5
7/16/2015
Stressors: (depend on
requirements…)
Temperature
Humidity
Altitude
Shock
Vibration
EMI
Cell phone, pagers
Brownouts
ESD
Spillage
Shipping
Power Surges
Page 6
7/16/2015
Figure 13-1 Requirements checklist
Page 7
7/16/2015
Customer Misuse:
• Coffee spillage
• Drop in toilet
• Use as a seat
• Cross-connecting
• Reverse polarity
• Dirty hands
• Unplugging via jerk on cable
Page 8
7/16/2015
Testing requires that you define
failure!
• Intended application?
• Anticipated environment?
• Lifetime?
• Value vs. time
• Sunken ships make good coral reefs in
time …
Page 9
7/16/2015
Types of testing:
• Event testing – cycling, time to failure
• Environmental – operating & storage
• Altitude
• Threshold (door, elevator, bricks… )
• Vibration
• ISTA shipping test
• EMI (standards!, see next slide: )
Page 10
7/16/2015
EN 1000 series standards
• Radiated electric field immunity
• Fast transients
• Surge, line, magnetic
• Dips, variations, V, F variation
• On/off, ESU
• Walkie-talkie, cell phone
• ESU, etc.
Page 11
7/16/2015
Failure Definition
For each test and for each device, a
failure must be defined. This definition
depends on the intended application
and the anticipated environment. What
is considered a failure for one
component or device may not be a
failure for another. The test protocol
should be as detailed as possible in
defining the failure.
Page 12
7/16/2015
We want to test some power supplies to prove a MTBF goal of 50,000
hours of operation. How many units do we test and for how long,
assuming one failure and 90% confidence level?
(sample size)(test time) = MTBF goal (7.779)/2
(sample size)(test time) = 50,000 (7.779)/2
= 194,475 unit-hours
Page 13
7/16/2015
Determining Sample Size and Test Length
MTBF goal = (sample size)(test time) (2)/Χ2 α;2r+2
Confidence level = 1 - α, where α is the risk factor and
r = the number of failures
Example--------------------------Using the 90% confidence level, α = 0.10 & no failures (r=0)
(Sample size)(test time) = (MTBF goal)( Χ2 α;2 )/2
(sample size)(test time) = MTBF goal (4.61)/2
to prove a MTBF goal of 50,000 hours of operation:
(sample size)(test time) = 50,000 (4.61)/2
= 115,250 unit-hours
Page 14
7/16/2015
Types of Testing:
Verification
Procedures that attempt to determine
that the product of each phase of the
development process is an
implementation of a previous phase,
i.e., it satisfies it. Each verification
activity is a phase of the testing life
cycle…
Page 15
7/16/2015
Types of Testing:
Validation
• Validation is the process of evaluating a
system or component during or at the
end of the development process to
determine whether it satisfies specified
requirements
Types of Testing: Black Box
Black box testing is a data driven testing
scheme. The tester views the device or
program as a black box, i.e., the tester is
not concerned about the internal behavior
and structure. The tester is only
interested in finding circumstances in
which the device or program does not
behave according to its specification
Page 17
7/16/2015
Types of Testing: White Box
White box testing is concerned with the
degree to which test cases exercise or
cover the structure of the device or
program. The ultimate while box test is
an exhaustive path test.
Page 18
7/16/2015
Types of testing: Software
• Module …
• Integration …
• System …
• Acceptance …
Page 19
7/16/2015
Types of testing:Fuctional
This type of testing verifies that given
all the expected inputs then all of the
expected outputs are produced. This
type of testing is termed success
oriented testing because the tests are
expected to produce successful results.
Page 20
7/16/2015
Types of testing - continued
• Robustness Testing
• Stress Testing
• Safety testing
• Regression Testing
Page 21
7/16/2015
Types of testing, continued
• Life test/reliability
• Customer misuse
• Fluid spillage
• Weight test
• Keyed connections
• Time related
• Failure related (MTBF)
Page 22
7/16/2015
Questions?
Page 23
7/16/2015