Transcript PowerPoint Sunusu
Software Quality Assurance & Testing
Mistake in coding is called error , Error found by tester is called defect, Defect accepted by development team then it is called bug , Product does not meet the requirements then it Is failure.”
Importance of Test
There are hundreds of stories about failures of computer systems that have been attributed to errors in software There are many reasons why systems fail but the issue that stands out the most is the lack of adequate testing. Example Pepsi - $42 Billion Error Philippines -1992 Due to a software error, 800,000 bottle caps were produced with number 349 instead of one.
It was equivalent to $42 billion in prize money instead of $40,0000
Testing Levels
Testing Goals Based on Test Process Maturity Bezier Testing Levels
Level 0 There is no difference between testing and debugging Level 1 The purpose of testing is to show the correctness (software works) .
Level 2 The purpose of testing is to show that the software doesn’t work.
Level 3 The purpose of testing is not to prove anything specific, but to reduce the risk of using the software.
Level 4 Testing is a mental discipline that helps all IT professionals develop
higher quality
software.
Bezier Testing Levels
Level 0 is the view that testing is the same as debugging.
In Level 1 testing, the purpose is to show correctness In Level 2 testing, the purpose is to show failures Level 3 testing shows the presence , not absence or failure. If we use software, we expose some risk . The risk may be small be great and unimportant and the consequences , or the risk may catastrophic, but
risk is always there.
In level 3 testing, both testers
together
to reduce risk.
and developers work
Bezier Testing Levels
Once the testers and developers are on the same “team,” an organization can progress to real Level 4 testing .
Level 4 testing means that the purpose of testing is to improve the ability of the developers to produce high quality software.
Debugging and Testing
Debugging is done in the development phase by the developer In debugging phase identified bugs are fixed Testing is done in the tester phase by the tester.
In testing, locating and identifying of the bugs are included.
Software Bug
A software bug is an error, flaw, mistake, failure, or fault in a computer program or system Bug is terminology of Tester Softare bug produces an incorrect or unexpected result, or causes it to behave in unintended ways
Software Fault
A static (physical)defect*, imperfection, flaw in the software.
short between wires break in transistor infinite program loop A fault means that there is a problem within the code of a program which causes it to behave incorrectly. * defect: Mismatch between the requirements
Software Error
Error is an incorrect internal state that is the demostration of some fault. In other words; Error is a deviation from correctness or accuracy.
Example Suppose a line is physically shortened to 0. (there is a fault ). As long as the value on line is supposed to be 0, there is no error .
Errors are usually associated with incorrect values in the system state.
Software Failure
A failure means that the program has not performed as expected, and the actual performance has not met the minimum standards for success.
External, incorrect behavior with respect to the requirements or other description of the expected behavior Example Suppose a circuit controls a lamp (0 = turn off, 1 = turn on) and the output is physically shortened to 0 (there is a fault). As long as the user wants the lamp off, there is no failure.
eBay Crash
eBay in 1999 was a very large Internet auction house –top 10 Internet business – Market value of $22 billion – 3.8 million users as of March 1999 – Access allowed 24 hours 7 days a week In June 6, 1999 eBay system was unavailable for 22 hours with problems ongoing for several days Stock drops by 6.5%, $billion lost revenues
Problems blamed on Sun server software
Ariane 5 Rocket Crash (June 4, 1996)
Ariane 5 rocket exploided 37 seconds after lift-off on Error was due to software bug Conversion of a 64-bit floating point number to a 16 bit integer resulted in an overflow Due to the overflow, the computer cleared its memory Ariane 5 interpreted the memory dump as an instruction to its rocket nozzles Testing of full system under actual conditions not
done due to budget limits
Estimated cost: 60 million $
Nine Causes of Software Errors
1. Faulty requirements definition 2. Client developer communication failures 3. Deliberate deviations 4. Logical design errors from software requirements 5. Coding errors 6. Noncompliance instructions with documentation and coding 7.
Shortcomings of the testing 8. User i nterface process and procedure errors 9 . Documentation errors
Example: Failure & Fault & Error
Consider a medical doctor making a diagnosis for a patient. The patient enters the doctor’s office with a list of
failures
(that is, symptoms). The doctor then must discover the
fault
, or root cause of the symptom. To aid in the diagnosis, a doctor may order tests that look for anomalous internal conditions.
In our terminology, these anomalous internal conditions correspond to errors.
Cause-and-effect relationship
Faults Errors Errors can result in errors. can lead to system failures are the effect of faults. Failures are the effect of errors Bug in a program is a fault.
Possible incorrect values caused by this bug is an error. Possible crush of the operating system is a failure
Cause & Effect Relationship
Origins of Faults
Specification Mistakes
– incorrect algorithms, incorrectly specified requirements (timing, power, environmental)
Implementation Mistakes
– poor design, software coding mistakes,
Component Defects
– manufacturing imperfections, random device defects, components wear-outs
External Factors
– operator mistakes, radiation, lightning,
class numZero {
Program Example
public static int numZero (int[] x) { // if x == null throw NullPointerException // else return the number of occurrences of 0 in x int count = 0; for ( int i = 1; i < x.length; i++) if (x[i] == 0) return count++ ; count; } }
The Fault in the Example
The fault in this program is that it starts looking for zeroes at index 1 necessary for arrays instead of index 0 , as is First Input:
numZero ([2, 7, 0])
evaluates to 1,
correctly
Second Input:
numZero ([0, 7, 2])
evaluates to 0.
incorrectly
In both of these cases the
fault
is executed. Both of these cases result in an error, Only the second case results in failure .
Error States in the Example
To understand the
error states,
we need to identify the state for the program.
The state
for
numZero
consists of values for the variables x, count , i and the program counter
(PC).
The state
at the if statement on the first iteration of the loop is ( x = [2, 7, 0], count = 0, i = 1, PC = if) This
state
is in error, because the value of i should be 0 on the first iteration.
Error States in the Example (for the first input)
However The value of count is correct The error state does not affect the output, The software does not fail . Finally: A state is in error if it is not the expected state , even if all of the values in the state are acceptable.
Error States in the Example (for the second input)
For the second input:
The state
(PC).
for
numZero
consists of values for the variables x, count , i and the program counter The
state
is (x = [0, 7, 2], count = 0, i = 1, PC = if) The error affects the variable count The error method. The is present in the
failure
results.
return value of the
Distinguish Testing from Debugging
The definitions of faul t and distinguish testing from failure debugging.
allow us to Big difference is that debugging is conducted by a programmer and the programmer fix the errors during debugging phase. Tester never fixes the errors, but rather find them and return to programmer.
Testing versus Debugging
Testing activity is carried down by a team of testers, in order to find the defect in the software. Test engineers run their tests on the piece of software and if they encounter any defect (i.e. actual results don't match expected results), they report it to the development team.
Testers also have to report at what point the defect occurred and what happened due the occurrence of that defect.
All this information will be used by development team to DEBUG the defect.
Testing versus Debugging
Debugging is the activity which is carried out by the developer.
After getting the test report from the testing team about defect(s), the developer tries to find the cause of the defect.
He has to go through lines of code and find which part of code in causing that defect.
After finding out the bug, he tries to modify that portion of code and then he rechecks if the defect has been finally removed. After fixing the bug, developers send the software back to testers.
What is Software Quality ?
According to the IEEE Software Quality is:
The degree to which a system, component, or process meets specified requirements .
The degree to which a system, component, or process meets customer or user needs or expectations.
Importance of Software Quality
Software is a major component of computer systems (about 80% of the cost) – used for – communication (e.g. phone system, email system) – health monitoring, – transportation (e.g. automobile, aeronautics), – economic exchanges (e.g. ecommerce), – entertainment, – etc.
Software defects are extremely costly in term of – money – reputation – loss of life
Software Quality Factors
Software Quality Factors
Correctness – accuracy, completeness of required output –datedness, availability of the information Reliability – maximum failure rate Efficiency – resources needed to perform software function Integrity – software system security, access rights Usability – ability to learn, perform required task
Software Quality Factors
Maintainability – effort to identify and fix software failures (modularity, documentation, etc) Flexibility – degree of adaptability (to new customers, tasks, etc) Testability – support for testing (e.g. log files, automatic diagnostics, etc)
Software Quality Factors
Portability – adaptation to other environments (hardware, software) Reusability – use of software components for other projects Interoperability – ability to interface with other components/systems
What is Software Quality Assurance?
According to the IEEE Software quality assurance is:
A planned and systematic pattern of all actions necessary to provide adequate confidence that an item or product conforms to established technical requirements.
A set of activities designed to evaluate the process by which the products are
developed or manufactured.
Contrast with: quality control.
Software Quality Assurance
Verification – are we building the product right ?
– performed at the end of a phase to ensure that requirements established
during previous phase
have been met Validation – are we building the right product ?
– performed at the end of the development process to ensure compliance
with product requirements
Three General Principles of Quality Assurance
Know what you are doing Know what you should be doing Know how to measure the difference
Verification & Validation
Verification: The process of determining whether the products of a given phase of the software development process fulfill the requirements
established during the previous phase.
Validation: The process of evaluating software at the
end of software development
to ensure compliance with intended usage.
Difference Between Verification &Validation
Verification is preventing mechanism to detect possible failures before the testing begin. It involves reviews, meetings, evaluating documents, plans, code, inspections, specifications etc. Validation occurs after verification and it's the actual testing to find defects against the functionality or the specifications
The Difference between Verification &Validation
Verification is usually a more technical activity that uses knowledge about the individual software artifacts, requirements, and specifications. Validation usually depends on domain knowledge; that is, knowledge of the application for which the software is written. For example, validation of software for an airplane requires knowledge from aerospace engineers and pilots.