Transcript Kein Folientitel

ECE 453 – CS 447 – SE 465
Software Testing &
Quality Assurance
Instructor
Kostas Kontogiannis
1
Welcome
• Welcome to the Software Testing and
Quality Assurance course
• Demanding, challenging and, rewarding
course
• A glimpse of what software testing in theory
and practice is all about
2
How to reach me
• Kostas Kontogiannis
–
–
–
–
Web: www.swen.uwaterloo.ca/~kostas
E-mail: [email protected]
Tel. (ext. 2840)
Office hours:
• Monday 17:30 – 18:30
• DC 2539
3
Software Testing
CHAPTER 1
Unit 1
4
Outline for today
Introduction
• Course description
• Software Testing, Quality Assurance,
and Evolution basics
5
Course components
• 3 lectures
– SE 465 MC 4045
• 14:30-15:20 MWF
– ECE 453 / CS 447 RCH 301
• 16:30 – 17:20 MWF
• 3 tutorials
– SE 465 MC 4045
• 15:30 – 16:20 MW
– ECE 453 / CS 447 RCH 305
• 8:00 – 8:50 W
• 1 project
– Evolution
– Testing
6
Course website
•
•
•
•
http://www.swen.uwaterloo.ca/~kostas/ECE453-06/
schedule
lecture notes & slides
recommended book
– Software Testing A Craftsman's Approach
2nd edition,
Paul C. Jorgensen,
CRC Press
•
•
•
•
past exams
project description
grade allocation
course news system
7
Project component
• Design and implement
software for an IP
Telephony application
• Groups of 4
• 30% of total grade
• Two parts:
– System Enhancements
• Requirements
• Design
• Implementation
– Testing
• Black Box
• White Box
8
Goals of this course
• Become familiar with software testing concepts
• Learn how to design and plan software testing
activities
• Experience software testing in a project that uses
several components
 Note:
– “Scratching the surface of software testing and QA”
– “Fitting you to become an amateur software tester”
9
Course outline
Unit 1: Overview of Software Testing (Chapter 1-2)
• Unit 2: Mathematics for S/W Testers (Chapters 3-4)
• Unit 3: Black Box / Functional Testing (Chapters 5-8)
• Unit 4: White Box / Structural Testing (Chapters 9-11)
• Unit 5: Integration and System Testing (Chapter 12-13)
• Unit 6: Software Reliability and Quality Assurance Basics
• Unit 7: Software Reengineering and Evolution Basics
• Unit 8: Project Planning and Management
10
What to do as a first Task
• Visit the Web site
• Go to the Project section
– Complete Part I
• Administration
– Complete Part II
• Preparation
11
Outline for today
• Introduction
• Course description
Software testing basics
12
Software Lifecycle
•
•
•
•
•
•
•
•
Requirements phase
Analysis phase
Design phase (System and Object)
Implementation phase
Testing phase
Integration phase
Maintenance phase
Retirement
13
Software Lifecycle Activities
...and their models
Requirements
Elicitation
Analysis
Expressed in
Terms Of
System
Design
Structured By
Object
Design
Implementation
Implemented
By
Realized By
Verified
By
class...
class...
class...
Use Case
Model
Application
Subsystems
Domain
Objects
Testing
Solution
Domain
Objects
Source
Code
?
class.... ?
Test
Cases
14
Software Testing : Definition
“The process of devising a set of inputs to a given piece of
software that will cause the software to exercise some
portion of its code.
The developer of the software can then check that the results
produced by the software are in accord with his or her
expectations.”
www.reactive-systems.com/esda-glossary.msp
15
Software Testing : Definition
“Software testing is a process used to identify the correctness,
completeness and quality of developed computer software.
Actually, testing can never establish the correctness of
computer software, as this can only be done by formal
verification (and only when there is no mistake in the
formal verification process). It can only find defects, not
prove that there are none.”
en.wikipedia.org/wiki/Software_testing
16
A Testing Cycle
•
Although testing varies between organizations, there is a cycle to testing:
1. Requirements Analysis: Testing should begin in the requirements phase of the
software life cycle (SDLC).
2. Design Analysis: During the design phase, testers work with developers in
determining what aspects of a design are testable and under what parameter those
testers work.
3. Test Planning: Test Strategy, Test Plan(s), Test Bed creation.
4. Test Development: Test Procedures, Test Scenarios, Test Cases, Test Scripts to use
in testing software.
5. Test Execution: Testers execute the software based on the plans and tests and report
any errors found to the development team.
6. Test Reporting: Once testing is completed, testers generate metrics and make final
reports on their test effort and whether or not the software tested is ready for
release.
7. Retesting the Defects
17
Some Software Testing Objectives
•
•
•
•
•
•
•
Find as many defects as possible.
Find important problems fast.
Assess perceived quality risks.
Advise about perceived project risks.
Advise about perceived quality.
Certify to a given standard.
Assess conformance to a specification
(requirements, design, or product claims).
18
Software Testing in RUP
Phases
Process Workflows
Inception Elaboration
Construction
Transition
Business Modeling
Requirements
Architecture & Design
Implementation
Test
Deployment
Supporting Workflows
Configuration Mgmt
Management
Environment
Preliminary
Iteration(s)
Iter.
#1
Iter.
#2
Iter.
#n
Iter. Iter.
#n+1 #n+2
Iterations
Iter.
#m
Iter.
#m+1
19
Software Testing in RUP
• RUP Testing Philosophy
–
–
–
–
Iterative development.
Low up-front documentation
Holistic approach.
Automation
• Actors and Roles
–
–
–
–
Test Manager
Test Analyst
Test Designer
Tester
20
Software Testing in XP
• The biggest difference between XP projects and most
‘traditional’ software development projects is the concept
of test-driven development.
• With XP, every chunk of code is covered by unit tests,
which must all pass all the time. The absence of unit-level
and regression bugs means that testers actually get to focus
on their job: making sure the code does what the customer
wanted.
• The acceptance tests define the level of quality the
customer has specified (and paid for!)
21
Software Testing in XP
Characteristics
Planning
Every 2-3
weeks
Write tests
Release
Pair Programming
+ Refactoring
Test
Integration
Min.
daily
• Evolutionary development
• Collection of “12 Best
Practices“
• Focus on working code that
implements customer needs
(rather than documents)
• Testing is a crucial element of
the process
• Focus on flexibility and
efficiency of the process
• Designed for small teams (<10)
22
Software Testing in XP
• Activities testers perform on XP teams.
– Negotiate quality with the customer (it’s not YOUR standard of
quality, it’s what the customer desires and is willing to pay for!)
– Clarify stories, flush out hidden assumptions
– Enable accurate estimates for both programming and testing tasks
– Make sure the acceptance tests verify the quality specified by the
customer
– Help the team automate tests
– Help the team produce testable code
– Form an integral part of the continuous feedback loop that keeps
the team on track
23
Software Quality Assurance : Definition
“The function of software quality that assures that the
standards, processes, and procedures are appropriate for
the project and are correctly implemented.”
www.hq.nasa.gov/office/codeq/software/umbrella_defs.htm
24
Software process improvement
initiatives
• Capability maturity model (CMM)
– http://www.sei.cmu.edu/cmm/cmms/cmms.html
• ISO 9000-series
• ISO/IEC 15504 – Standard for Software
Process Assessment (SPICE)
– http://www-sqi.cit.gu.edu.au/spice/
25
©Steven Schach 2002 [modified]
Capability Maturity Model
• A set of strategies for improving the software
process
–
–
–
–
–
SW–CMM for software
P–CMM for human resources (“people”)
SE–CMM for systems engineering
IPD–CMM for integrated product development
SA–CMM for software acquisition
• These strategies are being unified into CMMI
(capability maturity model integration)
• Developed by Software Engineering Institute
(SEI)
©Steven Schach 2002 [modified]
26
SW–CMM
• A strategy for improving the software process
• Put forward in 1986 by the SEI
• Fundamental ideas:
– Improving the software process leads to
• Improved software quality
• Delivery on time, within budget
– Improved management leads to
• Improved techniques
• Five levels of “maturity” are defined
• Organization advances stepwise from level to
level
©Steven Schach 2002
27
SW–CMM Summary
28
©Steven Schach 2002
Software Evolution : Definition
“A continuous change from a lesser, simpler, or worse state to
a higher or better state [for a software system]”
www.cc.gatech.edu/reverse/glossary.htm
29
Software Maintenance : Definition
“Software maintenance is one of the activities in software
engineering, and is the process of enhancing, porting, and
optimizing deployed software, as well as remedying
defects”
en.wikipedia.org/wiki/Software_maintenance
30
Average cost distribution
(1976–1981 data)
Object-Oriented and Classical Software Engineer 5th Edition, Schach (2002)
31
Summary
• Testing is an important part of the Software
Lifecycle
• Highly technical and challenging
• Is affected by the selected process
• Quality Assurance is paramount both for mission
critical and non-critical systems
• Software Evolution aims to keep systems
operational when environment changes occur
32