Transcript Quality Management and Peer Review

University of Southern California
Center for Systems and Software Engineering
Quality Management
Peer Review
Supannika Koolmanojwong
CSCI577
University of Southern California
Center for Systems and Software Engineering
Outline
• Quality Management
– In CMMI 1.3
– In ISO 15288
– In CSCI577ab
(c) USC-CSSE
2
University of Southern California
Center for Systems and Software Engineering
Objectives of QM
• To ensure the high quality process
• in order to deliver high quality products
(c) USC-CSSE
3
University of Southern California
Center for Systems and Software Engineering
Quality Management in CMMI 1.3
Process Areas
Configuration Management (CM)
Product Integration (PI)
Causal Analysis and Resolution (CAR)
Project Monitoring and Control (PMC)
Decision Analysis and Resolution (DAR)
Project Planning (PP)
Integrated Project Management (IPM)
Quantitative Project Management (QPM)
Measurement and Analysis (MA)
Requirements Development (RD)
Organizational Performance Management
Requirements Management (REQM)
(OPM)
Organizational Process Definition (OPD)
Organizational Process Focus (OPF)
Risk Management (RSKM)
Supplier Agreement Management (SAM)
Organizational Process Performance
(OPP)
Technical Solution (TS)
Organizational Training (OT)
Validation (VAL)
Process and Product Quality
Assurance (PPQA)
Verification (VER)
(c) USC-CSSE
4
University of Southern California
Center for Systems and Software Engineering
PPQA - Product and Process Quality Assurance
(c) USC-CSSE
5
University of Southern California
Center for Systems and Software Engineering
PPQA - Product and Process Quality Assurance
(c) USC-CSSE
6
University of Southern California
Center for Systems and Software Engineering
PPQA for Agile development
(c) USC-CSSE
7
University of Southern California
Center for Systems and Software Engineering
CM – Configuration Management
(c) USC-CSSE
8
University of Southern California
Center for Systems and Software Engineering
CM – Configuration Management
(c) USC-CSSE
9
University of Southern California
Center for Systems and Software Engineering
CM – Configuration Management
(c) USC-CSSE
10
University of Southern California
Center for Systems and Software Engineering
MA – Measurement and Analysis
(c) USC-CSSE
11
University of Southern California
Center for Systems and Software Engineering
VER - Verification
(c) USC-CSSE
12
University of Southern California
Center for Systems and Software Engineering
VER - Verification
(c) USC-CSSE
13
University of Southern California
Center for Systems and Software Engineering
VAL - Validation
(c) USC-CSSE
14
University of Southern California
Center for Systems and Software Engineering
VAL - Validation
(c) USC-CSSE
15
University of Southern California
Center for Systems and Software Engineering
Quality Management in ISO 15288
Activities
a) Plan quality management.
1.
2.
3.
Establish quality management policies
Establish organization quality management objectives
Define responsibilities and authority for implementation of
quality management.
b) Assess quality management.
1.
2.
3.
Assess customer satisfaction and report.
Conduct periodic reviews of project quality plans.
The status of quality improvements on products and services
is monitored.
c) Perform quality management corrective action.
1.
2.
Plan corrective actions when quality management goals are
not achieved.
Implement corrective actions and communicate results
through the organization.
(c) USC-CSSE
16
University of Southern California
Center for Systems and Software Engineering
Configuration Management in ISO 15288
Activities
a)Plan configuration management.
1) Define a configuration management strategy
2) Identify items that are subject to configuration
control.
b)Perform configuration management
a) Maintain information on configurations with an
appropriate level of integrity and security
b) Ensure that changes to configuration
baselines are properly identified, recorded,
evaluated, approved, incorporated, and
verified.
(c) USC-CSSE
17
University of Southern California
Center for Systems and Software Engineering
Quality Management in 577ab
•
•
•
•
•
•
•
•
•
•
IIV&V
Configuration Management
Defect Reporting and Tracking
Testing
Buddy Review
Architecture Review Board
Core Capability Drive through
Design Code Review
Document template
Sample artifacts
(c) USC-CSSE
18
University of Southern California
Center for Systems and Software Engineering
Quality Guidelines
• Design Guidelines
– Describe design guidelines on how to improve
or maintain modularity, reuse and maintenance
– How the design will map to the implementation
• Coding Guidelines
– Describe how to document the code in such as
way that it could easily be communicated to
others
(c) USC-CSSE
19
University of Southern California
Center for Systems and Software Engineering
Coding Guidelines
• C: http://www.gnu.org/prep/standards/standards.html
• C++ : http://geosoft.no/development/cppstyle.html
• Java: http://geosoft.no/development/javastyle.html
• Visual Basic:
http://msdn.microsoft.com/en-us/library/h63fsef3.aspx
(c) USC-CSSE
20
University of Southern California
Center for Systems and Software Engineering
Quality Guidelines
• Version Control and History
– Chronological log of the changes introduced to
this unit
• Implementation Considerations
– Detailed design and implementation for as-built
considerations
• Unit Verification
– Unit / integration test
– Code walkthrough / review / inspection
(c) USC-CSSE
21
University of Southern California
Center for Systems and Software Engineering
Quality Assessment Methods
• Methods, tools, techniques, processes that
can identify the problems
– Detect and report the problem
– Measure the quality of the software system
• Three methods of early defect identification
– peer review, IIV&V, Automated Analysis
(c) USC-CSSE
22
University of Southern California
Center for Systems and Software Engineering
Peer Review
• Reviews performed by peers in the
development team
– Can be from Fagan’s inspections to simple
buddy checks
– Peer Review Items
– Participants / Roles
– Schedule
(c) USC-CSSE
23
University of Southern California
Center for Systems and Software Engineering
Defect Classification
• Severity : Major/ Minor
• Type: Missing / Wrong / Extra
• Category
– Logic , Syntax, Clarity. Performance, Interface,
…
(c) USC-CSSE
24
University of Southern California
Center for Systems and Software Engineering
Defect terminologies
• Activity: This is the actual activity that was
being performed at the time the defect was
discovered.
• Trigger: The environment or condition that
had to exist for the defect to surface. What is
needed to reproduce the defect?
• Impact: For in-process defects, select the
impact which you judge the defect would have
had upon the customer if it had escaped to the
field.
(c) USC-CSSE
25
University of Southern California
Center for Systems and Software Engineering
Orthogonal Defect Classification
Target
Defect Type

Operational 
Concept /

Requirements 





Design / Code



Triggers
1. Design Conformance
Correctness
2. Logic/ Flow
Completeness
3. Backward Compatibility
Consistency
Clarity / Testability 4. Lateral Compatibility
5. Concurrency
Traceability
6. Internal Document
7. Language Dependency
8. Side Effect
9. Rare Situations
10.Simple Path
Assign/Init
11.Complex Path
Checking
12.Coverage
Alg/Method
13.Variation
Func/Class/Object 14.Sequencing
Timing/Serial
15.Interaction
Interface/O-O
16.Workload/Stress
Messages
17.Recovery/Exception
Relationship
18.Startup/Restart
19.Hardware Configuration
20.Software Configuration
21.Blocked Test (previously
Normal Mode)
http://www.research.ibm.com/softeng/ODC/DETODC.HTM
(c) USC-CSSE
Impact
Severity Qualifier
1. Installability
2. Serviceability
3. Standards
4. Integrity/Security
5. Migration
6. Reliability
 Major
7. Performance
 Minor
8. Documentation
9. Requirements
10.Maintenance
11.Usability
12.Accessibility
13.Capability
 Missing
 Wrong /
Incorrect
 Extra
26
University of Southern California
Center for Systems and Software Engineering
Opener Section
(These attributes are usually available when the defect is opened.)
Defect Removal
Triggers
Impact
Activities
1.Design Conformance
2.Logic/ Flow
3.Backward Compatibility
4.Lateral Compatibility
1.Installability
5.Concurrency
2.Serviceability
6.Internal Document
3.Standards
7.Language Dependency
4.Integrity/Security
8.Side Effect
5.Migration
9.Rare Situations
Design Rev, Code
6.Reliability
10.Simple Path
Inspection, Unit test,
7.Performance
11.Complex Path
Function Test, System
8.Documentation
12.Coverage
Test.
9.Requirements
13.Variation
10.Maintenance
14.Sequencing
11.Usability
15.Interaction
12.Accessibility
16.Workload/Stress
13.Capability
17.Recovery/Exception
18.Startup/Restart
19.Hardware Configuration
20.Software Configuration
21.Blocked Test (previously Normal Mode)
(c) USC-CSSE
27
University of Southern California
Center for Systems and Software Engineering
Closer Section
(These attributes are usually available when the defect is fixed.)
Target
Design/Code
Defect Type
Qualifer
1.Assign/Init
2.Checking
3.Alg/Method
1.Missing
4.Func/Class/Object
2.Incorrect
5.Timing/Serial
3.Extraneous
6.Interface/O-O Messages
7.Relationship
(c) USC-CSSE
Age
1.Base
2.New
3.Rewritten
4.ReFixed
Source
1.Developed InHouse
2.Reused
FromLibrary
3.Outsourced
4.Ported
28
University of Southern California
Center for Systems and Software Engineering
Requirement Defect Type (1/2)
• A requirements defect is an error in the definition of system
functionality
• Correctness: Wrongly stated requirements
• Examples:
– An incorrect equation, parameter value or unit specification
– A requirement not feasible with respect to cost, schedule and
technology
• Completeness: Necessary information is missing
• Examples:
– Missing attributes, assumptions, and constraints of the software
system.
– No priority assigned for requirements and constraints.
– Requirements are not stated for each iteration or delivery
(c) USC-CSSE
29
University of Southern California
Center for Systems and Software Engineering
Requirement Defect Type (2/2)
• Consistency: A requirements that is inconsistent
or mismatched with other requirements
• Examples:
– requirements conflict with each other
– Requirements are not consistent with the actual operation
environment (eg. Test, demonstration, analysis, or
inspection) have not been stated.
• Traceability: A requirement that is not traceable to
or mismatched with the user needs, project goals,
organization goals
(c) USC-CSSE
30
University of Southern California
Center for Systems and Software Engineering
Unavoidable defects
• Changes arising because of
– The dynamic of learning
– Exploration in IKIWIKI situations
– Code or screen content reorganization taken on
as an “afterthought”
– Replacement of stubs of placeholders in code
(c) USC-CSSE
31
University of Southern California
Center for Systems and Software Engineering
Avoidable defects
• Changes in analysis, design, code or
documentation arising from human error
• Can be avoided though better analysis,
design, training
(c) USC-CSSE
32
University of Southern California
Center for Systems and Software Engineering
Product Assurance
• Requirements Verification
• IIV&V
• Automated Analysis
(c) USC-CSSE
33
University of Southern California
Center for Systems and Software Engineering
Configuration Management
•
•
•
•
•
•
Configuration items and rationale
Identification systems
Storage of configuration items
Configuration Controls
Status and accounting
Baselining events
(c) USC-CSSE
34
University of Southern California
Center for Systems and Software Engineering
Defect and Change Management
• Processes
• Change Control Board
(c) USC-CSSE
35
University of Southern California
Center for Systems and Software Engineering
COQUALMO
• Cost, Schedule and Quality
(c) USC-CSSE
36
University of Southern California
Center for Systems and Software Engineering
Current COQUALMO System
COCOMO II
COQUALMO
Software Size Estimate
Software platform,
Project, product
and personnel
attributes
Defect removal
profile levels
Automation,
Reviews, Testing
Defect
Introduction
Model
Defect
Removal
Model
(c) USC-CSSE
37
Software
development effort,
cost and schedule
estimate
Number of residual
defects
Defect density per unit of
size
University of Southern California
Center for Systems and Software Engineering
Coding defects introduction range
(c) USC-CSSE
38
University of Southern California
Center for Systems and Software Engineering
(c) USC-CSSE
Defect Removal Profiles
39
University of Southern California
Center for Systems and Software Engineering
Partion of COQUALMO Rating
Scale
COCOMO II p.263
Very Low
Low
Nominal
High
Very High
Extra High
Automated
Analysis
Simple
compiler
syntax
checking
Basic
compiler
capabilities
Compiler
extension
Basic req. and
design
consistency
Intermediatelevel module
Simple
req./design
More
elaborate
req./design
Basic distprocessing
Formalized
specification,
verification.
Advanced
distprocessing
Peer Reviews
No peer
review
Ad-hoc
informal
walk-through
Well-defined
preparation,
review,
minimal
follow-up
Formal review
roles and
Well-trained
people and
basic checklist
Root cause
analysis,
formal follow
Using
historical data
Extensive
review
checklist
Statistical
control
Execution
Testing and
Tools
No testing
Ad-hoc test
and debug
Basic test
Test criteria
based on
checklist
Well-defined
test seq. and
basic test
coverage tool
system
More advance
test tools,
preparation.
Distmonitoring
Highly
advanced
tools, modelbased test
(c) USC-CSSE
40