Transcript Introduction to the Software Engineering Institute (SEI) and the Capability Maturity Model (CMM)

Introduction to the
Software Engineering Institute (SEI)
and the
Capability Maturity Model (CMM)
Paul Sesto, Software Engineering Manager
5/19/2016
1
Class Outline
• The Software Engineering Institute (SEI) and the
Capability Maturity Model (CMM)
• Importance of SEI & CMM in Industry
• Frame of Reference Project - IADS
• Major Components of the CMM
• Northrop Grumman Implementation of CMM
–
–
–
–
Software Engineering Process Organization (SEPO)
Organizational Standard Software Process (OSSP)
Software Process Asset Library (SPAL)
Software Project Planning
• Northrop Grumman Commercial
2
5/19/2016
Evolution of the SEI
• “Software Crisis” Developed over Past Few Decades
– Questionable Quality of Delivered Products
– Cost & Schedule Overruns Pervasive
– Department of Defense Catastrophes Persist
– Cost Overruns of Software Acquisition Rising by the
Millions
– Schedule Delays of Not Months but Years
– Multi-Billion-Dollar Systems Don’t Perform as Envisioned
– Projects Cancelled Due to Poor Software Quality
– Commercial Industry Suffered Similar Problems
• Complexity of Software Continues to Increase
• Industry Leaders Deemed Software Crisis to be a Management
Problem and Not a Technical Problem
3
5/19/2016
What is the SEI?
•
•
•
•
Software Engineering Institute
Federally Funded Research & Development Center
Sponsored by the U.S. Department of Defense
Contract Competitively Awarded to Carnegie Mellon University
in 1984
• CMU Staff is A Consortium of Government, Industry &
Academia
• SEI Mission
“Provide Leadership in Advancing the State of the Practice of
Software Engineering to Improve the Quality of Systems that
Depend on Software”
• Visit the SEI at www.sei.cmu.edu
4
5/19/2016
What is the Capability Maturity Model (CMM)
• Framework to Help Organizations Improve Their Processes
– Documented Practices for Software Development
– Model for Continuous Improvement
– Guidelines, not Rules, on how to Best Run Programs
– Evolving to Address Other Standards (ISO-9001)
• Method for Assessing “Maturity” of an Organization
– Five Maturity Levels
– Successive Foundations of Process Improvement
– Used by DoD to Evaluate Contractors
• Multiple CMMs Exist
– Software Development
– Integrated Product Team Development
– Systems Engineering
– Personal Software Process
• CMMI
– Initiative to Integrate Multiple Models
5
5/19/2016
Fundamental Concepts of Process Maturity
Software Process: set of
activities, methods, practices, and
transformations that people employ
to develop and maintain software
and the associated products.
internal SDP
SW Product
Underlying Premise: quality of a software
product is largely determined by the quality of the
process used to develop and maintain it.
6
5/19/2016
Five Levels of Software Maturity
Level 1: Initial
• Process is ad hoc, even chaotic;
• Success depends on individual
effort and heroics.
Level 2: Repeatable
• Basic project mgt processes established;
• Earlier successes can be repeated
on projects with similar applications.
Level 3: Defined
• Process for mgt and engineering
documented, standardized, integrated
at organization level;
• All projects use version of standard
software process.
Level 4: Managed
• Detailed measures of software process
& product quality collected;
• SW process & products are
quantitatively understood & controlled.
Level 5: Optimizing
• Continuous process improvement
enabled
• Quantitative feedback from the process
• Piloting innovative ideas & technologies.
7
5/19/2016
Practices at Each Level
2
3
4
5
X
X
X
Process
At project level only
At organization level
X
Problems
Recognized/corrected as they occur
Anticipated & prevented
Sources understood & eliminated
Common sources understood & eliminated
X
X
X
X
People
Success depends on individual
Training provided
IPTs used
Strong teamwork in projects
Strong teamwork across organization
All involved in process improvement
X
X
8
5/19/2016
X
X
X
X
X
X
X
X
X
X
Practices At Each Level
2
3
4
5
Technology Assessment
Technology support established,
stable activities
X
Planning & mgt data used by projects
X
New technologies - evaluated qualitatively
X
Data collected & used in all defined processes
X
X
X
Data systematically shared by projects
X
X
X
Data definition & collection standardized
across organization
X
X
Data used to understand process
quantitatively & stabilize it
X
X
Data used to evaluate & select
process improvements
X
9
5/19/2016
Class Outline
• The Software Engineering Institute (SEI) and the
Capability Maturity Model (CMM)
• Importance of SEI & CMM in Industry
• Frame of Reference Project - IADS
• Major Components of the CMM
• Northrop Grumman Implementation of CMM
–
–
–
–
Software Engineering Process Organization (SEPO)
Organizational Standard Software Process (OSSP)
Software Process Asset Library (SPAL)
Software Project Planning
• Northrop Grumman Commercial
10
5/19/2016
Benefits of Software Process Maturity
5
Optimizing
Continuous
Improvement
Predicted Performance
Target N-z
Focus
Probability
Level
Productivity
and
Quality
Probability
Quantitative
Product and
Process Control
Target N-y
Time/$/ . . .
4
Managed
Result
Standard
Engineering
Process
Probability
3
Defined
Target N-x
Time/$/ . . .
Project
Management
Probability
2
Repeatable
Target N+a
Time/$/ . . .
Heroes
Target N
1
Initial
Probability
Time/$/ . . .
Time/$/ . . .
11
5/19/2016
RISK
Why Bother With CMM Certification?
76% of Companies
are at Levels 1 & 2
Level 5 - 1.2%
Level 4 - 5.4%
NG ESSS
Level 3 - 17.4%
TRW
Logicon
Many USAF/Army Sites
Level 4 is necessary to
Level 2 - 24.8%
remain a leader in
today’s market!
Initial - 51.2%
12
5/19/2016
NG MASD
NG ESID
Hughes/Raytheon
LMTAS
LM - 3 Sites
NASA Ames
Boeing (Space
Shuttle)
Motorola (India)
STSC (Hill AFB)
CMM Level 3 is a DOD Requirement
100
90
80
Commerical
% Of Assessments
70
60
DOD/Fed
Contractor
50
Military/Federal
40
30
Other/Unknown
20
10
0
SEI Level
Initial
Repeatable
Defined
Managed
Optimizing
1
2
3
4
5
13
5/19/2016
Class Outline
• The Software Engineering Institute (SEI) and the
Capability Maturity Model (CMM)
• Importance of SEI & CMM in Industry
• Frame of Reference Project - IADS
• Major Components of the CMM
• Northrop Grumman Implementation of CMM
–
–
–
–
Software Engineering Process Organization (SEPO)
Organizational Standard Software Process (OSSP)
Software Process Asset Library (SPAL)
Software Project Planning
• Northrop Grumman Commercial
14
5/19/2016
IADS - The Real World
• Air Defense Requires Centralized Facility
– Integrates Country Wide Infrastructure
– Data & Voice Communications
• Network of Workstations & PCs
– Distributed Architecture & Displays
• 1.5 Million Lines of Software
– Written in Ada95, Java, PWB & SQL
• Object Oriented Design & Development
• 85 Software Engineers at Peak on 15 Integrated Product Teams
• Started with a Specification and Statement of Work of 600
Pages
• Standard Processes & Procedures Facilitated On Target
Completion
15
5/19/2016
Class Outline
• The Software Engineering Institute (SEI) and the
Capability Maturity Model (CMM)
• Importance of SEI & CMM in Industry
• Frame of Reference Project - IADS
• Major Components of the CMM
• Northrop Grumman Implementation of CMM
–
–
–
–
Software Engineering Process Organization (SEPO)
Organizational Standard Software Process (OSSP)
Software Process Asset Library (SPAL)
Software Project Planning
• Northrop Grumman Commercial
16
5/19/2016
Structure of the CMM
Maturity Levels
Contain
Indicate
Process
Capability
18 Total KPAs
Key Process Areas
Organized by
Achieve
5 Per KPA
Goals
Common Features
Contain
Address
Implementation or
Institutionalization
Key Practices
Describe
Activities or
Infrastructure
17
5/19/2016
CMM Defined Key Process Areas (KPAs)
Continuous
improvement
Measures collected
Process/products quantitatively
understood & controlled
4. Managed
3. Defined
Process standardized &
integrated at organ. level
2. Repeatable
1. Initial
•
•
•
•
•
•
5. Optimizing
•
•
•
•
•
•
•
• Quantitative process management
• SW quality management
Organization process focus
Organization process definition
Training program
Integrated software management
SW product engineering
Intergroup coordination
Peer reviews
Requirements management
SW project planning
SW project tracking & oversight
SW subcontract management
SW quality assurance
SW configuration management
Basic project
mgt processes
Success depends on
individual effort
18
5/19/2016
• Defect prevention
• Technology change management
• Process change management
Each KPA Has 5 Common Features
1. Commitment to Perform
• Written organizational policy required; addresses every KPA
• ESSS: Software Process Manual
2. Ability to Perform
• Adequate resources/training required for every KPA
• ESSS: WSAs at organization and project level
3. Activities Performed
• Activities, roles, procedures necessary to implement a KPA
• ESSS: management plans, procedures, technical directives
19
5/19/2016
Each KPA Has 5 Common Features
4. Measurement and Analysis
• Basic measurement practices necessary to determine status
related to the process
• ESSS: metrics (technical & management), Level 4 Software
Quantitative Management
5. Verifying Implementation
• Steps to ensure that the activities are performed in compliance
with the process that has been established.
• ESSS: Project and senior management reviews, software quality
audits, SEI and ISO audits, reports to SEPO
20
5/19/2016
KPA Example - Peer Reviews
• Peer review activities are planned
• Defects in the software work products are identified
and removed
Maturity Levels
Contain
Indicate
Process
Capability
Key Process Areas
Organized by
Achieve
Goals
Common Features
Contain
Address
Implementation or
Institutionalization
Key Practices
Describe
Activities or
Infrastructure
21
5/19/2016
Peer Review - Key Practices
1. Commitment to Perform
• The project follows a written organizational policy for performing
peer reviews.
Work Product
Inspections
ESSS
2. Ability to Perform
• Adequate resources and funding are provided for performing
peer reviews on each software work product to be reviewed.
• Peer review leaders receive required training in how to lead
peer reviews.
• Reviewers who participate in peer reviews receive required
training in the objectives, principles, and methods of peer
reviews.
22
5/19/2016
Peer Review - Key Practices
3. Activities Performed
• Peer reviews are planned, and the plans are documented.
• Peer reviews are performed according to a documented
procedure.
• Data on the conduct and results of the peer reviews are
recorded
4. Measurement and Analysis
• Measurements are made and used to determine the
status of the peer review activities
5. Verifying Implementation
• The SW Quality Group reviews and/or audits the activities and work
products for peer reviews and reports the results.
23
5/19/2016
KPA Summary
Key Process Area
KPAs Describe What
is to be Done to Satisfy
the Desired Goals
• Goals
Common Features
During an Assessment
Each KPA is Evaluated
for Objective Evidence
of the Common Features
for Organizational
Institutionalization
• Commitment to Perform
• Ability to Perform
• Activities Performed
• Measurement and Analysis
• Verifying Implementation
24
5/19/2016
Class Outline
• The Software Engineering Institute (SEI) and the
Capability Maturity Model (CMM)
• Importance of SEI & CMM in Industry
• Frame of Reference Project - IADS
• Major Components of the CMM
• Northrop Grumman Implementation of CMM
–
–
–
–
Software Engineering Process Organization (SEPO)
Organizational Standard Software Process (OSSP)
Software Process Asset Library (SPAL)
Software Project Planning
• Northrop Grumman Commercial
25
5/19/2016
Software Engineering Process Organization
• SEPO is the Enterprise Governing Organization
• Provides Project Support
– Software Project Planning
– Process Definition
– Process Improvement
– Artifact Collection
26
5/19/2016
Process Definition & Project Planning
CPP
Template
Organization’s
Standard
Software
Process
• Policies
• Procedures
• Templates
Software Process Group
• Process Training
• SEPO Representative
Integrated
Dev. Plan
Integrated
Dev. Plan
Command
Media
Project
Planning
Project
Web Page
• Tailor Standard Process
27
5/19/2016
Project’s
Comprehensive
Program
Plan (CPP)
Project’s
Internal
Software
Development
Plan
(iSDP)
• Management Plans
• Technical Directives
• Resource Estimates
SEPO Provides Process Improvement
Process
Improvements
Software Process
Group / PICT
Development / PICT
Team
• Analyze Metrics
• Develop and Improve Process
Project
Process
Improvements
Process
Updates
Process Pilots
Organization’s
Standard
Software
Process
Command
Media
Integrated
Dev. Plan
Integrated
Dev. Plan
• Policies
• Procedures
• Templates
Planning
and Process
Improvement
• Periodic Process
Evaluation
28
5/19/2016
Project
Web Page
Project’s
Internal
Software
Development
Plan
(iSDP)
• Management Plans
• Technical Directives
• Resource Estimates
Organizational Standard Software Process
• OSSP Facilitates Organizational Standard
Processes
– Accessible via Corporate Intranet Web-Site
– Supports SEI Level-4 Objectives
– Promotes Standardization
– Increases Software Productivity & Quality
29
5/19/2016
Software Process Asset Library (SPAL)
ORGANIZATION STANDARD SOFTWARE PROCESS
• Policies
Software Process Manual
Resources
Compliance
Checklist
(CCL)
Plan &
Directive
Templates
Software
Policies
• Procedures
Procedures
• Compliance Checklists
Work
Product
Specs
(WPS)
Misc.
Templates
Std. OSSP
Forms
iSDP Generation Manual
Help
Guides
Training
Acronyms
Definitions
Process
Pilots
• iSDP Generation
Tools
SOFTWARE RELATED DOCUMENTS
• Tailoring Process
• Training
Software Engineering
Process
Organization (SEPO)
Process &
Technology
Library
Software
Reuse Archive
(Future)
ORGANIZATION SOFTWARE PROCESS DATA BASES
Software Process
Measurement
Data Base
30
5/19/2016
Project
Examples
Software Training
Records
Data Base
Project
Compliance
Data Base
Software Project Planning is Simplified
Organization Standard Software Process
Internal SDP
Software
Software
Metrics/
Tool
Training
Standards
Databases
1. Project
IPT Org
2. System
Definition
(for planning)
Software
Policies and
Procedures
3. Project
Life Cycle
Organizatio
n
Standard
Process
Assets
Command
Media
4. Management
Plans
Project
Web Page
5. Technical
Directives
Project’s
Comprehensive
Program
Plan
6. Size, Cost,
Schedule
Estimates
Project Planning
Process
7. Approvals
. . . When Projects Tailor from the Standard Process
31
5/19/2016
Templates Exist for Each Plan
ESSS SPAL
Standard Procedures
iSDP
Planning Step 1 - Develop
Project IPT Operations Plan
IPT Organization Plan
Compliance
Checklist
(CCL)
Planning Step 2 - Define
System to Support Planning
Ref. Development Directives
Ref. Risk Management Plan
Procedures
Planning Step 3 - Define
Project Life Cycle
Project Life Cycle Definition
Planning Step 4 - Define
Management Plans
Requirements Management Plan
Project Indicators
Risk Management Plan
S/SEE Implementation Plan
Software Maintenance Plan
Software Reuse Plan
Subcontract Management Plan
Software CM Plan
Software QA Plan
Training Plan
Planning Step 5 - Define
Project Directives
Development and Review
Directives
Planning Step 6 - Estimate
Scope and Resources
Estimation Methods
Planning Step 7 - Review and
Deliver Planning Products
iSDP Compliance and
Approval
32
5/19/2016
Work
Product
Specs
(WPS)
Templates
Standard Tool Usage . . .
ESSS SPAL
Standard Procedures
Compliance
Checklist
(CCL)
iSDP
Planning Step 1 - Develop
Project IPT Operations Plan
IPT Organization Plan
Planning Step 2 - Define
System to Support Planning
Ref. Development Directives
Ref. Risk Management Plan
Planning Step 3 - Define
Project Life Cycle
Project Life Cycle Definition
Planning Step 4 - Define
Management Plans
Requirements Management Plan
Project Indicators
Risk Management Plan
Templates
S/SEE Implementation Plan
Planning Step 5 - Define
Project Directives
Software Maintenance Plan
Software Reuse Plan
Subcontract Management Plan
Software CM Plan
Software QA Plan
Training Plan
Development and Review
Directives
Planning Step 6 - Estimate
Scope and Resources
Estimation Methods
Planning Step 7 - Review and
Deliver Planning Products
iSDP Compliance and
Approval
• Supports Standard
Processes
• Reduces Tool
Licensing Costs
• Reduces Tool Training
& Support Costs
• Increase Software
Reuse
. . . Reduces Software
Development Costs
33
5/19/2016
Procedures
Work
Product
Specs
(WPS)
Training is CMM Requirement
SPAL
Standard Procedures
Planning Step 1 - Develop
Project IPT Operations Plan
IPT Organization Plan
Compliance
Checklist
(CCL)
Planning Step 2 - Define
System to Support Planning
Ref. Development Directives
Ref. Risk Management Plan
Procedures
Planning Step 3 - Define
Project Life Cycle
Project Life Cycle Definition
Planning Step 4 - Define
Management Plans
iSDP
Templates
Requirements Management Plan
Project Indicators
Risk Management Plan
S/SEE Implementation Plan
Software Maintenance Plan
Software Reuse Plan
Subcontract Management Plan
Software CM Plan
Software QA Plan
• New Technologies
• Software Process
• Defined by Roles
Training Plan
Planning Step 5 - Define
Project Directives
Development and Review
Directives
Planning Step 6 - Estimate
Scope and Resources
Estimation Methods
Planning Step 7 - Review and
Deliver Planning Products
iSDP Compliance and
Approval
.
34
5/19/2016
Work
Product
Specs
(WPS)
OSSP Core Curriculum
Introductory
Executives &
Program
Managers
Intermediate
Software Development Overview
for Program Managers
Specialty Topics
Quantitative Software Mgmt
for Executives
Requirements Management
Software
Managers &
Software
Technical Leads
Software Project Estimation
Integrated
Product
Teams
(IPTs)
(ESSS
Software
Development
Engineering
Course)
Lifecycle
Overview
Introduction
to the
CMM
and the
ESSS
Organizat’l
Standard
Software
Process
Managing
Software
Development
Software Project Planning
Training Plan Development
Subcontract Management
Work
Product
Inspections
Quantitative Software Mgmt
Software Creation
Engineers
Software Testing
Software Quality Assurance
Software Configuration Mgmt
35
5/19/2016
Software Standard Tools Web Page
Electronic Format of the Software Tools Web Page Is Located on the Intranet At:
http://spal.md.essd.northgrum.com
 click on SPAL
 click on Tools on the SPAL diagram
36
5/19/2016
Standard Metrics . . .
Project C Data
Project B Data
Refine Project
Process
Project A Data
Metrics
Database
Improve
Organizational
Process
Support New
Project Planning
Efforts
… Provide Project & Process Insight
37
5/19/2016
Class Outline
• The Software Engineering Institute (SEI) and the
Capability Maturity Model (CMM)
• Importance of SEI & CMM in Industry
• Frame of Reference Project - IADS
• Major Components of the CMM
• Northrop Grumman Implementation of CMM
–
–
–
–
Software Engineering Process Organization (SEPO)
Organizational Standard Software Process (OSSP)
Software Process Asset Library (SPAL)
Software Project Planning
• Northrop Grumman Commercial
38
5/19/2016
Summary
The SEI was established and funded by the DoD
to address the “Software Crisis”. It provides
leadership to advance the practice of software
engineering to improve the quality of systems.
The CMM represents the model
by which software organizations are
assessed to establish organizational
process maturity (5 Levels).
CMM
The quality of a software product
is largely determined by the
quality of the process used to
develop and maintain it.
Process
Maturity
Northrop
Grumman
Northrop Grumman’s CMM Implementation
is governed by SEPO and implemented
using OSSP & SPAL.
39
5/19/2016
SEI