Transcript Document

A Process Tale:
Experiences at
the NDIA CMMI
Technology
Conference &
User Group
17-20 November
2003
Presented to the Southern California SPIN
Warren Scheinin
Systems Engineer, NG Mission Systems
December 5, 2003
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
Topics
 Some highlights from the Conference
 My Presentation on A Project’s Transitioning
from CMM L3 to CMMI L3 S/W
–The Challenge
–Our Strategy
–Initial Steps
–Lessons Learned
 Networking Break
 Extracts from other presentations
Note: [Words in square brackets are my opinion]
Southern California SPIN 12/05/03
2
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
3RD Annual CMMI Technology Conference
 Held November 17 – 20, 2003 at Hyatt Regency Technical
Center, Denver, CO
– About a half an hour from the airport
– Room rates about 4 times the rate at hotels 2 to 4 miles away
– Very strange layout
– Very nice, business friendly rooms with refrigerator and coffee pot
– No electrical connections in meeting rooms for laptops!!!!!
 Conference did not seem to have lessons learned
– No registration Sunday night, November 16
– Registration lines first day very long since many tutorial signups were
not available, causing tutorials to start a half hour late
– Instructors did not know how many students to expect
– No handouts at tutorials
Southern California SPIN 12/05/03
3
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
Day 1 Tutorial 1
Going Beyond Death by Slides for Process
Suzie Garcia and Shaun Presson, SEI
 Methods for making training more interesting, thus increasing
you ability to keep the attention of the class and increase
retention of knowledge. Lots of good ideas for keeping your
class from falling asleep during your presentation while
managing your time. Also increases participation by and
interaction between attendees.
 Four types of training introduced
– How is my toy like <x> ?
– Chaos cocktail party - Why the training module died?
– Computer Based Training/Industrial Design, Video
– PowerPoint
Southern California SPIN 12/05/03
4
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
Day 1 Tutorial 2A – Business Case Analysis –
Achieving and Maintaining Buy In
Jacobs Sverdrup
 A way to assess your need for and how to scope your CMMI
effort.
 An investment decision for CMMI implementation which
assumes that the decision to adopt the CMMI has not yet been
made. Provides arguments for why (ROI) you should go for it.
 Designed for Senior Leaders and Process Group Leads.
 Consisted mostly of attendees filling out forms.
 Finished early.
Southern California SPIN 12/05/03
5
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
Day 1 Tutorial 2B
CMMI Maturity Level 3 Tutorial
Tim Kasse, Kasse Innitiatives, LLC
 Checked out other tutorials since second tutorial ended early
 Warned attendees that many areas appraised as CMMI Level 3
were not operating at that level.
– Cited a car company as an example
– One lady came up at end of presentation worried about her automatic
brakes failing
 Walked attendees through process areas showing how they
should be interpreted to be process improvement opportunities
 Great description of Product Integration process as a iterative
activity.
Southern California SPIN 12/05/03
6
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
Day 2 Plenary Session
 Registration went much better
 Met lots of old friends from Hughes/Raytheon,
Logicon/NG, and other parts of Northrop Grumman
 Introduction was a sales pitch on being an NDIA member
 Several presentations followed (see next TBD slides)
available at www.dtic.mil/ndia/2003CMMI/2003CMMI.html
 Lunch was excellent (New York strip I believe),
especially speak
 Mine was the third presentation after lunch
Southern California SPIN 12/05/03
7
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
Seven Plenary Presentations [took the whole morning]
•
Impact of CMMI® on Systems Engineering by Dev A. Banerjee
•
Fiscal Year 2003 Survey of Acquisition Project Managers by Mark Kasunic
•
CMMI® Today,,, the US Army Perspective by Dave Castellano
•
CMMI® Today,,, The Current State Ron Paulson
•
CMMI® Today: The Current State by Bob Rassa
•
Next Generation Systems Engineering and CMMI by Mark Schaeffer
•
CMMI® The Current State by Gregory Shelton
Southern California SPIN 12/05/03
8
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
Impact of CMMI on Systems Engineering
Dev A. Banerjee
Boeing – Integrated Defense Systems
November 18, 2003
Southern California SPIN 12/05/03
9
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
Outline
 Systems Engineering (SE) Evolution
 Clear Understanding of SE
 Detractors for Deploying CMMI
 Boeing is Committed to CMMI
 Boeing Enterprise Process Model
 CMMI is Means to Greater Commonality
 CMMI is an Opportunity for SE
Southern California SPIN 12/05/03
10
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
Systems Engineering is Evolving
1950
Implementations
Simple Analog
Hardware
Paradigm Shift
Individual Systems
Engineer
2003
Complex, Interactive,
Software Intensive,
Interoperable Systems
2030
Self-Organizing Ad Hoc
Network Systems
Multi National /
Multidisciplinary Teams
Company Teams
Following Integrated
Processes (CMMI)
Domains
C
C2
C3
C4ISR
Interconnected Systems, Counter
Command, Control,
Stand Alone
Intelligence, Security,
Communications, Computers,
Systems
Misinformation, Information
Intelligence, Surveillance
Isolation
SE Approaches
SoSE Considerations (JTA,
Structured Analysis &
UML
Interoperability, DoD
Functional Decomposition
Modeling
Architectural Views)
Process Standards
Ad Hoc
Processes
Tools
Simple, Disconnected
Models
Team Focus
IPT’S
MIL–STD
499
EIA–632 & 731
IEEE-1220
CMMI
Integrated Tool Suites
Interconnected Performance
Models, Data Bases & Data
Use of Self-Tailoring
Expert System /
Intelligent Process Tools
Intelligent / Adaptive
Tools
Southern California SPIN 12/05/03
11
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
Fiscal Year 2003 Survey of
Acquisition Project Managers
Army Strategic Software Improvement Program (ASSIP)
AAG Meeting
Nov. 5, 2003
Mark Kasunic
Software Engineering Measurement and Analysis [SEMA]
Software Engineering Institute
[email protected]
Southern California SPIN 12/05/03
12
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
Two Most Difficult Problems Facing Acquisition Managers - 1
63
Response Categories
Requirements Management
Project Management
22
Contractor processes
22
21
Unstable Funding
16
Required Skills
12
Interoperability
9
Integration
Testing
8
Mandates
8
6
Technology Change
4
Policy
8
Other
18
No Response
8
N/A
0
225 Responses
Southern California SPIN 12/05/03
10
20
30
40
50
60
70
Frequency
13
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
CMMI ® Today…
the US Army Perspective
Dave Castellano
US Army Representative to CMMI Steering Group
4 August 2003
US Army Armaments Research Development & Engineering Center
Southern California SPIN 12/05/03
14
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
The Army is committed to Process Improvement
 ARDEC – became the World’s First CMMI Level 3
(Armaments RDE Center)
– Based on CMMI-SE/SW/A, version 1.02d
 CERDEC – became DoD’s First CMMI Level 5
(Communication – Electronics RDE Center)
– Based on CMMI-SE/SW
 TARDEC – investigating migration to CMMI
(Tank-
Automotive RDE Center)
– Software Engineering Level 3 – August 2001 SW-CMM, version 1.1
 PM Abrams – became Army’s First SA-CMM application
(PEO Ground Combat Systems)
– Based on SA-CMM, version 1.02
 AMRDEC – currently transitioning from CMM to CMMI
(Aviation & Missile RDE Center)
– S/W Engineering Level 4 – Fiscal Year 2000 SW-CMM, version 1.1
Southern California SPIN 12/05/03
15
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
ARDEC Lessons Learned [All True IMHO]
 Active senior management involvement is a prerequisite.
 Communication, training and coaching are all essential and
must be integrated.
 Measurement effort should start early and be kept streamlined.
 Process group should include participation from all project
leaders.
 A central Process Asset Library and repository are basic and
must be actively stocked and maintained.
 Active integration of the quality assurance function
into the process improvement effort.
– Periodically assess project activities & artifacts.
– Target the audits to assess compliance with CMMI.
 CMMI does not address the quality of the framework.
Southern California SPIN 12/05/03
16
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
CMMI® Today – The Current State
CMMI® Technology Conference 2003
November 18, 2003
• Ron Paulson
Vice President, Engineering
Lockheed Martin Corporation
Southern California SPIN 12/05/03
17
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
Lockheed Martin M&DS* Award Fee
Award Fee vs CMM Level
SW CMM L2: 1993-1995
SW CMM L3: 1996-1997
SW CMM L4: 1998-1999
SW CMM L5: 2000-2001
CMMI L5: 2002
Additional
Award Fee
Achieved =
55% of the
Potential
Additional
Award Fee
Per Cent
CMMISM
0
SW CMM L2
SW CMM L3
SW CMM L4
SW CMM L5
CMMI L5
* Management & Data Systems
Southern California SPIN 12/05/03
18
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
How is Lockheed Martin implementing
CMMI®?
• Corporate policy requires CMMI® adoption
• CMMI® is being addressed in the context of our business
requirements
– Our Integrated Engineering Process (LM-IEP) standard includes CMMI®
– CMMI® complements our lean and six sigma initiatives
• SCAMPISM is being used to meet customer requirements
• An incremental appraisal approach is being deployed
– Lockheed Martin’s Continuous Appraisal Method (CAM) is efficient and
effective
– CAM meets ARC* Class A method requirements
Southern California SPIN 12/05/03
19
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
The Road Ahead
• CMMI® models need to be expanded for better
– Life cycle coverage, e.g., maintenance, acquisition
– Enterprise scope coverage, e.g., HW engineering, finance,
strategic planning, workforce management
– Mechanisms for adding specialty areas such as safety,
security and services to the model’s core
• Incremental, delta and multiple certificate appraisal
options are needed in SCAMPISM for more efficient
and effective appraisals
• Process improvement benefits are continuing with
CMMI® but better ROI data is needed
Southern California SPIN 12/05/03
20
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
CMMI Today:
The Current State
Bob Rassa, Raytheon
Industry CMMI Chair
Software Engineering Institute
Nov 18, 2003
Southern California SPIN 12/05/03
21
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
Introduction
•Characterize the adoption of CMMI
•Provide a first look at the results from the Standard CMMI Appraisal
Method for Process Improvement (SCAMPISM V1.1) Class A
appraisal method using Capability Maturity Model Integration
(CMMI) V1.1 *
•Provide latest information on CMMI current activities
•Discuss future planning for the CMMI Product Suite
•* Organizations previously appraised against CMMI V1.0 that have not reappraised against V1.1 are
• not included in this report.
•Please visit http://www.sei.cmu.edu/sema/profile_about.html for additional information or questions you may
have about this briefing before contacting the SEI directly.
Southern California SPIN 12/05/03
22
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
Summary Organizational Maturity Profile
100%
90%
% of
Organizations
80%
70%
60%
50%
37.9%
40%
27.3%
30%
19.7%
20%
10.6%
10%
4.5%
0%
Initial
Managed
Southern California SPIN 12/05/03
66
D efined
Quantitatively
Managed
Optimizing
23
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
Nu m b e r o f As se s sm e n ts Rep o rte d to th e S E I b y Y ea r
5 No v e m b e r 2 00 3
600
500
400
300
200
100
0
1987 1988 1989 1990 1 991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003
S PA
S EI C B A IPI
SC A M P I vX
Southern California SPIN 12/05/03
24
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
100 appraisals have been reported to the SEI in the 13 months since the April 2002
release of the SCAMPI Class A Version 1.1 appraisal method for CMMI Version 1.1.
Number of SCAMPI V1.1 Appraisals Conducted by Month
Reported as of 31 October 2003
16
14
12
10
8
6
4
2
0
Apr-02
May-02
Jun-02
Jul- 02
A ug-02
Sep-02
Oct-02
Nov-02
Dec-02
Jan-03
Feb-03
Mar-03
Apr-03
May-03
Jun-03
Jul-03
Aug-03
Sep- 03
Southern California SPIN 12/05/03
25
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
CMMI Future Plans
 CMMI Project made stability commitment at Dec 2001 release of V1.1
– No model changes for at least three years
– Designated nominal change cycle of five years
 Sunset of SW-CMM begins Dec 2003, completed by Dec 2005
– No more formal training
– No further Lead Assessor authorizations
 Planning process for CMMI v1.2 “next iteration” has started
– Open call for additional Change Requests (CRs) made
 Final submittal date is December 12, 2003 [Yikes]
– New CRs plus previously submitted CRs will form basis of update deliberations by
CMMI Steering Group and Product Development Team & Expert Groups
Southern California SPIN 12/05/03
26
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
CMMI Future Plans: The Process
• All CRs will be carefully evaluated:
– Steering Group (SG) establishes criteria
– Product Team/Expert Groups perform analysis
– Summary Recommendations made for candidate Change Packages (CPs)
• SG validates/confirms
– Initial CPs go to Configuration Control Board (CCB) [Not Level 1 !]
– CCB Approves/Rejects/Rejects With Comments
– Product Team acts
• Approved initial CPs are developed into Implementation Packages (IPs)
• Reject-With-Comment CPs are redirected per comment
– IPs submitted to CCB
• CCB Approves/Rejects individual IPs
– Approved IPs incorporated into Product Suite
– Planned release of update: between 4Q05 and 4Q06
Southern California SPIN 12/05/03
27
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
Next Generation Systems Engineering and
CMMI
November 2003
Mark Schaeffer
Principal Deputy, Defense Systems &
Director, Systems Engineering
USD(AT&L) Imperatives
 “Provide a context within which I can make decisions about
individual programs.”
 “Achieve credibility and effectiveness in the acquisition and
logistics support processes.”
 “Help drive good systems engineering practices back into the
way we do business.”
Southern California SPIN 12/05/03
28
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
Execute the “Big Picture” – A Cartoon
“He thinks we can do it.”
Southern California SPIN 12/05/03
29
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
“New Paradigm”
• DoD 5000 Series and CJCSI 3170.01C have been recast
• Both address capabilities-based approach to acquisition based
on joint integrated architectures
RGS
Integrated at
Department
Systems
Requirements
JCIDS
National
Military
Strategy
Joint
Vision
Joint Concept of Operations
Joint Concepts
Integrated Architectures
Joint Capabilities
Bottom Up, Often Stovepiped
Southern California SPIN 12/05/03
Top Down, Born Joint
30
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
DoD Architecture Framework
An architecture is
“the structure of components, their interrelationships, and the principles and guidelines
governing their design and evolution over time.”
Source: DoD Integrated Architecture Panel, 1995
Based on IEEE STD 610.12
Operational View
Systems View
What the warfighter wants
to do and how
Technical View
What systems to bring
together and how to
organize them to provide
capability
How to put the pieces
together
Southern California SPIN 12/05/03
31
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
Role of CMMI
• The initial vision for CMMI was to integrate the competing
maturity models and provide more consistent process
improvement
• What CMMI really does is cause integration of the functional
disciplines in their application in organizations and on
programs
• It has also caused tremendous increases in systems engineering
process maturity as organizations migrate from the sunsetting
SW-CMM to CMMI
Southern California SPIN 12/05/03
32
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
CMMI® – The Current State
Presented by Gregory Shelton
Corporate Vice President
Engineering, Technology, Manufacturing & Quality
3rd Annual CMMI Technology Conference & User Group
18 November 2003
Southern California SPIN 12/05/03
33
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
Our mechanisms for process and
technology enterprise integration
Integrated Product
Development System
(IPDS) provides an
integrated set of best
practices for the
entire product
development life
cycle through a
program tailoring
process.
Our technology will flourish with process discipline
Raytheon Six Sigma
guides us to use CMMI
and IPDS as tools to
deliver value to
customers and integrate
industry best practices.
Capability Model
Maturity Integration
(CMMI) provides the
process requirements
and appraisal methods
for creating, measuring,
managing, and
improving specific
processes.
Southern California SPIN 12/05/03
34
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
In Summary
 Advances in technology and systems integration are
driving CMMI implementation
 True adoption will improve our program performance
in Cost, Schedule, and Risk
 CMMI at Raytheon involves our whole Enterprise with
leadership support
 Raytheon has demonstrated productivity successes
[Examples in presentation] with the SW-CMM. Similar
(greater) successes are expected with CMMI.
 CMMI is our model for process excellence and will be
our Customers’ model
Southern California SPIN 12/05/03
35
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
[Lunch Time Speaker -something to chew on]
• 3rd Annual CMMI Technology Conference
and User Group
Kent Schneider, President
Defense Enterprise Solutions
Northrop Grumman Information Technology
November 18, 2003
Southern California SPIN 12/05/03
36
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
Hughes Radar
Me
Raytheon
Organizational Process Maturity Pedigree
1999
Logicon LISS
2001
2000
2002
L3
ENABLER
Logicon LAT
(to other units)
L3
Logicon LTS
Logicon LIS
LIEB
Northrop Grumman
Information Technology
CMMI
L5
Defense Enterprise Solutions
L5
Litton PRC
(to other units)
L3
SPII
TRW
Litton TASC
(to other units)
Southern California SPIN 12/05/03
37
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
Key Process Insertion Differences
• Two projects with:
–
–
–
–
–
Same customer (different divisions)
Similar applications
Both Client-Server (PowerBuilder / Oracle)
Similar level of team expertise and training
Estimated to be similar size and effort (~$2 Million)
• Project A:
– Disciplined team design process used to create sound developer work packets
– Personal Software Process used consistently by developers
 SEI developed course for software developers which provides process at an individual
level for producing software components and documentation (user & technical)
– Peer Reviews conducted on the most critical 20% of the software
• Project B:
– We decided that this project was “too far along” to benefit from process insertion
Southern California SPIN 12/05/03
38
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
Project Test Time / Cost / Quality
Project A
Integration / Acceptance Test Cost
Project B
$78.K
$612K
$0.95K
$4.05K
Time to Accept (months)
3.7
14.6
Normalized (months per 100 KLOC)
4.5
9.7
31.8
43.0
Size (KLOC)
82
151
Developer Defect Density
9.4
17.3
Peer Review Exit Density
4.78
17.3
Delivered Defect Density
1.55
5.27
Normalized (per KLOC)
Duration (months)
Southern California SPIN 12/05/03
39
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
Positive Effect of Process Insertion
•
•
•
Mature Processes & PSP led to
– Project A’s Integration and Acceptance Test Costs were 24% of Project B
– Project A’s Test Duration was 46% of Project B
– Project A’s delivered defect density was 29% of Project B
Process saved cost, reduced schedule, and improved quality
CMMI Practices have a direct and positive influence on cost, schedule, and quality
performance
– It helps to tie the quantitative goals and measures to customer needs and relevant project
problems
•
CMMI high maturity enables organizational and project agility in high priority areas
– Projects can improve dramatically in 6 months or less using defect prevention cycles of 1
week or less
•
The transition from SW-CMM to CMMI is not inherently difficult, costly, and timeconsuming, but…
– It pays to proactively improve against emerging models and standards
– It is much easier to attain the same maturity level the second time
– It is much easier to transition between models when you focus on performance
Southern California SPIN 12/05/03
40
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
Session C
Two Papers Before Mine
Southern California SPIN 12/05/03
41
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
Applying CMMI® Generic Practices
with Good Judgment
• CMMI Technology Conference and User Group
• 17-20 November 2003
Rick Hefner, Ph.D.
Northrop Grumman
Geoff Draper
Harris Corporation
Southern California SPIN 12/05/03
42
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
Implementing the CMMI Model
Process Architecture - 2
REQM
PP
Project
Management
Policy
Project
Management
Process
…
PMC
Estimating Process
Option #2: Thought and
Judgment
•
SAM
MA
Lifecycle Processes
Quality
Policy
PPQA
CM
Config Mgmt
Policy
Engineering
Development
Process
RD
TS
Engineering
Policy
Advantages:
•
•
•
O&M
Process
PI
…
VER
OPD
Process
Improvement
Policy
OT
IPM
Training
Policy
RSKM
• Processes
• Procedures
• Methods / Guidance
• Checklists
• Templates
• etc.
•
Indirect mapping and CMMI
tailoring could complicate
appraisal risk
Reduced visibility of PAbased generic practices for
objective evidence
…
DAR
•
…
OPF
Model tailoring based on
business value
Emphasize key subprocesses
Processes more intuitive to
implement and
institutionalize
Disadvantages:
R&D
Process
VAL
Processes organized to fit the
business and culture
Southern California SPIN 12/05/03
43
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
GP 2.2 – Plan the Process
Establish and maintain the plan for performing the process.
Implementation
Guidance
 Plan = description of activities + budget + schedule
– Includes other generic practices (e.g., process descriptions, resources,
responsibilities, CM)
 Must distinguish between plans (GP 2.2) and process descriptions
(GP 3.1)
– Much of the plan is replaced by the process description
Appraisal Guidance
 Schedules may be tied to program events as opposed to shown on
a Gantt chart
– E.g., DAR events
– Plans should make clear the conditions under which a DAR is to be conducted
[Replicated for all 12 Generic Practices, very practical]
Southern California SPIN 12/05/03
44
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
“Two Steps Forward – One Step Back:
A New Dance Step Or Interpreting The
CMMI Model”
[Presentation right before mine, now I’m really nervous]
Presented to National Defense Industrial Association
3nd Annual CMMI Technology Conference and User
Group
Denver, Colorado
Adrio DeCicco
Space and Airborne Systems
Raytheon El Segundo, CA
November 17-20, 2003
Southern California SPIN 12/05/03
45
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
•SAS Experienced Some Diversified Opinions Of
Interpreting The CMMI Model To Measure Process Maturity
And Implementing The Organization’s Standard Process To
Make Process Improvements
–Utilized Several Raytheon And Consultant Personnel In Attempting
To Understand The Meaning And Intent Of The Model
•Started With Evaluating How To Make The Existing
Organization Standard Process CMMI Compliant And
Defining The Architecture
Southern California SPIN 12/05/03
46
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
•The Interpretation Of The Meaning Of “Defined Process” Was Addressed
With Differing Opinions
–Processes Should Be Sufficiently Defined To Answer “What To Do” And “How To
Do It”
–Opinions Varied On The Level Of Detail Needed When Developing New Procedures, Work
Instructions, And Enablers
–On The One Hand, It Was Document Your Process The Way You Conduct Business
–Don’t Copy The CMMI Model As Part Of The Organization’s Standard Process
–On The Other Hand, It Was The Organization’s Standard Process Does Not Contain
Enough Information Showing How To Do Tasks (Both The Existing And CMMI Based)
–Enablers Being “Informational” Were Viewed As Not Acceptable By Some Because They
Are Not Directive
–Some Were Of The Opinion That All Sub-practices Had To Be Addressed To Be Compliant
•As A Result The Level Of Detail In Work Instructions Varied Widely As Did
The Criteria For Determining When A Work Instruction Was Needed
–Addressed Sub-practices In Terms Of Value Added To Process And Product
Development For SAS’ Businesses
Southern California SPIN 12/05/03
47
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
–From Process And Product Quality Assurance (PPQA): May Be Performed,
Partially Or Completely, By Peers
Everyone Performing QA Activities Should Be Trained In QA
Those Performing QA Activities Should Be Separate From Those Directly Involved
In Developing Or Maintaining The Work Product And Have An Independent Reporting
Channel To The Appropriate Level Of Organizational Management
•Two Steps Forward:
–Maintained The Position That Quality Is Everyone’s Job
Improved Objective Evidence (OE) Focus Within Disciplines
–Identified Strengths And Weaknesses
Perform OE On Products Very Well; Need To Improve OE Of Processes
–Developed Additional Audit Checklists
–Working With the QA Organization To Implement OE Across
Disciplines
•One Step Backward: Time And Effort Spent:
Developing An Integrated Approach To OE
Getting Buy In From Disciplines
Southern California SPIN 12/05/03
48
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
• Two Steps Forward:
– Implementing CMMI Based PIs On Legacy Programs
• Introduce Process Improvements As Legacy Programs Enter A Major Milestone
– Tailor The Organization’s Standard Process From That Point Forward
– Have A Senior Systems Engineer Or A Senior Software Engineer Be
Responsible For Applicable Systems Engineering Activities On
Predominantly Software Programs
– Increased Scope (Process And Product) Of Capturing Lessons Learned
– Developed An “Independent Review” Detailed Work Instruction
• Two Steps Backward:
– Implementing CMMI Based Process Improvements On Legacy Or
Predominantly Software Or Hardware Programs Is Difficult, And
Seems To Carry Little Value During Appraisals
– Time And Effort Trying To Produce Lessons Learned Artifacts For
Every CMMI Process Area
Southern California SPIN 12/05/03
49
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
And now, our featured presentation:
A Project’s Tale: Transitioning From SW-CMM to CMMI-SE/SW
The Challenge:
Bring an Existing Project up to CMMI Level 3
 The entire organization was transitioning from CMM
Level 3 to CMMI Level 3
– New command media
– Periodic redirection
– Fluctuating success criteria
 The project had other concerns
– Performance
– Shrinking delivery schedules
Southern California SPIN 12/05/03
50
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
The Organizational Environment
Quantitatively Measured
Enterprise-Wide Institutionalization
• Metrics Manual
• Measurement repository
• Policy & Requirements Manual
• Standard Process Manual
• Training
Projects
PRM
MET
SPM
CMMI Assessments
Six Sigma Teams
• DMAIC / DFSS
• Tools & methods
CMMI/Six Sigma Synergy
• Self-Assessment Tool
• Internal / external formal
assessments
• Project Reviews/Summits
• Integrated strategies
Southern California SPIN 12/05/03
51
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
Our Strategy
 Map the Path from CMM to CMMI
 Build on Proven Project Management Practices
 Use Change Management Tools
 Extend Software Development Activities to include
Systems Engineering
 Keep Score
Southern California SPIN 12/05/03
52
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
Map the Path from CMM to CMMI
Organization process focus
Organization process definition
Training program
Integrated software mgmt
LEVEL 3
DEFINED
Software product
engineering
Intergroup coordination
Peer reviews
LEVEL 2
REPEATABLE
Requirements management
Software project planning
Software project tracking & oversight
Software subcontract mgmt
Software quality assurance
Software configuration mgmt
Organization process focus
Organization process definition
Organizational training
Integrated project management
Risk management
Requirements development
Technical solution
Product integration
Verification
Validation
Decision analysis and resolution
Requirements management
Project planning
Project Monitoring and Control
Supplier Agreement Management
Product & Process Quality Assurance
Configuration Management
Measurement and Analysis
Southern California SPIN 12/05/03
53
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
Build on Proven Project Management Practices
 Establish and maintain a plan
– If you don’t know where you are going, you will probably end up where
you don’t want to be
 Engage relevant stakeholders
– Instituted weekly coordination working group
 Track progress
– Schedule, task list tracked to closure
 Integrated Management
– Software Quality Assurance is your friend
 Risk Management
Southern California SPIN 12/05/03
54
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
Use Change Management Tools
 Maintain senior management sponsorships
 Work with early adopters
 Use staff and all hands meetings as training
opportunities
 Let group leaders be your change agents
 Show constant progress
 Celebrate small victories
Southern California SPIN 12/05/03
55
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
Extend Software Development Activities to include
Systems Engineering
 Emphasize and build interfaces to Program and Software
Systems Engineering groups
– Program Systems Engineering acknowledged responsibility for
requirements allocation and acceptance of software baselines
– Software Systems Engineering maintained requirements evidence
books
– Project documented transfer of artifacts and completion of milestones
 Encourage all trade studies to use the DAR methodology
 View the project as a system, not a collection of
components
Southern California SPIN 12/05/03
56
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
Keep Score
Southern California SPIN 12/05/03
57
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
Initial Steps
 Identified points of contact for all CMMI process areas
 Developed a schedule with a simple format
 Held weekly coordination meeting
 Started the hard stuff first (requirements, project
planning)
 Defined project product development life cycle model
 Released initial updated program plans/processes
Southern California SPIN 12/05/03
58
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
Lessons Learned - 1
 Process group cannot do it all
– Flow down of information and training essential to implementation
 Take full advantage of Organizational resources
– Presentations by Process Assessment Organization lead clarified
principles and showed top management commitment
 Dig Early and Often
– Appeal to project people to save evidence - especially emails
– Need to document verbal orders
 Training is Essential
Southern California SPIN 12/05/03
59
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
Lessons Learned - 2
 Timelines are appropriate for communicating
expectations of urgency, but they must be realistic
– Identifying the gaps and adjust for changing strategy
– Allow adequate time to create, review and update documents,
evidence notebooks, train participants, audit products and processes
– Do peer reviews, including an informal appraisal
 Address resistance to change
Whining
– “I thought the organization did that.”
– “Our customer won’t let us do that.”
– “Why aren’t these projects included in the appraisal?”
– “I want to do CMMI – I just don’t want to change our process.”
Southern California SPIN 12/05/03
60
Copyright 2003 Northrop Grumman Space and Mission Systems Corp. All Rights Reserved.
Questions?
•
Two hour conference reception was great (Simi
Valley CS) except for hors d’ourve line
•
Several vendors to visit, including SLIM,
Raytheon and several Process
Improvement/CMMI providers with chotchkas
•
Other presentations will be presented after
networking break
Southern California SPIN 12/05/03
61