Transcript CMMI Technology Conf Presentation

Pittsburgh, PA 15213-3890
ASI CMMI PILOT
Status Briefing
Sponsored by the U.S. Department of Defense
© 2003 by Carnegie Mellon University
Version 1.0
CMMI Technology Conference 2003 - page 1
Topics
• Pilot Highlights
- Activities to Date
- Planned Activities
- Realized and Expected Benefits to ASI
- About The ASI Pilot Team
• Path Forward for ASI
- Expected Cost and Benefit of CMMI-based Process
Improvement
- Potential Post-pilot Activities
© 2003 by Carnegie Mellon University
Version 1.0
CMMI Technology Conference 2003 - page 2
Pittsburgh, PA 15213-3890
Pilot Highlights
Sponsored by the U.S. Department of Defense
© 2003 by Carnegie Mellon University
Version 1.0
CMMI Technology Conference 2003 - page 3
Activities To Date (1 of 6)
Site Kickoff
Meeting
Gap Analysis
Session
Action Plan
Implementation
Close Interaction
Between ASI
And Consultants
 CMMI Overview Tutorial that covered Process Areas of Maturity Levels
2 & 3 and focused upon the typical business impacts related to the
area
 Business Analysis to capture high impact/high need Process Areas for
ASI
Had simple “thumbs-up” voting technique to determine level of
impact/need
Also polled participates for the level of impact (H M L)
The high impact areas were obvious and it was easy to select the
Process Areas to focus on for the pilot
© 2003 by Carnegie Mellon University
Version 1.0
CMMI Technology Conference 2003 - page 4
Activities To Date (2 of 6)
Site Kickoff
Meeting
Gap Analysis
Session
Action Plan
Implementation
Close Interaction
Between ASI
And Consultants
 3 Process Areas were selected by the joint SED/SEI/ASI team :
Project Planning
Requirements Management
Measurement & Analysis
• Decided to “informally” work on Organizational Process Focus
© 2003 by Carnegie Mellon University
Version 1.0
CMMI Technology Conference 2003 - page 5
Activities To Date (3 of 6)
Site Kickoff
Meeting
Gap Analysis
Session
Action Plan
Implementation
Close Interaction
Between ASI
And Consultants
 Workshop to analyze the process areas selected - the
concepts/methods used for the SCAMPI B/C development project
were adapted for ASI:
Current ASI practices documented and mapped to relevant
CMMI model components
Interpretation of model intent for ASI made and gaps
documented
Developed Action Plan for ASI Team to address gaps found
© 2003 by Carnegie Mellon University
Version 1.0
CMMI Technology Conference 2003 - page 6
Activities To Date (4 of 6)
Site Kickoff
Meeting
Gap Analysis
Session
Action Plan
Implementation
Close Interaction
Between ASI
And Consultants
 ASI Team Members are in various stages of implementing their Action
Plan
 ASI Team Members developed Project-Specific Process Descriptions
Project Planning
Requirements Management
Measurement and Analysis
 ASI Quality Facilitator evaluating impact at the organizational level and
relationship to QSM
© 2003 by Carnegie Mellon University
Version 1.0
CMMI Technology Conference 2003 - page 7
Activities To Date (5 of 6)
Site Kickoff
Meeting
Gap Analysis
Session
Action Plan
Implementation
Close Interaction
Between ASI
And Consultants
Monthly Face-to-Face sessions to status progress and get
help/guidance from Consultants
Tutorial on writing process guidance documentation
Tutorial on business-oriented metrics
Tutorial on Process Capturing Techniques
Tutorial on CMMI Institutionalization Concepts
 Feedback on Action Plans and Process Descriptions
© 2003 by Carnegie Mellon University
Version 1.0
CMMI Technology Conference 2003 - page 8
Activities to Date (6 of 6)
Site Kickoff
Meeting
Gap Analysis
Session
Action Plan
Implementation
Close Interaction
Between ASI
And Consultants
Weekly teleconferences to keep the project progressing
 Review Action Items
 Provide feedback on recently reviewed material
 Provide guidance on any issues/problems encountered
© 2003 by Carnegie Mellon University
Version 1.0
CMMI Technology Conference 2003 - page 9
Planned Activities (1 of 5)
Execute New
Processes
Appraise
Pilot
Projects
Brief
SED
Share
Lessons
Learned
Publish
Pilot Results




Baseline Process Descriptions
Execute processes in Pilot Projects
Collect Metrics
Gather lessons learned (including benefits) and process
improvements
 Update processes to reflect process improvements
 Evaluate processes for standardization at the Organizational Level
© 2003 by Carnegie Mellon University
Version 1.0
CMMI Technology Conference 2003 - page 10
Planned Activities (1 of 5)
Execute New
Processes
Appraise
Pilot
Projects
Brief
SED
Share
Lessons
Learned
Publish
Pilot Results
 Prepare for Appraisal
Define Scope (Requirements Management, Project Planning,
Measurement and Analysis, and Organizational Process Focus are
candidate process areas)
Have regular preparation sessions with Lead Appraiser
Gather objective evidence to support appraisal activities
Set expectations
 Conduct Appraisal
ASI to provide one person for the Appraisal Team
Team Training and Readiness Review (tentative: 04/27-04/30)
Appraisal (tentative: 05/24-05/27)
© 2003 by Carnegie Mellon University
Version 1.0
CMMI Technology Conference 2003 - page 11
Planned Activities (3 of 5)
Execute New
Processes
Appraise
Pilot
Projects
Brief
SED
Share
Lessons
Learned
Publish
Pilot Results
 Briefing Scheduled for 02/05/04 02/05 at ASI
 Opportunity for ASI to provide
 capabilities briefing
 Pilot Status
 Pilot Benefits
 Lessons Learned
 Opportunity for ASI to discuss with Mr. Craig his plans after the pilot
© 2003 by Carnegie Mellon University
Version 1.0
CMMI Technology Conference 2003 - page 12
Planned Activities (4 of 5)
Execute New
Processes
Appraise
Pilot
Projects
Brief
SED
Share
Lessons
Learned
Publish
Pilot Results
 ASI invited to present at SE2 Conference
 Half-day tutorial – March 29
• Sharing materials and activities conducted with ASI and Cirrus with
interested members of HSV small business community
• No ASI-specific information will be provided without ASI permission
• Opportunity for ASI to share what it deems appropriate
 CMMI Panel – March 31
• Provide Small Business Perspective for CMMI Implementation
• Share Lessons Learned
 Possible Interviews with Huntsville Times for pre-conference articles
© 2003 by Carnegie Mellon University
Version 1.0
CMMI Technology Conference 2003 - page 13
Planned Activities (5 of 5)
Execute New
Processes
Appraise
Pilot
Projects
Brief
SED
Share
Lessons
Learned
Publish
Pilot Results
 Interpretive Guide for Using CMMI in Small Businesses
Focusing on general lessons learned/recommendations gathered during
implementation of the various processes at ASI/Cirrus
Materials used (i.e., tutorials, pilot processes) to be included as jumpstart
for other small businesses
 Case Study: ASI
Published jointly among ASI, SED, SEI
All three organizations agree on content
 Case Study: Cirrus
Same conditions as for Cirrus case study
© 2003 by Carnegie Mellon University
Version 1.0
CMMI Technology Conference 2003 - page 14
Realized and Expected Benefits to ASI
(1 of 8)
Patty…
• CMMI Implementation has significantly improved ability to
communicate status of the project
• Use of CMMI-recommended practices has reduced “lotterysensitivity” on the project
- Much of what was only in my head is now accessible to
other team members
• CMMI Implementation has positively affected the way I
organize my project
• Requirements Management Process Description developed
during the pilot works for my project and should tailor easily to
other “service” projects
• Will fully execute and document entire process with recently
received major requirements changes
• There is no turning back now. I am hooked!
© 2003 by Carnegie Mellon University
Version 1.0
CMMI Technology Conference 2003 - page 15
Realized and Expected Benefits to ASI
(2 of 8)
Larry…
• Documentation created as a result of using CMMI:
•
Fills in missing gaps in ISO documentation
•
Is a natural follow onto ISO and meets ISO goals for continual
improvement
•
Communicates program roles to all stakeholders
•
Provides (for the first time!) step-by-step instructions for new employees
(especially PMs and TLs)
•
Provides “bragging rights” in proposals and presentations
•
Supports greater (and needed!) degree of granularity in organizational
performance measurement
•
Helps to identify trouble spots (and strengths) in organizational
practices
•
Provides a consistent approach for implementing ASI processes and
procedures
© 2003 by Carnegie Mellon University
Version 1.0
CMMI Technology Conference 2003 - page 16
Realized and Expected Benefits to ASI
(3 of 8)
Jimi…
• Experience gathered in communication of practices has proven to be of
great benefit in clarifying expectations among team members
• Through CMMI Implementation, the ASI Team has gained knowledge of
existing ASI systems and practices that were not previously clear
• After the pilot is complete, ASI will have the knowledge and experience
necessary to implement process improvement more efficiently
• The pilot project has widened the ASI Team’s thoughts toward process
design and improvement
• ASI Team Members will now have insight, skills, and initiative to develop
new ideas into new practices, with confidence
• The fear of change has been released from those involved
• There seems to be a renewed sense of interest and understanding
towards process automation
• I see a future that includes many benefits currently unrealized
© 2003 by Carnegie Mellon University
Version 1.0
CMMI Technology Conference 2003 - page 17
Realized and Expected Benefits to ASI
(4 of 8)
Jack…
• CMMI has provided a best practice model to develop process and
procedures needed in ASI QMS. Pilot project has provided some
confidence that we are doing the right things.
• “Bottoms-up” encouraged by CMMI structure has made it easier to
develop a difficult procedure. Expect expansion to other programs to
work well.
• CMMI specifically addresses customer related processes, many of
which are very relevant to the services we provide.
• Integrated view of processes, through CMMI Process Area interactions,
has significantly improved our ability to define more effective processes
© 2003 by Carnegie Mellon University
Version 1.0
CMMI Technology Conference 2003 - page 18
Realized and Expected Benefits to ASI
(5 of 8)
Jack…
• The Pilot provided the ASI Team with tools an techniques that will make
implementation of improvements more cost-effective
• CMMI has helped me envision a path to “take the company to the next
level”
- Develop consistent, lasting capabilities that support our business
objectives and go beyond having good people
- Increase competitive advantage through specific capabilities,
documented as ASI processes, that provide solutions for customers
- Move away from being totally people dependent to a capabilitybased organization with proven processes that deliver desired
results to customers.
© 2003 by Carnegie Mellon University
Version 1.0
CMMI Technology Conference 2003 - page 19
Realized and Expected Benefits to ASI
(6 of 8)
Beth…
• The CMMI can be easily incorporated into ASI’s QMS
• The bottom-up approach encouraged by the CMMI, coupled with ASI’s
QMS, will connect those parts of the system which can benefit from
specific process improvements
• Incorporating CMMI practices into ASI’s QMS will give direction to our
improvement goals and, in turn, increase customer satisfaction
• CMMI will strengthen ASI’s QMS and support company objectives as we
grow
• If we incorporate strategic planning (both at he project and corporate
levels) and measurable goals while streamlining our processes to suit
our particular activities, our QMS would benefit
© 2003 by Carnegie Mellon University
Version 1.0
CMMI Technology Conference 2003 - page 20
Realized and Expected Benefits to ASI
(7 of 8)
Your Consultants see the following as key realized or expected
benefits …
• ASI can use CMMI implementation as risk avoidance for ISO audit
findings looking for evidence of continual improvement
• CMMI is providing a common point of reference for working towards
standardizing management approaches across different tasks
• Use of CMMI-referenced process descriptions has already started to
reduce training effort for new staff
• CMMI usage has added insight to development of a standard work
authorization process
• CMMI focus on measurement has added insight into risks related to
staffing variances
• ASI will achieve higher visibility of relationship of processes to revenue,
through quarterly reviews
© 2003 by Carnegie Mellon University
Version 1.0
CMMI Technology Conference 2003 - page 21
Realized and Expected Benefits to ASI
(8 of 8)
Your Consultants see the following as key realized or expected
benefits …
• ASI staff has gained understanding of how a bottom-up approach can
be used to improve processes in the QMS
• ASI has gained understanding of what it takes to implement CMMI
Process Areas and how to approach implementation
• ASI will get more visibility in the community as a company that pursues
and invests in quality
• ASI has started to use best practices to respond to new requirements
(Subcontractor Agreement Management is an example)
• ASI will gain CMMI appraisal experience and be counted as one of the
few that have gone through an “official” appraisal (see next set of
slides)
© 2003 by Carnegie Mellon University
Version 1.0
CMMI Technology Conference 2003 - page 22
Carnegie Mellon University
Software Engineering Institute
CMMI v1.1 Today -– The Current State
Highlights from “ CMMI
Today:
The Current State”
PRESENTED BY
Bob Rassa, Raytheon
Industry CMMI Chair
Clyde Chittister
Chief Operating Officer,
Software Engineering Institute
Nov 18, 2003
SUMMARIZED BY SANDRA CEPEDA
© 2003 by Carnegie Mellon University
November 2003
CMMI v1.1 Today - Page 23
Carnegie Mellon University
Software Engineering Institute
CMMI v1.1 Today -– The Current State
Current Appraisal Synopsis
Based on SCAMPI v1.1 appraisals conducted since April 2002
release through June 2003 and reported to the SEI by July 2003.
• 100 appraisals
• 93 organizations
• 52 participating companies
•
6 reappraised organizations
• 357 projects
• 54%offshore organizations
Please refer to: Terms Used in this Report on page 20
Additional charts providing different views and break down of this information will be added to this briefing
as more of these types of appraisals are reported to the SEI over time.
© 2003 by Carnegie Mellon University
November 2003
CMMI v1.1 Today - Page 24
Carnegie Mellon University
CMMI v1.1 Today -– The Current State
Software Engineering Institute
Reporting Organizational Types
Commercial/In-house
47.3%
DoD/Fed Contractor
45.1%
Military/Federal
7.7%
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
% of Organizations
Based on 91 organizations
© 2003 by Carnegie Mellon University
November 2003
CMMI v1.1 Today - Page 25
50%
Carnegie Mellon University
CMMI v1.1 Today -– The Current State
Software Engineering Institute
Types of Organizations
Based on Primary Standard Industrial Classification (SIC) Code
Finance, Insurance and
Real Estate
6.7%
Public Administration
(Including Defense)
8.9%
Business Services
28.9%
Transportation Equipment
2.2%
Industrial Machinery And
Equipment
4.4%
Manufacturing
26.7%
Services
57.8%
Instruments And Related
Products
8.9%
Electronic & Other Electric
Equipment
11.1%
Engineering & Management
Services
24.4%
Services, Nec
2.2%
Health Services
2.2%
Based on 45 organizations reporting SIC code. For more information visit: http://www.osha.gov/oshstats/sicser.html
© 2003 by Carnegie Mellon University
November 2003
CMMI v1.1 Today - Page 26
Carnegie Mellon University
CMMI v1.1 Today -– The Current State
Software Engineering Institute
Organizational Size
Based on the total number of employees within the area of the organization that was appraised
2000+
4.3%
25 or fewer
9.8%
1001 to 2000
10.9%
26 to 50
12.0%
1 to 100
39.1%
201 to 2000+
50.0%
501 to 1000
16.3%
101 to 200
10.9%
51 to 75
9.8%
76 to 100
7.6%
301 to 500
10.9%
201 to 300
7.6%
Based on 92 organizations reporting size data
© 2003 by Carnegie Mellon University
November 2003
CMMI v1.1 Today - Page 27
Carnegie Mellon University
CMMI v1.1 Today -– The Current State
Software Engineering Institute
Summary Organizational Maturity Profile
100%
90%
% of Organizations
80%
70%
60%
50%
37.9%
40%
27.3%
30%
19.7%
20%
10.6%
10%
4.5%
0%
Initial
Managed
Defined
Quantitatively
Managed
Optimizing
Based on most recent appraisal of 66 organizations reporting a maturity level rating
© 2003 by Carnegie Mellon University
November 2003
CMMI v1.1 Today - Page 28
Carnegie Mellon University
Software Engineering Institute
CMMI v1.1 Today -– The Current State
Summary
CMMI Adoption is widespread
• DoD and Commercial
• US and off-shore
CMMI Adoption is on much faster pace than SW-CMM at the same
period of its life
CMMI Training and Appraisals continue to increase at rapid pace
Application Notes have covered the software-only users
Update of content will occur under careful deliberation and
consideration in a controlled and open process
CMMI will always be sensitive to expressed user needs
CMMI ROI Data will be collected, aggregated and disseminated
CMMI Training will be updated as needed to meet user requirements
© 2003 by Carnegie Mellon University
November 2003
CMMI v1.1 Today - Page 29
About The ASI Pilot Team
• Dedicated, smart, creative, wonderful people!
• Worked extra hard to meet their project commitments as
well as their commitment to us
• Jack’s leadership and hands-on involvement has been a
significant contributor to the success of the pilot
• Thank you for giving us top-notch people!!!
© 2003 by Carnegie Mellon University
Version 1.0
CMMI Technology Conference 2003 - page 30
Pittsburgh, PA 15213-3890
Path Forward for ASI
Cost and Benefit of CMMI-based
Process Improvement
Potential Post-pilot Activities
Sponsored by the U.S. Department of Defense
© 2003 by Carnegie Mellon University
Version 1.0
CMMI Technology Conference 2003 - page 31
Value of CMMI
CMMI helps organizations…
• Improve delivery of promised performance, cost, and schedule
• Integrate stakeholders into project activities
• Provide competitive world-class products and services
• Implement an integrated, enterprise, business and engineering
perspective
• Use common, integrated, and improving processes for systems
and software
• Implement proactive program management techniques
• Enable staff members to move between projects and still use the
same processes
• Create and improve processes that adapt to a changing business
environment
© 2003 by Carnegie Mellon University
Version 1.0
CMMI Technology Conference 2003 - page 32
Pittsburgh, PA 15213-3890
Value Specific to Services
Organizations (1 of 4)
CMMI PAs in Project Management
Can Help Organization
• Project Planning
• Determine the staff and schedule
• Project Monitoring and Control
needed to support the customer
• Supplier Agreement Management
• Manage costs and stay on schedule
• Integrated Project Management
• Effectively manage their suppliers
• Risk Management
• Be concerned about current and future
• Quantitative Project Management
risks to their organizations
• Integrate their functions with other
project functions
• Know how to apply the predictability of
their service support functions
Sponsored by the U.S. Department of Defense
© 2003 by Carnegie Mellon University
Version 1.0
CMMI Technology Conference 2003 - page 33
Value Specific to Services
Organizations (2 of 4)
CMMI PAs in Support
Can Help Organization
• Configuration Management
• Know the measures of the cost of
• Process and Product Quality
providing services and profitability, and
Assurance
the cost of quality
• Measurement and Analysis
• Control technical and management work
• Causal Analysis and Resolution
products
• Decision Analysis and
• Ensure their services meet quality
Resolution
objectives
• Be able to make informed selections of
products or techniques for their
customers
• Track service issues to root causes
• Provide services, including establishing
work environments that mitigate the risks
of engineering and integration tasks
© 2003 by Carnegie Mellon University
Version 1.0
CMMI Technology Conference 2003 - page 34
Value Specific to Services
Organizations (3 of 4)
CMMI PAs in Engineering
• Requirements Management
• Requirements Development
• Technical Solution
• Product Integration
• Verification
• Validation
© 2003 by Carnegie Mellon University
Can Help Organization
• Develop and manage their support
requirements
• Provide services that provide
technical stability and support all
aspects of product development and
fielding
• Ensure interface compatibilities prior
to integration
• Provide technical support to ensure
that products are implemented
correctly
• Evaluate the suitability of acquired
products
Version 1.0
CMMI Technology Conference 2003 - page 35
Value Specific to Services
Organizations (4 of 4)
CMMI PAs in Process Management
• Organizational Process Focus
• Organizational Process Definition
• Organizational Training
• Organizational Process
Performance
• Organizational Innovation and
Deployment
© 2003 by Carnegie Mellon University
Can Help Organization
• Build and Maintain the Service
Organization and its Culture
• Implement processes to Support
the Service Staff
• Train staff members to perform their
service functions
• Measure the effectiveness and
performance of management and
service processes
• Introduce new service functions to
their customers
Version 1.0
CMMI Technology Conference 2003 - page 36
Notional Business Case for CMMI
•
•
•
•
Reduced Development/ Maintenance/Service
Costs
• Improved Productivity
• Less Rework
Improved Customer Satisfaction
• Reduced Post-Release Defects
• Measurable Improvements of
Reliability and Quality
Reduced Cycle Time
• Improved Process Performance
• Improved Service Time
Improved Professional Staff
• Improved Employee Morale
• Increased Developer/Maintainer/
Service Provider Confidence
•
Increased Revenue/Profitability
•
•
Repeat Business
Increased Product/Service Sales
•
•
•
Enhanced Time-to-Market Performance
Enhanced Service Delivery Time
Bonuses for Early Delivery
•
Reduced Employee Turnover and
Retraining Costs
Improved Competitive Advantage
•
Better Products and Services Out Sooner And Cheaper
Copyright 2003, CSSA, Inc. Used with permission.
Building on the Legacy of SW-CMM
Success for Product Development
Post-Release
Defect Reports
Time to
Market
Productivity
Percentage Improvement
Improvements From Adopting SW-CMM (SEI, 1994)
-39%
Savings vs.
40
+35%
cost of
35
software
process
30
improvement
-19%
25
(median) 5:1
20
15
10
5
0
Annual Medians
Current ROI Value
to Programs
(DACS, 1999)
Application of SPI to “Example
Organization With Example
Projects”:
Development Costs
Reduced 73%
Rework Costs
Reduced 96%
Average Schedule
Length
Reduced 37%
Post-Release
Defects
Reduced 80%
Weighted Risk
Likelihood
Reduced 92%
Return On
Investment
Expect Even Higher ROI For CMMI
21:1
+N%
+N%
Value of technical/
management services
-N%
Cost
Service
Times
-N%
Continuity of
Services
+N%
Customer
Satisfaction
Percentage Improvement
Expectations for Service Organizations
• Ensure stability, consistency, and quality of ASI services while
making a fair profit
• Protect ASI from “lottery sensitivity”
• Develop ability to sell “processes that make ASI capable”
instead of “the individual who can solve your problem”
© 2003 by Carnegie Mellon University
Version 1.0
CMMI Technology Conference 2003 - page 39
The Entire Supply Chain Is or Will Be
Involved in CMMI
• Improve Integration of
Products and Services
DoD
Customers
• Will Use
Organizational
Maturity or Process
Capability to Advise
Customers
• Will Use
Organizational
Maturity or Process
Capability to
Competitive
Advantage
Associate
Contractors
Suppliers
• May Require
Organizational
Maturity or Process
Capability in RFP,
Implicitly or
Explicitly
• May Use
Organizational
Maturity or Process
Capability
As a Discriminator
Competitors
• Improve Integration of
Products and Services
• Provide Insight Into
Supplier Performance
and Quality
DoD Contractors Have Additional Motivation To Transition To The CMMI
Cost of CMMI Implementation
•We don’t have any aggregate data for the CMMI yet
•Treat CMMI Implementation Like a Project
- Estimate based on previous process
improvement efforts
- Estimate based on pilot project cost
© 2003 by Carnegie Mellon University
Version 1.0
CMMI Technology Conference 2003 - page 41
Summary--Benefits
The bad news:
• Statistically-based cost/benefit is difficult to determine in
organizations without a history of performance measurements
that support the ROI case
The good news:
• Even without the statistically-based ROI case, solving known
(through the Business Analysis process) ASI problems using
CMMI as the approach reduces risk exposure
• Anecdotal benefits seen by ASI so far are consistent with
those seen in larger organizations, and costs (in terms of ASI
team participation as an improvement team) are consistent/low
percentage-wise in comparison to larger organizations
• Deployment of improved processes from the pilot is likely to be
easier/faster for ASI than for a larger organization
© 2003 by Carnegie Mellon University
Version 1.0
CMMI Technology Conference 2003 - page 42
Potential Post-Pilot Activities
• Manage ISO implementation and CMMI implementation
as an integrated activity to support QMS
• Formalize Process Improvement Infrastructure to sustain
and increase reach of improvement benefits
• Determine next set of process areas to implement
- Perform Business Analysis with larger management
group
- Pursue Maturity Level X in incremental fashion if
considered relevant to current business context
• Develop ASI-wide Process Improvement Plan
Leverage ASI Pilot Team Expertise
© 2003 by Carnegie Mellon University
Version 1.0
CMMI Technology Conference 2003 - page 43