Transcript Cornellis Human The pragmatic use of
Slide 1
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 2
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 3
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 4
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 5
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 6
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 7
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 8
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 9
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 10
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 11
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 12
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 13
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 14
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 15
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 16
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 17
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 18
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 19
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 20
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 21
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 22
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 23
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 24
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 25
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 26
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 27
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 28
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 29
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 30
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 31
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 32
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 33
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 34
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 35
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 36
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 37
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 38
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 39
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 40
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 41
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 42
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 43
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 44
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 45
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 46
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 47
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 48
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 49
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 50
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 51
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 52
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 53
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 54
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 55
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 56
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 57
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 58
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 59
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 60
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 61
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 62
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 63
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 64
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 65
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 66
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 67
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 68
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 2
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 3
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 4
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 5
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 6
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 7
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 8
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 9
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 10
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 11
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 12
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 13
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 14
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 15
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 16
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 17
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 18
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 19
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 20
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 21
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 22
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 23
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 24
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 25
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 26
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 27
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 28
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 29
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 30
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 31
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 32
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 33
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 34
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 35
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 36
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 37
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 38
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 39
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 40
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 41
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 42
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 43
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 44
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 45
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 46
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 47
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 48
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 49
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 50
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 51
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 52
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 53
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 54
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 55
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 56
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 57
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 58
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 59
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 60
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 61
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 62
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 63
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 64
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 65
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 66
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 67
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI
Slide 68
The pragmatic use of technology to achieve CMMi goals
Agenda
Part 1: Technology in support of achieving CMMI goals
Role of Automation in SPI initiatives
Choosing the right technology
Technology support for Level 2 Process Areas
Configuration Management (CMMI Level 2)
Requirements Management (CMMI Level 2)
Others
Part 2: Guidelines to Successful SPI
Part 3: Panel Discussion
2
Part 1 – Technology in support of CMMI
4
Mapping Methodologies to Tools
SW Development
Methodology
(How/Recipe)
Waterfall
Iterative
Extreme Programming
others
SW Mgmt Methodology
Tools
CaliberRM
(What/Menu)
Together
CMM/CMMI
StarTeam
ISO
MS Project
JBuilder
Others
SPICE
others
• Tools make the tasks of the SW Development Methodology more efficient
• Tools support the measurement of the SW Management Methodology
5
The “Tool Capability Spread”
Using a Requirements Management Tool as an example, let’s
plot the “Aggregate Need” for tool support throughout the
Application Development Lifecycle (Blue) and compare
against Tool Capabilities (Green, Red)
6
The “Requirements Capability Spread”
Gap analysis of Tool A capabilities
(Green) vs. Tool support Need (Blue)
Gap analysis of Tool B capabilities (Red)
vs. Tool support Need (Blue).
Note the difference in “coverage profile”, ie.
Tool B has less overall gap area than Tool A.
7
The “Requirements Capability Spread”
Represents the Capabilities of “Tool A” over and
above those provided by “Tool B” for the
DEFINITION ROLE ONLY.
Question: Are these extra features/capabilities worth it,
when you consider the next slide…
8
The “Requirements Capability Spread”
Represents the Capability Areas where “Tool B”
provides more coverage for roles other than just
“Definition”.
Question: Are the extra features from previous slide worth
it when key roles “downstream” are missing out on
improved RM process support?
9
Spreading the goodness… (RM Example)
Requirements affects what everyone does. Start
thinking beyond just your Analysts!
Don’t make Tool decisions solely based on the needs
of your Analysts (Definition Role), but consider the
impact of your choice on downstream Roles.
The key to real significant organisational
Requirements Management improvements, lies in
the wide internal adoption of RM tools &
processes across many key roles, not just catering
to the needs of the few. (analysts)
10
Thinking in a silo
The Requirements Silo (or “Tower of Smart People”)
• Is this
enough
for the overall
business?
"Requirements Capability Spread"
for good
a typical
Software
Development
Project
• Isn’t requirements for the rest of us too?
11
Quantified Need/Capability
10
9
• “Tool Silos” exist everywhere, placing a burden on smooth
8
7
Requirements Management Need
process enablement across multiple
roles
Tool A Capabilities
6
5
Tool B Capabilities
4
3
2
1
0
Inception
11
Definition
Design
Develop
Testing
Deploy
Tool Selection Criteria
Process enablement first, then process Enforcement
Make process “fun”, not “forced”
Focus on process participation as the only means of working
Tools should allow you to choose & grow your process
Tools with a specific process “designed in” often fail to penetrate
Lack of tool penetration results in non-process conformity
Coverage is more important than silo capabilities
Wide internal adoption is the secret, not Hero-worship
Look for cross-tool reporting capabilities
12
Tool Selection Criteria
Flexibility, Openness, Customisable
Industry Standard, Open API’s a must
Easy Forms customisation a big bonus
Be wary of proprietary-languages, -scripting and –repositories
Deep, Embedded Integrations is the way to go
Touchpoint integrations are a crutch for poor process
Minimal automated workflow is a VAST improvement
Be wary of “big bang” implementations
Humans evolve slower than technology does
Start with simple workflow, then improve it. Constantly.
Look for tools that make small incremental improvements easy
Focus on improving workflow, not always getting it right
(Business Environments changes so fast nowadays, by the time you
have the answer, the problem has changed…)
13
Technology Supporting RM
Manage Requirements-SG 1
Obtain an understanding of Requirements-SP 1.1
Obtain Commitment to Requirements-SP 1.2
Manage Requirements Changes-SP 1.3
Maintain Bidirectional Traceability of Requirements-SP 1.4
CaliberRM Capabilities
Visibility
Secure Approval Process
Baselining and versiioning
Traceability
Together Capabilities
Visualization
Traceability
14
Project Planning
Establish Estimates-SG 1
Develop a Project Plan-SG 2
Obtain Commitment to the Plan-SG 3
CaliberRM Capabilities
Project Plan integration
EstimatePro integration
StarTeam Capabilities
Project Plan integration
15
Project Monitoring and Control
Monitor Project against Plan- SG1
Monitor Project Planning Parameters-SP 1.1
StarTeam Capabilities
Task Assignment and Effort Tracking
StarTeam Datamart
16
Measurement and Analysis
Many Borland products support this Process Area:
CaliberRM/Datamart
Standard Reports
Full reporting module via Caliber Datamart
Full history of all Requirements
Together
Metrics
StarTeam/Datamart
Standard Reports
Customized reports as part of Consultancy
17
Process & Product Quality Assurance
Objectively Evaluate Processes and Work Products- SG1
CaliberRM & StarTeam Datamarts
Provide Objective Insight-SG 2
CaliberRM & StarTeam
Full history of previous versions and changes including who made the
change and what the change was.
StarTeam
Corrective actions assigned to tasks
Full version control of all process documentation and associated work
products, plus evaluation reports
18
Configuration Management
Establish Baselines –SG1
Identify Configuration Items-SP 1.1
Establish a Configuration Management System-SP 1.2
Create or Release Baselines-SP 1.3
Track and Control Changes-SG2
Track Change Requests-SP 2.1
Control Configuration Items-SP 2.2
Establish Integrity-SG3
Establish Configuration Management Records-SP 3.1
All 3 goals supported by StarTeam
19
Unified Repository For All Assets
Project and View definitions
provide structural flexibility
for sharing/restricting assets
All asset types are stored
within the same project
and folder structures
Look for tools that provide a single,
integrated interface for managing files,
change requests, requirements, tasks, and
topics
20
Integrated Change Management
Change requests record
defects, enhancements,
suggestions, etc.
Change requests should be
integrated, native objects to
the central repository
Change requests can be entered
directly or synchronized from
other defect tracking sources
21
Change requests definitions
can easily be customized
with custom fields and forms
Change requests can be linked to
other configuration items
manually or automatically
Integrated Requirements Management
Requirements should be
integrated, native objects to
the central repository
Requirements should be
entered directly or imported
from external tools or
documentation
Requirement definitions should
be easily exposed to other users
without a need to switch UI
22
Integrated Task Management
Tasks are native objects that
the StarTeam Server
understands
Tasks can be entered in
StarTeam or synchronized from
Microsoft Project
Assigning tasks and reporting on
progress of assigned Tasks should
be integrated. “Tasks” should be
deeply integrated to PM tools.
23
Customizable Workflow & Forms Example
Custom user interfaces should
be easy to setup for all object
types (CR’s, Tasks, etc.)
Underlying workflow design
should preferably have a
graphical UI for design work.
24
Graphical workflow definition
allows customization of
StarTeam process rules and
enforces standards
Make the process fit your
environment - not the other way
around!
Workflow definition stored in
StarTeam Server as
configuration items so client
deployment is automatic
Borland Java Studio Embedded Integration
Embedded IDE Client provides full
StarTeam functionality within the
developer’s environment without
and context-switching
Integrations are subject to all process rules
and workflow and security restrictions just
like any other StarTeam client type
25
Eclipse/WSAD Embedded Integration
Two new Eclipse perspectives accommodate
both novice and experienced StarTeam users
Changes
can be checked
in or
Label
decorations
add
checked outinformation
as a wholeto
for selected
server-side
artifacts
merged into the local
sharedor
workspace
versions
on a per-difference basis
resources
All comparisons are shown within the
Eclipse workbench so there is no
longer a need for external difference
and merge tools
26
1.) The “Classic” perspective is close to the
layout of the other clients
2.) The “Work-Centric” perspective arranges
the StarTeam views around the editor space
for highly productive daily operations
Overview of Borland technology in support of CMMI
Borland products directly support:
Requirements Management (CaliberRM)
Requirements Development (Together/CaliberRM)
Configuration Management (StarTeam)
Technical Solution (Together/IDE’s)
Product Integration (StarTeam)
Borland products assist in:
Project Planning (CaliberRM)
Project Monitoring and Control (StarTeam)
Measurement and Analysis (Datamarts)
Process & Product Quality Assurance (Datamarts)
27
Technology Change Management (TCM)
CMM Level 5 KPA
Identify new technologies (i.e., tools, methods, and processes)
and transition them into the organization in an orderly
manner.
Evaluate new technologies
Use pilot efforts to assess changes
Incorporate changes into projects and organization standards
28
Key Considerations
Just maintaining a CMM/CMMI level requires investment
Benefits result from operating at an improved level of maturity, not from just
getting there
Some benefits may not be financial, but can still can be “valued”
Weaknesses at lower levels of maturity increase risk and cost of achieving
higher levels of maturity
Costs and benefits must be determined separately for each scenario
29
Summary of Key Points
Focus on business drivers
Ensure fast returns
Do what you’re capable of
Controlled change
Build on what you’ve already got
Basic engineering disciplines are critical
Don’t ignore basic working practices
Your learning is important
Use technology to your (competitive) advantage
30
Part 2 – Guidelines to successful SPI
31
How an initiative typically starts …
Identify a process improvement initiative
Make someone responsible
Pick a model
CMMI, ISO 9000, SPICE etc.
Get an assessment
Strengths, Weaknesses, and Recommendations
Close the gap
Implement the recommendations
Write some processes and procedures
32
Factors affecting SPI success
Senior management monitoring of SPI
Compensated SPI responsibilities
SPI goals clear and well understood
Technical staff involved in SPI
SPI people well respected
Staff time resources dedicated to SPI
Source:
• CMM: CMU/SEI-95-TR009, Goldenson & Herbsleb op. cit. (www.sei.cmu.edu)
33
Common Causes Of Failure
Lack of management commitment
Lack of capability measures
Your business does not have time
You fail to support projects
You do not have buy in from practitioners
Pilot is successful but roll out fails
You try to become “Better” than you need to be
You get seduced into compliance with the model
Lack of quality assurance
34
Successful Implementation of CMMI
Management Participation
Communicate regularly and often the business drivers that have
initiated the project
Actively participate in review and approval of processes
Lead by example
35
Successful Implementation of CMMI
Study the Work
Analyse what development staff do currently
Implement Capability Measures, not Targets:
Measure process, not people
Aligned to the purpose of the work
Assess the strengths and weaknesses
Benefits:
Start gaining buy-in from development staff
Able to prioritise improvements
36
Successful Implementation of CMMI
Invest in Education
Formal Training in CMMI
Train project members in the models.
Optionally train selected staff as CMM
Evaluators
Informal Training:
Attend forums such as the Management
Forum for Excellence in Software
Development (MFESD)
Talk to other organisations
Trawl the organisation for people with
experience
37
Successful Implementation of CMMI
Process Development
Involve development staff at every step
Communicate the “What’s In It For Me” items
Avoid bureaucracy:
Minimise “red tape”
Eliminate unnecessary paperwork
Automate at every opportunity
Avoid following the Standard slavishly:
If you don’t need it, don’t do it
38
Successful Implementation of CMMI
Process Implementation
Create a Communication Plan and
communicate!
Recognise staff who:
Participate in process improvement
Adopt new and/or changed processes
Revise Capability Measures:
Measure process, not people
39
Sustaining Improvements
Involve those affected
Make it their program
Create an improvement culture
Make it clear what Is valued
Leadership by example (People will do what their manager
values)
Education and Training
Be very careful with reward mechanisms
They usually do damage
40
Part 3 – Panel Discussion
41
Panel Introduction
42
Thank you for participating!
43
APPENDIX SECTION
44
What is StarTeam ?
StarTeam is…
“…a comprehensive software configuration
management system to support the definition and
control of all assets and life cycle tasks from within a
single repository”
Unified repository for all enterprise assets
Highly optimized client-server interaction
Customizable workflow and process rules
45
What is StarTeam ?
Requirement Publication
Change Management
Team Discussion
Task Allocation & Tracking
File Management
Customizable Workflow
Customizable Forms
Automatic Linking
Open Customizable Platform
Web-Centric Architecture
Secure
46
Borland Application Lifecycle Management
Together
JBuilder
Delphi
StarTeam
OptimizeIt
CaliberRM
BES
Janeva
Interbase
47
Why CMMI –
Software Delivery Results are Poor
66%
NOT
SUCCESSFUL
54%
OVER
BUDGET
70%
LATE
30%
CANCELLED
Software development is an art, not a science. How are predictability
and reliability achieved in software delivery?
48
Source: The Standish Group 2003.
Why CMMI –
Rework Devastates Productivity
Without attention to processes, most
application development organizations can
easily spend 40% of their effort on rework
Initial benefits from process improvement
result from dramatically reducing rework
# of developers
49
Burdened cost
Wasted on rework
30
$3,000,000
$1,200,000
100
$10,000,000
$4,000,000
300
$30,000,000
$12,000,000
500
$50,000,000
$20,000,000
1000
$100,000,000
$40,000,000
3000
$300,000,000
$120,000,000
What is CMMI –
Background
CMMI—an evolutionary roadmap for implementing the
vital practices needed to continuously improve system
development
Developed by the Software Engineering Institute at
Carnegie Mellon University
Created to help U.S. Department
of Defense evaluate the capability
of bidders on system development
contracts
50
What Is CMMI –
Conceptual Foundations
System Development
context and objectives
best practices
Total Quality
Management
quantitative management
continuous improvement
Organizational Change
and Development
culture & maturity
change management
Characteristics of CMMI
Guidelines for improvement – not a prescriptive method
Indicates the what’s, not the how’s
A benchmark for improvement progress
Not just another process standard, but a unique model of
organizational development
51
What Is CMMI –
Ancestry of CMMI
Shewart,
Deming
Juran
Crosby
System
Eng. CMM
Humphrey’s
Process
Maturity
Framework
CMM for
Software
People
CMM
Acquisition
CMM
CMMI
CMMI was needed to integrate multiple engineering
disciplines into one organizational improvement model
52
What Is CMMI –
Two Representations
Staged
Continuous
Focuses on transforming an organization
through a series of evolutionary stages by
implementing new process areas at each
maturity level
Focuses on improving one or more process
areas by implementing more sophisticated
techniques for managing and performing the
process at each capability level
Most organizations use the staged representation
Borland’s Software Delivery Optimization practice most
frequently employs the staged model when
implementing improvement programs
53
What is CMMI –
Structure of CMMI
Maturity Level
An evolutionary plateau in an
organization’s development that
establishes a new capability for
performing its business functions
Process Areas
A cluster of interrelated practices
and objectives that contribute to
the capability established at a
maturity level
Goals
An organizational state that
results from implementing some
of the practices of a process area;
the objectives to be achieved by a
process area
consists of
scoped by
implemented by
Practices
54
A subprocess of a process area
that contributes to achieving a
process area goal
What Is CMMI –
Two Kinds of Goals and Practices
Specific
Goals &
Practices
The practices that are typically incorporated into
an effective process that achieves the objectives
described by the goals for the process area
Generic
Goal &
Practices
The practices generic to every process area that
are associated with the goal of institutionalizing
the specific practices of the process area in order
to sustain them as routine activities
2.1) Organizational policy
2.7) Involve stakeholders
2.2) Plan the process
2.8) Monitor & control
2.3) Provide resources
2.9) Evaluate adherence
2.4) Assign responsibility
2.10) Mgt. status review
2.5) Train people
3.1) Define process
2.6) Manage configurations 3.2) Collect Improvement
55
What Is CMMI –
Process Areas by Maturity Level
56
Level 5 – Optimizing
Causal Analysis & Resolution
Organizational Innovation & Deployment
Level 4 – Quantitatively
Managed
Organizational Process Performance
Quantitative Project Management
Level 3 – Defined
Requirements Development
Technical Solution
Product Integration
Verification
Validation
Organizational Process Focus
Organizational Process Definition
Organizational Training
Integrated Project Management for IPPD
Risk Management
Integrated Teaming
Decision Analysis & Resolution
Organizational Environment for Integration
Level 2 – Managed
Requirements Management
Project Planning
Project Monitoring and Control
Supplier Agreement Management
Measurement and Analysis
Process & Product Quality Assurance
Configuration Management
Understanding CMMI –
Level 1 – Initial
Most Level 1 organizations do some of the practices expected at
Level 2, but do not get repeatable results
Typical problems:
Untrained project managers
Disciplined practices sacrificed to crisis schedules
Developers rely on personal methods (hero worship)
57
Understanding CMMI –
Level 2 – Managed
Managed control of interfaces to the external environment
Supplier
Agreement
Management
Requirements
Management
Project
Planning
Project
Monitoring
and Control
Measurement
and Analysis
Configuration
Management
Managed control of the project environment
Process and Product
Quality Assurance
58
Governance
of the process
Understanding CMMI –
Life at Level 2
Manager characteristics:
Balances commitments with
resources
Says ‘no’ when necessary
Involves teams in commitments
Plans time to repeat practices
that worked in the past
Does not sacrifice disciplined
practices to schedule pressures
Takes corrective action when
progress deviates from plan
Manages requirements changes
Eliminates blame
59
Developer characteristics:
Participates in planning
Provides realistic estimates
Take commitments seriously
Experience less burnout from
working nights and weekends
Uses disciplined practices that
have worked in the past
Observes configuration
management practices
Requests advice from process
and quality assurance
Understanding CMMI –
Level 3 – Defined
60
Organizational support for
standardized processes
Engineering
best practices
Process-based project
management
Organizational
Process Focus
Requirements
Development
Integrated Project
Management
Organizational
Process Definition
Technical
Solution
Risk Management
Organizational
Training
Product
Integration
Integrated Supplier
Management
Decision Analysis
and Resolution
Verification
Integrated Teaming
Organizational
Environment for
Integration
Validation
Understanding CMMI –
Level 3 – Supporting Standard Processes
Organization’s Process Asset Repository
Organization's standard Guidelines
development processes for tailoring
standard
Process Architecture
processes
for use on
projects
Reqts.
Analysis
Process
Specifications of
Process
Elements
Best practices
61
Lessons
learned
Database of
standard
project
measures
Repository
of reusable
project
artifacts
Project 1
Size
$$$
Defects
Results
Lessons
Historical
measures
Reusable
history
Understanding CMMI –
Level 4 – Quantitatively Managed
P
r
o
j
e
c
t
1
Goal 2: Manage
subprocess
performance
statistically
Inspection problem reports
Sev 1 only
Sev 1 + 30%
Sev 1 + 40%
Model
Criterion
Actual
500
400
300
Module completion times
100
200
100
0
1
3
5
7
9
11 13 15 17 19
Weeks of testing
10
0%
20%
40%
60%
80%
100%
System test defect reports
Goal 1: Manage
the project
quantitatively
Quantitative Project
Management
62
Inspection preparation times
1000
600
of testing
Defects per 1000 hours
P
r
o
j
e
c
t
2
Organization’s Process
Performance Baselines
Mean time to failure in test
Organizational Process
Performance
Understanding CMMI –
Predicting Outcomes from Stable Processes
Predictive equation for project outcome
a0 + a1X1
Project
input
+
a2X2
+
a3X3 + e = Youtcome
Subprocess
1 result
Subprocess
2 result
Subprocess
3 result
Subprocess
1 behavior
Subprocess
2 behavior
Subprocess
3 behavior
Project
outcome
Stable processes produce more accurate predictors
63
Understanding CMMI –
Level 5 – Optimizing
Proactively improve process
capability to close gaps
between executive business
objectives and current results
Organizational
Innovation and
Deployment
new
method
business
objective
{
Causal Analysis
and Resolution
Capable process
64
Understanding CMMI –
Proactive Continuous Improvement
Evaluate candidate
improvements
Select candidate
improvements
Do
Act
Plan
Check
Project’s defined
process
Process Architecture
Process Architecture
Process Elements
Process Elements
Project’s defined
processes
65
Organization
standard process
Organization’s
standard processes
Deploy proven
improvements
Understanding CMMI –
Maturity Transformations
Level 5
Optimizing
Capable
processes
Change
management
Experimental
learning
Quantitatively
Managed
Stable
processes
Capability
management
Statistical
learning
Level 3
Defined
Standard
processes
Process
management
Organizational
learning
Level 2
Managed
Project
procedures
Project
management
Project
learning
Level 1
Initial
Inconsistent
practices
Inconsistent
management
Individual
learning
Level 4
66
Source: Humphrey (1989)
Validating CMMI –
SEI Reported Results
Improvement benefit
Annual
range
Productivity growth
35%
9% - 67%
Pre-test defect detection
22%
6% - 25%
Time to market
19%
15% - 23%
Field error reports
39%
10% - 94%
5.0:1
4:1 - 8.8:1
Return on investment
67
Annual
median
Source: Software Engineering Institute, CMU/SEI-94-TR-13
Validating CMMI –
Boeing Information Services
Level change
Level 1 to 2
34 months
Level 2 to 3
25 months
Level 3 to 4
30 months
L1L2
L2L3
L3L4
Reduced Defects
12%
40%
85%
Reduced Time
10%
38%
63%
Reduced Cost
8%
35%
75%
145%
25%
15%
Benefits
Schedule Variance
68
Average months
Source: Vu (1996)
# of months to change levels
Validating CMMI –
Time to Achieve Maturity Levels
Based on assessments from 1992 - present
100
80
upper range
Interquartile range
median
60
40
24
20
22
33
18
0
1 to 2
2 to 3
3 to 4
4 to 5
Change between levels
69
Source: SEI