Transcript Trends in Software Process and Quality

Adoption of Agile Methods
by High Maturity Organizations
Dr. Mark C. Paulk
2015 ASEE Annual Conference, 21 February 2015
________________________________________________________________________
Jonsson School of Engineering and Computer Science
Software CMM v1.1 (1987-2005)
Level
Focus
Key Process Areas
5 Optimizing
Continuous
process
improvement
Defect Prevention
Technology Change Management
Process Change Management
4 Managed
Product and
process quality
Quantitative Process Management
Software Quality Management
3 Defined
Engineering
processes and
organizational
support
Organization Process Focus
Organization Process Definition
Training Program
Integrated Software Management
Software Product Engineering
Intergroup Coordination
Peer Reviews
Requirements Management
Software Project Planning
Software Project Tracking & Oversight
Software Subcontract Management
Software Quality Assurance
Software Configuration Management
2 Repeatable Project
management
processes
1 Initial
Competent people (and heroics)
2
Implications of Maturity
Better predictability… Less variability… Improved performance…
Level
5 Optimizing
Process Characteristics
Predicted Performance
Process improvement is
institutionalized
Time/$/...
4 Managed
Product and process are
quantitatively controlled
Time/$/...
3 Defined
2 Repeatable
1 Initial
Software engineering and
management processes
defined and integrated
Project management
system in place;
performance is repeatable
Time/$/...
Time/$/...
Process is informal and
unpredictable
Time/$/...
3
Empirical Data on Improvement
By Maturity Level
Quality (reliability, defects) improves by roughly
a factor of 2 (or more)
- C. Jones, “Software Benchmarking,” IEEE Computer,
October 1995.
- K.D. Williams, "The Value of Software Improvement…
Results! Results! Results!" SPIRE97, June 1997.
- R. Yacobellis, “Panel: Does SEI Level 5 Lead to High
Quality Software?” COMPSAC 2001.
Effort for a given product decreases 15-21%,
productivity increases, cycle time decreases
- B.K. Clark, “Quantifying the Effects on Effort of Software
Process Maturity,” IEEE Software, November/December
2000.
- D.E. Harter, M.S. Krishnan, and S.A. Slaughter, “Effects of
Process Maturity on Quality, Cycle Time, and Effort in
Software Product Development,” Management Science,
April 2000.
- L.H. Putnam, “Linking the QSM Productivity Index with the
SEI Maturity Level,” QSM, 2000.
4
CMMI-DEV v1.3
Level
5
Optimizing
Process
Characteristics
Focus is on quantitative
continuous process
improvement
4
Process is measured
Quantitatively and controlled
Managed
3
Defined
2
Managed
1
Initial
Process Areas
Causal Analysis & Resolution
Organizational Performance Management
Organizational Process Performance
Quantitative Project Management
Process is characterized Requirements Development
for the organization and Technical Solution
Product Integration
is proactive
Verification
Validation
Process is characterized Requirements Management
for projects and is often Project Planning
Project Monitoring & Control
reactive
Organizational Process Focus
Organization Process Definition
Organizational Training
Integrated Project Management
Risk Management
Decision Analysis & Resolution
Supplier Agreement Management
Product & Process Quality Assurance
Process is unpredictable,
poorly controlled, and
reactive
5
Configuration Management
Measurement & Analysis
A Scrum Adoption Survey
184 distinct organizations responded to the 2011
Scrum adoption survey
• 24 ML5
• 2 ML4
• 44 ML3
• 61 ML2
• 125 ML1 and unknown
128 projects adopting Scrum
M.C. Paulk, “A Scrum Adoption Survey,” ASQ
Software Quality Professional, March 2013.
6
Organizational Size
20
18
16
14
12
10
8
6
4
2
0
7
Organizational Size by Level
Org Size
≤25
26-50
51-75
76-100
101-200
201-300
301-500
501-1000
1001-2000
>2000
ML5 ML4 ML3 ML2 Unknown
1
0
0
2
15
0
0
0
1
14
0
0
0
0
5
0
0
0
2
9
2
0
2
3
10
2
0
0
2
10
4
1
0
1
9
3
0
3
2
6
3
0
4
1
9
0
0
0
0
0
8
Software Engineering Methods
Other methods
Unified Process (including
RUP, AUP, OUP)
Team Software Process (TSP)
Crystal methods (including
Crystal Clear)
Feature Driven Development
(FDD)
Extreme Programming (XP)
Scrum
0
9
50
100
150
Methods by Level
A high percentage of high maturity organizations
use Scrum.
Many high maturity organizations use XP, FDD,
and the Unified Process.
Method
ML5 ML4 ML3 ML2 Unknown
Scrum
17
0
12
16
86
Extreme Programming (XP)
5
0
6
6
16
Feature Driven Development (FDD)
5
0
2
4
11
Crystal methods (including Crystal Clear)
0
0
1
0
5
Team Software Process (TSP)
1
0
2
1
0
Unified Process (including RUP, AUP, OUP)
7
2
4
6
18
Other methods
8
1
10
8
87
10
Requirements Volatility Per Month
>50% per month
20-50% per month
10-20% per month
5-10% per month
3-5% per month
1-3% per month
<1% per month
0
10
11
20
30
40
Requirements Volatility by Level
Requirements
Volatility
ML5 ML4 ML3 ML2
<1% per month
4
0
6
4
1-3% per month
3
0
3
1
3-5% per month
3
0
0
1
5-10% per month
4
0
3
3
10-20% per month 7
1
0
6
20-50% per month 1
0
0
0
>50% per month
5
0
1
1
12
Unknown
13
5
5
17
17
14
7
Use of Scrum
Scrum is the normal way we build software
Scrum is one of the standard methods we…
We are currently deploying Scrum across…
We have piloted Scrum but no decision…
We are currently piloting Scrum
We are aware that Scrum exists
We never heard of Scrum before
0
13
10
20
30
40
50
60
Use of Scrum by Level
Few high maturity organizations are unaware of
Scrum or uninterested in it.
Scrum Adoption
We never heard of Scrum before
We are aware that Scrum exists
We are currently piloting Scrum
We have piloted Scrum but no
decision about adoption has been
made
We are currently deploying Scrum
across the organization
Scrum is one of the standard methods
we use
Scrum is the normal way we build
software
14
ML5
1
2
5
ML4
0
1
0
ML3
1
2
0
ML2 Unknown
0
4
1
32
1
9
1
1
4
1
9
2
0
2
3
16
6
0
3
11
30
4
0
3
0
25
Scrum Team Size by Level
Team
Size
ML5 ML4 ML3 ML2 Unknown
≤3
0
1
4
3
6
4 to 6
5
0
2
2
22
7 to 9
5
0
1
4
27
10 to 12
2
0
0
3
6
13 to 15
4
0
1
1
2
≥16
6
0
2
2
14
15
Quality of the Software
much higher
higher
about the same
lower
much lower
0
20
16
40
60
80
Quality by Level
Quality tends to be higher for organizations
adopting Scrum in general.
Quality
much lower
lower
about the same
higher
much higher
ML5
0
1
2
12
5
ML4
0
0
1
0
0
17
ML3
0
1
2
5
2
ML2
0
2
4
5
2
Unknown
1
3
14
37
15
Cost
much higher
higher
about the same
lower
much lower
0
10
20
18
30
40
50
60
Cost by Level
Cost does not appear to be significantly
affected for most organizations adopting
Scrum.
• lower for a noticeable percentage
Cost
ML5 ML4 ML3 ML2 Unknown
much lower
0
0
0
0
3
lower
7
0
3
5
22
about the
same
13
1
5
5
29
higher
0
0
1
1
9
much higher
0
0
0
2
7
19
Meeting Schedule Expectations
much higher
higher
about the same
lower
much lower
0
20
20
40
60
Schedule Expectations by Level
Meeting schedule expectations appears to be
higher for organizations adopting Scrum
in general.
Schedule
Expectations
ML5 ML4 ML3 ML2 Unknown
much lower
0
0
0
0
4
lower
2
0
0
1
3
about the same
3
1
1
4
12
higher
12
0
6
6
33
much higher
3
0
2
3
18
21
Customer Satisfaction
much higher
higher
about the same
lower
much lower
0
20
22
40
60
Customer Satisfaction by Level
Customer satisfaction tends to be higher
for organizations adopting Scrum in general.
Customer
Satisfaction
ML5 ML4 ML3 ML2
much lower
0
0
0
0
lower
0
0
0
2
about the same 1
0
1
4
higher
11
1
5
3
much higher
8
0
3
5
23
Unknown
1
2
9
28
26
Concluding Thoughts
It appears that most high maturity organizations are
aware of, have piloted, and have adopted agile
methods.
Most particularly Scrum.
With good success in terms of customer satisfaction,
quality, and meeting schedule expectations… and
some impact on improving cost.
This survey is only a first step in exploring the
adoption of agile methods by high maturity
organizations…
24
Questions and Answers
25