Transcript Document

SE 470
Software Development Processes
James Nowotarski
19 May 2003
Course Map
Week
1
2
3
4
5
6
7
8
9
Content
. Rational Unified Process
. Extreme Programming
Implementation
. Tools, Training, Roles
. CMM, Metrics
. Selection & Evaluation
Briefings (Term Papers)
Assignments
Quizzes
Memorial Day
Overview
. Introduction
. History
10
11
Today’s Objectives
• Recap the Capability Maturity Model (CMM)
• Understand key issues of methodology selection
and evaluation
• Discuss term papers (last chance)
Today’s agenda
Topic
Duration
• CMM Recap
15 minutes
• CMM Jeopardy
45 minutes
• CMM Quiz
35 minutes
• *** Break
15 minutes
• Methodology Selection: Organization
50 minutes
• Methodology Selection: Project
20 minutes
• Project Methodology Adequacy
15 minutes
Today’s agenda
Topic
Duration
• CMM Recap
15 minutes
• CMM Jeopardy
45 minutes
• CMM Quiz
35 minutes
• *** Break
15 minutes
• Methodology Selection: Organization
50 minutes
• Methodology Selection: Project
20 minutes
• Project Methodology Adequacy
15 minutes
CMM Levels
Optimized
(5)
Managed
(4)
Defined
(3)
Repeatable
(2)
Initial
(1)
Key process areas (KPAs)
Maturity levels
Indicate
Contain
Process capability
Key process areas
Achieve
Goals
Contain
Key practices
CMM Appraisal Method
Team
Selection
Maturity
Questionnaire
Response
Analysis
3
2
1
KPA
Profile
On-site visit
Interviews &
document
reviews
4
Findings
based on the
CMM
5
6
Appraisal Methods
• Software Process Assessments (SPA)
– Performed in open, collaborative environment
– Focuses on improving the organization’s software
process
– Now called CMM-Based Appraisal for Internal Process
Improvement (CBA-IPI)
• Software Capability Evaluations (SCE)
– Performed in a more audit-oriented environment
– Focuses on identifying risks associated with a
contractor
– Team’s recommendation will help select contractors or
set fees
CMM Issues in the Real-World
• “Level envy”
• Areas not addressed
–
–
–
–
–
–
Business strategy and linkage to IT
Operations, help desk, support
Management of the IT human resource
Application portfolio
Tools
Risk
• Many question whether it is worth the effort to pursue
levels 4 and 5
CMM Maturity Profile
August 2002
100%
90%
% of Organizations
80%
70%
60%
43.2%
50%
40%
23.4%
30%
19.3%
20%
7.3%
6.8%
10%
0%
Initial
Repeatable
Defined
Managed
Based on assessments from 1998-2002 of 1124 organizations
Optimized
USA and Offshore Profiles
August 2002
100%
90%
% of Organizations
80%
70%
60%
50%
47.4%
37.6%
40%
25.3%
30%
20%
23.3%
22.0%
14.0%
13.2%
10.0%
10%
5.3%
2.0%
0%
Initial
Repeatable
Defined
USA
Managed
Offshore
Based on 645 U.S. organizations and 479 offshore organizations
Optimized
CMM
Approximately 50 of the 70 or so publicly-acknowledged
Level 5 CMM-certified organizations are in India
• Marketing tool to win clients, who are based
predominantly in US and Europe
• Clients using Indian service providers should have
certain key processes in place:
–
–
–
–
service level agreements
identifying business requirements
scoping requirements
managing changes
Time to Move Up
100
Number of
months to
move to next
maturity level
75
50
Largest observed value that
is not an outlier
28
Recommended time
between appraisals
(18-30 mos)
23
25
75th percentile
22
17
0
1 to 2
2 to 3
3 to 4
4 to 5
Median (50th percentile)
25th percentile
Smallest observed value that
is not an outlier
CMM-based Software Process
Improvement (SPI)
• Time and cost often exceed expectations
– 18-24 months to advance 1 level
– Can cost $2K per software engineer per year
– 1-2% full-time resources (e.g., 5-10 in a 500-person
organization)
– 2-4% of rest of organization’s time
• Difficult KPAs
– Planning and tracking
• Key success factors
– Senior management is engaged
– Participation and buy-in at all levels, including middle
management and technical staff
– Clearly stated, well understood SPI goals
– Clear assignment of responsibility
– SEPG staffed by highly respected people
Software Process Improvement Models
A number of models enable software development
organizations to compare their practices to a set of
“best practices”
IT specific models
• ISO 15504
• ISO 9000-3
• TickIT
General models
• Total Quality Management (TQM)
• Six Sigma
Software Process Improvement Models
ISO 15504
• International collaborative effort (including SEI)
• Sparked by an investigative study sponsored by the
U.K. Ministry of Defense (MOD)
• Objective: To develop a standard in the area of
software process assessment
– establish a common framework for expressing the
process capability ratings resulting from a 15504conformant assessment
– provide a migration path for existing assessment
models and methods wishing to become 15504conformant
Software Process Improvement Models
The Integrated CMM (CMMI) has emerged as the latest thinking
from SEI
• Over time, proliferation of CMMs:
–
–
–
–
Capability Maturity Model for Software (SW-CMM®)
Systems Engineering Capability Model (SECM) (may or may not include software)
Integrated Product Development Capability Maturity Model (IPD-CMM)
Software acquisition
• Many organizations wish to integrate improvement efforts across
disciplines
• Differences among these multiple models made integration
difficult
• SEI developed common framework to support integration of
current and future discipline-specific maturity models
• The common framework is called the Integrated CMM (CMMI)
–
“Each CMMI model is designed to be used in concert with other CMMI models, making
it easier for organizations to pursue enterprise-wide process improvement at their own
pace”
CMMI
CMMI integrates process improvement models for
product and service development and maintenance
• Incorporates and extends:
– Capability Maturity Model for Software (SW-CMM®)
– Systems Engineering Capability Model (SECM)
– Integrated Product Development Capability Maturity Model
(IPD-CMM)
– Supplier sourcing
• CMMI-SW model released August 2002
• SW-CMM® to be sunsetted by end of 2003 (stay tuned)
• What’s different about CMMI-SW:
– Stronger linkage to business objectives and customer needs
– Greater alignment with relevant ISO standards
– Standard CMMI Appraisal Method for Process Improvement
(SCAMPISM) V1.1 as a replacement for CMM-Based
Appraisal for Internal Process Improvement (CBA IPI) and
Software Capability Evaluation (SCESM)
CMMI
• What’s different about CMMI-SW:
– Stronger linkage to business objectives and customer needs
– Greater alignment with relevant ISO standards
– Standard CMMI Appraisal Method for Process Improvement
(SCAMPISM) V1.1 as a replacement for CMM-Based
Appraisal for Internal Process Improvement (CBA IPI) and
Software Capability Evaluation (SCESM)
– New process areas
– New names for maturity levels
• Still outside scope of CMMI:
– People CMM (P-CMM)
– Software Acquisition CMM (SA-CMM)
CMMI
• New names for maturity levels:
Level Old
New
1
Initial
Initial
2
Repeatable
Managed
3
Defined
Defined
4
Managed
Quantitatively Managed
5
Optimizing
Optimizing
CMMI
For more information
• http://www.sei.cmu.edu/publications/documen
ts/02.reports/02tr029.html (600+ page pdf)
Today’s agenda
Topic
Duration
• CMM Recap
15 minutes
• CMM Jeopardy
45 minutes
• CMM Quiz
35 minutes
• *** Break
15 minutes
• Methodology Selection: Organization
50 minutes
• Methodology Selection: Project
20 minutes
• Project Methodology Adequacy
15 minutes
Today’s agenda
Topic
Duration
• CMM Recap
15 minutes
• CMM Jeopardy
45 minutes
• CMM Quiz
35 minutes
• *** Break
15 minutes
• Methodology Selection: Organization
50 minutes
• Methodology Selection: Project
20 minutes
• Project Methodology Adequacy
15 minutes
Today’s agenda
Topic
Duration
• CMM Recap
15 minutes
• CMM Jeopardy
45 minutes
• CMM Quiz
35 minutes
• *** Break
15 minutes
• Methodology Selection: Organization
50 minutes
• Methodology Selection: Project
20 minutes
• Project Methodology Adequacy
15 minutes
Methodology Selection Process
Methodology Selection Process
1. Pre-RFP
2. RFP
3. Proposal
Submissions
4. Proposal
Evaluations
5. Vendor
Selection
6. Procurement
Method
7. ROI Analysis
8. Negotiate
Contract
Objective: Identify best
solution to meet stated
business need while
minimizing cost and risk
1. Pre-RFP
• Also known as Requirements Definition
• Preliminary analysis for management (not given to vendor)
• Serves as basis for Request for Proposal (RFP) and
evaluation criteria
• May be a simple presentation (small firm) or a formal report
• Most important step in the system procurement process
1. Pre-RFP
• Factors to consider
– Organization and project sizes
– Locations/Degree of distribution
– Diversity of application developers
– IT culture
– Cost/Budget
• Request for Information (RFI)
– Vendors are called/consultants consulted/research
conducted
– Breadth of alternatives is identified
– Vendors identified to participate in future stages
1. Pre-RFP
Sections of the Pre-RFP Report
• Problem statement
– Current state
– Gaps
– Risks
• Alternative solutions
• Ratings (of each alternative)
• Range of costs and benefits
• Recommended alternative and rationale
1. Pre-RFP
Categories of Methodology Providers
• Developed and delivered by consultants
• Software tool vendors
• Industry consortia or other groups
2. RFP
• Blueprint
• Confirms in detail the exact requirements stated in both
business and technical terms
• Limited distribution (e.g., 3-5 vendors)
– Protect confidentiality
– Keep selection process manageable
2. RFP
Sample Contents of an RFP
• Business need/Requirements
• Statement of Work to be done
– Methodology characteristics
– Implementation plan
– Training strategy
– Maintenance and support
– Cost/Budget
• Procedural details
– Form and structure of proposal
– Schedule (meetings, demos, selection)
– Key contacts
• Selection criteria
2. RFP
When should an RFP be used?
• Multiple solutions available that will fit the need
• Multiple vendors can provide the same solution
• Corporate policy requires it
2. RFP
Advantages of Using an RFP
• RFP team develops better understanding of their needs
from both a technical and business perspective
• Compels vendors to create competitive solutions
• Does not favor one vendor over another (in theory)
• “Everybody singing from same hymn book”
– Vendors working from same set of rules and
requirements
• Facilitates evaluation of competitive solutions
– Provides a foundation on which to base a more
rigorous evaluation of a vendor
3. Proposal Submissions
• Forums to answer vendor questions (written, oral)
– Vendor conferences before proposal submission
• Response content and format
• Sometimes requires "proof" statements, such as "This
feature was implemented 12 months ago and is currently
installed at 10 sites. Names and addresses are provided in
the reference section."
4. Proposal Evaluations
• Business and Technical Solutions
– Rating scale: (0=unresponsive, . . ., 5=exceptional)
• Vendor qualifications (site visits, reference checks)
• Preliminary cost, value, and risk analysis
• Cost proposal may be a separate document from
technical proposal
• Vendor demo
• Personnel assignment
• May be a two-stage process, with only a “short list” of 2-3
vendors doing demos and making “best and final” offers
• Question: Who are the key stakeholders in this process?
4. Proposal Evaluations
In selecting a vendor, there are major
management and technical considerations
Management considerations
• Ability and track record of vendor to meet commitments?
• Satisfaction levels of vendor’s current customers,
particularly long-term customers?
• Vendor’s track record for providing support?
• Any litigation pending against vendor?
• Is the vendor financially stable?
4. Proposal Evaluations
In selecting a vendor, there are major management
and technical considerations (cont.)
Technical considerations
• Ability and track record of vendor to meet technical
challenges of project?
• Level of vendor’s expertise in your industry (e.g., Financial
Services)? With your type of IT organization?
• Quality of vendor’s past work? Are metrics available?
What does a methodology product
consist of?
• Content
– building blocks (processes, deliverables, etc.)
– pre-defined routes
• Delivery vehicle (e.g., browser)
• Tools for authoring and publishing content
• Tools for applying the methodology to a specific project
– project planning and estimating tools
– process management tools
– project management tools
• Deliverable templates tightly coupled with a development
and/or execution platform
• Training and support
• Maintenance
Small groups
Identify criteria to be considered in each of the
following categories when considering methodology
products for your organization
•
•
•
•
•
•
Content
Methodology tools
Fit with development tools
Deployment and Support
Vendor
Cost
Small groups
Examples:
Content
- Which life cycle models are supported?
- Does the methodology support component-based
development?
5. Vendor Selection
• Vendor site visits
• Weighted score method
• Final cost, value, and risk analysis
– Costs
-- one-time vs. recurring
-- fixed vs. variable
– Benefits
-- tangible vs. intangible
6. Procurement Methods
• Purchase
– not that popular because of fear of obsolescence
– longest-term commitment of these 3 methods
• Subscription
– usually 12-36 months in duration
– often done with an option to buy
7. ROI Analysis
Earnings on investment
ROI 
Total investment
• Metrics to justify investment in methodology remains a
weak link
• Vendors supply very little help for customers developing
ROI models for methodology
• Managers must commit to developing custom cost, benefit,
and risk models to justify their methodology investments
8. Contract Negotiation
• Do’s
– Include vendor responses to RFP in the contract
– Consider intellectual property issues
– Leverage outside expertise in negotiations
– Provide incentives/penalties
• Don’ts
– Buy vaporware instead of proven solutions
– Purchase low bid unless the value is there
– Settle on final offer prematurely
Today’s agenda
Topic
Duration
• CMM Recap
15 minutes
• CMM Jeopardy
45 minutes
• CMM Quiz
35 minutes
• *** Break
15 minutes
• Methodology Selection: Organization
50 minutes
• Methodology Selection: Project
20 minutes
• Project Methodology Adequacy
15 minutes
Methodology Selection: Project
• The Course of Action (the selection of the system
development process) is driven by:
– Content factors: the result to be achieved
– Context factors: internal and external influences
Context
Course of
Action
Content
Decision Tree
Packaged
Buy
vs.
Build
See Table 1
Context 1
Asset-Based
Asset
vs.
NonAsset
See Table 2
Context 2
Legacy
Technology
Generation
See Table 3
Context 3
Agile
vs.
Traditional
See Table 4
Context 4
Agile
Trad.
Table 1 - Buy versus Build
• Content Factors
Shorter initial time to market
Higher responsiveness to market changes
Lower project risk
Lower budget for development
Greater variety of users (discretionary,
consumers, suppliers, etc.)
Greater need for competitive advantage
Greater need for multi-media content
Greater acceptability of constraints on
organization and processes
Buy Build
Comments:
X
X
X
(X)
In case of some enterprise wide packaged software
this is not/should not be a driving factor
(X) Although some packages can also support
X
Not a distinguishing factor in this decision
X
Context 1 - Buy versus Build
• Context Factors
– Is a suitable package available?
– Availability of tools (development tools, plugins, etc.)
– Previous success with system development
process
– Management understanding of and
commitment to one of the approaches
Table 2 - Asset versus Non-Asset
• Content Factors
Shorter initial time to market
Higher responsiveness to market changes
Lower project risk
Lower budget for development
Asset
X
Comments:
X
X
(X)
Greater variety of users (discretionary,
consumers, suppliers, etc.)
Greater need for competitive advantage
Greater need for multi-media content
Greater acceptability of constraints on
organization and processes
NonAsset
Sometimes the asset is considered an evolving
asset, and requires an eventual total development
cost equal to custom developed systems.
Not a distinguishing factor in this decision
(X)
X
Assets can sometimes be modified over time to
provide as much of a competitive advantage as a
custom developed system
Not a distinguishing factor in this decision
Context 2 - Asset versus Non-Asset
• Context Factors
– Is suitable asset available?
– Availability of tools (development tools, plugins, etc.)
– Availability of asset-experienced resources
– Previous success with system development
process
– Management understanding of and
commitment to one of the approaches
Table 3 - Technology Generation
• Content Factors
NC
Shorter initial time to market
Higher responsiveness to market changes
Lower project risk
Lower budget for development
Greater variety of users (discretionary,
consumers, suppliers, etc.)
Greater need for competitive advantage
Greater need for multi-media content
Greater acceptability of constraints on
organization and processes
C/S or
Host
Comments:
Not a distinguishing factor in this decision
Not a distinguishing factor in this decision
X
Not a distinguishing factor in this decision
X
Not a distinguishing factor in this decision
X
Not a distinguishing factor in this decision
Context 3 - Technology Generation
• Context Factors
– Availability of tools (development tools, plugins, etc.)
– Previous success with system development
process
– Need for the client to be considered leading
edge
– Management understanding of and
commitment to one of the approaches
Table 4 - Agile vs. Traditional
• Content Factors
Agile
Shorter initial time to market
Higher responsiveness to market changes
Lower project risk
Lower budget for development
Greater variety of users (discretionary,
consumers, suppliers, etc.)
Greater need for competitive advantage
Greater need for multi-media content
Greater acceptability of constraints on
organization and processes
Trad.
Comments:
Not a distinguishing factor in this decision
X
X
X
Not a distinguishing factor in this decision
Not a distinguishing factor in this decision
Not a distinguishing factor in this decision
Not a distinguishing factor in this decision
Context 4 - Agile vs. Traditional
• Context Factors
–
–
–
–
–
–
Availability of tools (development tools, plug-ins, etc.)
Previous success with system development process
Experience with object technology
Need for the client to be considered leading edge
Result will be used as an asset
Management understanding of and commitment to one
of the approaches
Words of Caution
• A system development process is not all-exclusive. In
all probability the project will use concepts, tasks,
deliverables from more than one system
development process
• One project, however, will follow primarily one system
development process
• For specific projects some factors may not apply, or
additional factors can have an influence on the
decisions.
Today’s agenda
Topic
Duration
• CMM Recap
15 minutes
• CMM Jeopardy
45 minutes
• CMM Quiz
35 minutes
• *** Break
15 minutes
• Methodology Selection: Organization
50 minutes
• Methodology Selection: Project
20 minutes
• Project Methodology Adequacy
15 minutes
Project Methodology Adequacy
Key Questions
Content
• Was there an adequate set of systems development
methodology and standards to guide project activities?
– What existed?
– Was it adequate?
Deployment
• Was the set of systems development methodology and
standards rolled out to developers so that they had
awareness and understanding of how to use them?
Usage
• Did project personnel adhere to the standards and methodology?
Next Class: June 2
• First half of term papers to be presented
Extra Slides