Transcript No Slide Title

University of Southern California
Center for Systems and Software Engineering
Software and System
Acquisition
CS 577b Software Engineering II
Supannika Koolmanojwong
April 11, 2011
University of Southern California
Center for Systems and Software Engineering
Outline
•
•
•
•
Definition of Software Acquisition
Life Cycle of Software Acquisition
Net-Centric Systems of Systems (NCSOS)
Top 10 risks of Software-Intensive SOS
(SISOS)
04/11/2011
© 2011 USC-CSSE
2
University of Southern California
Center for Systems and Software Engineering
Definition
• CMMI – ACQ
– Acquisition = The process of obtaining products
or services through supplier agreements. (See
also “supplier agreement.”)
04/11/2011
© 2011 USC-CSSE
3
University of Southern California
Center for Systems and Software Engineering
Software Acquisition
• US firms spend $250 annually acquiring software
(1996)
• Software investment results in productivity gains
and strategic advantages
• Consideration
–
–
–
–
–
–
04/11/2011
Determine the software requirements
Identify potential suppliers
Prepare contract requirements
Evaluate and select supplier
Manage supplier performance
Accept the software
© 2011 USC-CSSE
4
University of Southern California
Center for Systems and Software Engineering
Software Acquisition
• Acquisition Approach
– Custom
– Package
• Acquisition Source
– Internal Resource
– External Provider
04/11/2011
© 2011 USC-CSSE
5
University of Southern California
Center for Systems and Software Engineering
Software Acquisition Decision
Paul Nelson , William Richmond , Abraham Seidmann, Two dimensions of software acquisition,
Communications of the ACM, v.39 n.7, p.29-35, July 1996
04/11/2011
© 2011 USC-CSSE
6
University of Southern California
Center for Systems and Software Engineering
Deciding to Develop or Acquire: Options
Criteria
Develop
Outsource
Development cost, speed
OTS
**
• - **
• - **
Scarce staff
*
*
Non- core competency
function
Core competency function
**
*
**
Unique needs
**
*
Control of evolution
**
*
Life cycle cost
04/11/2011
*
© 2011 USC-CSSE
7
University of Southern California
Center for Systems and Software Engineering
Process Pattern Decision Driver
Decision Criteria
Importance
Alternatives
More than 30% of features available in NDI/NCS
Has a single NDI/NCS that satisfies a complete solution
Very unique/ inflexible business process
Life Cycle
Need control over upgrade / maintenance
Rapid Deployment; Faster time to market
Architecture
Critical on compatibility
Internet Connection Independence
Need high level of services / performance
Need high security
Asynchronous Communication
Access Data anywhere
Resources
Critical mass schedule constraints
Lack of Personnel Capability
Little to no upfront costs (hardware and software)
Low total cost of ownership
Not-so-powerful local machines
Architected
Agile
Use Single
NDI-Intensive
NDI
ServicesIntensive
0–1
0–1
2–4
2–3
4
0–1
3–4
2–3
0–1
3–4
2–3
0–1
2–4
0–1
0–1
4
0–1
2- 4
0–1
2–3
2–4
0–4
0–4
2–4
0–4
0–4
3–4
0–4
0–3
0 -4
0–4
0–4
1–3
0–4
0–3
0–4
0–4
0–4
2–4
0
0–2
0–2
0
4
0–1
0–2
0–2
0–1
1–4
3–4
3–4
2–4
0–3
1–3
2–3
2–4
2–4
0–3
0–4
2–4
2–3
3–4
3–4
3–4
Note: Decision importance scale varies from project to project
Decision Criteria Rating Scale; 0:Very Low; 1:Low; 2: Medium; 3:High; 4:Very High
Importance Rating Scale: 1:Low; 2: Medium; 3:High
09/03/2010
@USC CSSE
8
University of Southern California
Center for Systems and Software Engineering
ICSM Common Cases
Special Case
Example
1. Use NDI
Small Accounting
2. Agile
Simple business
application
Competitive
cellphone feature
Security kernel; Safetycritical LSI chip
Multisensor control device
Size,
Complexity
Change Rate
(%/Month)
Criticality
NDI Support
Org, Personnel
Capability
Complete
Low
1 – 30
Low-High
3. Architected
Agile
4. Formal
Methods
5 SW embedded
HW component
6. Indivisible IOC Complete vehicle platform
Med
1 – 10
Med-High
Low
Low
Extra High
Low
0.3 – 1
Med – High
0.3 – 1
Good;
In place
Some in place
7. NDI- Intensive
Supply Chain
Management
Med – High
0.3 – 3
Med-Very
High
High-Very
High
Med- Very
High
NDI-driven
architecture
Agile-ready
High
Agile-ready
High
Strong formal
methods experience
Experienced; medhigh
Experienced; medhigh
NDI-experienced;
Med-high
8. Hybrid agile /
plan-driven
system
9. Multi-owner
system of
systems
10. Family of
systems
11. Brownfield
Major Upgrade
12. Maintenance
Power grid control
Med – Very
High
Mixed parts:
1 – 10
Mixed parts
Mixed parts
Crisis management
Very High
Mixed parts:
1 – 10
Mixed parts;
Med-Very
High
Very High
Many NDIs;
some in place
Related experience,
med-high
Medical Device Product
Line
Incremental legacy
phase-out
On-going maintenance/
small hardware and/or
software upgrades to
legacy system
Med – Very
High
High-Very
High
Changes: Low
System: Low
–Very High
1–3
Some in place
Low
Med – Very
High
Med-High
Low-medium
average over
time
Changes:
Low-Very
High
Related experience,
med – high
Legacy reengineering
Maintenance with
some overall
knowledge of legacy
system
9
04/11/2011
© 2011 USC-CSSE
Good;
in place
Good;
most in place
None
NDI as legacy
replacement
Depends on
legacy system
University of Southern California
Center for Systems and Software Engineering
Comparison of process areas
CMMI-DEV
CMMI - SVC
Causal Analysis and Resolution (CAR)
Configuration Management (CM)
Decision Analysis and Resolution (DAR)
Measurement and Analysis (MA)
Organizational Process Definition (OPD)
Organizational Process Focus (OPF)
CMMI - ACQ
Organizational Performance Management (OPM)
Organizational Process Performance (OPP)
Organizational Training (OT)
Process and Product Quality Assurance (PPQA)
Requirements Management (REQM)
Risk Management (RSKM)
Project Planning (PP)
Work Planning (WP)
Project Planning (PP)
Project Monitoring and Control
Work Monitoring and Control (WMC)
Project Monitoring and Control (PMC)
Integrated Project Management
Integrated Work Management (IWM)
Integrated Project Management (IPM)
Quantitative Project Management
Quantitative Work Management (QWM)
Quantitative Project Management (QPM)
Supplier Agreement Management (SAM)
Product Integration (PI)
Requirements Development (RD)
Technical Solution (TS)
Validation (VAL)
Verification (VER)
03/02/2011
Capacity and Availability Management
(CAM)
Incident Resolution and Prevention (IRP)
Service Continuity (SCON)
Service Delivery (SD)
Service System Development (SSD)
Service System Transition (SST)
Strategic Service Management (STSM)
© 2011 USC-CSSE
Agreement Management (AM)
Acquisition Requirements Development
(ARD)
Acquisition Technical Management (ATM)
Acquisition Validation (AVAL)
Acquisition Verification (AVER)
Solicitation and Supplier Agreement
Development (SSAD)
10
University of Southern California
Center for Systems and Software Engineering
Outline
•
•
•
•
Definition of Software Acquisition
Life Cycle of Software Acquisition
Net-Centric Systems of Systems (NCSOS)
Top 10 risks of Software-Intensive SOS
(SISOS)
04/11/2011
© 2011 USC-CSSE
11
University of Southern California
Center for Systems and Software Engineering
Software acquisition phase milestones
Phase
Phase initiation
milestone
Phase completion
milestone
Planning
Idea is developed
Release the request for
proposal (RFP)
Contracting
RFP is released
Sign the contract
Product
Contract is signed
implementation
Product
Software product is
acceptance
received
Software product is
Follow-on
accepted
Steps in software acquisition
process
1.Planning organizational strategy
2.Implementing organizational process
3.Determining the software requirements
4.Identifying potential suppliers
5.Preparing contract requirements
6.Evaluating and selecting the supplier
Receive the software
product
7.Managing supplier performance
Accept the product
8.Accepting the software
Product is no longer in
use
9.Using the software
http://cs.wallawalla.edu/~aabyan/Colloquia/Acquisition/process.html
04/11/2011
© 2011 USC-CSSE
12
University of Southern California
Center for Systems and Software Engineering
Planning phase
Process steps -- key inputs and outputs
Phase
Planning
Step
Inputs*
1.Planning
organizational
strategy
•Acquirer's objectives
•Strategic areas
2.Implementing
organizational
process
•Steps 3-9 of the process
•Organizational strategy
•Contracting practices
•Organization's policies
3.Determining the
software
requirements
•Software definition
•Supplier evaluation criteria
•Acquirer and supplier
obligations
•Quality plan and maintenance
plan content
Outputs
•Quality characteristics of
software
•Organizational strategy for
acquiring software
•General practices
•Establish a software
acquisition process for
organization
•Supplier qualification and
selection process
•Software being acquired
defined
•Quality and maintenance plans
defined
•Proposal evaluation standards
•Contingency plan
•Request for proposal (RFP)
http://cs.wallawalla.edu/~aabyan/Colloquia/Acquisition/process.html
04/11/2011
© 2011 USC-CSSE
13
University of Southern California
Center for Systems and Software Engineering
Contracting phase
Process steps -- key inputs and outputs
Phase
Step
4.Identifying
potential
suppliers
5.Preparing
contract
Contracting
requirements
6.Evaluating
and selecting
the supplier
Inputs*
•Supplier performance data from prior
contacts
•Supplier evaluation criteria
•Definition of software
•Results being acquired
•User survey questionnaire
Outputs
•Information of software
•MOTS software/suppliers
•Candidate list
•User survey
•Supplier and acquirer responsibilities
•Supplier performance standards
•Acquirer's terms and conditions
•Quality assurance clauses
•Payment provisions
•Acceptance criteria
•Supplier performance criteria
•Evaluation and test criteria
•Tie payments to deliverables
•Prepared contract
•Legal counsel review
•Supplier proposals
•Proposal evaluation standards
•Supplier qualification and selection
process
•Visit supplier facilities
•User survey results
•Quality assurance clauses
•Evaluation of proposals
•Evaluation of suppliers
•Qualified suppliers list
•Supplier selection
•Negotiated contract
http://cs.wallawalla.edu/~aabyan/Colloquia/Acquisition/process.html
04/11/2011
© 2011 USC-CSSE
14
University of Southern California
Center for Systems and Software Engineering
Product Implementation –
Acceptance – Follow-on phases
Phase
Inputs*
Outputs
7.Managing
Product
supplier
implementation
performance
•Negotiated contract
•Contract milestones
•Acquirer's deliverables provided to
supplier
•Monitor supplier progress
•Supplier performance criteria
•Work segments approved
•Completed milestones
•Software deliverables
•Reliability and quality
measurements
•Feedback to supplier
Product
acceptance
8.Accepting the
software
•Acceptance criteria
•Evaluation criteria
•Test criteria
•Maintenance plan
•Supplier performance criteria
•Establish acceptance process
•Acceptance process
•Acceptable software
•Usable documentation
9.Using the
software
•Software deliverables
•Documentation
•Support available
•Quality plan
•Maintenance plan
•Contracting practices evaluated
•Practices to change
•Practices to retain
•User satisfaction assessment
•Supplier performance data
Follow-on
Step
http://cs.wallawalla.edu/~aabyan/Colloquia/Acquisition/process.html
04/11/2011
© 2011 USC-CSSE
15
University of Southern California
Center for Systems and Software Engineering
Checklists for software acquisition
• Checklist 1: Organizational strategy
– Establish the rights and responsibilities of supplier and acquirer
based on the capabilities of each organization
• Checklist 2: Define the software
– Quality characteristics
– Included deliverables
– Support
• Checklist 3: Supplier evaluation
–
–
–
–
–
04/11/2011
Financial soundness
Experience and capabilities
Development and control processes
Technical assistance
Quality practices
© 2011 USC-CSSE
−
−
−
−
−
Maintenance service
Product usage
Produce warranty
Costs
Contracts
16
University of Southern California
Center for Systems and Software Engineering
Checklists for software acquisition
• Checklist 4: Supplier and acquirer obligations
– Determine interm rights and responsibilities for each milestone
• Checklist 5: Quality and maintenance plans
– What the quality plan should contain
– What the maintenance plan should contain
• Checklist 6: User survey
–
–
–
–
Operation
Reliability
Maintenance service
Performance
• Checklist 7: Supplier performance standards
–
–
–
–
04/11/2011
Performance criteria
Evaluation and test
Correction of discrepancies
Acceptance criteria
© 2011 USC-CSSE
17
University of Southern California
Center for Systems and Software Engineering
Checklists for software acquisition
•
Checklist 8: Contract payments
–
•
Checklist 9: Monitor supplier progress
–
•
Payment provisions for maximum chance for success and reward for the supplier
achieving satisfactory progress
Actions that ensure adequate visibility of supplier progress
Checklist 10: Software evaluation
–
–
–
–
–
–
–
–
–
–
Functionality
Performance
Reliability
Availability
Ease of modification
Serviceability
Ease of installation
Ease of use
Adequacy of documentation
Cost to acquire and use
04/11/2011
Checklist 11: Software test
−Actions needed to maintain adequate control over the
software test
Checklist 12: Software acceptance
−Actions for certification process
−Remedies needed in case the supplier fails to perform
© 2011 USC-CSSE
18
University of Southern California
Center for Systems and Software Engineering
Context: “The World is Flat”
-Friedman, 2005
• Information processing functions can be
performed almost anywhere in the world
– Low-cost global fiber-optic communications
– Overnight global delivery services
• Significant advantages in outsourcing to
low-cost suppliers
– But significant risks also
• Competitive success involves pro-actively
pursuing advantages
– While keeping risks manageable
04/11/2011
© 2011 USC-CSSE
19
University of Southern California
Center for Systems and Software Engineering
Example Success Patterns
- e.g. Walmart, Dell
• Identify corporate focus and strategy
– Focus on core competitive discriminators
• Develop value chain linking inputs to
outputs and values
– Eliminate non-value-adding activities
– Identify which can be outsourced
• Select, engage outsourcing partners
• Monitor progress vs. plans
– Adapt to changes in marketplace, technology,
environment
04/11/2011
© 2011 USC-CSSE
20
University of Southern California
Center for Systems and Software Engineering
Preparing for Outsourcing (I)
-Gartner, 2004
• Self-Awareness
– Strengths, weaknesses, opportunities, threats
(SWOT)
• Resource Management
– Future vs. current allocation of resources,
accountability
• Competences
– Competitive core competencies; outsourceable
competencies
• Organization
– Multimodal: skills, operations, cross-cutting
improvements
04/11/2011
© 2011 USC-CSSE
21
University of Southern California
Center for Systems and Software Engineering
Preparing for Outsourcing (II)
-Gartner, 2004
• Roles
– Less on skills; more on business processes and
objectives
• Change Readiness
– Not just more adaptive, but more pro-active
• Communication
– Less hierarchical, open
04/11/2011
© 2011 USC-CSSE
22
University of Southern California
Center for Systems and Software Engineering
Core Competencies Include
Outsourcing
• Sourcing strategy and framework
– Business strategy and Op Con, technical architecture,
parts outsourced
• Source selection
– Request for proposals; evaluation criteria, evaluation
processes, selection processes
• Contact negotiation
– Statement of work, deliverables, award fees, change
adaptiveness, principled negotiation
• Outsourcing management
– Teambuilding, expectations management, change
management, progress monitoring, corrective action
• Success-critical stakeholder win-win
04/11/2011
© 2011 USC-CSSE
23
University of Southern California
Center for Systems and Software Engineering
Life Cycle Evolution Risks:
-Outsourcing or OTS
• Divergence of objectives
– Accommodation of special needs
• Erosion of core competencies
• Continuing-change costs
– COTS products: new release every 8-10 months;
unsupported after 3 releases
– Renegotiating supplier agreements
• Lack of technology scalability, evolvability
04/11/2011
© 2011 USC-CSSE
24
University of Southern California
Center for Systems and Software Engineering
Outline
•
•
•
•
Definition of Software Acquisition
Life Cycle of Software Acquisition
Net-Centric Systems of Systems (NCSOS)
Top 10 risks of Software-Intensive SOS
(SISOS)
04/11/2011
© 2011 USC-CSSE
25
University of Southern California
Center for Systems and Software Engineering
21st Century Acquisition Challenges
• Transformational net-centric systems of
systems (NCSOS)
– With broad and deep supplier chains
• Increasingly rapid change, need for agility
– Technology, competition, environment changes
– Requirements emergent rather than pre-specifiable
• Increasing need for discipline and high
assurance
– Security, safety, performance, interoperability, …
• Increasing dependence on COTS, reused, and
legacy components
04/11/2011
© 2011 USC-CSSE
26
University of Southern California
Center for Systems and Software Engineering
Complexity of NCSOS Solution Spaces
• Size: 10-100M lines of software code
• Number of external interfaces: 30-300
• Number of “Coopetitive” suppliers: 20-200
– Even more separate work locations
• Depth of supplier hierarchy: 6-12 levels
• Number of coordination groups: 20-200
– Reviews, changes, risks, requirements, architecture,
standards, procedures, technologies, -ilities, integration, test,
deployment, personnel, infrastructure, COTS,…
– Key personnel spend 60 hours/week in meetings
• Unprecedentedness
• Emergence
• Rapid change
04/11/2011
© 2011 USC-CSSE
27
University of Southern California
Center for Systems and Software Engineering
Rapid Change Trends
• Global connectivity and competition accelerate
change
– More ripple effects of technology, marketplace changes
• Increased need for agility, continuous learning
– Need to balance agility and plan-driven dependability
– Decline of THWADI (That’s how we’ve always done it)
– Avoid technical agility, administrative THWADI
• Hybrid agile/plan-driven processes needed for larger
systems
• Need for incremental processes, methods, tools,
skills
• Need for pro-active technology, marketplace
monitoring
• Education: Need to learn how to learn
04/11/2011
© 2011 USC-CSSE
28
University of Southern California
Center for Systems and Software Engineering
Failure Mode I: Build-to-Spec Deliverables
- Purchasing agent metaphor
• Rapid change: heavy spec change traffic, slow
contract changes
• Plus deep supplier chain: slowdowns multiply,
changes interact
• Plus emergent rqts: many initial specs wrong;
more changes
• Plus build-to-spec award fee: supplier inertia
• Bottom line: late rework, overruns, mission
shortfalls
04/11/2011
© 2011 USC-CSSE
29
University of Southern California
Center for Systems and Software Engineering
Failure Mode II: Sequential Document-Driven
Milestones
- Waterfall, V-model, MIL-STD-1521B
• Rqts. Emergence, COTS: freeze rqts. too early
• Plus document-completion progress metrics:
hasty point solutions, undiscovered risks
• Plus rapid change: problems with Failure Mode I
• Bottom line: more late rework, overruns, mission
shortfalls
04/11/2011
© 2011 USC-CSSE
30
University of Southern California
Center for Systems and Software Engineering
Failure Mode III: Hasty Best-of-Breed Source
Selection
- Overambitious startup schedule
• Complexity, emergence: incomplete, unvalidated
system architecture, solicitation SOWs
• Deep, wide supplier chains: incompatible legacy
solutions, COTS; critical-path modeling & simulation
needs
• Rapid change: rapid COTS evaluation, version
obsolescence
– GSAW: 8-11 months/release; 3 supported releases
• Bottom line: serious integration problems, overruns,
mission shortfalls
04/11/2011
© 2011 USC-CSSE
31
University of Southern California
Center for Systems and Software Engineering
Failure Mode IV: Incremental Document-Driven
Development
- Risk-insensitive “spirals”; ambitious increment schedules
• High assurance of –ilities: deferral to later increments
• Complex NCSOS: early-increment architecture
inadequate for later-increment –ilities.
• Rapid change: destabilization of ambitious increment
schedules; increment completion delays; next
increment destabilized
• Bottom line: serious security, safety, scalability
problems; destabilized development; more rework,
overruns, shortfalls
04/11/2011
© 2011 USC-CSSE
32
University of Southern California
Center for Systems and Software Engineering
Conclusions So Far
• 20th century contracting practices have many failure
modes for 21st century NCSOS
– High likelihood of serious overruns, mission shortfalls
– Still good for 20th century type acquisitions
• Purchasing agent metaphors a poor match to NCSOS
acquisition
– Acquirers need to operate in OODA loops
• Observe, Orient, Decide, Act
• New acquisition policies, processes, and practices
needed
– Empower acquisition corps to deal with NCSOS challenges
– Being worked out in various programs
– Example provided next
04/11/2011
© 2011 USC-CSSE
33
University of Southern California
Center for Systems and Software Engineering
Emerging Scalable Spiral Process
- For 21st century systems engineering and acquisition
• The C4ISR Metaphor for NCSOS Acquisition
– Role of OODA loops
– Example: Internet Spiral
– Example: FCS Win Win Spiral Model
• Risk-Driven Scalable Spiral Model
– Increment view
– Life-cycle view
– Role of anchor point milestones
• Acquisition management implications
• People management implications
04/11/2011
© 2011 USC-CSSE
34
University of Southern California
Center for Systems and Software Engineering
Acquisition C4ISR Via Spiral OODA Loops
– Example: ARPANet/Internet Spiral
Observe new/updated objectives,
Orient with respect to stakeholders
• Usage monitoring
• Risk/Opportunity analysis
• Competition, technology,
marketplace ISR
• Business case/mission analysis
constraints, alternatives
priorities, feasibility, risks
• Prototypes, models, simulations
Operate as current system
Accept new system
Act on plans, specifications
• Keep development stabilized
• Change impact analysis,
preparation for next cycle (miniOODA loop)
Decide on next-cycle capabilities,
architecture upgrades, plans
• Stable specifications, COTS
upgrades
• Development, integration, V&V, risk
management plans
• Feasibility rationale
04/11/2011
Life
Cycle Architecture Milestone for Cycle
© 2011 USC-CSSE
35
University of Southern California
Center for Systems and Software Engineering
Risk-Driven Scalable Spiral Model:
Increment View
Unforseeable Change
(Adapt)
Rapid
Change
Short
Development
Increments
Agile
Future Increment Baselines
Rebaselining for
Future Increments
Deferrals
Foreseeable
Change (Plan)
Increment N Baseline
Short, Stabilized
Development
of Increment N
Stable Development
Increments
Current V&V
High
Assurance
04/11/2011
Resources
Artifacts
Increment N Transition/O&M
Concerns
V&V
of Increment N
Future V&V
Resources
Continuous V&V
© 2011 USC-CSSE
36
University of Southern California
Center for Systems and Software Engineering
Acquisition Management Implications - I
• 20th century build-to-spec contracting
practices usable in part
– Good fit for stabilized-increments team
– But not for rebaselining, V&V teams
• Time & materials or equivalent
• Award fee based on cost/effectiveness
• These apply all the way down the supplier
chain
• Need top-level award fee for cost-effective
team balancing
– No stable distribution of effort
04/11/2011
© 2011 USC-CSSE
37
University of Southern California
Center for Systems and Software Engineering
Acquisition Management Implications - II
• Don’t skimp on system definition phases
– But avoid analysis-paralysis
– Use Feasibility evidence generation as progress metric
• Use more evidence-based source-selection processes
– Competitive exercise as proof of capability
– Preceded by multistage downselect
• Use Schedule/Cost as Independent Variable processes
– Prioritized features as dependent variable
• Top priority: transformational empowerment of acquisition corps
– Education, mentoring, tools, techniques
04/11/2011
© 2011 USC-CSSE
38
University of Southern California
Center for Systems and Software Engineering
Outline
•
•
•
•
Definition of Software Acquisition
Life Cycle of Software Acquisition
Net-Centric Systems of Systems (NCSOS)
Top 10 risks of Software-Intensive SOS
(SISOS)
04/11/2011
© 2011 USC-CSSE
39
University of Southern California
Center for Systems and Software Engineering
Top 10 SISOS Risks and Spiral Mitigation Strategies
• Risk 1:Acquisition Management and Staffing
– Over-commitment, especially requirements from legacy systems
• predetermined and allocated to hardware, software, and humans before
architecting the SISOS and contracting
– Mitigation:
• risk-driven concurrent engineering and evolutionary development
• may need several spiral cycles of risk resolution to get the right
combination of requirements, architecture, system elements, and lifecycle plans
– lack of rapid response to change; software expertise and decision authority
are scattered at low management levels across various project elements
– Mitigation:
• a project needs an integrated software and information processing leader
reporting directly to the project manager
– Key staff shortages and burnout; evolutionary acquisition project can go on
for years
– Mitigation:
• Career path development, mentoring junior staff to provide replacements
for key personnel, incremental completion bonuses
04/11/2011
© 2011 USC-CSSE
40
University of Southern California
Center for Systems and Software Engineering
Risk 2: Requirements/Architecture Feasibility
• Committing to a set of requirements or
architecture without validating feasibility
– E.g. one-second-response time requirement
– a custom $100 million system to meet the one-second
requirement when a prototype belatedly showed that a $30
million COTS-based system with a four-second-response time
would be sufficient
• Mitigation:
– Win-Win Spiral Model’s anchor-point milestone pass/fail
criteria and feasibility rationale explicitly address this risk
04/11/2011
© 2011 USC-CSSE
41
University of Southern California
Center for Systems and Software Engineering
Risk 3:Achievable Software Schedules
• More complex, longer schedule, higher risk
• Mitigation:
– Schedule feasibility, software cost and schedule estimation
– Explicit development and critical-path analysis of project
activity networks
– software reuse, COTS, reducing rework, top personnel, and
better tools
– SAIV
04/11/2011
© 2011 USC-CSSE
42
University of Southern California
Center for Systems and Software Engineering
Risk 4: Supplier Integration
• Integration team cohesion
• Mitigation:
– making them first-class stakeholders in
negotiating
– establishing early training and team-building
activities for selected suppliers; proactively
identifying needs for supplier collaboration and
networking
04/11/2011
© 2011 USC-CSSE
43
University of Southern California
Center for Systems and Software Engineering
Risk 5:Adaptation to Rapid Change
• Continuous adaptation to change across dozens of
suppliers, IPTs, and external interoperators can be
completely destabilizing
• Mitigation:
– balancing change and stability is incremental development
– Feasibility Evidence, milestone stabilization, synchronization
– COTS-watch, and interoperability-watch activities; crosssupplier and cross IPT networking; change-anticipatory
architectures; and agile change control and version control
capabilities
04/11/2011
© 2011 USC-CSSE
44
University of Southern California
Center for Systems and Software Engineering
Risk 6: Software Quality Factor Achievability
• the most difficult sources of SISOS
requirements/architecture feasibility risk
• Scenario-dependent and complex
• Mitigation:
– establish a quality factor trade space by replacing
single value quality factor requirements with a
range between acceptable and desired values
– Quality Trade-Off Analysis
04/11/2011
© 2011 USC-CSSE
45
University of Southern California
Center for Systems and Software Engineering
Risk 7: Product Integration and Electronic Upgrade
• SISOS - integrated across supplier hierarchies, IPT domains,
computing platforms, vehicle platforms, critical scenarios,
operational profiles, system modes, and friendly-to adversarial
environments
– Integration can cause significant delays and rework
– Potential mistakes: wrong version’s upgrades, update mismatches,
out-of-synchronization data
• Mitigation:
– up-front involvement of software-oriented integration, test,
supportability, and maintenance stakeholders in win-win
negotiations
– architecting the software to accommodate continuous
operation and synchronized upgrades
04/11/2011
© 2011 USC-CSSE
46
University of Southern California
Center for Systems and Software Engineering
Risk 8: Software COTS and Reuse Feasibility
• synchronizing COTS upgrades
– A) Update COTS during implementation
– B) continue to use unsupported version of
COTS
• Mitigation:
– establishing key COTS vendors as strategic
partners and success-critical stakeholders;
– proactive COTS-watch experimentation and
participation in user groups
04/11/2011
© 2011 USC-CSSE
47
University of Southern California
Center for Systems and Software Engineering
Risk 9: External Interoperability
• Large SISOS - more than 100 independently
evolving external systems
• Mitigation:
– establishing proactive stakeholder win-win
– relationships with critical interoperability
– systems, including memoranda of agreement on
interoperability preservation
– Joint Capabilities Integration and Development
System
04/11/2011
© 2011 USC-CSSE
48
University of Southern California
Center for Systems and Software Engineering
Risk 10:Technology Readiness
• Compatible technology maturity
– Mobile device, software middleware services
• Mitigation:
– satisfying a feasibility rationale for the key
advanced technologies, including the exercise
of models, simulations, prototypes
– Identifying fallback technology capabilities in
case key new technologies prove inadequate for
usage
04/11/2011
© 2011 USC-CSSE
49
University of Southern California
Center for Systems and Software Engineering
Conclusions
• 20th century acquisition practices inadequate to
address 21st century system acquisition challenges
– Net-centric systems of systems, emergent requirements,
rapid change, high assurance
• Risk-driven Scalable Spiral Process provides
framework for coping with 21st century challenges
– Stabilized increments using 20th century practices
– New practices for agile rebaselining, thorough V&V
• Transformational empowerment of acquisition corps a
top priority
– Education, mentoring, tools, techniques
04/11/2011
© 2011 USC-CSSE
50
University of Southern California
Center for Systems and Software Engineering
References
• Paul Nelson , William Richmond , Abraham Seidmann, Two
dimensions of software acquisition, Communications of the ACM,
v.39 n.7, p.29-35, July 1996
04/11/2011
© 2011 USC-CSSE
51