Transcript Slide 1

USC
C S E
University of Southern California
Center for Software Engineering
COSYSMO
COnstructive SYStems Engineering Cost MOdel
INCOSE IW Workshop
Dr. Barry Boehm
Ricardo Valerdi
Tampa, FL
February 3, 2003
University of Southern California
Center for Software Engineering (CSE)
February 2003
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
Outline
• Workshop Agenda
• Background on CSE, COSYSMO, and COCOMO II
• COSYSMO Overview
– Operational concept and scope
– Prototype demo
• Model Progress to Date
– Front end sizing and drivers
– Full life cycle sizing and drivers
• New proposed drivers
• Action items
February 2003
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
Workshop Agenda
• Proposed drivers
– Process maturity
– # of recursive levels of design
– Technology risk (different ratings on the
viewpoints)
– Length of life cycle
– Quality Attributes (ie., documentation)
– Manufacturability/Producibility
– Degree of Distribution
• Discontinuity of members
• ISO/IEC 15288 expertise in the working group
• Preparation for Delphi Round 2
• Preliminary data collection
February 2003
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
USC Center for Software Engineering (CSE)
• Researches, teaches, and practices CMMI-based
Software engineering
– Systems and software engineering fully integrated
• Focuses on better models to guide integrated systems
and software engineering
– Success models: stakeholder win-win, business
cases
– Product models: requirements, architectures, COTS
– Process models: spiral extensions, value-based
RUP extensions
– Property models: cost, schedule, quality
• Applies and extends research on major
programs (DARPA/Army, FCS, FAA ERAM,
NASA Missions)
February 2003
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
USC-CSE Affiliates (34)
• Commercial Industry (15)
– Daimler Chrysler, Freshwater Partners, Galorath, Group
Systems.Com, Hughes, IBM, Cost Xpert Group, Microsoft,
Motorola, Price Systems, Rational, Reuters Consulting, Sun,
Telcordia, Xerox
• Aerospace Industry (6)
– Boeing, Lockheed Martin, Northrop Grumman, Raytheon,
SAIC, TRW
• Government (8)
– DARPA, DISA, FAA, NASA-Ames, NSF, OSD/ARA/SIS, US
Army Research Labs, US Army TACOM
• FFRDC’s and Consortia (4)
– Aerospace, JPL, SEI, SPC
• International (1)
– Chung-Ang U. (Korea)
February 2003
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
INCOSE Companies actively
involved with COSYSMO
• Commercial Industry (1)
– Galorath
• Aerospace Industry (5)
– Lockheed Martin, Northrop Grumman*,
Raytheon, SAIC, TRW
• Government (1)
– US Army Research Labs
*Most recent
addition
• FFRDC’s and Consortia (2)
– Aerospace, SPC
February 2003
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
Key Members of the COSYSMO
Working Group
Aerospace Corp.
Galorath
LMCO
Raytheon
SAIC
SPC
US Army/PSSM
USC
Karen Owens, Marilee Wheaton
Evin Stump
Garry Roedler, Gary Hafen
Gary Thomas, John Rieff
Tony Jordano, Don Greenlee
Chris Miller
Cheryl Jones
Barry Boehm, Elliot Axelband,
Don Reifer, Ricardo Valerdi
underline = will participate in Monday’s Workshop
Italics = SE experience
February 2003
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
USC-CSE Cost, Schedule, and Quality Models
• Build on experience with COCOMO 1981, COCOMO II
– Most widely used software cost models worldwide
– Developed with Affiliate funding, expertise, data
support
• Collaborative efforts between Computer Science (CS) and
Industrial Systems Engineering (ISE) Depts.
– 3 CS PhD’s, 2 ISE PhD’s to date
– Valerdi an ISE PhD student
– Boehm joint appointment in CS, ISE
• COCOMO Suite of models
– Cost, schedule: COCOMO II, CORADMO, COCOTS
– Quality: COQUALMO
– Systems Engineering: COSYSMO
• Uses mature 7-step model development methodology
February 2003
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
7-step Modeling Methodology
Analyze Existing
literature
1
Perform
Behavioral Analysis
2
Identify Relative
Significance
3
A-PRIORI MODEL
+
SAMPLING DATA
=
A-POSTERIORI MODEL
Perform ExpertJudgement, Delphi
Assessment
4
Gather Project Data
5
Determine Bayesian
A-Posteriori Update
6
Gather more data;
refine model
7
February 2003
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
Parametric Cost Model Critical Path
Usual #
Months*
Critical Path Task
6
Converge on cost drivers, WBS
6
Converge on detailed definitions and rating scales
12
Obtain initial exploratory dataset (5-10 projects)
6
Refine model based on data collection & analysis
experience
12+
Obtain IOC calibration dataset (30 projects)
9
Refine IOC model and tool
*Can be shortened and selectively overlapped
February 2003
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
COSYSMO: Overview
• Parametric model to estimate system
engineering costs
• Covers full system engineering lifecycle
• Focused on use for Investment Analysis,
Concept Definition phases estimation and
tradeoff analyses
– Input parameters can be determined in
early phases
February 2003
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
COSYSMO Operational Concept
# Requirements
# Interfaces
# Scenarios
# Algorithms
+
Volatility Factor
Size
Drivers
Effort
Multipliers
- Application factors
-5 factors
- Team factors
-7 factors
- Schedule driver
February 2003
Effort
COSYSMO
Duration
Calibration
WBS guided by
EIA/ANSI 632 &
ISO/IEC 15288
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
Previous COSYSMO Evolution Path
Inception
Elaboration
Construction
IP (Sub)system
1. COSYSMO-IP
C4ISR System
2. COSYSMO-C4ISR
Physical Machine
System
System of
Systems (SoS)
February 2003
Oper Test
& Eval
Transition
3. COSYSMO-Machine
4. COSYSMO-SoS
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
Revised View of COSYSMO Evolution Path
(Results from last week’s meeting)
Conceptualize
Develop
Oper Test
& Eval
Transition to
Operation
Global Command
and Control
System
1. COSYSMO-IP
Satellite Ground
Station
2. COSYSMO-C4ISR
Joint Strike Fighter
3. COSYSMO-Machine
Future Combat
Systems
4. COSYSMO-SoS
Operate,
Maintain, or
Enhance
Replace or
Dismantle
Include ISO/IEC 15288 Stages
Initiate data collection for all and let
the amount of data received
determine what is included.
February 2003
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
Breadth and Depth of Key SE Standards
Process
description
High level
practices
Detailed
practices
ISO/IEC 15288
EIA/ANSI 632
IEEE 1220
Level of detail
System life
Conceptualize Develop
Input to
632/1220
Transition to
Operation
Operate,
Maintain,
or Enhance
Replace
or Dismantle
Purpose of the Standards:
ISO/IEC 15288 - Establish a common framework for describing the life
cycle of systems
EIA/ANSI 632 - Provide an integrated set of fundamental processes to
aid a developer in the engineering or re-engineering of a system
IEEE 1220 - Provide a standard for managing systems engineering
February 2003
Source : Draft Report ISO Study Group May 2, 2000
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
ISO/IEC 15288 Key Terms
• System
– a combination of interacting elements organized to achieve one or more
stated purposes
• System-of-Interest
– the system whose life cycle is under consideration in the context of this
International Standard
• System Element
– a member of a set of elements that constitutes a system
– NOTE: A system element is a discrete part of a system that can be
implemented to fulfill specified requirements
• Enabling System
– a system that complements a system-of-interest during its life cycle
stages but does not necessarily contribute directly to its function during
operation
– NOTE: For example, when a system-of-interest enters the production
stage, an enabling production system is required
February 2003
Source: ISO/IEC 15288.
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
ISO/IEC 15288 System of Interest
Structure
Systemof-interest
System
System
element
System
System
element
System
System
element
February 2003
System
element
System
element
System
System
element
System
element
System
element
System
System
element
System
element
System
element
System
element
System
element
System
element
System
element
System
System
element
System
System
element
System
element
System
element
Source: ISO/IEC 15288.
Make or
buy
System
element
System
element
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
Outline
• Workshop Agenda
• Background on CSE, COSYSMO, and COCOMO II
• COSYSMO Overview
– Operational concept and scope
– Prototype demo
• Model Progress to Date
– Front end sizing and drivers
– Full life cycle sizing and drivers
• New proposed drivers
• Action items
February 2003
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
Recent developments
• Developed a project plan
• Reduced drivers from 24 to 18
• Expanded the model to cover the later phases
of the life cycle
• Replaced EIA 632 with ISO 15288
• Introduced new drivers to reflect full life cycle
scope
• Changed name from COSYSMO-IP (Information
Processing) to COSYSMO
• Revised the evolution path to allow the data to
drive the scope of the model
February 2003
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
4 Size Drivers
1.
2.
3.
4.
Number of System Requirements
Number of Major Interfaces
Number of Operational Scenarios
Number of Unique Algorithms
• Each weighted by complexity, volatility, and degree of reuse
February 2003
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
Number of System Requirements
This driver represents the number of requirements that are typically taken from
the system or marketing specification. A requirement is a statement of capability
containing a normative verb such as “shall” or “will.” It may be functional,
performance, feature, or service-oriented in nature depending on the
methodology used for specification. System requirements can typically be
quantified by counting the number of applicable “shall’s” or “will’s” in the system or
marketing specification.
Easy
No. of System
Requirements
February 2003
Nominal
Difficult
- Well specified
- Loosely specified
- Poorly specified
- Traceable to source
- Can be traced to source with some
effort
- Hard to trace to source
- Simple to understand
- Takes some effort to understand
- Hard to understand
- Little requirements overlap
- Some overlap
- High degree of requirements
overlap
- Familiar
- Generally familiar
- Unfamiliar
- Good understanding of what’s
needed to satisfy and verify
requirements
- General understanding of what’s
needed to satisfy and verify
requirements
- Poor understanding of what’s
needed to satisfy and verify
requirements
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
Number of Major Interfaces
This driver represents the number of shared major physical and logical boundaries
between system components or functions (internal interfaces) and those external
to the system (external interfaces). These interfaces typically can be quantified by
counting the number of interfaces identified in either the system’s context diagram
and/or by counting the significant interfaces in all applicable Interface Control
Documents.
Easy
No. of Major
Interfaces
February 2003
Nominal
Difficult
- Well defined
- Loosely defined
- Ill defined
- Uncoupled
- Loosely coupled
- Highly coupled
- Cohesive
- Moderate cohesion
- Low cohesion
- Well behaved
- Predictable behavior
- Poorly behaved
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
Number of Operational Scenarios
This driver represents the number of operational scenarios that a system must
satisfy. Such threads typically result in end-to-end test scenarios that are developed
to validate the system satisfies all of its requirements. The number of scenarios can
typically be quantified by counting the number of end-to-end tests used to validate
the system functionality and performance. They can also be calculated by counting
the number of high-level use cases developed as part of the operational architecture.
Easy
No. of
Operational
Scenarios
February 2003
Nominal
Difficult
- Well defined
- Loosely defined
- Ill defined
- Few end-to-end scenarios
(< 10)
- Modest no. of end-to-end scenarios
(10 < OS < 30)
- Many end-to-end
scenarios (> 30)
- Timelines not an issue
- Timelines a constraint
- Tight timelines through scenario
network
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
Number of Unique Algorithms
This driver represents the number of newly defined or significantly altered functions
that require unique mathematical algorithms to be derived in order to achieve the
system performance requirements. As an example, this could include a complex
aircraft tracking algorithm like a Kalman Filter being derived using existing
experience as the basis for the all aspect search function. Another example could be
a brand new discrimination algorithm being derived to identify friend or foe function
in space-based applications. The number can be quantified by counting the number
of unique algorithms needed to support each of the mathematical functions specified
in the system specification or mode description document (for sensor-based
systems).
Easy
No. of Unique
Algorithms
February 2003
Nominal
Difficult
- Existing algorithms
- Some new algorithms
- Many new algorithms
- Basic math
- Algebraic by nature
- Difficult math (calculus)
- Straightforward structure
- Nested structure with decision logic
- Recursive in structure
with distributed control
- Simple data
- Relational data
- Persistent data
- Timing not an issue
- Timing a constraint
- Dynamic, with timing issues
- Library-based solution
- Some modeling involved
- Simulation and modeling
involved
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
12 Cost Drivers
Application Factors (5)
1.
2.
3.
4.
5.
February 2003
Requirements understanding
Architecture complexity
Level of service requirements
Migration complexity
Technology Risk
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
Requirements understanding
This cost driver rates the level of understanding of the system requirements by all
stakeholders including the systems, software, hardware, customers, team
members, users, etc…
Very low
Poor,
unprecedented
system
February 2003
Low
Minimal, many
undefined areas
Nominal
Reasonable, some
undefined areas
High
Strong, few
undefined areas
Very High
Full understanding of
requirements, familiar
system
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
Architecture complexity
This cost driver rates the relative difficulty of determining and managing the system
architecture in terms of platforms, standards, components (COTS/GOTS/NDI/new),
connectors (protocols), and constraints. This includes tasks like systems analysis,
tradeoff analysis, modeling, simulation, case studies, etc…
Very low
Poor understanding
of architecture and
COTS,
unprecedented
system
February 2003
Low
Nominal
High
Very High
Minimal
understanding of
architecture and
COTS, many
undefined areas
Reasonable
understanding of
architecture and
COTS, some weak
areas
Strong understanding
of architecture and
COTS, few undefined
areas
Full understanding of
architecture, familiar
system and COTS
2 level WBS
3-4 level WBS
5-6 level WBS
>6 level WBS
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
Level of service requirements
This cost driver rates the difficulty and criticality of satisfying the Key Performance
Parameters (KPP). For example: security, safety, response time, the “illities”,
etc…
Very low
Low
Nominal
High
Very High
Difficulty
Simple
requirements
Low difficulty
Moderately
complex
Difficult
requirements
Really complex
Criticality
Slight
inconvenience
Easily
recoverable
losses
Some loss
High financial
loss
Risk to human
life
February 2003
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
Migration complexity
This cost driver rates the complexity of migrating the system from previous system
components, databases, workflows, etc, due to new technology introductions,
planned upgrades, increased performance, business process reengineering etc…
Very low
Low
Nominal
Introduction of
requirements is
transparent
February 2003
High
Difficult to upgrade
Very High
Very difficult to
upgrade
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
Technology Risk
The maturity, readiness, and obsolescence of the
technology(ies) being implemented. This may include
Viewpoint
Rating
VL
L
N
H
VH
Technology
Maturity Level
Technology proven
and widely used
throughout industry
Proven through actual use
and ready for widespread
adoption
Proven on pilot projects
and ready to roll-out for
production jobs
Ready for pilot use
Still in the laboratory
Technology
Readiness Level
Mission proven (TRL
9)
Concept qualified (TRL 8)
Concept has been
demonstrated (TRL 7)
Proof of concept validated
(TRL 5 & 6)
Concept defined (TRL 3 &
4)
- Technology is the stateof-the-practice
- Emerging technology
could compete in future
- Technology is stale
- New and better technology is
on the horizon in the nearterm
- Technology is outdated and
use should be avoided in new
systems
- Spare parts supply is scarce
Technology
Obsolescence
Level
We need to supply guidelines on how to compose the
readiness and obsolescence into a single factor.
February 2003
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
12 Cost Drivers (cont.)
Team Factors (7)
1.
2.
3.
4.
5.
6.
7.
February 2003
Stakeholder team cohesion
Personnel capability
Personnel experience/continuity
Process maturity
Multisite coordination
Formality of deliverables
Tool support
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
Stakeholder team cohesion
Represents a multi-attribute parameter which includes leadership, shared vision,
diversity of stakeholders, approval cycles, group dynamics, IPT framework, team
dynamics, trust, and amount of change in responsibilities. It further represents the
heterogeneity in stakeholder community of the end users, customers,
implementers, and development team.
VL
Highly
heterogeneous
stakeholder
communities with
diverse objectives
Multiple
stakeholders with
diverse expertise,
task nature,
language, culture,
infrastructure
System involving
major changes in
stakeholder roles &
responsibilities
February 2003
L
Heterogeneous
stakeholder community
with converging
organizational
objectives
System involves some
changes in stakeholder
roles & responsibilities
N
Common shared
organizational
objectives
Functional team
H
Strong team cohesion
High stakeholder trust
level
Clear roles &
responsibilities
VH
Virtually homogeneous
stakeholder communities
Institutionalized, shared
project culture
Strong team dynamics
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
Personnel capability
Systems Engineering’s ability to perform in their duties and the quality of human
capital.
Very Low
15th percentile
Low
35th percentile
Nominal
55th percentile
High
75th percentile
Very High
90th percentile
Personnel experience/continuity
The applicability and consistency of the staff over the life of the project with respect
to the customer, user, technology, domain, etc…
Very low
Less than 2 months
February 2003
Low
Nominal
High
1 year continuous
experience, other
technical experience
in similar job
3 years of continuous
experience
5 years of continuous
experience
Very High
10 years of
continuous
experience
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
Revisit, should include more detail
Process maturity
Under process levels, process improvement
Maturity per EIA/IS 731, SE CMM or CMMI.
commitment
Very low
Level 1 (lower
half)
Low
Nominal
Level 1 (upper
half)
Level 2
High
Level 3
Very High
Level 4
Extra High
Level 5
Multisite coordination
Location of stakeholders, team members, resources (travel).
Very low
Low
Nominal
High
Very High
Extra High
SITE:
Collocation
International
Multi-city and
multi-national
Multi-city or
multicompany
Same city or
metro area
Same building
or complex
Fully located
SITE:
Communications
Some phone,
mail
Individual
phone, FAX
Narrowband
e-mail
Wideband
electronic
communicatio
n
Wideband
electronic
communication
, occasional
video
conference
Interactive
multimedia
Add Collaboration barriers row (time zone, security, export restrictions, language,
culture, competition, Intellectual property, contractual)
February 2003
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
Right size of documentation to the life cycle needs
The
Formality of deliverables
The breadth and depth of documentation required to be formally delivered.
Very low
Low
Nominal
High
Very High
General goals
Some conformity
Some relaxation
Partially streamlined
process, occasional
relaxation
Rigorous, follows
customer
requirements
Breadth depth
Right size
Reviews?
Tool support
Use of tools in the System Engineering environment.
Very low
No SE tools
February 2003
Low
Simple SE tools, little
integration
Nominal
Basic SE tools
integrated
(throughout the
systems process)
High
Strong, mature SE
tools, integrated
(integrated with other
disciplines)
Very High
Strong, mature
proactive SE tools
integrated with
process (Modelbased SE and
management
systems)
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
Outline
• Workshop Agenda
• Background on CSE, COSYSMO, and COCOMO II
• COSYSMO Overview
– Operational concept and scope
– Prototype demo
• Model Progress to Date
– Front end sizing and drivers
– Full life cycle sizing and drivers
• New proposed drivers
• Action items
February 2003
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
# and diversity of installations/platforms
The number of different platforms that the system will be hosted and installed on.
The complexity in the operating environment (space, sea, land, fixed, mobile,
portable, information assurance/security). For example, in a wireless network it
could be the number of unique installation sites and the number of types of fixed
clients, mobile clients, and servers.
Include phase out
Viewpoint
N
H
Sites/installations
Few # of installations or many
similar installations
Moderate # of installations or some
amount of multiple types of
installations
Numerous # of installations with
many unique aspects
Operating environment
Not a driving factor
Moderate environmental
constraints
Multiple complexities/constraints
caused by operating environment
Few types of platforms (< 5)
Modest types of no. of platforms
(5 < P <10)
Many types of platforms (> 10)
Homogeneous
Mixed
Heterogeneous
Typically networked using a
single protocol
Typically networked using several
consistent protocols
Typically networked using
different protocols
No. of Different
Platforms
February 2003
VH
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
# of recursive levels in the design
New cost driver. Can capture the number of boundaries in
the SOI (System of Interest). Number of recursive levels that
require SE. For example, a system with a single level of
design may have lower numbers of recursive levels in the
design. On the other hand, a system with a system
integrator (system of systems level), a prime (system level),
a subcontractor (segment level), second-tier subcontractor
(subsystem level), and third-tier contractor (component/box
level) would have 5 levels of recursive design. Includes
internal design/subsystem levels. Note: this driver depends
on what level your enabling system lies.
Revision: Has system-wide multiplicative effect vs.
incremental additive effect. Better handled in two cost
drivers: Architecture Complexity (technical aspects) &
Multisite Coordination (management aspects).
February 2003
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
# and diversity of installations/platforms
New size driver. Can capture the number of
varied and multiple installations and/or platforms.
This is in light of the discussion to cover the full
system life cycle. This driver is especially
important in later implementation/deployment
stages.
Revision: Its effect on effort is additive but the
amount of added effort per type of
installation/platform is a multiplier of the baseline
effort. Probably best covered by a formula like:
(# of diverse installation/platform types) * (Avg
extra % of systems engineering effort per type)
February 2003
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
The number of different processing platforms that the
system will be hosted and installed on. For example, in
a network it could be the number of unique installation
sites and the number of workstations and servers.
Viewpoint
N
H
VH
Sites/installations
Few # of
installations or
many similar
installations
Moderate # of
installations or
some amount of
multiple types of
installations
Numerous # of
installations with
many unique
aspects
Operating
environment
Not a driving
factor
Moderate
environmental
constraints
Multiple
complexities/cons
traints caused by
operating
environment
Few platforms
(< 5)
Modest no. of
platforms
(5 < P <10)
Many platforms
(> 10)
Homogeneous
Mixed
Heterogeneous
Typically
networked using
a single protocol
Typically
networked using
several consistent
protocols
Typically
networked using
different protocols
# of Different
Platforms
February 2003
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
# of years in operational life cycle
Revision: Another additive factor where the
added effort is a multiplier of the baseline
effort. Probably best covered by a formula
like:
(# of years in operational life cycle) * (Avg
annual % of continuing systems engineering
effort)
• Evolutionary acquisition
• # of independently evolving interoperating systems
• Upgrade to prior models
February 2003
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
# and diversity of
installations/platforms phased out
Revision: Its effect on effort is additive but
the amount of added effort per
installation/platform being phased out is a
multiplier of the baseline effort. Probably
best covered by a formula like:
(# of diverse installation/platform types) *
(Avg extra % of systems engineering effort
per type being phased out)
February 2003
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
Quality Attributes
Rationale: Reliability, availability and maintainability requirements for
certain classes of systems often drive their costs. This is true for
military as well as commercial systems. A cost driver aimed at
assessing the impact of varying these requirements is therefore needed.
Quality Attributes: Represents a measure of the extent to which the
system must perform its intended requirements (reliability), be available
(availability) and facilitate change, modification and repair
(maintainability) over time.
Viewpoint
Rating
VL
Reliability
Errors led to
slight
inconvenience
Availability
< 70%
availability
Maintainabili
ty
February 2003
Difficult to
change, modify
and/or repair
L
Errors led to
easily
recoverable
loss
Between 70%
and 90%
availability
Can change,
modify and/or
repair with
effort
N
Errors led to
moderate
recoverable
loss
> 90%
availability
Easily
changed,
modified and
repaired
H
Errors led to high
financial loss
VH
Safety
critical
24/7 with
schedulable
downtime (for
PM)
24/7 with
negligible
downtime
Reconfigurable
Self
healing
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
Manufacturability/Producibility
Rationale: The ability to manufacture/produce the system
can drive the costs especially when advances in
manufacturing technology are needed to field the system.
Manufacturability/Producibility: Represents the ability of contractors
to manufacture/produce the system in specific quantity to meet the market
demand.
Viewpoint
Rating
VL
Manufactura
bility
New
technology
needed to
manufacture
system
Producibility
Can meet less
than 70% of
peak demand
L
Facilities
needed to
manufacture
systems
Can meet
between 70
and 90% of
peak demand
N
Technology/faci
lities in place to
manufacture
system
Can meet 90%
of peak
demand
H
Can meet 100%
of peak
manufacturing
demand
Can meet 100%
of peak demand
VH
Have
excess
manufactur
ing
capability
DFMA
Design for “x”
Design trades
Have
excess
production
capability
Revisit
February 2003
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
Degree of Distribution
Rationale: The degree of partitioning and distribution of the
system impacts its cost. This cost driver how interoperable
the system must be.
Degree of Distribution: Characterizes how distributed the
architecture is.
Viewpoint
Rating
VL
February 2003
L
Distribution
Selfcontained
with limited
distribution
Distribution
via
LANs/direct
links
Interoperability
Selfcontained
with no
interoperabilit
y
Interoperates
with fewer
than 3
systems
N
Distribution via
LANs/WANs
Interoperates
with between 3
to 5 systems
H
Network of
networks
Interoperates
with 5 to 10
systems
VH
System of
systems
Interoperates
with more
than 10
systems
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
Levels & complexity of V&V
Revisit
Might be under # of Requirements (under VH)
February 2003
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
Size Drivers vs. EIA/ANSI 632 & ISO/IEC
15288 Stages Late in the Life Cycle
632 - 20
15288
(Implement- 632 - 21
632 - 31
632 - 32
632 - 33b
15288
(Maintenance
15288
ation)
(Transition) (Verification) (Readiness) (Validation) (Operations) or Support) (Retirement)
# System Requirements
# Major Interfaces
# Unique Algorithms
# Operational Scenarios
# Recursive Levels in the Design
# Systems being Phased Out
(Covered by Migration Complexity?)
# Operators / Maintainers
# Training Courses
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
# Installations
# System Elements
# Length of Lifecycle
Legend
Bold = existing driver
Italics = proposed addition
February 2003
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
Cost Drivers vs. EIA/ANSI 632 & ISO/IEC
15288 Stages Late in the Life Cycle
Application factors
–Requirements understanding
–Architecture complexity
–Level of service requirements,
criticality, difficulty
–Migration complexity
–Technology risk (maturity &
obsolescence)
632 - 20
15288
(Implement- 632 - 21
632 - 31
632 - 32
632 - 33b
15288
(Maintenance
15288
ation)
(Transition) (Verification) (Readiness) (Validation) (Operations) or Support) (Retirement)
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
- Operational Complexity
Legend
Bold = existing driver
Italics = proposed addition
February 2003
x
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
Cost Drivers vs. EIA/ANSI 632 & ISO/IEC
15288 Stages Late in the Life Cycle
Team factors
–Stakeholder team cohesion
–Personnel capability
–Personnel experience/continuity
–Process maturity
–Multi-site coordination
–Formality of deliverables
–Tool support
632 - 20
15288
(Implement- 632 - 21
632 - 31
632 - 32
632 - 33b
15288
(Maintenance
15288
ation)
(Transition) (Verification) (Readiness) (Validation) (Operations) or Support) (Retirement)
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
Legend
Bold = existing driver
Italics = proposed addition
February 2003
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
Action Items from Last WG Meeting
1. Develop a project Plan
2. Address technology maturity/obsolescence
3. Refine driver definitions to incorporate
ISO/IEC 15288 definitions
4. Incorporate System and People idea
5. Refine drivers applicability matrix
6. Develop data collection strategy
7. Generate Data Collection Form
8. Update Stakeholder Cohesion to include
diversity, identification and trust
February 2003
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
Calendar of Activities: 2003
Paper &
tutorial
submitted
USC CSE Annual Research Review
INCOSE 2003
ISPA
COCOMO Forum
J
F
M
A
M
J
J
A
2003
S
O
N
D
2004
INCOSE
Fall Workshop
Practical Software &
Systems Measurement
Workshop
Conference on
Systems Integration
February 2003
Paper
submitted
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
Points of Contact
Dr. Barry Boehm
[[email protected]]
Dr. Elliot Axelband
[[email protected]]
Don Reifer
[[email protected]]
Ricardo Valerdi
[[email protected]]
Website
http://sunset.usc.edu
February 2003
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
Backup Charts
February 2003
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
USC-CSE Research ($10M backlog)
• DARPA/Army: Model applications and extensions for
Future Combat Systems
• DARPA: Architectures for mobile distributed
systems (DASADA)
• FAA: Acquisition processes; COCOMO security
extensions
• NASA: Empirical methods for High Dependability
Computing
• NSF: Center for Empirically-Based Software
Engineering (with U. of Maryland)
• NSF: Strategic Design (with CMU, Virginia,
Washington)
• Industry Affiliates’ program
February 2003
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
General Affiliate Benefits
•
•
•
•
•
•
•
•
•
•
February 2003
Affiliates-only Web portal
– Early access to tools, methods, papers, talks,
student resumes
Tools: COCOMO Suite, Architecture tools, WinWin
Technical Report series
Workshops on Affiliate-prioritized topics
Annual Research Review and Steering Group
meeting
Annual one-day professor-visit
Bilateral visit arrangements; internships
Conferences and special workshops
Monthly LA SPIN meetings
Tutorials and eWorkshops
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
Collaboration Modes and Special Benefits
•
•
•
•
•
•
February 2003
Software architecting assistance
-Aerospace, Hughes, JPL, Northrop Grumman, TACOM, TRW,
Xerox
Software process/cost/quality/cycle time assistance
-Aerospace, Litton, Microsoft, Northrop Grumman, Raytheon, SAIC,
Sun, TACOM, TRW, Xerox
Management reviews of critical projects
-Litton, Motorola, SAIC, SEI, TRW
Reviews of corporate research programs
-Daimler Chrysler, Draper Labs, Lockheed Martin, SAIC, SEI, SPC,
Telcordia, TRW
Joint research contracts
-Aerospace, Lockheed Martin, Northrop Grumman, SEI, SPC, TRW
Aid in commercializing USC-CSE research
-C-Bridge, Galorath, Group Systems.com, Marotz, Price Systems,
Rational
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
Collaboration Modes and Special Benefits - II
•
•
•
•
•
February 2003
Special Projects
-Aerospace, Auto Club, FAA, Fidelity, IBM, JPL, Litton, Northrop
Grumman, Telcordia
Joint workshops on key topics
-Aerospace, Motorola, Rational, DOD/SIS, SEI, SPC
Focused working groups (COSYSMO)
-Aerospace, Galorath, Lockheed Martin, Raytheon, SAIC, SPC, TRW
Visiting collaborators
-Aerospace, Chung-Ang, C-Bridge, IBM, JPL, Litton, Northrop
Grumman, SEI, TRW
Corporate State-of-the-art tutorials
-Boeing, Chung-Ang, Daimler Chrysler, Draper, EDS, FAA, Fidelity,
IBM, JPL, Litton, Lockheed Martin, Lucent, Motorola, Microsoft,
Raytheon, SAIC, SEI, SPC, Sun, TRW, Xerox
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
Size Factors
Mapping of Old to New COSYSMO-IP Drivers
Old (7)
New (4)
Number of System Requirements
Number of Major Interfaces
Number of Technical Performance
Measures
Number of Operational Scenarios
Number of Modes of Operation
Number of Different Platforms
Number of Unique Algorithms
Number of System Requirements
Number of Major Interfaces
Number of Operational Scenarios
Number of Unique Algorithms
Level of Service Requirements
Architecture complexity
February 2003
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
Delphi Round 1 Highlights
Range of sensitivity for Size Drivers
6.48
5.57
6
4
February 2003
2.21
2.23
# Modes
# TPM’s
2.54
# Algorithms
# Interfaces
1
# Requirements
2
2.10
# Platforms
Effort
# Scenarios
Relative
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
Mapping of Old to New COSYSMO-IP Drivers
New (5)
Old (9)
Application Cost Factors
# of TPMs
# of Platforms
Requirements understanding
Architecture complexity
Level of service requirements
Legacy Transition complexity
COTS assessment complexity
Platform difficulty
Required business process
reengineering
Technology Maturity
Physical system/information
subsystem tradeoff analysis
complexity
February 2003
Requirements understanding
Architecture complexity
Level of service requirements
Migration complexity
Technology Maturity
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
Delphi Round 1 Highlights (cont.)
Range of sensitivity for Cost Drivers (Application Factors)
4
February 2003
Legacy transition
2.81
Level of service reqs.
1.93
2.43
Requirements und.
1.74
Platform difficulty
1.13
COTS
2.13
Bus. process reeng.
2
2.24
Architecture und.
EMR
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
Mapping of Old to New COSYSMO-IP Drivers
New (7)
Old (8)
Team Cost Factors
Reqs Und
Number and diversity of stakeholder
communities
Stakeholder team cohesion
Personnel capability
Personnel experience/continuity
Process maturity
Multisite coordination
Formality of deliverables
Tool support
February 2003
Stakeholder team cohesion
Personnel capability
Personal experience/continuity
Process maturity
Multisite coordination
Formality of deliverables
Tool support
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
Delphi Round 1 Highlights (cont.)
Range of sensitivity for Cost Drivers (Team Factors)
February 2003
Process maturity
2.16
2.46
Personnel capability
1.84
1.94
Personal experience
1.78
Formality of deliv.
1.28
1.91
Tool support
1.25
Stakeholder cohesion
2
Stakeholder comm.
EMR
Multisite coord.
4
INCOSE IW
USC
C S E
University of Southern California
Center for Software Engineering
COSYSMO/COCOMO II Mapping
Previous candidate starting point
EIA Stage
TBD
Development
Inception
Elaboration
Construction
Transition
14
12
10
14
10
8
5
5
38
18
8
4
19
36
16
4
8
13
34
19
8
10
24
24
3
3
3
30
Management
Environment/CM
Requirements
Design
Implementation
Assessment
When doing
COSYSMO-IP and
COCOMOII,
Subtract grey areas
prevent double
counting.
Deployment
TBD
TBD
TBD
= COCOMOII
February 2003
= COSYSMO-IP
INCOSE IW