Transcript Slide 1

Screening for Real Options Opportunities
“In” an Engineering System:
A Step Towards Flexible Weapon System Development
Jason E. Bartolomei, Capt, USAF
Committee:
D. Hastings (chair), R. de Neufville, D. Rhodes
© 2005 Jason Bartolomei, Engineering Systems Division, Massachusetts Institute of Technology
Observed
Phenomena
Engineering
System
Military weapon development programs are “inflexible” engineering
systems which are unable to easily adjust in ways that either avoid
downside consequences or exploit upside opportunities
Physical
Technology
Infrastructure
Property
of Interest
Theoretical
Frameworks
Non Physical
Organization
Processes
Flexibility
Real Options
© 2005 Jason Bartolomei, Engineering Systems Division, Massachusetts Institute of Technology
System Engineering
Product Design
Complexity
Theory
Proposed Theoretical Contribution:
Advance Real Options Theory “In” Engineering Systems by developing a
methodology for identifying real option opportunities
Real Options Theory:
Identification
Valuation
intellectual
gap
Desired Practical Contributions:
-
-
Demonstrate how real options strategies can be applied to an actual
weapon system development program
Identify the enablers & impediments for flexibility in an engineering system
Methodological Framework:
System Engineering
Product Design
Decomposition
Real Options Analysis
Identification
Complex
Systems
Analysis
© 2005 Jason Bartolomei, Engineering Systems Division, Massachusetts Institute of Technology
Valuation
Methodology: C-DSM Design Structure Matrix
STAKEHOLDERS
SYSTEM BOUNDARY
OBJECTIVES
ATTRIBUTES
FUNCTIONAL DECOMPOSITION/ PROCESSES
FEEDBACK
OBJECTS/DETAILS
Map Mental Models
Causal Loops
Multidisciplinary
Inputs
ACTIVITIES
Game Theory
Network Analysis
Real Options
MDO
System Dynamics
AGENTS
FEEDFORWARD
SYSTEM DRIVERS
Quantitative Insights
Qualitative Understanding
Design Framework for Representing an Engineering System
(Social, Political,
Economic, System
Constraints &
Alterables)
C-DSM Decomposition Pushes the Limits of the “Knowable”
Reduces Epistemic Uncertainty
•
•
•
•
Represents Qualitative “World” Map of System
Captures Salient Information
– Model variables
– Casual Relations
Defines system boundaries
Identifies Observation Points
© 2005 Jason Bartolomei, Engineering Systems Division, Massachusetts Institute of Technology
•
Contains Essential Information Required for
Multiple Quantitative Methods
– Game Theory/Stakeholder Analysis
– Social Network/Network Analysis
– Agent-Based Methods
– MDO
– System Dynamics
Constructing a C-DSM
STAKEHOLDERS
SYSTEM BOUNDARY
OBJECTIVES
•
All matrices are n2—
share the same column
and row headings
•
Decomposition begins
northwest to southeast
•
Off-diagonal cells
represent relationships
between variables
•
Goal is to represent
traceability between
multiple views of the
system
•
The C-DSM maps
represents physical and
non-physical relations
ATTRIBUTES/SYSTEM PARAMETERS
FUNCTIONAL DECOMPOSITION/ PROCESSES
OBJECTS/DETAILS
ACTIVITIES
AGENTS
SYSTEM DRIVERS
© 2005 Jason Bartolomei, Engineering Systems Division, Massachusetts Institute of Technology
Stakeholder Matrix
STAKEHOLDERS
SYSTEM BOUNDARY
OBJECTIVES
ATTRIBUTES/SYSTEM PARAMETERS
FUNCTIONAL DECOMPOSITION/ PROCESSES
OBJECTS/DETAILS
ACTIVITIES
Rows and Columns
•
Stakeholders are individuals or organizations
define system objectives and/or the system value
proposition
•
External stakeholders (outside the dashed line)
have no control over the aspects of the system
•
Internal stakeholders (inside the dashed line)
interpret the objectives for the system, control
resources, manpower, and have decision making
authority
•
The extent of internal stakeholders control
defines the system boundary of the system
AGENTS
SYSTEM DRIVERS
Cells
•
•
Can be used to represent several different types of relationships between stakeholders
such as:
–
Hierarchy (put an X in a row cell if reports to a column)
–
Linkages could represent transfer of money, information, product, other
The result is a social network of organization/individual stakeholder interactions
© 2005 Jason Bartolomei, Engineering Systems Division, Massachusetts Institute of Technology
Objectives Matrix
STAKEHOLDERS
SYSTEM BOUNDARY
OBJECTIVES
ATTRIBUTES/SYSTEM PARAMETERS
FUNCTIONAL DECOMPOSITION/ PROCESSES
Rows and Columns
•
The combined objectives of all
stakeholders in the system otherwise
known as their value propositions
•
The objectives includes all
articulated and unarticulated
customer needs, system
requirements, and goals/objectives
OBJECTS/DETAILS
ACTIVITIES
AGENTS
SYSTEM DRIVERS
Cells
•
Can be used to represent whether the row and column objectives are in competition (-), cooperation
(+), or unrelated (0)
•
An additional matrix under the stakeholders and to the left of objectives is formed with stakeholder
columns and objectives rows. The cells in the matrix are marked with an “X” represent traceability
between stakeholders and objectives. Also, each cells can be codes as the stakeholder view towards
that objective, for instance: positive, negative, neutral
© 2005 Jason Bartolomei, Engineering Systems Division, Massachusetts Institute of Technology
Attributes/System Parameters Matrix
STAKEHOLDERS
SYSTEM BOUNDARY
OBJECTIVES
ATTRIBUTES/SYSTEM PARAMETERS
OBJECTS/DETAILS
Rows and Columns
•
Captures the system attributes and
parameters
•
Attributes are the measures for which the
system objectives are measured
–
ACTIVITIES
AGENTS
SYSTEM DRIVERS
•
For example an objective for a Mini UAV
might require the system to be manpackable. System weight is one attribute
for this objective
Parameters are the variables used to
describe elements of a system
–
For example: Wing length and root chord
length are examples of parameters that
describe a wing
Cells
•
Cells can be used to represent links between attributes and parameters
•
The matrices to the left and right of the attributes/system parameters shows
traceability with the variables in the other matrices
•
Links can be represent causal and non-causal relations between nodes
© 2005 Jason Bartolomei, Engineering Systems Division, Massachusetts Institute of Technology
Functions Matrix
STAKEHOLDERS
SYSTEM BOUNDARY
OBJECTIVES
ATTRIBUTES/SYSTEM PARAMETERS
FUNCTIONAL DECOMPOSITION/ PROCESSES
OBJECTS/DETAILS
Rows and Columns
•
Represent the functions required in
order to achieve the objectives
•
Functions represent the “What and
Why” for the variables in an
engineering system
•
Traditional functional decomposition
methods are used to create the
functions matrix, such as: Functional
Analysis Systems Technique, Quality
Function Deployment, etc.
ACTIVITIES
AGENTS
SYSTEM DRIVERS
Cells
•
Can be used to represent a sub-hierarchical relationship between the functions with an “X” signifying if
“row” falls under “column.”
•
The cells can be color codes to signify the type of relation between the functions, such as static,
dynamic, information, or material
•
The light orange matrices representing the mapping between function and form (objects and activites).
Object-Process Methodology (OPM) relations can be captured in these matrices
© 2005 Jason Bartolomei, Engineering Systems Division, Massachusetts Institute of Technology
Objects Matrix
STAKEHOLDERS
SYSTEM BOUNDARY
OBJECTIVES
ATTRIBUTES/SYSTEM PARAMETERS
FUNCTIONAL DECOMPOSITION/ PROCESSES
OBJECTS
Rows and Columns
•
The physical architectural entities
required to carry out the functions
•
The physical “form” of the system
ACTIVITIES
AGENTS
SYSTEM DRIVERS
Cells
•
•
•
Can be used to represent a physical network between subsystem and
components.
The cells can also be colored codes to signify the type of relation between the
functions, such as static, dynamic, information, or material
The light orange matrices representing the mapping between function and form
© 2005 Jason Bartolomei, Engineering Systems Division, Massachusetts Institute of Technology
Activities Matrix
STAKEHOLDERS
SYSTEM BOUNDARY
OBJECTIVES
ATTRIBUTES/SYSTEM PARAMETERS
FUNCTIONAL DECOMPOSITION/ PROCESSES
OBJECTS
Rows and Columns
•
The activities required to be
performed by the system
•
Activities flow from the functions and
objects matrices
•
Represents information generally
found in a Work Breakdown
Structure
ACTIVITIES
AGENTS
SYSTEM DRIVERS
Cells
•
•
•
Can be used to represent if the activities are dependent/interdependent
The light orange matrix below the functions matrix maps function to form
The light orange matrix below the objects matrix is used to represent if there is a
relationship between an activity and a physical object
© 2005 Jason Bartolomei, Engineering Systems Division, Massachusetts Institute of Technology
Agents Matrix
STAKEHOLDERS
SYSTEM BOUNDARY
OBJECTIVES
ATTRIBUTES/SYSTEM PARAMETERS
FUNCTIONAL DECOMPOSITION/ PROCESSES
Rows and Columns
•
Represent the human agents
responsible for executing the
activities within the engineering
system
•
Represents the organization in place
to fulfill system objectives
OBJECTS
ACTIVITIES
AGENTS
SYSTEM DRIVERS
Cells
•
•
Can be used to represent links between agents (social network analysis)
The light orange matrix represents traceability between the organization and the
activities to be performed. An “X” signifies the specific agents associated to a
particular task
© 2005 Jason Bartolomei, Engineering Systems Division, Massachusetts Institute of Technology
System Drivers Matrix
STAKEHOLDERS
SYSTEM BOUNDARY
OBJECTIVES
ATTRIBUTES/SYSTEM PARAMETERS
FUNCTIONAL DECOMPOSITION/ PROCESSES
Rows and Columns
•
Captures the exogenous variables
that influence the system or are
beyond the internal stakeholders
control
•
In particular, these are the social,
political, economic, and technical
constraints, enablers that affect the
system
OBJECTS
ACTIVITIES
AGENTS
SYSTEM DRIVERS
Cells
•
•
Can be used to represent links between drivers to show interrelationships
More importantly, the additional matrix above the system drivers matrix
represents how system drivers affect the parameters within the engineering
system…the matrix to the left represents how the engineering system can
influence the system drivers
© 2005 Jason Bartolomei, Engineering Systems Division, Massachusetts Institute of Technology
Using the C-DSM
To Screen for “Hot” and “Cold” Spots
STAKEHOLDERS
“Cold” Spots
SYSTEM BOUNDARY
OBJECTIVES
ATTRIBUTES
FUNCTIONAL DECOMPOSITION/ PROCESSES
STAKEHOLDERS
SYSTEM BOUNDARY
OBJECTIVES
ATTRIBUTES
FUNCTIONAL DECOMPOSITION/ PROCESSES
OBJECTS/DETAILS
OBJECTS/DETAILS
ACTIVITIES
Static
Analyses:
“Hot” Spots
AGENTS
SYSTEM DRIVERS
ACTIVITIES
Dynamic
Analyses:
AGENTS
SYSTEM DRIVERS
(Social, Political,
Economic, System
Constraints &
Alterables)
(Social, Political,
Economic, System
Constraints &
Alterables)
Working Definitions:
- Nodes in an engineering system are “Hot” if future states are uncertain (change is likely) and changes have
significant upside or downside consequences
- Nodes are “Cold” if the are unlikely to change and likely changes have insignificant consequences on system
Finding the Spots:
Qualitative Examination
•
Nodal characteristics
•
Link Characteristics
•
Historical Trends
–
–
•
Static Analysis
•
Network Analysis
CONOPs Changes
Tech Innovation Clockspeeds
Perform Change Modes &
Effects Analysis (CMEA)
© 2005 Jason Bartolomei, Engineering Systems Division, Massachusetts Institute of Technology
–
–
–
–
–
Centrality
Criticality
Clustering
Path Length
Long Chains
Dynamic Analysis
•
Nonlinearities (Hot)
•
Accumulation Points (Cold)
•
Retrodict Past Behavior
•
Predictive Value
Current Case Study:
Mini-UAV
Mini-UAV
Current AF Ground Operator
Lighter Power Supplies
Battlefield
Communications
Transform
• Too much weight: 150-lb pack
–
(75-lbs special equipment PLUS weapon, rations,
etc.)
• Too many opportunities for error
• Too much time to transmit data to aircraft
© 2005 Jason Bartolomei, Engineering Systems Division, Massachusetts Institute of Technology
Lighter Targeting System
Future AF Ground Operator
C-DSM DSM
“World View” of Micro UAV Development System
Network Statistics:
~400 nodes: ~80% complete
~200 links: ~20% complete
© 2005 Jason Bartolomei, Engineering Systems Division, Massachusetts Institute of Technology
Micro UAV M-DSM
As of Oct 2005
Qualitative Analysis:
Change Modes and Effects Analysis (CMEA)* for “Hot” Spot Identification
Actuator
Empennage
Fuselage
Physical System
Servo
Skins
Ribs
Camera #2
Wing
Camera #1
Motor
Prop
• Examine each element in the physical structure for:
– Probability of change occurrence, readiness for change, ease of
change
• Expand the CMEA methodology to examine other
aspects of the engineering system
*Rajan et al, “An Empirical Foundation for Product Flexibility,” Design Studies 26 (2005), 405-438
© 2005 Jason Bartolomei, Engineering Systems Division, Massachusetts Institute of Technology
Static Analysis:
Searching for “Hot” and “Cold” Spots with MDO
Xo1
Xo4
Xo2
Xo5
Xo3
Xo6
© 2005 Jason Bartolomei, Engineering Systems Division, Massachusetts Institute of Technology
Static Analysis:
MDO Pareto Frontier Results and Implications to Physical Architecture
Endurance vs. Longest Linear Dim ension
1.2
1.1
Endurance (hr)
1
0.9
Xo1
0.8
Xo2
0.7
Xo3
Xo4
0.6
Xo5
Xo6
0.5
Approx. Pareto Front
0.4
18
28
38
48
58
68
78
Longest Linear Dim ension (cm )
Cold Spots
Platform
Opportunities
Endogenous Interfaces
Exogenous Interfaces
Hot Spots
Real Options
Opportunities
Exogenous Interfaces
© 2005 Jason Bartolomei, Engineering Systems Division, Massachusetts Institute of Technology
88
Back-Up
© 2005 Jason Bartolomei, Engineering Systems Division, Massachusetts Institute of Technology
Qualitative Analysis:
MAV Stakeholder Network
System Drivers?
Contract Award
Operational Needs
Monies Available
Threat Landscape
Development Product
ASC
SOF PEO
Production Product
Money
USAF /63 Money
Information
3600 production)
3010 (production)
ARMY 62/63 Money
System
Drivers
ASC
SOF PM
Accumulator
USER
AFSOC/CC
USER:
ARMY
Exogenous System Drivers?
Motivations for
Capital Investment
Goal
Setting
USER
AFSOC/XP
USER
AFSOC/DO
USER
AFSOC/XP
R
USER
AFSOC/DO
X
Freq/Quality
of Comm
Budget $
Stock Price
KTR
Shareholder
Profit $
KTR
Business
Unit
Manager
R&D $
KTR
ARA PM
SPO
MAV PM
Profit $
Endogeneous System Drivers?
Budget Performance
Award Fee?
USER
AFSOC/XP
T
AFRL
MAV ATD
PM
USER
AFSOC/OS
S
USER
AFSOC/ST
S
AFRL
BAO PM
Path Length
Shorteners
Target
AFRL
XP
Comm.
Satellite
AOC
POC
AFRL
CC
USAF 61 Money
MAV
USER
AFSOC/CCT
Briefings and
Reports
© 2005 Jason Bartolomei, Engineering Systems Division, Massachusetts Institute of Technology
GPS
Satellite
Fighter/Bomb
er
Pilot
Static Analysis:
Stakeholder DSM to UCINET Network Analysis
Organizational Hot Spot
© 2005 Jason Bartolomei, Engineering Systems Division, Massachusetts Institute of Technology
Dynamic Analysis:
Hypothesis: Time scale nonlinearities create “Hot” spots
New stakeholders
Changing need
STAKEHOLDERS
SYSTEM BOUNDARY
OBJECTIVES
ATTRIBUTES
FUNCTIONAL DECOMPOSITION/ PROCESSES
OBJECTS/DETAILS
Technology Innovation
New payloads
ACTIVITIES
New processes
Supplier changes
AGENTS
Management turnover
Retirements
Funding Instability
Administration Change
Acquisitions regulation
© 2005 Jason Bartolomei, Engineering Systems Division, Massachusetts Institute of Technology
SYSTEM DRIVERS
Draft Hot spot calculation
Change Temperature = f(Likelihood of Change Occurrence, Readiness for Change, Node
Centrality, Number of Possible Change Modes, Cost to Change)
•
Likelihood of Change Occurrence:
–
–
•
Readiness for Change:
–
–
•
–
–
Is a measure of the contribution of network position to the importance, influence, prominence of an actor in a
network. Measure of an actor's potential for these things based on network position alone
Centralization refers to the extent to which a network revolves around a single node. More specifically,
measured as share of all centrality possessed by the most central node. In a star network, the central point
has complete centrality, and all other points have minimum centrality: the star is a maximally centralized
graph
Network Analysis
Number of Possible Change Modes
–
•
Measures the readiness to accept a change from a organizational, financial, technical, or supply-chain
perspective
Qualitative Analysis
Node Centrality: Definition from: http://www.analytictech.com/networks/centrali.htm
–
•
Measures how likely a node in the system will change or require change. Measure is at the intersection of
system drivers and internal nodes
MDO Analysis, Dynamic Analysis
MDO, Qualitative Analysis
Cost of Change
–
Financial Analysis
© 2005 Jason Bartolomei, Engineering Systems Division, Massachusetts Institute of Technology
Qualitative Search for “Hot/Cold” Spots
STAKEHOLDERS
What would be affected is a
stakeholder was removed?
SYSTEM BOUNDARY
OBJECTIVES
Which stakeholder objectives are
most likely to change over time?
ATTRIBUTES/SYSTEM PARAMETERS
FUNCTIONAL DECOMPOSITION/ PROCESSES
OBJECTS/DETAILS
Which objects technologies are
advancing rapidly/slowly?
ACTIVITIES
AGENTS
How is my organization expected to change over
time? Who is up for promotion? Who is ready to
retire/leave? What is the effect?
SYSTEM DRIVERS
Working Definitions:
- Nodes in an engineering system are “Hot” if future states are uncertain (change is likely) and changes
have significant upside or downside consequences
- Nodes are “Cold” if the are unlikely to change and likely changes have insignificant consequences on
system
© 2005 Jason Bartolomei, Engineering Systems Division, Massachusetts Institute of Technology