Transcript Production Functions - Massachusetts Institute of Technology

FLAW OF AVERAGES
Richard de Neufville
Professor of Engineering Systems
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Flaw of Averages
Slide 1 of 29
Outline
• What is the concept?
• Why is it important in practice?
• When does it occur?
• How to avoid
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Flaw of Averages
Slide 2 of 29
THE CONCEPT
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Flaw of Averages
Slide 3 of 29
Flaw of Averages
• Presentation explains a fundamental problem
in the design and evaluation of systems
• This problem is the pattern of designing and
evaluating systems based on the “average” or
“most likely” future projections
• Problem derives from misunderstanding of
probability and systems behavior, known as
FLAW OF AVERAGES
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Flaw of Averages
Slide 4 of 29
Flaw of Averages

Named by Sam Savage (“Flaw of
Averages, Wiley, New York, 2009)
It is a pun. It integrates two concepts:
 A mistake => a “flaw”
 The concept of the “law of averages”,
that that things balance out “on average”

Flaw consists of assuming that design or
evaluation based on “average” or “most
likely” conditions give correct answers
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Flaw of Averages
Slide 5 of 29
Mathematics of Flaw




Jensen’s law:
E [ f(x) ] ≤ f [ E(x)] if f(x) is convex function
Notation: E(x) = arithmetic average, or
“expectation” of x
In words:


E[ f(x)] = average of possible outcomes of f(x)
f [ E(x)] = outcome calculated using average x
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Flaw of Averages
Slide 6 of 29
Example
Given:
And:




f(x) = √x + 2
x = 1, 4, or 7 with equal probability
E(x) = (1 + 4 + 7) / 3 = 4
f[E(x)] = √4 + 2 = 4
f(x) = 3 , 4, or [√7 + 2] ~ 4.65
with equal probability
E[f(x)] = (3 + 4 + 4.65) / 3 ~ 3.88 ≤ 4 = f[E(x)]
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Flaw of Averages
Slide 7 of 29
More generally…
E [ f(x) ]  f [ E(x)]
Example:
 Given:
f(x) = x2 + 2
 And:
x = 1, 2, or 3 with equal probability
 E(x) = (1 + 2 + 3) / 3 = 2
 = 4 + 2 = 6
 f(x) = 3 , 6, or 11 with equal probability
 E[f(x)] = (3 + 6 + 11) / 3 = 6⅔  6 = f[E(x)]
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Flaw of Averages
Slide 8 of 29
When equal?
E [ f(x) ] = f [ E(x)] when f(x) linear
This is rarely the case!
Example:
 Given:
f(x) = x + 2
 And:
x = 1, 2, or 3 with equal probability
 E(x) = (1 + 2 + 3) / 3 = 2
 f[E(x)] = 2 + 2 = 4
 f(x) = 3 , 4, or 5
with equal probability
 E[f(x)] = (3 + 4 + 5) / 3 = 4 = f[E(x)]
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Flaw of Averages
Slide 9 of 29
In Words



Average of all the possible outcomes
associated with uncertain parameters,
generally does not equal
the value obtained from using the average
value of the parameters
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Flaw of Averages
Slide 10 of 29
Practical Consequences
Because Engineering Systems not linear:
 Unless you work with distribution, you get
wrong answer
 answer from a realistic description differs –
often greatly – from the answer you get from
average or any single assumption

This is because gains when things do well,
do not balance losses when things do not
(sometimes they’re more, sometimes less)
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Flaw of Averages
Slide 11 of 29
WHY IMPORTANT
IN PRACTICE
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Flaw of Averages
Slide 12 of 29
Why does Flaw occur?
Flaw is a pattern in systems design. Why?
Several reasons:
 Management fixes design parameters, thus
limiting designers



Designers deliberately choose simplicity
Need, desire to focus on a single scenario,
given limited design resources
Professional desire for ‘certainty’
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Flaw of Averages
Slide 13 of 29
Management fixes parameters
Designers often constrained by management
or client to focus on one scenario



Management specifies price of product
(copper, oil) => opportunities that might be
valuable for a higher price are neglected
Client tells designers to work toward a
forecast (1 M customers for Iridium…)
Client specifies performance requirements
(a different way of defining forecast needs),
this is typical for military…
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Flaw of Averages
Slide 14 of 29
Designers choose simplicity
Designers often choose single scenario, even
when possible ranges are available

Example: design of oil platforms based on
“best estimate” ( = P50) of “oil in place”



Recall data on variability of estimates of oil
reserves (uncertainty presentation)
Geologists present ranges on their estimates (P10
to P90, for example). Uncertainty is clear
Yet design process focus on a single number!
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Flaw of Averages
Slide 15 of 29
Need, desire
to focus on 1 scenario
Detailed design of a system (automobile, oil
platform) requires great effort. Yet limited
time and resources available



Hard to create one design => desire not to
design many systems for different scenarios
Computer capabilities have changed, but
practice has not adapted (yet)
Desire to “optimize” => single scenario
(Example: design of Iridium satellite system)
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Flaw of Averages
Slide 16 of 29
Desire for certainty
Engineering practice often more comfortable
with certainties, precision


Tradition of scientific rigor, desire for
precision – not immediately compatible
with vagueness, uncertainty
A deep issue: designers have selected
engineering because it offers rigor, and
they feel uncomfortable with vagueness…
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Flaw of Averages
Slide 17 of 29
Practical Consequences
Organizational, personal resistance to
recognizing and dealing with uncertainty

Not current practice for



Much management of design work
Many client relationships
Uncomfortable personally for many
individuals
Although forecast uncertainty demonstrably
great, this reality is resisted
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Flaw of Averages
Slide 18 of 29
How issue arises in practice
• In practice, design rarely focuses specifically
on “average” future conditions
• If you do not recognize distribution of
uncertainly, you cannot calculate an average
• Focus typically on “most likely” scenario.
• Formally, not the same as “average”
• Conceptually equivalent however. Mental
model is Normal distribution around best
estimate, so “most likely” = “average”
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Flaw of Averages
Slide 19 of 29
REASONS FOR
SYSTEM NON-LINEARITY
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Flaw of Averages
Slide 20 of 29
3 Reasons for Non-Linearity



System response is non-linear
System response involves some
discontinuity (step change)
Management rationally imposes a
discontinuity
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Flaw of Averages
Slide 21 of 29
System Response is Non-Linear



Economies of Scale: Unit costs decrease
with scale of production
Large initial costs prorated over volume,
so that unit costs decrease as scale
increases toward capacity
Increasing marginal costs as scale
increases (labor, material costs higher)
Unit
Cost
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Scale
This is Usual
Situation!
Richard de Neufville ©
Flaw of Averages
Slide 22 of 29
System involves Discontinuity
Discontinuities = special form of non-linearity
Discontinuities are Common:
 Expansion of a Project might only occur in
large increments (new runways, for example)
 A System may be capacity constrained, so
that profitability or values increases with
demand up to a point, and then levels off
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Flaw of Averages
Slide 23 of 29
Management Creates Discontinuity




Managers or System Operators may decide
to take some major decision about a
project …
to enlarge it or change its function
this creates a step change in the
performance of the system.
This can happen often – and does!
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Flaw of Averages
Slide 24 of 29
A practical example
Design of oil platform and wells, Golf of Mexico
Reference: Babajide, A., de Neufville, R. and
Cardin, M.-A. (2009) “Integrated Method for
Designing Valuable Flexibility in Oil
Development Projects,” Paper 122710-PA,
Society of Petroleum Engineers, Projects,
Facilities and Construction, Vol.4, no. 2, June
2009
http://www.spe.org/ejournals/jsp
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Flaw of Averages
Slide 25 of 29
0.60
0.70
0.50
0.60
0.50
0.40
Probability
Probability
Gulf of Mexico Platform
Probability Mass Functions
0.30
0.20
0.40
0.30
0.20
0.10
0.10
0.00
0.00
80
150
200
100
150
220
Expected Ultimate Recov ery (MMBO)
Expected Ultimate Recovery (MMBO)
Sample Field PMF
Rother Field PMF
Note: “Most likely” scenarios are 150 and 100
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Flaw of Averages
Slide 26 of 29
Combined PMF
0 .4 0
0 .3 5
Probability
0 .3 0
0 .2 5
0 .2 0
0 .1 5
0 .1 0
0 .0 5
0 .0 0
180
230
250
300
350
370
420
Expect ed Ult imate Recovery (MMBO)
C ombined Sample and Rother O il Res erve P M F
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Flaw of Averages
Slide 27 of 29
Comparison of Values
1 .0 0
Cumulative Probability
0 .9 0
0 .8 0
0 .7 0
0 .6 0
0 .5 0
Based on
“most
likely”
estimates
Based on actual
distribution of
possibilities
0 .4 0
0 .3 0
0 .2 0
0 .1 0
0 .0 0
1 5 ,0 0 0 1 7 ,0 0 0 1 9 ,0 0 0 2 1 ,0 0 0 2 3 ,0 0 0 2 5 ,0 0 0 2 7 ,0 0 0 2 9 ,0 0 0 3 1 ,0 0 0 3 3 ,0 0 0
NPV ($, million)
I nflexible
E N P V - I nflex.
Bas ic E c on. M odel
Actual ENPV  Value based on Mostly Likely Conditions
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Flaw of Averages
Slide 28 of 29
Take-Aways




Do not be a victim of Flaw of Averages
Do not value projects or make design
decisions based on average or most likely
forecasts – your results will be WRONG
Do consider the range of possible events
and examine distribution of consequences
This will be hard – standard paradigm locks
on single, “best” estimates. Shift to new,
correct paradigm is difficult for many.
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Flaw of Averages
Slide 29 of 29