Transcript Production Functions - Massachusetts Institute of Technology

Constrained Optimization


Objective of Presentation: To present the
basic conceptual methods used to optimize
design in real situations
Essential Reality: In practical situations,
the designers are constrained or limited by



physical realities
design standards
laws and regulations, etc.
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Constrained Optimization
Slide 1 of 29
Outline


Unconstrained Optimization (Review)
Constrained Optimization


Approach
Equality constraints
 Lagrangeans
 Shadow

prices
Inequality constraints
 Kuhn-Tucker
conditions
 Complementary
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
slackness
Richard de Neufville ©
Constrained Optimization
Slide 2 of 29
Unconstrained Optimization: Definitions



Optimization => Maximum of desired quantity,
or => Minimum of undesired quantity
Objective Function = Expression to be
optimized
= Z(X)
Decision Variables = Variables about which
we can make decisions
= X = (X1….Xn)
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Constrained Optimization
Slide 3 of 29
Unconstrained Optimization: Graph
D
B
F(X)
A


C
B and D are maxima
A, C and E are minima
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Constrained Optimization
E
X
Slide 4 of 29
Unconstrained Optimization: Conditions


By calculus: if F(X) continuous, analytic
Primary conditions for maxima and minima:
F(X) / Xi = 0
I
( symbol means: for all i)

Secondary conditions:
2F(X) / Xi2 < 0
= > Max
(B,D)
2F(X) / Xi2 > 0
= > Min
(A,C,E)
These define whether point of no change in Z is a
maximum or a minimum
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Constrained Optimization
Slide 5 of 29
Unconstrained Optimization: Example

Example: Housing insulation
Total Cost = Fuel cost + Insulation cost
x = Thickness of insulation
F(x) = K1 / x + K2x
Primary condition: F(x) / x = 0 = -K1 / x2 + K2
=> x* = {K1 / K2} 1/2
(starred quantities are optimal)
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Constrained Optimization
Slide 6 of 29
Unconstrained Optimization:
Graph of Solution to Example

If: K1 = 500 ; K2 = 24
Then: X* = 4.56
Optimizing Cost Example
600
Cost
500
400
Fuel
Insulation
Total
300
200
100
0
1
3
5
7
9
11
Inches of Insulation
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Constrained Optimization
Slide 7 of 29
Constrained Optimization: General


“Constrained Optimization” involves the
optimization of a process subject to
constraints
Constraints have two basic types


Equality Constraints -- some factors have to
equal constraints
Inequality Constraints -- some factors have to
be less less or greater than the constraints
(these are “upper” and “lower” bounds)
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Constrained Optimization
Slide 8 of 29
Constrained Optimization:
General Approach

To solve situations of increasing complexity,
(for example, those with
equality, inequality constraints) …
Transform more difficult situation into one we
know how to deal with
Note: this process introduces new variables!

Thus, transform



“constrained” optimization to “unconstrained”
optimization
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Constrained Optimization
Slide 9 of 29
Equality Constraints: Example



Example: Best use of budget
Maximize: Output = Z(X) = aox1a1x2a2
Subject to (s.t.):
Total costs = Budget = p1x1 + p2x2
Z(x)
Z*
X
Budget
Note: Z(X) / X  0 at optimum
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Constrained Optimization
Slide 10 of 29
Lagrangean Method: Approach


Transforms equality constraints into
unconstrained problem
Start with:
Opt: F(x)
s.t.: gj(x) = bj

=>
gj(x) - bj = 0
L = F(x) - j j[gj(x) - bj]
j = Lagrangean multipliers (lambdas) -- these
are unknown quantities for which we must solve
Get to:
Note: [gj(x) - bj] = 0 by definition, thus
optimum for F(x) = optimum for L
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Constrained Optimization
Slide 11 of 29
Lagrangean: Optimality Conditions



Since the new formulation is a single
equation, we can use formulas for
unconstrained optimization.
We set partial derivatives equal to zero
for all unknowns, the X and the 
Thus, to optimize L:
L / xi = 0
I
L / j = 0
J
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Constrained Optimization
Slide 12 of 29
Lagrangean: Example Formulation


Problem:
Opt: F(x) = 6x1x2
s.t.: g(x) = 3x1 + 4x2 = 18
Lagrangean:
L = 6x1x2 - (3x1 + 4x2 - 18)
Optimality Conditions:
L / x1 = 6x2 - 3 = 0
L / x2 = 6x1 - 4 = 0
L / i = 3x1 + 4x2 -18 = 0
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Constrained Optimization
Slide 13 of 29
Lagrangean: Graph for Example
Isoquants for F(X) = 20 and 40 and
Constraint
X (sub 2)
8.00
7.00
6.00
5.00
4.00
3.00
2.00
1.00
0.00
-1.00
-2.00
1
2
3
4
5
X (sub 1)
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Constrained Optimization
Slide 14 of 29
Lagrangean: Example Solution




Solving as unconstrained problem:
L / x1 = 6x2 - 3 = 0
L / x2 = 6x1 - 4 = 0
L / i = 3x1 + 4x2 -18 = 0
so that:  = 2x2 = 1.5x1 (first 2 equations)
=> x2 = 0.75x1
=> 3x1 + 3x1 - 18 = 0 (3rd equation)
x1* = 18/6 = 3 x2* = 18/8 = 2.25
* = 4.5
F(x)* = 40.5
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Constrained Optimization
Slide 15 of 29
Lagrangean: Graph for Solution
Isoquants for F(X) = 20 and 40 and
Constraint
X (sub 2)
8.00
7.00
6.00
5.00
4.00

3.00
2.00
1.00
0.00
-1.00
-2.00
1

2
3
4
5
X (sub 1)
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Constrained Optimization
Slide 16 of 29
Shadow Prices



Shadow Price is the Rate of change of
objective function per unit change of
constraint
= F(x) / bj
It is extremely important for system design
It defines value of changing constraints, and
indicates if worthwhile to change them


Should we buy more resources?
Should we change environmental constraints?
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Constrained Optimization
Slide 17 of 29
Lagrangean Multiplier is a
Shadow Price


The Lagrangean multiplier is interpreted
as the shadow price on constraint
SPj = F(x)*/ bj = L*/ bj
=  {F(x) - j j[gj(x) - bj] } / bj
= j

Naturally, this is an instantaneous rate
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Constrained Optimization
Slide 18 of 29
Lagrangean = Shadow Prices
Example



Let’s see how this works in example, by
changing constraint by 0.1 units:
Opt: F(x) = 6x1x2
s.t.:
g(x) = 3x1 + 4x2 = 18.1
The optimum values of the variables are
x1* = (18.1)/6 x2* = (18.1)/8
* = 4.5
Thus F(x)* = 6(18.1/6)(18.1/8) = 40.95
F(x) = 40.95 - 40.5 = 0.45 = * (0.1)
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Constrained Optimization
Slide 19 of 29
Inequality Constraints: Example

Example: Housing insulation
Min: Costs = K1 / x + K2x
s.t.: x 8 (minimum thickness)
Optimizing Cost Example
600
Cost
500
400
Fuel
Insulation
Total
300
200
100
0
1
3
5
7
9
11
Inches of Insulation
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Constrained Optimization
Slide 20 of 29
Inequality Constraints: Approach



Transform inequalities into equalities,
then proceed as before
Again, this introduces new variables – in
this case, the “Slack” variables that
define “slack” or distance between
constraint and amount used
The resulting equations are known as the
“Kuhn-Tucker conditions”
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Constrained Optimization
Slide 21 of 29
Inequality Constraints -- putting
slack variables in Lagrangean




A “slack variable”, sj , for each inequality
gj(x)  bj => gj(x) + sj2 = bj
gj(x)  bj => gj(x) - sj2 = bj
These are “squared” to be positive
Start from:
opt: F(x)
s.t.: gj(x)  bj
Get to:
L = F(x) - jj[gj(x) + sj2 - bj]
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Constrained Optimization
Slide 22 of 29
Inequality Constraints: Complementary
Slackness Conditions


The optimality conditions are:
L / xi = 0
L / j = 0
plus: L / sj = 2sjj = 0
These new equations imply:
sj = 0
j  0
Or:
sj  0
j  0
These are “complementary slackness”
conditions.
Either slack or lambda (or both) = 0 i
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Constrained Optimization
Slide 23 of 29
Interpretation of Complementary
Slackness Conditions



Interpretation:
If there is slack on bj, (j  0 )
(i.e. more than enough of it)
=> No value to objective function
to having more: j = F(x) / bj = 0
If j  0, then all available bj used
=> sj = 0
In this case, it would be worthwhile to
have more of this constraint available
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Constrained Optimization
Slide 24 of 29
Example: Solution



Min: Costs = K1 / x + K2x
s.t.: x 8 (minimum thickness)
Lagrangean: L = K1/x + K2x - [x - s2 - b]
= 500/x + 24x - [x - s2 - 8]
Optimality Conditions:




L / xi = 500/x2 + 24 -  0
L / sj = 2s = 0
L / j = 0 ==> x - s2 = 8
If s = 0, x = 8,  = 31.8 (at that point)
Min cost = 254.5
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Constrained Optimization
Slide 25 of 29
Example: Interpretation of Slack


Since  = 31.8 > 0
Therefore, worth changing constraint to get
better solution


In this case, the constraint is on minimum, so
easing or relaxing it means that we would
lower the constraint, setting a lower
requirement on thickness of insulation
Unconstrained solution for lifetime cost is:
x* = 4.56 Optimum = 221 < 254.5
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Constrained Optimization
Slide 26 of 29
Slack Variable Example: Graph


The minimum cost with 8 inch constraint
on thickness = 254.5
If the constraint were less, optimum could
be better (until unconstrained minimum)
Optimizing Cost Example
600
Cost
500
400
Fuel
Insulation
Total
300
200
100
0
1
3
5
7
9
11
Inches of Insulation
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Constrained Optimization
Slide 27 of 29
Another application to Example





Min: Costs = K1 / x + K2x
s.t.: x 4 (NEW MINIMUM)
Lagrangean = K1/x + K2x - [x + s2 - b]
= 500/x + 24x - [x + s2 - 4]
Optimality Conditions:
500/x2 + 24 -  02s = 0 x - s2 = 4
If  = 0, x = 4.56, slack, s2 = 1.56
Optimum = 221
As  = 0, not worth changing constraint
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Constrained Optimization
Slide 28 of 29
Summary of Presentation

Important mathematical approaches




Lagreangeans
Kuhn- Tucker Conditions
Important Concept: Shadow Prices
THESE ANALYSES GUIDE DESIGNERS
TO CHALLENGE CONSTRAINTS
Engineering Systems Analysis for Design
Massachusetts Institute of Technology
Richard de Neufville ©
Constrained Optimization
Slide 29 of 29