Transcript Optimization of Multivariate Functions

Constrained Optimization
Chapter 16
16.2 Constraints
Consider a function z=f(x,y).
The two arguments x and y were considered to be independent before.
Now suppose there is a functional relationship between the two.
Example. Let A be apples, O be oranges. Then U  A  O  is a utility function for
someone consuming apples and oranges. This someone wants to maximize his/her
utility.
However, this consumer has to stay within his budget. Assuming the price of apples is p A
and the price of oranges is p O , total expenditure on apples and oranges may not exceed
consumer’s income I. This condition is called budget constraint:
p A A  pO O  I
Assuming the consumer is rational, meaning he is spending all of his money, the
constraint becomes p A  p O  I
A
O
Constrained Maximization Problem
This is how we formalize consumer’s attempts to maximize his utility under the constraint
that his total expenditures on apples and oranges may not exceed his income:
 MaxU  A , O 

 s .t
 p A  p B  I
A
B

Maximize the objective function
subject to (s.t.)
the constraint
Graphical Illustration
We are maximizing z=f(x,y).
Variable y is assumed to be dependent on x as
y=100-0.5x
Maximization of z=f(x,y) boils down to choosing
a point (a few points) on the line CF such that z
satisfies the definition of a local maximum.
Objective Function
Definition. The function that is being maximized or minimized is called the objective
function.
It is easy to think of objective functions as hills whose tops and troughs are maxima and
minima, respectively.
This function has an absolute maximum at R.
At R it also has a local maximum, check the
definition.
Constrained Maximum
The constraint is given by line CJTF.
This constraint defines an iso-section JPT for
which we are finding a maximum in the same
way as we find a maximum of a function of
one variable.
The iso-section JPT reaches its maximum at
point P.
The constrained maximum z* is lower than
the unconstrained maximum at R.
Solution by Implicit Differentiation
Line FC represents the original constraint.
Plane CDEF contains point P on the function’s
surface—this is the constrained maximum.
Condition 1. At point P the iso-z contour must
have the same slope as the constraint.
Condition 2. At the projection of point P, i.e.
point A, the constraint itself must be satisfied.
Recipe:
1. Find the slope of an iso-z contour at an arbitrary point (x,y):
this slope will be the first-order derivative of an implicit
function y=y(x)
2. Plug in the slope function into the original constraint.
Example
Suppose we are maximizing function z=f(x,y) subject to
the linear constraint y=100-0.5x
The iso-z sections are described by z-f(x,y)=0 where z is
any real number.
The slope of an iso-z section is given by implicit
differentiation:
dy
dx
 
fx
fy
The slope of an iso-z section at the point P of constrained maximum must be equal to
the slope of the constraint:

fx
fy
  0 .5
From this equation we get y as a
function of x.
We then plug this function into y=100-0.5x to obtain x at point P.
Maxima and Minima
The conditions to distinguish maxima and
minima exist, but we do not consider them.
If a function is concave or convex, it is clear
which kind of extremum we have found.
Example 16.1
Consider the following maximization problem:  x 2  y 2  Max

 y  10  x
Find the slope of an iso-z contour:
dy
 
dx
fx
fy
 
2x
2y
 
x
y
The slope of this iso-z contour must be equal to
the slope of the constraint -1:
dy
dx
 
x
 1  y  x, x  0
y
Plugging y=x into the constraint, we obtain:
y=10-y implying y=x=5 at point P.
Direct Substitution
Plug in the constraint into the objective function, and do the unconstrained optimization.
Example:
 x 2  y 2  Max

 y  10  x
Replace y in the objective function with the y from the constraint:
z  x  10  x   Max
2
2
The first-order conditions imply
dz
dx
 2 x  2 10  x   4 x  20  0  x  5
Plug in the x=5 into the constraint y=10-x to obtain y=5.
We arrive at the same result.
Why Constrained Optimization?
• Direct substitution is generally impossible
when the constraint is an implicit function
• Direct substitution conceals the structure of
the problem, and the nature of its solution
16.5 Lagrange Multiplier Method
Consider the general form of an optimization problem:  Maxf  x , y 

 g x, y   0
Form a Lagrangian function corresponding to the above problem:
L  f x, y   g x, y 
The factor
 is called Lagrange multiplier.
We now find stationary points of this Lagrangian by looking at three FOCs:
 L
 fx  g x  0

x

 L
 f y  g y  0


y

 L
 g x, y   0

 
By solving three equations in three variables, we obtain the
optimum values of x, y and  .
The Lagrangian multiplier value
meaning.
 has an economic
The third FOC equation is just the constraint itself.
Example 16.2
 x 2  y 2  Max

 y  10  x
The FOC for this Lagrangian are:
The Lagrangian corresponding to this problem will be:
L  x  y   10  x  y 
2
2
 L
 fx  g x  2 x    0

x

 L
 f y  g y  2 y    0


y

 L
 g  x , y   10  x  y  0

 
It follows from the first two equations that x=y since 2 x    2 y .
Plugging y=x into the third FOC, i.e. the constraint itself, we obtain x=5 and y=5.
Again, the result is the same.
Example 16.3
Suppose a farmer has fencing of length 100 meters, and he wants to make a chicken run of
the maximum rectangular area. What is this area going to be?
 xy  y

 2 x  2 y  100
The Lagrangian corresponding to this problem will be:
Consider the first-order conditions:
L  xy    2 x  2 y  100

 L
 f x  g x  y  2  0

x

 L
 f y  g y  x  2  0


y

 L
 g  x , y   2 x  2 y  100  0

 
From the first two equations, x=y. Plugging y=x into the constraint one obtains 4x=100.
The solution is x=y=25.
The maximum area is going to be z=625.
Lagrange Method Does Not Result in
an Unconstrained Optimum
 xy  Max

x  y  2
Let’s solve it by direct substitution first: plug y=2-x into the objective
function, and find an unconstrained maximum.
xy  x  2  x   Max
FOC : 2  2 x  0  x  1
x  y 1 y 1
Now consider a Lagrangian: L  xy    x  y  2 
Lx  y    0

Ly  x    0

 L  x  y  2  0
It follows that x=y=1 with   1
is a stationary point of the
Lagrangian.
However, x=y=1 does not maximize the Lagrangian L=xy-1(x+y-2). Indeed, at x=y=1 the
value of this Lagrangian is 1.
L(2,2)=2>1, so at x=y=1 the Lagrangian is not maximized!
In fact, it has a saddle point at x=y=1.
Interpretation of Lagrange Multiplier
 xy  y

 2 x  2 y  101
This is the same fencing problem.
However, we now have an extra meter of fencing.
The three FOCs result in x=y as before, and 2x+2y=101, not 2x+2y=100.
Solving, we obtain x=y=25.25, and the maximum chicken run area becomes 637.5265.
The increase in the maximum area that became possible due to one extra meter is equal
to 12.5625.
This is approximately equal to the solution value of
  12 .
Rule. The solution value of  is approximately equal to the change in the value of the
objective function when constrained is “relaxed” by one unit.