Transcript Calculation of Lagrange multipliers and their use for

Optimality conditions for
constrained local optima,
Lagrange multipliers and their
use for sensitivity of optimal
solutions
Constrained optimization
• Inequality constraints
m inim ize
f (x)
such that
g
j
x  0
g2(x)
x2
g1(x)
Infeasible regions
Optimum
Decreasing f(x)
x1
Feasible region
Equality constraints
• We will develop the optimality conditions
for equality constraints and then
generalize them for inequality constraints
m inim ize
such that
f (x)
h
j
x  0
• Give an example of an engineering
equality constraint.
Lagrangian and stationarity
• Lagrangian function
L (x,  )  f (x) 
ne

j 1
where j are unknown Lagrange multipliers
• Stationary point conditions for equality
constraints:
L
 xi
L
 j

f
 xi
h j
ne


j 1
 hj  0
j
 xi
0
j
h j (x)
Example
• Quadratic objective and constraint
• Lagrangian L  x  10 x
• Stationarity conditions
2
1
L
 x1
 2 x1  2  x1  0
L
x2
2
f  x 1  10 x 2
s .t
100  ( x 1  x 2 )  0
2
2
2
2
   100  x 1  x 2 
 20 x 2  2  x 2  0
• Four stationary points
2
M in
2
L

2
 100  ( x1  x 2 )  0
2
x1  0, x 2   10,   10 (f= 1000, m axim a)
x1   10, x 2  0,   1 (f= 100, m inim a)
2
Problem Lagrange multipliers
• Solve the problem of minimizing the
surface area of a cylinder of given value V.
The two design variables are the radius
and height. The equality constraint is the
volume constraint.
Inequality constraints
• Inequality constraints require transformation to
equality constraints:
g j  x   t  0 j  1,
2
j
, ng
• This yields the following Lagrangian:
ng
L (x, t,  )  f (x) 

j 1
j
g j x  t 


• Why is the slack variable squared?
2
j
Karush-Kuhn-Tucker conditions
• Conditions for stationary points are then:
L
 xi
L
 j
L
t j

f
 xi


g j
j
 xi
0
 g j  tj  0
2
 2 jt j  0
• If inequality constraint is inactive (t ≠ 0) then
Lagrange multiplier = 0
• For minimum, non-negative multipliers
Convex problems
• Convex optimization problem has
– convex objective function
– convex feasible domain if the line segment
connecting any two feasible points is entirely feasible.
• All inequality constraints are convex (or gj = convex)
• All equality constraints are linear
– only one optimum
• Karush-Kuhn-Tucker conditions necessary and
will also be sufficient for global minimum
• Why do the equality constraints have to be
linear?
Example extended to inequality
constraints
• Minimize quadratic objective in a ring
M in
s .t
2
2
f  x 1  10 x 2
2
2
1
2
2
2
o
ri  x  x  r
ri  10, ro  20
• Is feasible domain convex?
• Example solved with fmincon using two
functions: quad2 for the objective and ring
for constraints (see note page)
Message and solution
Warning: The default trust-regionreflective algorithm does not solve
…. FMINCON will use the activeset algorithm instead.
Local minimum found ….
Optimization completed because
the objective function is nondecreasing in feasible directions,
to within the default value of the
function tolerance, and
constraints are satisfied to within
the default value of the constraint
tolerance.
x =10.0000 -0.0000
fval =100.0000
lambda = lower: [2x1 double]
upper: [2x1 double]
eqlin: [0x1 double]
eqnonlin: [0x1 double]
ineqlin: [0x1 double]
ineqnonlin: [2x1 double]
lambda.ineqnonlin’=1.0000 0
What assumption Matlab likely
makes in selecting the default
value of the constraint tolerance?
Problem inequality
• Solve the problem of minimizing the
surface area of the cylinder subject to a
minimum value constraint as an inequality
constraint. Do also with Matlab by defining
non-dimensional radius and height using
the cubic root of the volume.
Sensitivity of optimum solution to
problem parameters
• Assuming problem objective and constraints
• depend on parameter p
m inim ize f ( x , p )
such that g j ( x , p )  0
• The optimum solution is x*(p)
• The corresponding function value f*(p)=f(x*(p),p)
Sensitivity of optimum solution to
problem parameters (contd.)
We would like to obtain derivatives of f* w.r.t. p
After manipulating governing equations we obtain
df *
dp

f
p

T
g a
p
Lagrange multipliers called “shadow prices” because they
provide the price of imposing constraints
Why do we have ordinary derivative on the left side and
partial on the right side?
Example
• A simpler version of ring problem
2
2
M in
f  x 1  10 x 2
s .t
g  p  ( x1  x 2 )  0
2
2
• For p=100 we found   1
• Here it is easy to see that solution is
x1 
*
p , x 2  0, f * ( p )  p , df * / dp  1
*
• Which agrees with
df *
dp

f
p

g
p
 0  11
Problems sensitivity of optima
• For f ( x , p )  sin x  px 0  x  2  find the
optimum for p=0, estimate the derivative df*/dp
there, and check by solving again for p=0.1 and
comparing to finite difference derivative
• Check in a similar way the derivative of the
surface area with respect to 1% change in
volume (once from the Lagrange multiplier, and
once from finite difference of exact solution).