Transcript Calculation of Lagrange multipliers and their use for

Optimality conditions for
constrained local optima,
Lagrange multipliers and their
use for sensitivity of optimal
solutions
Constrained optimization
• Inequality constraints
minimize
f ( x)
such that
g j x  0
g2(x)
x2
g1(x)
Infeasible regions
Optimum
Decreasing f(x)
x1
Feasible region
Equality constraints
• We will develop the optimality conditions
for equality constraints and then
generalize them for inequality constraints
minimize
f ( x)
such that
h j x  0
• Give an example of an engineering
equality constraint.
Lagrangian and stationarity
• Lagrangian function
ne
L(x,  )  f (x)    j h j
j 1
where j are unknown Lagrange multipliers
• Stationary point conditions for equality
constraints:
ne
h j
L f

 j
0
xi xi j 1 xi
L
 hj  0
 j
Example
• Quadratic objective and constraint
Min
f  x12  10x22
s.t
100  (x12  x22 )  0
• Lagrangian L  x12  10x22   100  x12  x22 
• Stationarity conditions
L
 2 x1  2 x1  0
x1
L
 20 x2  2 x2  0
x2
• Two stationary points
L
 100  ( x12  x22 )  0

x1  0, x2  10,   10 (maximum)
x1  10, x2  0,   1(minimum)
Inequality constraints
• Inequality constraints require transformation to
equality constraints:
g j  x   t  0 j  1,
2
j
, ng
• This yields the following Lagrangian:
ng
L(x, t,  )  f (x)    j  g j  x   t 
j 1
• Why is the slack variable squared?
2
j
Karush-Kuhn-Tucker conditions
• Conditions for stationary points are then:
g j
L f

 j
0
xi xi
xi
L
 g j  t 2j  0
 j
L
 2 j t j  0
t j
• If inequality constraint is inactive (t ≠ 0) then
Lagrange multiplier = 0
• For minimum, non-negative multipliers
Convex problems
• Convex optimization problem has
– convex objective function
– convex feasible domain if
• All inequality constraints are convex
(or gj = convex)
• All equality constraints are linear
– only one optimum
• Karush-Kuhn-Tucker conditions necessary and
will also be sufficient for global minimum
• Why do the equality constraints have to be
linear?
Example extended to inequality
constraints
• Minimize quadratic objective in a ring
• Will use
Min
f  x12  10x22
s.t
ri2  x12  x22  ro2
ri  10, ro  20
Matlab’s fmincon
function f=quad2(x)
f=x(1)^2+10*x(2)^2;
function [c,ceq]=ring(x)
global ri ro
c(1)=ri^2-x(1)^2-x(2)^2;
c(2)=x(1)^2+x(2)^2-ro^2;
ceq=[];
x0=[1,10];ri=10.; ro=20;
[x,fval,exitflag,output,lambda]=fmincon(@quad2,x0,[],[],[],[],[
],[],@ring)
Output message
Warning: The default trust-region-reflective algorithm does
not solve problems with the constraints you have specified.
FMINCON will use the active-set algorithm instead.
Local minimum found that satisfies the constraints.
Optimization completed because the objective function is
non-decreasing in feasible directions, to within the default
value of the function tolerance, and constraints are satisfied
to within the default value of the constraint tolerance.
What assumption do you think Matlab makes in selec ting
the default value of the constraint tolerance?
Solution
x =10.0000 -0.0000
fval =100.0000
lambda =
lower: [2x1 double]
upper: [2x1 double]
eqlin: [0x1 double]
eqnonlin: [0x1 double]
ineqlin: [0x1 double]
ineqnonlin: [2x1 double]
lambda.ineqnonlin’=1.0000 0
Sensitivity of optimum solution to
problem parameters
Assuming problem fitness and constraints
depend on parameter p
minimize f ( x, p)
such that g j ( x, p)  0
The solution is
and the corresponding fitness value
Sensitivity of optimum solution to
problem parameters (contd.)
We would like to obtain derivatives of f* w.r.t. p
After manipulating governing equations we obtain
df * f
T g a


dp p
p
Lagrange multipliers called “shadow prices” because they
provide the price of imposing constraints
Why do we have ordinary derivative on the left side and
partial on the right side?
Example
• A simpler version of ring problem
Min
f  x12  10x22
s.t
g  p  (x12  x22 )  0
• For p=100 we found   1
• Here it is easy to see that solution is
x1* 
p , x2*  0, f *( p )  p, df * / dp  1
• Which agrees with
df * f
g


 0  1 1
dp p
p