Transcript Engineering Optimization - Center of Mechanics, ETH Zurich

Engineering Optimization
Concepts and Applications
WB 1440
Fred van Keulen
Matthijs Langelaar
CLA H21.1
[email protected]
WB1440 Engineering Optimization – Concepts and Applications
Geometrical interpretation
● For single equality constraint: simple geometrical
interpretation of Lagrange optimality condition:
f
h

 0T
x
x
x2
Meaning:
f
h
h
f h
//
f
x x
Gradients parallel 
tangents parallel  h tangent to isolines
WB1440 Engineering Optimization – Concepts and Applications
x1
Summary
● First order optimality
x2
condition for equality
constrained problem:
f
h
h
– Zero reduced gradient:
f
1
f f h h

 0T
d s s d
– Equivalent: stationary
Lagrangian:
L  f (x)  λT h(x)
WB1440 Engineering Optimization – Concepts and Applications

 L
T

0
 x
 L
  0T
 λ
x1
Contents
● Constrained Optimization: Optimality Criteria
– Reduced gradient
– Lagrangian
– Sufficiency conditions
– Inequality constraints
– Karush-Kuhn-Tucker (KKT) conditions
– Interpretation of Lagrange multipliers
● Constrained Optimization: Algorithms
WB1440 Engineering Optimization – Concepts and Applications
Sufficiency?
● Until now, only stationary points considered. Does not
guarantee minimum!
● Lagrange
f h
f h
f
h
f
condition:
f  T h  0
maximum
h
h
f
minimum
WB1440 Engineering Optimization – Concepts and Applications
f h
h
no extremum
h
f
minimum
f
Constrained Hessian
● Sufficiency conditions follow from 2nd order Taylor
approximation
● Second order information required: constrained Hessian:
 2  2 f   2 f  s   s 
    
 2  
2
d
d  ds  d   d 
T
  2 f   s 

   
 sd   d 
T
 2 f
 2
 s
 s   f   2s 
     2 
 d   s  d 
with
T
T
T
2
2
2
2

 s
 h   h   h  s   s    h   s    h  s 
 2      2  
     
     2  
 s   d  ds  d   d   sd   d   s  d 
 d 
2
1
obtained by differentiation of the constrained gradient, and
T
second-order constraint perturbation:
  h 
  0
d  d 
WB1440 Engineering Optimization – Concepts and Applications
Sufficiency conditions
● Via 2nd order Taylor approximation, it follows that at a
minimum the following must hold:

 0T
d
and
d 
T
 2
d  0
2
d
(Constrained Hessian
positive definite)
● Lagrangian approach also yields:
d 
T
2
 2

L
T
d  x 
x
2
2
d
x
with
2
2L 2 f

h
T
 2 λ
2
x
x
x 2
Perturbations only in
tangent subspace of h!
WB1440 Engineering Optimization – Concepts and Applications
Summary
● Optimality conditions for equality constrained problem:
1. Necessary condition: stationary point when:
L  0
2. Sufficient condition: minimum when (1) and:
2

L
T
x 2 x  0
x
on tangent subspace.
WB1440 Engineering Optimization – Concepts and Applications
Example
min f (x)  x1  x2
2
x2
f
2
h
x1 , x2
s.t. h(x)  x1  2 x2  1  0
2
2

x1

 L(x)  x  x2   x  2x2 1
2
1
2
2
1
2
dL  2 x1 (1   ) 
dL
2
2


,

x

2
x

1
2 1
dx 2 x2 (1  2 )
d
1. Necessary condition: stationary point when L  0
T
x1  0,
x2  0,
x2   1/ 2,
x1  1,
WB1440 Engineering Optimization – Concepts and Applications
  1/ 2
  1
Contents
● Constrained Optimization: Optimality Criteria
– Reduced gradient
– Lagrangian
– Sufficiency conditions
– Inequality constraints
– Karush-Kuhn-Tucker (KKT) conditions
– Interpretation of Lagrange multipliers
● Constrained Optimization: Algorithms
WB1440 Engineering Optimization – Concepts and Applications
Inequality constrained problems
● Consider problem with only inequality constraints:
min
f ( x)
x  n
s. t.
g i ( x)  0
i  1 m
x
● At optimum, only active constraints matter:
min
f ( x)
x  n
s. t.
g j ( x)  0
j  1 k  m
x
● Optimality conditions similar to equality constrained
problem
WB1440 Engineering Optimization – Concepts and Applications
Inequality constraints
● First order optimality:
g0
1
f  g 
μ   
x  x 
f
T g
μ
 0T
x
x
T
L(x)  f (x)  μT g(x)

● Consider feasible local variation around optimum:
f
T g
μ
 0T
x
x

f
T g
x  μ
x  0
x
x
0
(boundary optimum)
WB1440 Engineering Optimization – Concepts and Applications
0
(feasible perturbation)
Optimality condition
● Multipliers must be non-negative:
f
T g
x  μ
x  0
x
x
 0
0
0
x2
g2
f
g1
k m
f    j g j  0
j 1
 f    j g j
g2 -f
g1
x1
● Interpretation: negative gradient (descent direction) -f
lies in cone spanned by positive constraint gradients
WB1440 Engineering Optimization – Concepts and Applications
Optimality condition (2)
f    j  g j 
● Feasible direction:
g  s  0 j  1k  m
  g  s  0
T
j
g2
x2
 g1
Feasible cone
g1
f  g
2
T
j
● Descent direction:
f  s  0
T
  f  s  0
g2 -f
T
g1
● Equivalent interpretation: no descent direction exists
within the cone of feasible directions
WB1440 Engineering Optimization – Concepts and Applications
x1
Examples
f    j g j  0
g2

g2
f
g2
1  0
2  0
f
-f
-f
g1
-f
f    j g j
g1
g1
1  0
2  0
WB1440 Engineering Optimization – Concepts and Applications
f
1  0
2  0
Optimality condition (3)
L (x)  f (x)  μT g(x)
● Active constraints:
Inactive constraints:
j  0
g j  0,
gi  0
● Formulation including all inequality constraints:
L(x)  f (x)  μT g(x)
L f
T g

μ
 0T
x x
x
and
WB1440 Engineering Optimization – Concepts and Applications
ii ggii  00
μi 0 0
Complementarity
i  1m
condition
i  1m
Example
min f  m g x22,
gh  L(1  cos x1 )  x2 
0
xx11,,xx22
x11
L
L
L( x1 , x2 )  f ( x1 , x2 )  g ( x1 , x2 )
x2
L( x1 , x2 )  m g x2   L(1  cos x1 )  x2 
m
L  L sin x1 
T


0

x  m g   
T

  mg
m gL sin x1  0

x1  0
x2  0
WB1440 Engineering Optimization – Concepts and Applications
m
x2
Mechanical application: contact
● Lagrange multipliers also used in:
– Contact in multibody dynamics
– Contact in finite elements
WB1440 Engineering Optimization – Concepts and Applications
Contents
● Constrained Optimization: Optimality Criteria
– Reduced gradient
– Lagrangian
– Sufficiency conditions
– Inequality constraints
– Karush-Kuhn-Tucker (KKT) conditions
– Interpretation of Lagrange multipliers
● Constrained Optimization: Algorithms
WB1440 Engineering Optimization – Concepts and Applications
Karush-Kuhn-Tucker conditions
● Combining Lagrange conditions for equality and
inequality constraints yields KKT conditions for general
problem: min
f ( x)
x
s. t.
g ( x)  0
h( x)  0
Lagrangian:
L  f (x)  μT g(x)  λT h(x)

g i
hi
L f

  i
  i
0
x x
x
x
(optimality)
and
g  0, h  0
(feasibility)
λ  0, μ  0, i gi  0
WB1440 Engineering Optimization – Concepts and Applications
(complementarity)
Sufficiency
● KKT conditions are necessary conditions for local
constrained minima
● For sufficiency, consider the sufficiency conditions
based on the active constraints:
x
T
2L
x  0
2
x
on tangent subspace of h and active g.
● Interpretation: objective and feasible domain
locally convex
WB1440 Engineering Optimization – Concepts and Applications
Additional remarks
● Global optimality:
– Globally convex objective function?
– And convex feasible domain?
Then KKT point gives global optimum
● Pitfall:
– Sign conventions for Lagrange multipliers in
KKT condition depend on standard form!
– Presented theory valid for negative null form
WB1440 Engineering Optimization – Concepts and Applications
Contents
● Constrained Optimization: Optimality Criteria
– Reduced gradient
– Lagrangian
– Sufficiency conditions
– Inequality constraints
– Karush-Kuhn-Tucker (KKT) conditions
– Interpretation of Lagrange multipliers
● Constrained Optimization: Algorithms
WB1440 Engineering Optimization – Concepts and Applications
Significance of multipliers
● Consider case where optimization problem depends on
parameter a:
min
f (x; a)
s. t.
h(x; a)  0
x
Lagrangian:
KKT:
L(x, λ; a)  f (x; a)  T h(x; a)
L f
T h


0
x x
x
T

df f f dx


da a x da
T
Looking for:
WB1440 Engineering Optimization – Concepts and Applications
f
h


x
x
T
Significance of multipliers (2)
df f f dx


da a x da
T
Looking for:
f
h


x
x
T
KKT:
df f h dx



da a x da
T
h dx h
h
h dx
h(x, a)  0 

0 

x da a
a
x da
T
df f
h


da a
a
WB1440 Engineering Optimization – Concepts and Applications
T
Significance of multipliers (3)
● Lagrange multipliers describe the sensitivity of the
objective to changes in the constraints:
df f
h


da a
a
● Similar equations can be derived for multiple
constraints and inequalities
● Multipliers give “price of raising the constraint”
● Note, this makes it logical that at an optimum,
multipliers of inequality constraints must be positive!
WB1440 Engineering Optimization – Concepts and Applications
Example
A, sy
Minimize mass (volume):
N
f ( A)  V  Al
Stress constraint:
l
N

L  Al     s y 
A

N

l   A2  0

 N  s  0
y
 A
N
g ( A; )   s y  0
A
A2 l
Nl
V



2
N
s y  s y
 A
WB1440 Engineering Optimization – Concepts and Applications
N
s y
,V
Nl
s y
Example (2)
Stress constraint:
N
g ( A; )   s y  0
A
V
Nl

s y
V
s y
Constraint sensitivity:
f
V V g


  g 
 s y   
V

   s y
Check:
V
Nl
s y

Nl 1
sy 
V
Nl 1
Nl 1
V




2

sy 
s y 

WB1440 Engineering Optimization – Concepts and Applications