Transcript Engineering Optimization

Engineering Optimization
Concepts and Applications
Fred van Keulen
Matthijs Langelaar
CLA H21.1
[email protected]
Engineering Optimization – Concepts and Applications
Covered so far …
1. Introduction:
2. Optimization problem:
–
Negative null form
–
Definition
–
Applications
–
Characteristics
3. Problem checking:
4. Optimization model:
–
Boundedness
–
Model simplification
–
Monotonicity analysis
–
Approximation
Engineering Optimization – Concepts and Applications
Upcoming topics
Optimization problem
Negative null form
Special topics
Model
Linear / convex problems
Definition
Sensitivity analysis
Checking
Topology optimization
Solution methods
Unconstrained problems
Constrained problems
Optimality criteria
Optimality criteria
Optimization algorithms
Optimization algorithms
Engineering Optimization – Concepts and Applications
Contents
● Unconstrained problems
– Transformation methods
– Existence of solutions, optimality conditions
– Nature of stationary points
– Global optimality
Engineering Optimization – Concepts and Applications
Unconstrained Optimization
● Why?
– Elimination of active constraints  unconstrained
problem
– Develop basic understanding useful for constrained
optimization
– Transformation of constrained problems into
unconstrained problems
– Relevant engineering problems
(potential energy minimization)
Engineering Optimization – Concepts and Applications
Transforming constrained problem
● Reformulation through barrier functions:
f  x
g
g  x2 1  0
Transformation:
~
1
f  f  ln(  g )
r
~
1
f   x  lnx 2  1
r
Engineering Optimization – Concepts and Applications
f
x
Transformed problem
~~
f~f(r(r1)32)
f (r  10)
g
f
Engineering Optimization – Concepts and Applications
Transformed problem
● Barrier functions result in feasible, interior optimum:
r = 100
r = 200
r = 400
r = 800
Engineering Optimization – Concepts and Applications
Penalization
● Alternative reformulation: penalty functions
~ ~~
f (ffp(p 0.412)))
f  x
~
f ( p  10)
g
g  x2 1  0
Transformation:
~
2
f  f  pmax(0, g )


2
~
2
f   x  p max(0, x 1)
Engineering Optimization – Concepts and Applications
f
Penalization (2)
● Penalty functions result in infeasible, exerior optimum:
p = 40
p = 20
p = 10
p=4
Engineering Optimization – Concepts and Applications
Problem transformation summary
Barrier function
Penalty function
Yes
No
Nature of optimum
Interior
(feasible)
Exterior
(infeasible)
Type of constraints
g
g, h
Need feasible
starting point?
Engineering Optimization – Concepts and Applications
Unconstrained Optimization
● Why?
– Elimination of active constraints  unconstrained
problem
– Develop basic understanding useful for constrained
optimization
– Transformation of constrained problems into
unconstrained problems
– Relevant engineering problems
(potential energy minimization)
Engineering Optimization – Concepts and Applications
Unconstrained engineering problem
● Example: displacement of loaded structure
10 cm
k1 = 8N/cm
Fy = 5N
Fx = 5N
10 cm
x1
x2
k2 = 1N/cm
● Equilibrium: minimum potential energy
Engineering Optimization – Concepts and Applications
Unconstrained engineering problem
● Potential energy:
k1
1
1
2
2
  k1u1  k 2u2  Fx x1  Fy x2
2
2
 4 x1  (10  x2 )  10


2
2
 0.5 x1  (10  x2 ) 2  10


 5 x1  5 x2
● Equilibrium:
min 
x1 , x 2
Engineering Optimization – Concepts and Applications
Fx
x2
2
2
x1
2
Fy
k2
Unconstrained engineering problem
x2

x2
x1
Engineering Optimization – Concepts and Applications
x1
Contents
● Unconstrained problems
– Transformation methods
– Existence of solutions, optimality conditions
– Nature of stationary points
– Global optimality
Engineering Optimization – Concepts and Applications
Theory for solving unc. problems
● Assumptions:
– Objective continuous and differentiable (C1)
– Domain closed and bounded (compact)

f
]a, b[
Engineering Optimization – Concepts and Applications
Existence of minima
● Weierstrass Theorem:
“A continuous function on a compact set has
a maximum and a minimum in that set”
● Sufficient condition for existence!
● Interior optima only exist for non-monotonic functions
Engineering Optimization – Concepts and Applications
One-dimensional case
● Calculus: conditions for minimum of f
– Derivative zero:
f ' 0
(necessary)
– Second derivative positive:
f " 0
(sufficient)
● Interpretation through Taylor series:
f
f ( x  h)  f  f ' h   (h2 )
-h
f ( x  h)  f  f ' h   (h2 )  0
f ( x  h)  f   f ' h   (h2 )  0
Engineering Optimization – Concepts and Applications
+h
x
 f ' 0
One-dimensional case (2)
● Condition for minimum: f”(x) > 0
1
12
f ( x  h)  f  f ' hf "h f " h(h2 3) (0h 3 )  0
2
2
 f " 0
1
12
f ( x  h)  f  f ' hf "h f " h( h2 3) (0h 3 )  0
2
2
● Other possibilities:
f”(x) < 0
maximum
f”(x) = 0
? Check higher order derivatives
Engineering Optimization – Concepts and Applications
Geometrical interpretation
● Positive
f "  f ' locally increasing
f
f ' 0
f ' 0
f ' 0
x
Engineering Optimization – Concepts and Applications
One-dimensional case (3)
● Possible situations for stationary points (f’ = 0):
f
f " 0
f " 0
f " 0
x
Engineering Optimization – Concepts and Applications
Example
● Aspirin pill revisited: worst pill ever!
– Maximize dissolving time  minimize surface area
h
r

min 2r 2  2rh
r ,h
s.t. r 2 h  1
● Equality constraint active  eliminate h
1
h 2
r

Engineering Optimization – Concepts and Applications
2
min 2r 
r
r
2
Example (2)
2
f  2r 
r
2
2
f '  4r  2  0
r
4
f "  4  3
r
 0 r  0
hD
r  2 

Engineering Optimization – Concepts and Applications
1
3
 0.542
2 

h

2
3
 2r
Multidimensional case
● Local approximation to multidimensional minimum by
multidimensional Taylor series:
 f 
f ( x  h )  f  f h    h 
 x 
 1
 f 
Gradient f   x 
2
  
T
 f 
f ( x  h )  f  f h    h   0


T
 xn 
f (x  h)  f  f h    h   0
T
Condition for minimum: f  0
Engineering Optimization – Concepts and Applications
Multidimensional case (2)
● For minimum, consider second-order approximation:
 
1 T
f (x  h)  f  f h  h Hh   h
2
T
 2 f

2

x
 21
  f
Hessian H   x x
 2 1
 
 2 f
 x x
 n 1
Engineering Optimization – Concepts and Applications
2 f
x1x2
2 f
2
x2
2 f
xn x2
2
2 f 


x1xn 
2 f 
x2 xn 


 
2 f 

2 
xn 
Multidimensional case (3)
● Second order approximation:
1
2
f (x  y )  f  y T Hy   y  0
2
1
2
f (x  z )  f  zT Hz   z  0
2
1
2
f (x  q)  f  qT Hq   q  0
2
 
 
 
yT Hy  0 y  0
● Local minimum:
– First Order Necessity Condition:
f  0
– Second Order Sufficiency Condition:
y Hy  0 y
Engineering Optimization – Concepts and Applications
T
Positive definite
●
yT Hy  0 y  0

Hessian positive definite
● Test for positive definiteness:
– Evaluate yTHy for all
– All eigenvalues li of
y (impractical)
H positive
– Sylvester’s rule: all determinants
of H and its principal submatrices
are positive
Engineering Optimization – Concepts and Applications
Example
uy
k1
● Another loaded structure:
1
1
2
2
  k1u x  k 2u y  Fx u x  Fy u y  f
2
2
● Equilibrium:
min f
x1 , x2
● First order necessity condition:
 f 
 Fx 

u x  
 u x 
  k1u x  Fx 
 k1 

f  

0    
f  k 2u y  Fy 
Fy
u
y

   

 


 k2 

 u y 

Engineering Optimization – Concepts and Applications
Fy
Fx
k2
ux
Example (2)
● Second order sufficiency: H positive definite?
1
1
2
2
f  k1u x  k 2u y  Fx u x  Fy u y
2
2
 2 f

2
u x

H
 2 f
 u u
 y x
2 f 

u x u y  k1

2
 f  0
2 
u y 
0

k2 

k1  0

k 2  0
● Note: H diagonal constant matrix.
Separable objective function: f  f1 (ux )  f 2 (u y )
Engineering Optimization – Concepts and Applications
Quadratic functions
● Polynomial terms up to 2nd order:
f ( x1, x2 , x3 )  3  4x2  2x1x2  x  x3
2
1
2
● General form:
1 T
f (x)  x Ax  bT x  c
2
T
T
 x1   2 4 0  x1  0  x1 
1  
     

 x 2   0 0 0   x2    4   x 2   3
2 
 x  0  x 


x
0
0
2
 3 
 3     3 
● Note: 2nd order Taylor series is exact
Engineering Optimization – Concepts and Applications
Quadratic functions (2)
1 T
f (x)  x Ax  bT x  c
2
● Optimality conditions:
– Gradient:




1 T
f  A  A x  b  0
2
– Hessian:
● Stationary points:
1 T
H A A
2


1
x  2 A  A b
Engineering Optimization – Concepts and Applications
T
Example
T
1  x1 
● Consider f ( x)   
2  x2 
3 1   x1   1  x1 
1 2  x    4   x   2

 2     2 
T
3 1  x1   1
 f  
    0

1 2  x2   4 
1
3 1   1  1.2 
 x*   
 
 Stationary point

1 2  4   2.6
3 1 3  0
H


1 2  H  6  1  0
Engineering Optimization – Concepts and Applications
Hessian positive definite
Example (2)
● Result:
minimum
at (1.2, -2.6):
f
x2
x1
Engineering Optimization – Concepts and Applications
Example: least squares
● Least squares fitting: unconstrained optimization problem



T ~
~
min L  ε ε  f  Ma f  Ma
a
1 T
~
~T ~
T
 f (a)  a 2M M a  2 f Ma  f f
2
● Stationary point:
T


f
1
x  2 A  A b
T


~
 a  2 4M M  2M f
1
T
T~
 M M M f

T

1
Engineering Optimization – Concepts and Applications
T
~
f
x
Contents
● Unconstrained problems
– Transformation methods
– Existence of solutions, optimality conditions
– Nature of stationary points
– Global optimality
Engineering Optimization – Concepts and Applications
Nature of stationary points
y M M  y
● Hessian H positive definite:
T
T
  My   My 
T
– Quadratic form
– Eigenvalues
● Local nature: minimum
Engineering Optimization – Concepts and Applications
y Hy  0
T
li  0
 My  0
2
Nature of stationary points (2)
● Hessian H negative definite:
– Quadratic form
– Eigenvalues
● Local nature: maximum
Engineering Optimization – Concepts and Applications
yT Hy  0
li  0
Nature of stationary points (3)
● Hessian H indefinite:
– Quadratic form
yT Hy  0
– Eigenvalues
● Local nature: saddle point
Engineering Optimization – Concepts and Applications
li  0
Nature of stationary points (4)
● Hessian H positive semi-definite:
– Quadratic form
– Eigenvalues
● Local nature: valley
Engineering Optimization – Concepts and Applications
yT Hy  0
li  0
H singular!
Nature of stationary points (5)
● Hessian H negative semi-definite:
– Quadratic form
– Eigenvalues
● Local nature: ridge
Engineering Optimization – Concepts and Applications
yT Hy  0
li  0
H singular!
Stationary point nature summary
yT Hy li
Definiteness H
Nature x*
0
Positive d.
Minimum
0
Positive semi-d.
Valley
0
Indefinite
Saddlepoint
0
Negative semi-d.
Ridge
0
Negative d.
Maximum
Engineering Optimization – Concepts and Applications
Structural example
● Compressed pin-jointed truss with rotational springs:
dz  l  l cos1 cos 2
1
1
2
2
  k11  k 2 2  Fdz
2
2
F
2
l
k1
F  6, l  2
2
Maximum (unstable)
k2
Minima (stable)
Saddles (unstable)
k1  10, k2  9.5
Engineering Optimization – Concepts and Applications
1
Contents
● Unconstrained problems
– Transformation methods
– Existence of solutions, optimality conditions
– Nature of stationary points
– Global optimality
Engineering Optimization – Concepts and Applications
Global optimality
● Optimality conditions for unconstrained problem:
– First order necessity:
f (x*)  0 (stationary point)
– Second order sufficiency:
H positive definite at x*
● Optimality conditions only valid locally:
 x * local minimum
● When can we be sure x* is a global minimum?
Engineering Optimization – Concepts and Applications
Convex functions
● Convex function: any line connecting any 2 points on
the graph lies above it (or on it):
● Equivalent: tangent lines/planes stay below the graph
● H positive (semi-)definite  f locally convex
(proof by Taylor approximation)
Engineering Optimization – Concepts and Applications
Convex domains
● Convex set:
“A set S is convex if for every two points x1, x2 in S, the
connecting line also lies completely inside S”
Engineering Optimization – Concepts and Applications
Convexity and global optimality
If:
●
Objective f = (strictly) convex function
●
Feasible domain = convex set
(Ok for unconstrained
optimization)
Stationary point = (unique) global minimum
● Special case: f, g, h all linear  linear optimization
programming
● More general class: convex optimization
Engineering Optimization – Concepts and Applications
Example
● Quadratic functions with A positive definite are strictly
convex:
T
1  x1 
f ( x)   
2  x2 
3 1   x1   1  x1 
1 2  x    4   x   2

 2     2 
 Stationary point
(1.2, -2.6) must be
unique global
optimum
T
f
x1
Engineering Optimization – Concepts and Applications
x2
Summary optimality conditions
● Conditions for local minimum of unconstrained problem:
– First Order Necessity Condition:
f  0
– Second Order Sufficiency Condition:
H positive definite
● For convex f in convex feasible domain:
condition for global minimum:
– Sufficiency Condition:
Engineering Optimization – Concepts and Applications
f  0