Transcript Nonlinear Optimization
Nonlinear Optimization
Review of Derivatives Models with One Decision Variable Unconstrained Models with More Than One Decision Variable Models with Equality Constraints: Lagrange Multipliers Interpretation of Lagrange Multiplier Models Involving Inequality Constraints
Nonlinear Optimization
Review of 1
st
Derivatives
Notation: y =
f
(x), dy/dx =
f’
(x)
f
(x) = c
f
(x) = x n
f
(x) = x
f
(x) = x 5
f’
(x) = 0
f’
(x) = n*x (n-1)
f’
(x) = 1*x 0 = 1
f’
(x) = 5*x 4
f
(x) = 1/x 3
f
(x) = c*g(x)
f
(x) = 10*x 2
f
(x) = u(x)+v(x)
f
(x) = x 2 - 5x
f
(x) = x -3
f’
(x) = c*g’(x)
f’
(x) = 20*x
f’
(x) = u’(x)+v’(x)
f’
(x) = 2x - 5
f’
(x) = -3*x 4
Nonlinear Optimization
Review of 2
nd
Derivatives
Notation: y =
f
(x), d(
f’
(x))/dx = d 2 y/dx 2 =
f’’
(x)
f
(x) = -x 2
f
(x) = x -3
f’
(x) = -2x
f’
(x) = -3x -4
f’’
(x) = -2
f’’
(x) = 12x -5
Nonlinear Optimization
Nonlinear Optimization
Review of Derivatives Models with One Decision Variable Unconstrained Models with More Than One Decision Variable Models with Equality Constraints: Lagrange Multipliers Interpretation of Lagrange Multiplier Models Involving Inequality Constraints
Nonlinear Optimization
Models with One Decision Variable
Requires 1 st & 2 nd derivative tests
Nonlinear Optimization
1
st
& 2
nd
Derivative Tests
Rule 1 (Necessary Condition): df/dx = 0 Rule 2 (Sufficient Condition): d 2 f/dx 2 > 0 d 2 f/dx 2 < 0 Minimum Maximum
Nonlinear Optimization
Maximum Example
Rule 1:
f
(x) = y = -50 + 100x – 5x 2 dy/dx = 100 – 10x = 0, x = 10 Rule 2: d 2 y/dx 2 = -10 Therefore, since d 2 y/dx 2 < 0:
f
(x) has a Maximum at x=10
Nonlinear Optimization
Maximum Example – Graph Solution
500 450 400 350 300 250 200 150 100 50 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
x Nonlinear Optimization
Minimum Example
Rule 1:
f
(x) = y = x 2 – 6x + 9 dy/dx = 2x - 6 = 0, x = 3 Rule 2: d 2y /dx 2 = 2 Therefore, since d 2 y/dx 2 > 0:
f
(x) has a Minimum at x=3.
Nonlinear Optimization
Minimum Example – Graph Solution
400 350 300 250 200 150 100 50 0 1 4 7 3 10 13 16 19 22 25 28 31 34 37
x Nonlinear Optimization
Max & Min Example
Rule 1:
f
(x) = y = x 3 /3 – x 2 dy/dx =
f
’(x) = x 2 – 2x = 0; x = 0, 2 Rule 2: d 2 y/dx 2 =
f
’’(x) = 2x – 2 = 0 2(0) – 2 = -2,
f
’’(x=0) = -2 Therefore, d 2 y/dx 2 < 0: Maximum of
f
(x) at x=0 2(2) – 2 = 2,
f
’’(x=2) = 2 Therefore, d 2 y/dx 2 > 0: Minimum of
f
(x) at x=2
Nonlinear Optimization
Max & Min Example – Graph Solution
6 4 2 0 -2 1 -4 -6 -8 4 7 0 10 13 16 19 2 22 25 28 31 34 37
x Nonlinear Optimization
Example: Cubic Cost Function Resulting in Quadratic 1
st
Derivative
Rule 1:
f
(x) = C = 10x 3 – 200x 2 – 30x + 15,000 dC/dx =
f
’(x)= 30x 2 – 400x – 30 = 0 Quadratic Form: ax 2 + bx + c
x
b
[
b
^ 2 4
ac
]
x
400 2
a
[( 400 )^ 2 4 ( 30 )( 30 )] 2 ( 30 )
x
13 .
4 , .
07
Nonlinear Optimization
Rule 2: d 2y /dx 2 =
f
’’(x) = 60x – 400 60(13.4) – 400 = 404 > 0 Therefore, d 2 y/dx 2 > 0: Minimum of
f
(x) at x = 13.4
60(-.07) – 400 = -404.2 < 0 Therefore, d 2 y/dx 2 < 0: Maximum of
f
(x) at x = -.07
Nonlinear Optimization
Cubic Cost Function – Graph Solution
50000 45000 40000 35000 30000 25000 20000 15000 10000 5000 0 1 4
x = Units Produced Nonlinear Optimization
Economic Order Quantity – EOQ
Assumptions: Demand for a particular item is known and constant Reorder time (time from when the order is placed until the shipment arrives) is also known The order is filled all at once, i.e. when the shipment arrives, it arrives all at once and in the quantity requested Annual cost of carrying the item in inventory is proportional to the value of the items in inventory Ordering cost is fixed and constant, regardless of the size of the order
Nonlinear Optimization
Economic Order Quantity – EOQ
Variable Definitions: Let Q represent the optimal order quantity, or the EOQ C h represent the annual carrying (or holding) cost per unit of inventory C o represent the fixed ordering costs per order D represent the number of units demanded annually
Nonlinear Optimization
Economic Order Quantity – EOQ
Note: If all the previous assumptions are satisfied, then the number of units in inventory would follow the pattern in the graph below:
EOQ Model Q Time Nonlinear Optimization
Economic Order Quantity – EOQ
At time = 0 after the initial delivery, the inventory level would be Q. The inventory level would then decline, following the straight line since demand is constant. When the inventory just reaches zero, the next delivery would occur (since delivery time is known and constant) and the inventory would instantaneously return to Q. This pattern would repeat throughout the year.
Nonlinear Optimization
Economic Order Quantity – EOQ
Under these assumptions: Average Inventory Level = Q/2 Annual Carrying (or Holding) Cost = (Q/2)*C h The annual ordering cost would be the number of orders times the ordering cost: (D/Q)* C o Total Annual Cost = TC = (Q/2)*C h +(D/Q)* C o
Nonlinear Optimization
Economic Order Quantity – EOQ
To find the Optimal Order Quantity, Q take the first derivative of TC with respect to Q: (dTC/dQ) = (C h /2) – DC o Q -2 = 0 Solving this for Q, we find: Q * = (2DC o /C h )^(1/2) Which is the Optimal Order Quaintly Checking the second-order conditions (Rule 2 in our text), we have: (d 2 TC/dQ 2 )= (2DC o /Q 3) Which is always > 0, since all the quantities in the expression are positive. Therefore, Q * gives a minimum value for total cost (TC)
Nonlinear Optimization
Restricted Interval Problems
Step 1: Find all the points that satisfy rules 1 & 2. These are candidates for yielding the optimal solution to the problem.
Step 2: If the optimal solution is restricted to a specified interval, evaluate the function at the end points of the interval.
Step 3: Compare the values of the function at all the points found in steps 1 and 2. The largest of these is the global maximum solution; the smallest is the global minimum solution.
Nonlinear Optimization
Nonlinear Optimization
Review of Derivatives Models with One Decision Variable Unconstrained Models with More Than One Decision Variable Models with Equality Constraints: Lagrange Multipliers Interpretation of Lagrange Multiplier Models Involving Inequality Constraints
Nonlinear Optimization
Unconstrained Models with More Than One Decision Variable
Requires partial derivatives
Nonlinear Optimization
Example Partial Derivatives
If z = 3x 2 y 3
∂
z/
∂
x = 6xy 3
∂
z/
∂
y = 9y 2 x 2 If z = 5x 3 – 3x 2 y 2 + 7y 5
∂
z/
∂
x = 15x 2 – 6xy 2
∂
z/
∂
y = -6x 2 y + 35y 4
Nonlinear Optimization
2
nd
Partial Derivatives
2 nd Partials (
∂
/
∂
x) (
∂
z/
∂
x)
= ∂
2 z/
∂
x 2 (
∂
/
d
y) (
∂
z/
∂
y)
= ∂
2 z/
∂y
2 Mixed Partials (
∂
/
∂
x) (
∂
z/
∂
y)
= ∂
2 z/(
∂
x
∂y)
(
∂
/
∂
y) (
∂
z/
∂
x)
= ∂
2 z/(
∂y∂
x)
Nonlinear Optimization
Example 2
nd
Partial Derivatives
If z = 7x 3 + 9xy 2 + 2y 5
∂
z/
∂
x = 21x 2 + 9y 2
∂
z/
∂
y = 18xy + 10y 4
∂
2 z/(
∂
y
∂x) =
18y
∂
2 z/(
∂x∂
y) = 18y
∂
2 z/
∂
x 2 = 42x
∂
2 z/
∂y
2 = 40y 3
Nonlinear Optimization
Partial Derivative Tests
Rule 3 (Necessary Condition):
∂
f/
∂
x 1 = 0,
∂
f/
∂
x 2 = 0, Solve Simultaneously Rule 4 (Sufficient Condition): If
∂
2 f/
∂
x 1 2 > 0 And (
∂
2 f/
∂
x 1 2 )*(
∂
2 f/
∂
x 2 2 ) – (
∂
2 f/(
∂
x 1
∂
x 2) ) 2 > 0 Then Minimum If
∂
2 f/
∂
x 1 2 < 0 And (
∂
2 f/
∂
x 1 2 )*(
∂
2 f/
∂
x 2 2 ) – (
∂
2 f/(
∂
x 1
∂
x 2) ) 2 > 0 Then Maximum
Nonlinear Optimization
Partial Derivative Tests
Rule 4, continued: If (
∂
2 f/
∂
x 1 2 )*(
∂
2 f/
∂
x 2 2 ) – (
∂
2 f/(
∂
x 1
∂
x 2) ) 2 < 0 Then Saddle Point If (
∂
2 f/
∂
x 1 2 )*(
∂
2 f/
∂
x 2 2 ) – (
∂
2 f/(
∂
x 1
∂
x 2) ) 2 = 0 Then no conclusion
Nonlinear Optimization
Partial Derivative Tests
Rule 5 (Necessary Condition): All
n
partial derivatives of an unconstrained function of
n
variables,
f
(x 1 , x 2 , …, x
n
), must equal zero at any local maximum or any local minimum point.
Nonlinear Optimization
Nonlinear Optimization
Review of Derivatives Models with One Decision Variable Unconstrained Models with More Than One Decision Variable Models with Equality Constraints: Lagrange Multipliers Interpretation of Lagrange Multiplier Models Involving Inequality Constraints
Nonlinear Optimization
Lagrange Multipliers
Nonlinear Optimization with an equality constraint Max or Min
f
(x 1 , x 2 ) ST:
g
(x 1 , x 2 ) = b Form the Lagrangian Function: L =
f
(x 1 , x 2 ) + λ[
g
(x 1 , x 2 ) – b]
Nonlinear Optimization
Lagrange Multipliers
Rule 6 (Necessary Condition): Optimization of an equality constrained function, 1 st order conditions:
∂
L/
∂
x 1
∂
L/
∂
x 2 = 0 = 0
∂
L/
∂
λ = 0
Nonlinear Optimization
Lagrange Multipliers
Rule 7 (Sufficient Condition): If rule 6 is satisfied at a point (x * 1 , x function with λ fixed at a value of λ * * 2 , λ * ) apply conditions (a) and (b) of rule 4 to the Lagrangian to determine if the point (x * 1 , x * 2 ) is a local maximum or a local minimum.
Nonlinear Optimization
Lagrange Multipliers
Rule 8 (Necessary Condition): For the function of subject to
m n
variables,
f
(x 1 , x 2 , …, x
n
), constraints to have a local maximum or a local minimum at a point, the partial derivatives of the Langrangian function with respect to x 1 , x 2 , …, x
n
and λ 1 , λ 2 , …, λ
m
must all equal zero at that point.
Nonlinear Optimization
Interpretation of Lagrange Multipliers
The value of the Lagrange multiplier associated with the general model above is the negative of the rate of change of the objective function with respect to a change in
b
. More formally, it is negative of the partial derivative of
f
(x 1 , x 2 ) with respect to b; that is, λ = -
∂f
/
∂b
or
∂f
/
∂b
= - λ
Nonlinear Optimization
Nonlinear Optimization
Review of Derivatives Models with One Decision Variable Unconstrained Models with More Than One Decision Variable Models with Equality Constraints: Lagrange Multipliers Interpretation of Lagrange Multiplier Models Involving Inequality Constraints
Nonlinear Optimization
Models Involving Inequality Constraints Step 1: Assume the constraint is not binding, and apply the procedures of “Unconstrained Models with More Than One Decision Variable” to find the global maximum of the function, if it exists. (Functions that go to infinity do not have a global maximum). If this global maximum satisfies the constraint, stop. This is the global maximum for the inequality-constrained problem. If not, the constraint may be binding at the optimum. Record the value of any local maximum that satisfies the inequality constraint, and go on to Step 2.
Step 2: Assume the constraint is binding, and apply the procedures of “Models with Equality Constraints” to find all the local maxima of the resulting equality-constrained problem. Compare these values with any feasible local maxima found in Step 1. The largest of these is the global maximum.
Nonlinear Optimization