Transcript Optimization in Matlab

Fin500J: Mathematical Foundations in Finance
Topic 5: Numerical Methods for Optimization
Philip H. Dybvig
Reference: Optimization Toolbox User’s Guide in Matlab, 2008 by the MathWorks, Inc.
Slides designed by Yajun Wang
Fin500J Topic 5
Fall 2010 Olin Business School
1
Recall, with optimization, we are seeking f '(x) = 0
f(x)
f '(x) = 0
f "(x)< 0
f '(x) = 0
f "(x)>0
Fin500J Topic 5
x
Fall 2010 Olin Business School
2
One Dimension Unconstrained Optimization
(Example)
Find the maximum of
x2
f ( x)  2 sin x 
10
To solve the root problem for
x
f ( x)  2 cos x   0
5
'
and the second condition is satisfied at the root
1
f ( x*)  2 sin x *   0
5
''
Fin500J Topic 5
Fall 2010 Olin Business School
3
One Dimension Unconstrained Optimization
(Example)
 We can solve f '(x) =0 by
2
Bisection using initial interval
[1,2], Newton’s with initial
point 1.2 or Secant method
with initial points 1.2 and 2
presented in Topic 3.
1
y=2sinx-x2/10
0
-1
-2
 We can also solve it in Matlab.
-3
-4
-5
-6
-4
Fin500J Topic 5
-2
0
x
2
4
6
Fall 2010 Olin Business School
>> f=@(x) 2*cos(x)-1/5*x;
>> fzero(f,[1,2])
ans =1.4276
4
Objectives : Using Optimization Toolbox in
Matlab to
 Solve unconstrained optimization with multiple variables
 Solve linear programming problem
 Solve quadratic programming problem (for example: optimal
portfolio)
 Solve nonlinear optimization with constraints
 Mostly, we will focus on minimization in this topic, max f(x) is
equivalent to min –f(x)
Fin500J Topic 5
Fall 2010 Olin Business School
5
Linear Programming/Quadratic
Programming/Nonlinear Programming
 If f(x) and the constraints are linear, we have linear
programming
 If f(x) is quadratic, and the constraints are linear, we have
quadratic programming
 If f(x) in not linear or quadratic, and/or the constraints are
nonlinear, we have nonlinear programming
Fin500J Topic 5
Fall 2010 Olin Business School
6
Recall the Optimality Conditions for Multiple
Variables
• Unconstrained Minimization Problem:
min f(x1, x2,…xn)
• Optimality Condition: x* is a local minimum if
f ( x* )  0 and  2 f(x* ) is positive definite.
• Example: min f(x)  e x1  x2 1  e x1  x2 1  e -x1 1 ,
 e x1  x2 1  e x1  x2 1  e - x1 1 
,
f ( x)   x  x 1
 e 1 2  e x1  x2 1



•What values of x make
f ( x)  0 ?
Fin500J Topic 5
Fall 2010 Olin Business School
7
Unconstrained Optimization with Multiple
Variables in Matlab
 Step 1: Write an M-file objfun.m and save under the work
path of matlab
function f=objfun(x)
f=exp(x(1)+x(2)-1)+exp(x(1)-x(2)-1)+exp(-x(1)-1);
 Step 2: >>optimtool in the commend window to open the
optimization toolbox
Fin500J Topic 5
Fall 2010 Olin Business School
8
Unconstrained Optimization with Multiple
Variables in Matlab (cont.)
 We use the
function fminunc
to solve
unconstrained
optimization
problem “objfun”
Fin500J Topic 5
Fall 2010 Olin Business School
9
Quasi-Newton Method is an Algorithm used
in function fminunc
f ' ( x k  x )  f ' ( x k )  f ' ( x k )
f (x ) 

x
x
f ' (xk )
 x   '' k . Multiple variablecase :
f (x )
''
k
x  - 2 f ( x k ) 1 f ( x k ).
Fin500J Topic 5
Fall 2010 Olin Business School
10
Recall the Algorithm of Newton Method
 Newton’s method may fail if Hessian is not positive
definite
Fin500J Topic 5
Fall 2010 Olin Business School
11
Quasi-Newton Methods Replace the Hessian with
some Positive Definite Matrix H
 The function “fminunc” uses BFGS (Broyden, Fletcher, Goldfarb and
Shanno) Hessian Update in the Quasi-Newton algorithm. The formula
given by BFGS is
Fin500J Topic 5
Fall 2010 Olin Business School
12
Linear Programming
 Both the objective function and the constraints are linear
 Example: maximizing profit or minimizing cost
 Objective function
Max or Min Z = c1x1 +c2x2 +…..cnxn
where cj = payoff of each unit of the jth activity
xj = magnitude of the jth activity
 The constraints can be represented by
ai1x1 +ai2x2+…..ainxn  bi
where aij = amount of the ith resource that is consumed for each
unit of the jth activity, bi = amount of the ith resource available
 Finally, we add the constraint that all activities have a positive
value, xi  0
Fin500J Topic 5
Fall 2010 Olin Business School
13
Example
Product
Resource
Raw Gas
Regular
7
Premium
11
Resource Availability
77
Production Time
(hr/tonne)
10
8
120
Storage
(tonne)
9
6
Profit (/tonne)
150
175
3
(m /tonne)
Total Profit = 150 x1 + 175 x2
Maximize Z = 150 x1 + 175 x2
7x1 + 11x2  77
10x1 + 8x2  120
x1  9
x2  6
x1,x2  0
Fin500J Topic 5
x1 = amount
of regular and
x2 = amount
of premium
Objective function
(material constraint)
(time constraint)
(storage constraint)
(storage constraint)
(positivity constraint)
Fall 2010 Olin Business School
14
Graphical Solution
(1) 7x1 + 11x2  77
→x2  -7/11 x1 +7
(2) 10x1 + 8x2  120
→ x2  -5/4x1 + 15
Constraint 2
14
x2
(3) x1  9
Constraint 1
16
12
Constraint 3
10
Constraint 4
8
(4) x2  6
6
(5) x1,x2  0
2
4
0
0
5
10
15
x1
Fin500J Topic 5
Fall 2010 Olin Business School
15
Graphical Solution
Now we need to add the objective function to the plot.
Start with
Z = 0 (0=150x1 + 175x2)
and
Z = 500 (500=150x1 + 175x2)
Z=1200
Z=1550
Still in feasible region
x1*= 9
x2*  1.5
Fin500J Topic 5
Fall 2010 Olin Business School
16
Linear Programming in Matlab
Example:
 Step 1: >>optimtool in the commend
window to open the optimization
toolbox
 Step 2: Define matrices A, Aeq and the
vectors f, b, lb, ub
Fin500J Topic 5
Fall 2010 Olin Business School
17
Linear Programming in Matlab (Example)
 File->export to
workspace
 can export the results
including
lambda,etc.
Fin500J Topic 5
Fall 2010 Olin Business School
18
Quadratic Programming in Matlab
 Step 1: >>optimtool in the commend window to open the optimization
toolbox
 Step 2: Define matrices H,A and the vectors f, b
Fin500J Topic 5
Fall 2010 Olin Business School
19
Quadratic Programming in Matlab (Example:
Portfolio Optimization)
Fin500J Topic 5
Fall 2010 Olin Business School
20
Quadratic Programming in Matlab (quadprog)
H=[0.017087987 0.003298885
0.001224849; 0.003298885
0.005900944 0.004488271;
0.001224849 0.004488271
0.063000818]
f=[0; 0; 0]
A=[1 1 1; -0.026 -0.008 -0.074; -1 0 0; 0
-1 0; 0 0 -1]
b=[1000; -50; 0; 0; 0]
The function ‘quadprog ’ uses an active
set strategy. The first phase involves
the calculation of a feasible point.
The second phase involves the
generation of an iterative sequece
of feasible points that converge to
the solution.
Fin500J Topic 5
Fall 2010 Olin Business School
21
Nonlinear Programming in Matlab
( Constrained Nonlinear Optimization)
Formulation
Fin500J Topic 5
Fall 2010 Olin Business School
22
Nonlinear Programming in Matlab (Example)
Find x that solves
 Step 1: Write an M-file objfunc.m for the objective function.
function f=objfunc(x)
f=exp(x(1))*(4*x(1)^2+2*x(2)^2+4*x(1)*x(2)+2*x(2)+1);
 Step 2: Write an M-file confun.m for the constraints.
function [c, ceq]=confun(x)
%Nonlinear inequality constraints
c=[1.5+x(1)*x(2)-x(1)-x(2); -x(1)*x(2)-10];
%Nonlinear equality constraints
ceq=[];
 Step 3: >>optimtool to open the optimization toolbox
Fin500J Topic 5
Fall 2010 Olin Business School
23
Nonlinear Programming in Matlab (Example)
Fin500J Topic 5
Fall 2010 Olin Business School
24
Sequential Quadratic Programming is an Algorithm Used
in Function ‘fmincon’ (Basic Idea)
min f(x),
 The basic idea is analogous to Newton’s
method for unconstrained optimization.
subject to
g i(x)  0 i  1,...me .
 In unconstrained optimization, only the
objective function must be approximated,
in the NLP, both the objective and the
constraint must be modeled.
x
g i(x)  0
i  me  1,...m.
 An sequential quadratic programming
method uses a quadratic for the objective
and a linear model of the constraint ( i.e., a
quadratic program at each iteration)
Fin500J Topic 5
Fall 2010 Olin Business School
25