Transcript Document

MA409: Continous-time
Optimization
Adam Ostaszewski
1
17/07/2015
Lecture times for 2011/12

Lectures: Monday in TW1.U208 : 4-5 pm
and Thursday in NAB.1.14 : 12-1pm

Classes: Wednesday in TW1.U101: 6-7pm
… starting Week 1
all in Michaelmas Term

Plus: Revision in Summer Term!
2
17/07/2015
Q: What is this course about?
A: How to …
Select a curve x(t) for 0 < t < T
 … subject to constraints holding over
some or over all time t
 To minimize (maximize) some
performance index

3
17/07/2015
About the course…
geodesics &
other performance index
E.g. … subject to x(0) being given, and
also x(T)
 …or x(t) must lie on some fixed surface
 E.g. is a geodesic = minimizes distance
along the given fixed surface between
initial and terminal positions

4
17/07/2015
About the course …
time optimality
E.g. … subject to x(0) being given and
maybe x(T) only to lie on a specified
surface
 …or x(t) must lie on some varying surface
 E.g. minimize time taken to reach the
prescribed surface: T

5
17/07/2015
Notes:
n
The variable x(t) may be in R
 The constraint may be f(x(t)) = 0

… but could be a differential equation like
this ‘canonical’ one: x′(t)=a(t,x(t))
 Where dash is defined by: x′=dx/dt


…so dx = a(t,x(t)) dt
6
17/07/2015
State equation in differential
form (first order)

… Or even like x′(t)=a(t,x(t),u(t))
here u(t) is to be selected and is known as
a control (costly, usually!)
 (Think: selecting car speed with car
constrained to a particular road.)

7
17/07/2015
A state equation in vector
differential form – first order
…indeed … despite Newton’s law …

x′′ = u (accn = force)


Rewritten, with x1=x, and with x2=x1′,

…as the linear matrix first-order system
x1′ = x2, x2′ = u
8
17/07/2015
…or in stochastic form
… could be a ‘stochastic’ differential
equation
 dx(t)=a(t,x(t),u(t)) dt + b(t,x(t),u(t)) dWt


..where for each t the term Wt represents a
continuously varying standard `random
variable’, modelling uncertainty
9
17/07/2015
… risky assets

Model of asset price evolution St
‘Instantaneous’ return on the asset over the
time interval dt (from t to t+dt )
is defined to be:

(St+dt - St)/St , or in compact form = dSt/ St
10
17/07/2015
… Black-Scholes model for
asset price St

Return on the asset modelled by
dSt / St =  dt +  dWt
…anticipated ‘rate’ of return 
 …PLUS ‘volatility factor’  times a
standard volatility term, namely dWt …


nb dWt = (Wt+dt - Wt) the increments are
independent, normal, with variance dt
11
17/07/2015
Format of Performance Index:
Deterministic Case



Bequest/gift: g(x(T)) – a Meyer problem
Running cost format:
Lagrange problem
T
0
L(x(t),…)dt – a
Mix of above: g(x(T)) + 
– a Bolza problem
T
0
L(x(t),…) dt
12
17/07/2015
…Actually they’re equivalent
…proved by fiddling with the number, n,
of variables, e.g. introduce xn+1 and adjoin
 … a diff. eqn.:
dxn+1 = L(x(t),…)dt
 … initial condition: xn+1 (0) = 0


Then g(x(T))= xn+1 (T) = 
T
0
L(x(t),…)dt
13
17/07/2015
.. Stochastic case
Expected performance

Thus might be

E[g(x(T))] – a Meyer problem …

So we can solve the investor’s optimal
consumption problem
14
17/07/2015
Expected performance

At what rate to consume wealth, at what
rate to invest in a risky asset and how
much to salt away in a safe bank deposit
T

Max Ew[ 0 exp(-rt)U(u(t))]dt

maximize expected utility of lifetime
consumption, where u(t) =fraction of
wealth x(t) consumed and x(0)=w.
15
17/07/2015
Note the implicit ‘stopping time’

…in the upper limit of integration
T
0

Max Ew[
exp(-rt)U(u(t))] dt

T denotes the first time t when x(t) = 0, so
T may be a special sort of random variable
16
17/07/2015
Solution technique
Optimality as with real-variables where
 Max f(x)
 is solved using the differential condition:
df(x) = 0, the ‘first order condition’ (f.o.c.)


We develop a higher level calculus, which
converts df(x) = 0 to …a differential eqn.
17
17/07/2015
…f.o.c. leads to a …differential
equation
…satisfied by the optimal trajectory (curve)

And so the technique amounts to setting up
the problem (modelling) and then solving
the d.e.
18
17/07/2015
Reading List
To begin with we use A.O.’s
“Advanced Mathematical Methods”


Notes will be provided throughout the
course by way of Moodle (& the maths
web-site)
19
17/07/2015
Literature – Mathematical
Basics
(♥) Troutman, Variational Calculus,
Springer
 (♥) Pinch, Optimal Control and the
Calculus of Variations, OUP
 Fleming & Rishel, Deterministic &
Stochastic Control, Springer
 (♥ ♥) Burghes & Graham, Control &
Optimal Control Theories with
Applications, Horwood.

20
17/07/2015
Mathematical Literature Advanced

Fleming & Soner, Controlled Markov
Processes and Viscosity Solutions,
Springer

Oksendal & Sulem, Applied Stochastic
Control of Jump Diffusions, Springer

Vinter, Optimal Control, Birkhauser
21
17/07/2015
Literature – Very Advanced
Rigorous Mathematical Background
…well beyond our course:
 Rogers & Williams, Diffusions, Markov
Processes and Martingales: Volumes 1& 2,
Itô Calculus, (Cambridge Mathematical
Library)


But you should know it’s there
22
17/07/2015
P.S. My secret place:
http://www.maths.lse.ac.uk/Courses/MA409/
This also yields access to my course
materials
23
17/07/2015