Vinay_proj_pres.ppt
Download
Report
Transcript Vinay_proj_pres.ppt
Learning Chaotic Dynamics
from Time Series Data
A Recurrent Support Vector Machine
Approach
Vinay Varadan
Primary Motivation
Understand the biological cell as a complex dynamical
system
Recent developments allow for in vivo post-translation
protein modification measurements along with gene
expression levels
Very expensive still, thus forcing only relatively sparse
sampling of the modified protein concentrations in time
We invariably measure only a small number of variables
of the system - in most cases just one or two variables
Develop modeling techniques to learn underlying
dynamics with short time series without knowing the
exact structure of the nonlinear differential equation
Even in the absence of noise, trajectory learning is still a
difficult problem
Problem Statement
Given the time series of one variable in a
multidimensional nonlinear differential equation (NDE)
Learn the number of dimensions, viz. number of
interacting variables in the underlying NDE
Given a few samples, be able to generate all future
samples exactly matching the trajectory of the variable
Do this for all possible NDEs, including ones at the edge
of chaos and also chaotic systems
In this project we concentrate on chaotic systems
because the rest would be easier to learn, for a given
dimensionality
Previous Attempts At Chaotic Time
Series Prediction
Taken’s delay embedding theorem (1981) – can recreate
the geometry of the state-space using just delayed samples
of the single observable
Thus for the time series measurement, y(t),
y(t) = f(y(t-1), y(t-2), … , y(t-m))
Nonlinear functions with universal approximation capability
employed for f such as RBF, polynomial functions, rational
functions, local methods
One-step predictors - these methods learn to predict one
time step ahead when given past samples of the
observable
Not good enough – not learning to follow trajectories of the
dynamical system thus not learning the geometry of the
state space well
We need to learn Recurrent models
Recurrent Models - SVM
Consider learning models of the form
yˆ k f yˆ k 1 , yˆ k 2 ,..., yˆ k p
where yˆ k denotes estimated output
In order to estimate the function f, we use Recurrent Least Squares
Support Vector Machines
We can rewrite the above equation in terms of the given data and the
error variables as
Using the parameteri zation :
yˆ k 1
ˆ
y
k 2
yˆ k wT
b
yˆ k p
We can rewrite the model as :
yk ek wT xk 1|k p k 1|k p b
where
ek yk yˆ k , xk 1|k p yk 1 ; yk 2 ;...; yk p , k 1|k p ek 1 ; ek 2 ;...; ek p
Recurrent Training using SVM
The training of the network is formulated as
1 T
1 Np 2
min w, b, e w w ek
w ,b , e
2
2 k p 1
subject to the equality constraint s
yk ek wT xk 1|k p k 1|k p b,
k p 1,..., N p
where is a positive constant
The final term of the equation to be minimized
refers to the Least Squares formulation
We can now define the Lagrangian and derive
the optimality conditions appropriately
Further, we can eliminate the calculation of w
explicitly and use just the Kernel formulation
Recurrent Training using SVM
The resulting recurrent simulation model is given as
yˆ k 1
yˆ
Np
k 2
yˆ k l p K zl 1|l p ,
b
l p 1
yˆ k p
where :
zl 1|l p xk 1|k p k 1|k p
and l p are Lagrange multiplier s
For the Recurrent SVM case, the parameter estimation
problem becomes nonconvex
We thus have to use sequential quadratic programming
Recurrent Model Performance
Performance of different prediction algorithms on a chaotic Predator-Prey
model
1.6
Single Step Iterated
1.4
1.2
1
Error
0.8
0.6
Recurrent Network
0.4
Single-Step Non-iterated
0.2
0
0
50
100
150
Step
200
250
300
Conclusion and Pending Work
Recurrent
SVM models are able to capture
the underlying dynamics much better
compared to other models
In the past, we have developed an
Improved Least Squares (ILS) formulation
for use in modeling chaotic systems
Need to explore how that can be
integrated with SVMs