Transcript Forecasting
Box-Jenkins models
A stationary times series can be modelled on basis of the serial correlations in it.
A non-stationary time series can be
transformed
into a stationary time series, modelled and back-transformed to original scale (e.g. for purposes of forecasting)
ARIMA
– models This part has to do with the transformation Auto Regressive, Integrated, Moving Average These parts can be modelled on a stationary series
AR-models (for stationary time series)
Consider the model
Y t = δ +
·Y t
–1
+ e t
with {
e t
} i.i.d with zero mean and constant variance = noise) and where
δ
(delta) and
σ 2
(white (phi) are (unknown) parameters
Autoregressive
process of order 1:
AR(1)
Set
δ
= 0 by sake of simplicity
E
(
Y t
) = 0
k
=
Cov
(
Y t , Y t-k
) =
Cov
(
Y t , Y t+k
) =
E
(
Y t ·Y t-k
) =
E
(
Y t ·Y t+k
)
Now: 0 =
E
(
Y t ·Y t
) =
E
((
·Y t-
1
+ e t
)
Y t
)=
· E
(
Y t-
1
·Y t
) +
E
(
e t
Y t
) = =
·
1 +
E
(
e t
(
·Y t-
1
+ e t
) ) =
·
1 +
· E
(
e t
Y t-
1 ) +
E
(
e t ·e t
)= =
·
1 + 0 +
σ 2
(for
e t
is independent of
Y t-
1 ) 1 =
E
(
Y t-
1
·Y t
) =
E
(
Y t-
1
·
(
·Y t-
1
+ e t
) =
· E
(
Y t-
1
·Y t-
1 ) +
E
(
Y t-
1
·e t
) = =
·
0 + 0 (for
e t
is independent of
Y t-
1 ) 2 =
E
(
Y t -
2
·Y t
) =
E
(
Y t-
2
·
(
·Y t-
1
+ e t
) =
· E
(
Y t-
2
·Y t-
1 ) + +
E
(
Y t-
2
·e t
) =
·
1 + 0 (for
e t
is independent of
Y t-
2 )
0 = 1 +
σ 2
1 =
·
0 2 =
·
1 …
Yule-Walker equations
k
=
·
k
-1 =…=
k ·
0 0 =
2 ·
0 +
σ 2
0
1
2 2
Note that for 0 to become positive and finite (which we require from a variance) the following must hold: 2 1 1 This in effect the condition for an AR(1)-process to be weakly stationary Now, note that
Corr
(
Y t
,
Y t
k
)
k
k
k
0 0
k Cov
(
Y t Var
(
Y t
) ,
Y t
k
)
Var
(
Y t
k
) 0
k
0 0
k
Recall that
k
is called the autocorrelation function (ACF) ”auto” because it gives correlations
within
the same time series.
For pairs of different time series one can define the cross correlation function which gives correlations at different lags
between
series.
By studying the ACF it might be possible to identify the approximate magnitude of .
Examples:
The general linear process
Y t
e t
1
e t
1 2
e t
2
e t
white noise
i
1
i
2 AR(1) as a general linear process:
Y t
e t
Y t
1
e t Y t
3
e t e t
2
e t
1
Y t
2
e t
1
e t
e t
1 2
e t
2
If |
|
< 1 The representation as a linear process is valid |
|
< 1 is at the same time the condition for stationarity of an AR(1)-process
Second-order autoregressive process
Y t
1
Y t
1 2
Y t
2
e t E
t
is assumed to be zero Otherwise, replace
Y t
by
Y t
,
Y t
1 by
Y t
1 and
Y t
2 by
Y t
2
Characteristic equation
Write the AR(2) model as
Y t
1
Y t
1 2
Y t
2
e t
Let
BY t
Y t
1 ;
BBY t
B
2
Y t
Y t
2 ; ;
B p Y t
Y t
p
The " backshift" operator
Y t
1 1 1
B BY t
2 2
B
2
B Y t
2
Y t
e t e t
1 1
x
2
x
2 0 is called the characteri stic equation of AR(2)
Stationarity of an AR(2)-process
The characteristic equation has two roots (second-order equation).
(Under certain conditions there is one (multiple) root.) The roots may be complex-valued If the absolute values of the roots both exceed 1 the process is stationary.
Absolute value > 1 Roots are outside the unit circle
i
1
1 1
x
2
x
2 0 x 1 1 2 2 2 4 2
x
1 1 2 1 ; 2 1 1 ; 2 1 Requires ( 1 , 2 ) to lie within the blue triangle.
Some of these pairs define complex roots.
Finding the autocorrelation function
Yule-Walker equations:
Y t
1
Y t
1 2
Y t
2
e t Y t
k
Y t
E
Y t
k
Y t
1
Y t
k
1
Y t
1
E
Y t
k
2
Y t
k
Y t
1
2
Y t
2
E
Y t
k
Y t
k
e t
Y t
2 (
Y t
k
independen t of
k
1
k
1 2
k
2
k
1
k
1 2
k
2
e t Y t
0
)
0 divide by 0
E
k Y t
k
e t
1
k
1 2
k
2 Start with 0 = 1
For any values of 1
exponentially
with
k
and 2 the autocorrelations will decrease For complex roots to the characteristic equation the correlations will show a
damped sine wave
behaviour as
k
increases.
Se figures on page 74 in the textbook
The general autoregressive process, AR(p)
Y t
1
Y t
1
p
Y t
p
e t
Characteri stic equation : 1 1
x
p x p
0 Stationary if all roots exceed 1 in absolute value Yule Walker equations : 1 2 1 1 1 2 2 1 3 2 3 1
p
p
1
p
p
2
p
1
p
1 2
p
2
p
k
1
k
1 2
k
2
p
k
p
Exponentially decaying Damped sine wave fashion if complex roots
Moving average processes, MA
Y t Y t
e t
1 1 1
B e t
1
q
q
B
q e t
e t q
MA(
q
) Always stationary
MA(1)
Y t
e t
e t
1 0 1
Var Cov
t
Y t
,
Y t
1
Var
t
2
Cov
e t
e t Var
1 ,
t
1
e t
1
e t e
2 2
1 2
e
2 1
k
1 0 ;
k
2 0 for
k
1
General pattern:
Y t
e t
1
e t
1
q
e t
q
k
k
1
k
1 1 1 2 2
k
2 2 2
q
2
q
k
q
0
k
1 , 2 , ,
q k
q
“cuts off” after lag
q
Invertibility
(of an MA-process)
Y t
e t
1
e t
1
q
e t
q
e t
Y t
1
q
e t
Y t
1
q
q
q
1
e t
q e t
q
1
Y t
1
Y t
1 2
e t
2
Y t
1
Y t
1 2
Y t
2
e t
i.e. an AR( )-process provided the rendered coefficients 1 , 2 , … fulfil the conditions of stationarity for
Y t
They do if the
characteristic equation
of the MA(
q
)-process 1 1
x
q
x q
0 has all its roots outside the unit circle (modulus > 1)
Autogregressive-moving average processes
ARMA
(
p
,
q
)
Y t
1
Y t
1
p
Y t
p
e t
1
e t
1
q
e t
q Y t
1 1 1
Y t
1
B
p
p
B
p Y t
Y t
p
e t
1 1 1
B e t
1
q
q B
q e t
e t q
Stationary if 1 1
x
p
x p
has all roots outside the unit circle Invertible if 1 1
x
q
x q
has all roots outside the unit circle If stationary
k
1
k
1
p
k
p
for
k
q
Specific equations needed for
k
,
k
q
Non-stationary processes
A simple grouping of non-stationary processes: • • • Non-stationary in mean Non-stationary in variance Non-stationary in both mean and variance Classical approach: Try to “make” the process stationary before modelling Modern approach: Try to model the process in it original form
Classical approach
Non-stationary in mean
Example
Random walk
Y t
Y t
1
e t Y t W t
Y t
1
Y t
e t Y t
1 becomes stationary (" first
Y t
order difference s"
Y t
1 can also be denoted
Y t
" Difference operator" ) or 1
B
Y t
using the backshift operator
More generally… If
Y t
satisfies
Y t
e t
1
e t
1 2
e t
1 (i.e.
the general linear model) we can try to model
Y t
as an ARMA(
p
,
q
) process If
Y t
t
is still non 2
Y t
Y t
stationary we can try
Y t
1
Y t
1
Y t
2
Y t
2
Y t
1
Y t
2 etc.
First-order non-stationary in mean Use first-order differencing Second-order non-stationary in mean Use second order differencing …
ARIMA(
p
,
d
,
q
)
W t
1 1
B d Y t
satisfies
p B p
t
1 1
B
q
B q
e t
Common:
d
≤ 2
p
≤ 3
q
≤
3
Non-stationarity in variance Classical approach: Use power transformations (Box-Cox)
g
t
t
log 1 0 0 Common order of application: 1.
Square root 2.
3.
4.
Fourth root Log Reciprocal (1/
Y
) For non-stationarity both in mean and variance: 1.
2.
Power transformation Differencing