Autocorrelation in Time Series
Download
Report
Transcript Autocorrelation in Time Series
Autocorrelation in Time Series
KNNL – Chapter 12
Issues in Autocorrelated Data
• When error terms are correlated (not independent),
problems occur when using ordinary least squares
(OLS) estimates
– Regression Coefficients are Unbiased, but not Minimum
Variance
– MSE underestimates s2
– Standard errors of regression coefficients based on OLS
underestimate the true standard error
– Inflated t and F statistics and artificially narrow
confidence intervals
Autocorrelated Errors (1st Order) t t 1 ut
where ut uncorrelated disturbances (typically assumed to be normal)
First-Order Model - I
First-Order Autoregressive Model ( AR(1)) :
Simple Regression: Yt 0 1 X t t
t t 1 ut
t 1,..., n
autoregression parameter with 1
ut ~ N 0, s 2 and independent
Generalizes to Multiple Regression:
Yt 0 1 X t1 ... p 1 X t , p 1 t
t t 1 ut
t 1,..., n
Properties of Errors (assumption regarding 1 for model consistency):
s2
1 ~ N 0,
2
1
2 1 u2 E 2 E 1 E u2 0
s2
s2
2
s 2 s 1 s u2
s
2
1 2
1
2
2
2
2
s 2
Covariance: s 2 , 1 s 1 u2 , 1 s 2 s u2 , 1 s 2 0
1 2
2
Corrrelation: 2 , 1
s 2 , 1
s 2 s 1
2
s 2
1 2
s
2
s
2
1 2 1 2
2
First-Order Model - II
In General:
t t 1 ut t 2 ut 1 ut t 2 ut 1 ut ... s ut s
2
s 0
E t 0
s2
s
2s 2
2
2s
s t s ut s s ut s s
1 2
s 0
s 0
s 0
2
2
ss 2
Covariance: s t , t s
1 2
Correlation: t , t s s
s0
s0
1
2
1
2
s
2
σ 2 ε
1
2
1
n 1 n 2 n 3
AR(2) : t 1 t 1 2 t 2 ut
n 1
n2
n 3
1
Even Higher or models can be fit as well.
Test For Independence - Durbin-Watson Test
Yt 0 1 X t1 ... p 1 X t , p 1 t
t t 1 ut ut ~ NID 0, s 2
1
H 0 : 0 Errors are uncorrelated over time
H A : 0 Positively correlated
1) Obtain Residuals from Regression
2) Compute Durbin-Watson Statistic (given below)
3) Obtain Critical Values from Table B.7, pp. 1330-1331 (R will provide a p-value)
If DW d L p 1, n Reject H 0
n
Test Statistic: DW
et et 1
If DW dU p 1, n Conclude H 0
2
t 2
n
e
t 1
2
t
s 2
E t 0 E t t 1
1 2
n
e e
t 2
t
t 1
2
s 2
e e 2 et et 1 2 e 2n
1 2
t 2
t 2
t 2
t 1
n
n
2
t
n
2
t 1
Under H 0 , expect DW 2
n
2
t
Otherwise Inconclusive
Autocorrelation - Remedial Measures
• Determine whether a missing predictor variable can
explain the autocorrelation in the errors
• Include a linear (trend) term if the residuals show a
consistent increasing or decreasing pattern
• Include seasonal dummy variables if data are quarterly
or monthly and residuals show cyclic behavior
• Use transformed Variables that remove the (estimated)
autocorrelation parameter (Cochrane-Orcutt and
Hildreth-Lu Procedures)
• Use First Differences
• Estimated Generalized Least Squares
Transformed Variables
Suppose is known: Yt 0 1 X t t
t t 1 ut
Let Yt ' Yt Yt 1 0 1 X t t 0 1 X t 1 t 1
0 1 1 X t X t 1 t t 1 0 1 1 X t X t 1 ut
Yt ' 0' 1' X t' ut
(Standard Simple linear regression with independent errors)
where:
Yt ' Yt Yt 1
X t' X t X t 1
0' 0 1
1' 1
In Practice, we need to estimate with a sample based value r
Yt ' Yt rYt 1
X t' X t rX t 1
^
Fit: Y ' b0' b1' X ' and if errors are uncorrelated, back transform to:
b0'
Y b0 b1 X where: b0
1 r
^
s b0
s b0'
1 r
b1 b1'
s b1 s b1'
Cochrane-Orcutt Method
• Start by estimating in Model: t = t-1 + ut by
regression through the origin for residuals (see below)
• Fit transformed regression model (previous slide)
• Check to see if new residuals are uncorrelated (DurbinWatson test), based on the transformed model
• If uncorrelated, stop and keep current model
• If correlated, repeat process with new estimate r based
on current regression residuals from the original (back
transformed) model
n
r
e
t 2
n
e
t 1 t
2
e
t 1
t 2
Hildreth-Lu and First Difference Methods
• Hildreth-Lu Method
– Find value of r (between 0 and 1) that minimizes the SSE for
the transformed model by grid search
– Apply the transformed analysis based on the estimated r
• First Differences Method
– Uses = 1 in transformed model (Yt’ = Yt – Yt-1 Xt’ = Xt – Xt-1 )
– Set b0’ = 0 and fits regression through origin of Y’ on X’
– When back-transforming:
b0 Y b X
'
1
b1 b
'
1
Forecasting with Autocorrelated Errors
Makes use of any of the 3 estimation techniques (C-O, H-L, First Differences):
Yt 0 1 X t t
t t 1 ut
Yt 0 1 X t t 1 ut Yn 1 0 1 X n 1 n un 1
3 Elements:
^
1. Expected Value: 0 1 X n 1 Estimated as Y n 1 b0 b1 X n 1
2. Multiple of period n Error Term: n
Estimated as ren
3. Current disturbance un 1 ~ N 0, s 2
Forecast for period n 1 (note the notation is "Forecast", not F -distribution :
^
Fn 1 Y n 1 ren
Standard Error of the Prediction (based on transformed model):
'
X n 1 X '
1
2
s pred MSE ' 1
n
2
n
1
X i' X '
i 2
Approximate 95% PI: Fn 1 t 1 2 ; n 3 s pred
(First Differences has n - 2 df)