Transcript The Value-Line Dow-Jones Model: Does It Have Predictive Content? Tom Fomby and Limin Lin Department of Economics SMU, Dallas, TX ISF 2004

The Value-Line Dow-Jones Model:
Does It Have Predictive Content?
Tom Fomby and Limin Lin
Department of Economics
SMU, Dallas, TX
ISF 2004
A USEFUL RULE OF THUMB
• “Data Mining” by Michael Lovell (1983,
RES) The Lovell Pretesting Rule for
Coefficient Significance
• Start MLR with c candidate variables.
Use A Best Subset Method to obtain
A MLR with k final variables
P-Value(actual) = (c/k)*P-Value(stated)
A MLR PREDICTION
RULE OF THUMB
• “On the Usefulness of Macroeconomic Forecasts
as Inputs to Forecasting Models” Richard Ashley
Jo. of Forecasting. 1983
• Var(x(hat))/Var(x) versus 1
• Ratio greater than one, x generally not useful
• Ratio less than one, x possibly useful
• It seems a lot of practitioners ignore this rule at
their peril
The Value-Line Dow Jones Stock
Evaluation Model
 Regression model used by the Value-Line Corporation in
its end-of-year report (Value Line Investment Survey) to
provide its readers a forecast range for the Dow-Jones
Index in the coming years. (Model builder: Samuel
Eisenstadt)
1
2
3
 EPt   DPt 
 BYt 
DJ t   DJ t 1  
 
 

EP
DP
BY
 t 1   t 1 
 t 1 
DJ — Dow Jones Industrial Average,
EP —Earnings Per Share on the Dow Jones,
DP — Dividends Per Share on Dow Jones, and
BY — Moody’s AAA Corporate Bond Yield
 logarithm transformation linear form:
 ln( DJt )  0  1 ln( EPt )  2 ln( DPt )  3 ln( BYt )
Motivation
 No evaluation of the model in existing literature, although
the model is in use for over twenty years and possibly by
millions of readers who may have made decisions upon
forecasting results from the model. It would be interesting
and useful to see how precise and reliable these forecasts
are.
 Arguments in the literature about the forecasting
competence of regression model vs. univariate models, eg.
Ashley (1983). Accuracy of the model depends on the
accuracy of the forecasts of the independent variables. Are
the independent variables making the forecast better or
worse?
Outline of Presentation
Data
Stability Analysis
Out-of-Sample Forecast Evaluation
(Predictive Content of Input Variables)
Conclusions
Data
 Annual observations (1920-2002) on
•
•
•
•
DJ: Dow Jones Industrial Average, annual averages
EP: Earnings Per Share on the Dow (data point 1932 adjusted
for convenience of log transformation)
DP: Dividends Per Share on the Dow
BY: Moody’s AAA Corporate Bond Yield
 Data source: “Long Term Perspective” chart of the Dow
Jones Industrial Average, 1920-2002, published by the
Value Line Publishing, Inc. in Value Line Investment
Survey
 Logarithm transformation used to obtain linear regression
 Comparisons are made among forecasts of DJ
Dow Jones Industrial Average, Annual Average 1920-2002
12000
10000
8000
6000
DJ
In-Sample
(1920-1972)
Holdout Sample
(1973-2002)
4000
2000
Year
2001
1998
1995
1992
1989
1986
1983
1980
1977
1974
1971
1968
1965
1962
1959
1956
1953
1950
1947
1944
1941
1938
1935
1932
1929
1926
1923
1920
0
Earnings Per Share and Dividends Per Share on Dow Jones Industrial Average,
1920-2002
600
500
400
EP
300
DP
200
100
0
1920
1925
1930
1935
1940
1945
1950
1955
1960
Year
1965
1970
1975
1980
1985
1990
1995
2000
Moody's AAA Corporate Bond Yield, 1920-2002
16
14
12
10
% 8
BY
6
4
2
0
1920
1925
1930
1935
1940
1945
1950
1955
1960
Year
1965
1970
1975
1980
1985
1990
1995
2000
Stability Analysis for VLDJ Model:
Recursive Coefficients Diagrams
.20
.5
.15
.4
.10
.3
.05
.2
.00
.1
-.05
.0
-.10
-.1
-.15
-.2
1940
1950
1960
1970
1980
Recursive C(1) Estimates
1990
2000
1940
± 2 S.E.
1950
1960
1970
1980
Recursive C(2) Estimates
1.6
1990
2000
± 2 S.E.
3
2
1.2
1
0.8
0
-1
0.4
-2
0.0
-3
-0.4
-4
1940
1950
1960
1970
1980
Recursive C(3) Estimates
1990
± 2 S.E.
2000
1940
1950
1960
1970
1980
Recursive C(4) Estimates
1990
2000
± 2 S.E.
As reported in end of year ValueLine Investment Survey, coefficients are estimated as follows:
2002: (1.030, 0.210, 0.350, -0.413);
1999: (1.034, 0.217, 0.332, -0.468);
2001: (1.032, 0.218, 0.336, -0.463);
1998: (1.032, 0.216, 0.335, -0.473), and so on.
2000: (1.033, 0.214, 0.340, -0.480);
Stability Analysis for VLDJ Model:
CUSUM and CUSUMSQ Test Results
30
1.2
20
1.0
0.8
10
0.6
0
0.4
-10
0.2
-20
0.0
-30
30
40
50
60
70
80
90
00
-0.2
30
CUSUM
40
50
60
70
CUSUM of Squares
The CUSUM test is based on the statistic:
Wt 
Where
t
90
00

r
t
T
Wt  (   ) /(  r2 )
/s
is recursive residual defined as
5% Significance
The CUSUMSQ test is based on the statistic:
t
r  k 1
80
5% Significance
r  k 1
t 
2
r
r  k 1
( yt  xt ' b)
1  x '( X
t
' X t ) xt 
1
t
S is the standard error of the regression fitted to all T sample points.
1/ 2
Test for Structural Change of Unknown Timing:
Wald Test Sequence as a Function of Break Date
Andrews (1993, 2003) critical values
The Models for “DLDJ”
(specified using in-sample data only)
 Transfer function model (in same form as the Value-Line
Model): DLDJ at time t is a function of DLEP, DLDP and
DLBY at time t ; where
DLEP ~ MA(2) , DLDP ~ MA(1) and DLBY ~ AR (1)
 Box-Jenkins univariate model: DLDJ ~ MA(1).
 Note: Transform Predictions for DLDJ to DJ in two steps:
ˆ  LDJ  DLDJ
ˆ
ˆ
Step 1:
LDJ
 DLDJ
t ,h
t
t ,1 
t  h 1,1
^
Step 2:

2
ˆ
ˆ
DJt ,h  exp LDJt ,h  0.5 h

Ex-Ante Forecast Accuracy—
Transfer Function vs. Box-Jenkins
(Imperfect Foresight)
Forecast Horizon
Horizon1
Horizon2
Horizon3
Horizon4
Horizon5
Horizon6
RMSFE
TF
605.30
1,046.46
1,405.36
1,645.72
1,884.64
2,090.61
Increase in RMSE
(H6-H1)/H1
245.38%
BJ
528.40
999.61
1,355.96
1,470.20
1,719.23
1,870.55
254.00%
Accuracy Ranking
TF
BJ
2
1
2
1
2
1
2
1
2
1
2
1
Usefulness of Explanatory Variables
in the Transfer Function Model—
MSFE( xˆt ) / VAR( xt )
Forecast Horizon Explanatory Variables in the TF
DLEP
DLDP
Horizon1
0.90621
1.04771
Horizon2
1.06544
1.14441
Horizon3
1.05522
1.13803
Horizon4
1.02157
1.15946
Horizon5
1.00511
1.18731
Horizon6
0.96534
1.22971
Ashley(1983 )
1 82
MSFE ( xˆt )   ( xt 1  xˆt ,1 )2
30 t 52
1 83
2
VAR( xt )    xt  x 
30 t 53
Model
DLBY
0.95627
1.11007
1.03032
0.96935
0.97723
1.00496
Forecast Accuracy--RMSFE: Assuming
Perfect Foresight for Leading Indicators in
Transfer Function Model
Forecast Horizon TF-Perfect
TF-Imperfect
RMSFE RMSFE Disadvantage*
Horizon1
607
605
99.69%
Horizon2
980
1,046
106.73%
Horizon3
1,175
1,405
119.59%
Horizon4
1,295
1,646
127.04%
Horizon5
1,490
1,885
126.46%
Horizon6
1,575
2,091
132.77%
RMSFE
528
1,000
1,356
1,470
1,719
1,871
BJ
Accuracy Ranking
Disadvantage* TF-Perfect BJ
87.02%
2
1
101.95%
1
2
115.38%
1
2
113.49%
1
2
115.36%
1
2
118.80%
1
2
*Disadvantage: Loss of forecast accuracy relative to TF-Perfect
Value Line Forecasts vs. TF and BJ Forecasts
Year
actual DJ
1983
1984
1985
1986
1987
1988
1989
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
MAFE*
RMSFE
1190
1178
1330
1797
2264
2062
2510
2670
2933
3282
3565
3735
4494
5740
7448
8631
10482
10731
10209
9214
ValueLine
DJ Forecast
1180
1390
1290
1435
2035
2425
2190
2710
2940
3445
N/A
N/A
N/A
5110
6415
8400
9800
11800
11800
10900
510
727
1-step-ahead
TF
BJ
972
906
1,335
1,368
1,230
1,202
1,421
1,451
2,002
2,034
2,355
2,497
2,107
2,086
2,645
2,819
2,779
2,817
3,234
3,172
3,647
3,546
3,785
3,824
3,872
3,975
4,674
4,977
6,001
6,412
7,757
8,368
9,327
9,396
11,056
11,681
11,178
11,327
11,096
10,700
593
555
807
704
2-step-ahead
TF
BJ
1,020
1,051
1,069
964
1,414
1,465
1,286
1,283
1,517
1,550
2,149
2,185
2,455
2,689
2,175
2,239
2,780
3,035
2,956
3,030
3,590
3,414
3,844
3,819
4,055
4,118
4,029
4,278
4,896
5,368
6,322
6,935
8,201
9,076
10,022
10,197
11,701
12,701
11,922
12,297
985
928
1,386
1,322
*The MAFE and RMSFE are computed based on years 1983-2002 except 1993-1995
Combination Forecasts of TF and BJ
 Simple Average (CF1)
 Nelson Combination (CF2)
 Granger-Ramanathan Combination (CF3)
 Fair-Shiller Combination (CF4)
 Note: We apply dynamic weights
Forecast Accuracy (RMSFE)—
Box-Jenkins vs. Combinations
RMSFE
Horizon1
Horizon2
Horizon3
Horizon4
Horizon5
Horizon6
(H6-H1)/H1
rank in increase
BJ
528
1,000
1,356
1,470
1,719
1,871
254%
1
CF1
559
1,011
1,368
1,549
1,790
1,973
253%
2
CF2
711
1,459
1,964
2,245
2,321
2,373
234%
3
CF3
627
1,134
1,656
1,394
1,548
1,279
104%
5
CF4
649
1,058
1,291
1,288
1,569
1,381
113%
4
RMSFE Ranking
Horizon1
Horizon2
Horizon3
Horizon4
Horizon5
Horizon6
Overall Score
Overall Ranking
BJ
1
1
2
3
3
3
2.17
1
CF1
2
2
3
4
4
4
3.17
4
CF2
5
5
5
5
5
5
5.00
5
CF3
3
4
4
2
1
1
2.50
3
CF4
4
3
1
1
2
2
2.17
1
Ways of Combating Weak Input Variables
 Drop input variables that don’t satisfy Ashley’s Criterion
(Forecast could have bias but less variance)
 Use improved input variables: Combination of sample
mean and forecasts of input variable
-- Simple average
-- Ashley (1985) combination
Forecast Accuracy—
Dropping Inadequate Input Variables
TF
Forecast Horizon Input Variables
Horizon1 DLEP and DLBY
Horizon2
N/A
Horizon3
N/A
Horizon4
DLBY
Horizon5
DLBY
Horizon6
DLEP
MAE
RMSE
394.56 633.69
N/A
N/A
N/A
N/A
917.41 1,535.99
1,118.73 1,755.99
1,426.34 2,139.76
Compare with BJ
MAE RMSE
351.91 528.4
604.64 999.61
764.41 1,355.96
895.69 1,470.20
1,092.55 1,719.23
1,198.63 1,870.55
Forecast Accuracy—
Input Variables From Combination Forecasts
Forecast
Leading Indicators:
Leading Indicators:
Compare with
Leading Indicators:
Horizon
Simple Average
MAFE RMSFE
376.43
596.93
641.95 1,058.13
824.00 1,385.89
997.90 1,616.70
1,241.54 1,912.19
1,452.05 2,159.12
Ashley Combination
MAFE RMSFE
389.28
611.46
721.76 1,112.25
834.19 1,386.82
995.05 1,596.70
1,261.50 1,886.64
1,458.71 2,129.24
ARIMA
MAFE RMSFE
378.85
605.30
630.54 1,046.46
822.84 1,405.36
998.43 1,645.72
1,189.42 1,884.64
1,385.33 2,090.61
Horizon1
Horizon2
Horizon3
Horizon4
Horizon5
Horizon6
Conclusions
 In the absence of perfect foresight, TF (Value Line) forecasts are less
accurate than the BJ benchmark forecasts for any forecast horizons.
 Ashley (1983) criterion shows that the leading indicators are very
noisy and inhibit ex ante forecasting accuracy of TF model.
 If future values of leading indicator variables are assumed known,
(perfect foresight), TF forecasts improve considerably--beat the BJ
forecast for 2-6 step-ahead forecast horizons, but do not for the 1-stepahead forecast horizon.
Conclusion (cont.)
 With respect to Ex Ante combination forecasting, BJ forecasts perform
better for short horizons and combinations of the TF and BJ are best
for longer horizons.
 For Ex Ante forecasts, differences in accuracy between TF forecasts
and the most accurate forecasts are not statistically significant. Ashley
(2003)
 Dynamic Combination forecasts perform better than combinations with
fixed weights.
 Dropping inadequate input variables did not improve forecast
accuracy. Using combination forecasts for the input variables only
improved the forecast accuracy of some horizons.
Conclusion (cont.)
 Evidently, the Value Line personnel have been pretty astute with
respect to choosing future values of the independent variables of their
model. Their published 1-step-ahead forecasts have smaller MAFE
than the ex ante TF model and the BJ model. With respect to the
RMSFE, however, the BJ model provides a more accurate 1-stepahead-forecast.
 Remember forecasting accuracy is only one way to evaluate the VLDJ
model. Irrespective of its forecasting powers, it should be recognized
that the VLDJ model is potentially quite useful for examining “what
if” scenarios and understanding historical causal factors in the stock
market.
 It would be interesting to compare competing models based on interval
forecast accuracy and density forecast accuracy.
Thank you!
References
• Andrews, D. W. K. (1993): “Tests for Parameter Instability and
Structural Change with Unknown Change Point,” Econometrica, 61,
821-856.
• Andrews, D. W. K. (2003): “Tests for Parameter Instability and
Structural Change with Unknown Change Point: A Corrigendum,”
Econometrica, 71 (1), 395-397.
• Ashley, R. (1983): “On the Usefulness of Macroeconomic Forecasts as
Inputs to Forecasting Models,” Journal of Forecasting, 2, 211-223.
• Ashley, R. (2003): “Statistically Significant Forecasting
Improvements: How Much Out-of-Sample Data Is Likely Necessary?”
International Journal of Forecasting, 19(2), 229-239.
• Bai, J. (1997): “Estimation of A Change Point in Multiple Regression
Models,” Review of Economics and Statistics, 79 (4), 551-563.
References (cont.)
• Brown, R. L., J. Durbin, and J. M. Evans (1975): "Techniques for
Testing the Constancy of Regression Relationships Over Time,"
Journal of the Royal Statistical Society, Series B, 37, 149-192.
• Diebold, F. X. and R. S. Mariano (1995): “Comparing Predictive
Accuracy,” Journal of Business and Economic Statistics, 13 (3), 253263.
• Fair, R. C. and R. J. Shiller (1990): “Comparing Information in
Forecasts from Econometric Models,” American Economic Review, 80
(3), 375-389.
• Nelson, C. R. (1972): “The Prediction Performance of the FRB-MITPENN Model of the U.S. Economy,” American Economic Review, 62
(5), 902-917.
Combinations of the TF and BJ models
• Naïve combination: simple average (weight=0.5)
CF1t ,h  0.5BJt ,h  0.5TFt ,h
In-sample
Out-of-sample
(obs. 1-53)
(obs. 54-83)
Combinations of the TF and BJ models
(dynamic weights applied)
• Dynamic Nelson combination (weights sum to 1)
CF 2t ,h  ˆt ,h BJt ,h  (1 ˆt ,h )TFt ,h
where weight is obtained from LS regression
(DJt h  TFt ,h )  t ,h (BJt ,h  TFt ,h )  t
15 obs.
Training
Test Data
Validation
(Out-of-sample)
15 obs.
Validation
…
Combinations of the TF and BJ models
(dynamic weights applied)
• Dynamic Granger-Ramanathan combination (weights
obtained from unrestricted regression)
CF 3t ,h  ˆt ,h  ˆt ,h BJt ,h  ˆt ,hTFt ,h
where weights are obtained from regression
DJt h  t ,h  t ,h BJt ,h   t ,hTFt ,h  t
• Dynamic Fair and Shiller Combination
CF 4t ,h  DJ t   t ,h  t ,h  BJ t ,h  DJ t    t ,h TFt ,h  DJ t 
where weights are obtained from regression
DJ t  h  DJ t   t ,h  t ,h  BJ t ,h  DJ t    t ,h TFt ,h  DJ t    t
Data --LDJ
Dow Jones Industrial Average, Annual Average in
Logarithm
10
9
8
7
6
LDJ
5
4
3
2
1
Year
2000
1996
1992
1988
1984
1980
1976
1972
1968
1964
1960
1956
1952
1948
1944
1940
1936
1932
1928
1924
1920
0
Year
2000
1996
1992
1988
1984
1980
1976
1972
1968
1964
1960
1956
1952
1948
1944
1940
1936
1932
1928
1924
1920
Data --LEP
Earnings Per Share on the Dow in Logarithm
7
6
5
4
LEP
3
2
1
0
Year
2000
1996
1992
1988
1984
1980
1976
1972
1968
1964
1960
1956
1952
1948
1944
1940
1936
1932
1928
1924
1920
Data --LDP
Dividends Per Share on the Dow in Logarithm
6
5
4
3
LDP
2
1
0
Year
2000
1996
1992
1988
1984
1980
1976
1972
1968
1964
1960
1956
1952
1948
1944
1940
1936
1932
1928
1924
1920
Data --LBY
Moody's AAA Bond Yields in Logarithm
3
2.5
2
1.5
LBY
1
0.5
0
Figure 12. First Difference of Log Dow Jones Industrial Average
0.4
0.2
-0.4
DLDJ
-0.6
-0.8
-1
Year
2001
1997
1993
1989
1985
1981
1977
1973
1969
1965
1961
1957
1953
1949
1945
1941
1937
1933
1929
1925
-0.2
1921
0
0
-1
-2
-3
-4
-5
Year
2001
1997
2
1993
1989
1985
1981
1977
1973
1969
1965
1961
1957
1953
1949
1945
1941
1937
1933
1929
1925
1921
Figure 13. First Difference of Log Earnings Per Share
4
3
DLEP
1
Figure 14. First Difference of Log Dividends Per Share
0.6
DLDP
0.4
0.2
-0.4
-0.6
-0.8
Year
2001
1997
1993
1989
1985
1981
1977
1973
1969
1965
1961
1957
1953
1949
1945
1941
1937
1933
1929
1925
-0.2
1921
0
Figure 15. First Difference of Log AAA Bond Yield
0.25
0.2
0.15
DLBY
0.1
-0.15
-0.2
-0.25
-0.3
Year
2001
1997
1993
1989
1985
1981
1977
1973
1969
1965
1961
1957
1953
1949
1945
1941
1937
1933
1929
1925
-0.05
-0.1
1921
0.05
0