Transcript EUROPEAN COMMISSION DIRECTORATE GENERAL ECONOMIC AND FINANCIAL AFFAIRS EU WORKSHOP ON RECENT DEVELOPMENTS IN BUSINESS AND CONSUMER SURVEYS Are the Representative Agent’s Beliefs Based.

EUROPEAN COMMISSION
DIRECTORATE GENERAL ECONOMIC AND FINANCIAL AFFAIRS
EU WORKSHOP ON RECENT DEVELOPMENTS
IN BUSINESS AND CONSUMER SURVEYS
Are the Representative Agent’s Beliefs Based on
Efficient Econometric Models?
Maurizio Bovi
Brussels
15 November 2012
Plan
 Motivation: several assumptions but little evidence on how
laypeople form expectations
 Occasion: Real-Time Data thanks to BoE, Survey Expectations
thanks to European Commission
 Data Analysis:
Survey: Heterogeneous expectations as Signal/Noise Ratios (SNR)
 Real Time “Hard” Data: Econometric models and MSE horse race
 Contributions:
i) Representative agent’s expectations may be not grounded in
optimal econometric models
ii) VAR Analysis of the Expectations Feedback System
(beliefsrealizations) resulting in: “SNR => MSE”
 Concluding Remarks and a Tentative
Agenda
2
Are the Representative Agent’s beliefs
based on efficient statistical models?
 People form rational expectations (Muth, 1961):
agents know and use the same “true” model.
 People learn the correct model (Evans and Honkapohja, 2001):
agents act as econometricians and relentlessly estimate models.
 People are “infected” by professional forecasters’ models (Carroll, 2003):
economists produce forecasts => mass media report forecasts => (more or
less frequently) people read forecasts.
 People use the model with the highest fitness (Brock and Hommes, 1997):
People examine different forecasting models switching from one model to
another after a cost-benefit analysis based on relative mean squared
errors. Model uncertainty (i.e. the kind of the optimal model changes with
high frequency), preferences and inertia in the dynamic switching can
create heterogeneous expectations (even to a greater extent wrt sticky
information, Branch, 2004 & 2007).
3
Issues
 People form rational expectations (Muth, 1961):
Several authors find evidence rejecting this assumption.
 People act as smart econometricians (Evans and Honkapohja, 2011):
This is the typical assumption maintained by the adaptive learning
literature. Finding evidence on that is one of my goals.
 People are “infected” by professional forecasters (Carroll, 2003):
If so, why household surveys are still among the most watched market
movers even among professional forecasters?
 Predictor Choice Approach (Brock and Hommes, 1997):
To be able to calculate the relative success of their own choice, agents
must know the success of all competing models: individuals have already
paid the computational cost. Then, the question can be asked whether the
determinant of the choice should deal with forecasting accuracy only.
Also, “preferences” towards a single model are at odds wrt both the
dynamic switching and learning activities.
4
Alternative Views on the Expectations Feedback System
 Pigou
(1927),
Keynes
(1936),
Simon
(1957),
Tversky
and
Kahneman, (1974)

Psychology matters and not-econometrically-based factors is what
surveys could/should capture (Katona, 1944, 1975; Bovi, 2009)
 Ludvigson (2004)

Many empirical papers have been looking, with some success, for
the additional information content of survey expectations, whereby
the adjective “additional” stands exactly for extra economic
elements and/or independent information
 Cass and Shell (1983)

Heterogeneous beliefs – e.g. in models with self-fulfilling prophecies
sunspot equilibria - can drive macroeconomic outcomes.
5
Key Questions and Logic
 Are representative agents’ (=>laypeople) beliefs based on
optimal econometric models? Do heterogeneous beliefs
Granger-cause the predictive power of efficient econometric
models, or vice versa? Here the logic:
 Think about an economy whereas a simple model turns out
to be the best predictor for many years, but survey-declared
expectations do not converge.
 Think about an economy whereas evidence points out that
SNR “precede” MSE, but not vice versa.
 All in all, the general validity of assuming best model-based
expectations would be weakened.
6
2. Survey Data
 Source: Business Surveys Unit of the European Commission
 Sample Period: January 1985 – March 2009 (simple mean for quarterly data)
 Features: No genuine panel (bad), continuously refined (good), Laypeople
(good: it is likely that economists use econometric models to forecast)
 Query: “How do you expect the general economic situation in the country
to develop over the next 12 months?”
 Reply Options: It will…
…get a lot better (=LB);
…get a little better (=B);
…stay the same (=E);
…get a little worse (=W);
…get a lot worse (=LW);
don't know
(=N).
LB, B, E, etc., are the shares of respondents having chosen the corresponding
option so that they sum up to one.
7
2.1 Survey Expectations Signal/Noise Ratios
8
2.1a
Survey Expectations Signal/Noise Ratios
9
2.1b Survey Expectations Signal/Noise Ratios
Unlike the previous methods, the IQV does not account for the ordered nature of the data.
10
2.2 Survey Expectations and Reality
8.0
1.00
6.0
0.96
4.0
0.92
2.0
0.88
0.0
0.84
-2.0
0.80
UK GDP annual grow th rate
Signal
Noise (rhs, 1=fully heterog. expect.)
-4.0
0.76
-6.0
0.72
86
88
90
92
94
96
98
11
00
02
04
06
08
3. Real Time Data
 When comparing econometric models and survey
expectations, the former should be estimated in real
time (and known by the representative agent: VAR
before 1980?)
 Why? Because otherwise one is assuming that
representative agents use information which will be
available only in future dates or that they have
remarkably good foresight about data revisions
 Trivial? Not so much: attention to actually available data
is becoming widespread in the literature only recently
(Croushore, 2010). As for the techniques, sometimes
laypeople are asked to act as leading econometricians.
12
3.1 BoE Real Time Data Set
 Real GDP:
the first disposable vintage (released in 1976Q1)
covers the period 1955Q1-1975Q4,
the second was published a quarter later and covers the
period 1955Q1-1976Q1...
the last release I use here covers the period 1955Q12009Q1
 Prices (GDP and Private Consumption Deflators):
the first release includes data from 1970Q1 to 1989Q4
(released in Jan. 1990)…
the last 1970Q1 to 2009Q1
 Interest rate (3M Treasury Bill Rate):
no data revision, available since 1957Q1
13
3.2 The Competing Models
All models estimated recursively, AR and VAR models via rolling regressions, too
14
3.3 Real-Time Estimation
 I sequentially estimate the mentioned models using real time data:
 Example:
in 1976Q1 (=t+1) the very first yt time series, running from 1956Q1
to 1975Q4 (=t), is made available. The 1st four-steps-ahead prediction refers
to 1976Q4 (=t+4). To compare this with its realization we have to wait
until 1977Q1 (=t+5), when the actual data for 1976Q4 is eventually released.
According to survey and hard data availability,
• Univariate models generate 97 recursive forecasts, from 85Q1 to 09Q1
• VAR models produce 74 recursive forecasts, from 90Q4 to 09Q1
15
3.3a Real-Time Estimation
 AE model. Each quarter grid search over all (0,1], with step size 0.04,
choosing the value of gamma that minimizes the squared forecasting
errors.
 Rolling windows estimation:
• Minimum window size = 32 quarters;
• Max window size = 56 quarters.
So, to find the optimal (MSE-minimizing) window size, I perform 24 separate
rolling regressions for each (non naïve) model.
16
3.4 Real-Time Forecasting Metric
 To mimic the forecasting exercise elicited in the surveys, I forecast “the
next twelve months”, computing the quarterly squared errors as follows:
MSEt+5= (t+5yt+4 –
e
2
y
t+1 t+4)
where:
yt = (GDPt-GDPt-4)/GDPt-4
e
t+1y t+4 = Expected value of y in t+4 based on the vintage released in t+1
t+5yt+4 = Actual value of y in t+4 as reported by the vintage released in t+5
Due to data availability, the first useful squared error is
MSE85:Q1 for univariate models
MSE90:Q1 for multivariate models
17
3.5 Real-Time Econometric Models Forecasting Ability
Model
85Q1-89Q4
90Q1-96Q2
96Q3-02Q4
03Q1-09Q1
90Q1-09Q1
RW
3.73
5.34
1.73
4.31
3.8
AE
1.17
2.49
0.42
3.09
2.01
AR1
1.98
5.11
0.98
3.72
3.27
AR1
rolling
1.8
3.52
0.9
3.71
2.73
VARPC1 (y,)
NA
6.64
1.45
4.24
4.11
VARPC1 rolling
NA
4.29
1.27
3.79
3.12
VARPC2 (y,)
NA
7.07
1.66
4.56
4.43
VARPC2 rolling
NA
3.71
1.34
3.74
2.94
VAR1
(y,, r)
NA
6.97
3.16
5.82
5.32
VAR1
rolling
NA
4.1
1.01
3.12
2.75
VAR2
(y,, r)
NA
6.52
3.18
5.98
5.24
VAR2
rolling
NA
3.45
0.95
3.48
2.64
18
Sample averages of the MSE of the corresponding models
Naive Expectations Forecasting Accuracy (MSE)
as a Proxy of the Great Moderation
12
MSE_RW
MSE_AE
10
8
6
4
2
0
86
88
90
92
94
96
19
98
00
02
04
06
08
3.6 Relative Forecasting Ability wrt RW
AE
AR1
-1
-2
-3
-4
94
96
98 00
02
04
06 08
2
2
1
1
1
0
0
0
-1
-1
-1
-2
94
96
98
02 04
-2
06 08
1
0
-1
-2
-3
-4
0
-1
98 00
02
04
06 08
-2
94
96
98
94
96
98 00
02
04
00
02 04
06 08
94
VAR2
06 08
4
3
2
1
0
-1
-2
94
96
98
00
02 04
96 98
00 02
04
06
08
-2
94
96
98
VARPC2 rolling
1
96
94
VARPC2
VAR1 rolling
1
0
-1
-2
-3
-4
00
2
94
VARPC1
2
VARPC1 rolling
1
0
-1
-2
-3
-4
AR1 rolling
96 98
00 02
04
00
02 04
06
08
06
08
VAR1
06
08
06
08
4
3
2
1
0
-1
-2
94
96
98
00
02 04
VAR2 rolling
1
0
-1
-2
-3
-4
06 08
94
96 98
00 02
04
t-stat of the constant term in the “(MSE_J-MSE_RW)=const” regression. When a curve is below
the -1.69 horizontal line (indicating the 5% p-value), then the corresponding model “J” dominates
the benchmark.
20
3.6a Relative Forecasting Ability wrt AE
AR1
AR1 rolling
6
4
2
0
-2
94
96
98
00
02
04
06
08
4
3
2
1
0
-1
-2
94
96
VARPC2
4
3
2
1
0
-1
-2
94
96
98
00
02
04
94
96
98
00
02
00
02
04
06
4
3
2
1
0
-1
-2
08
06
08
94
96
98
00
02
04
VARPC1 rolling
6
4
2
0
94
96
98
VARPC2 rolling
4
3
2
1
0
-1
-2
VAR2
4
3
2
1
0
-1
-2
98
VARPC1
00
02
04
06
08
-2
94
96
VAR1
06
08
06
08
4
3
2
1
0
-1
-2
94
96
98
00
02
98
00
02
04
06
08
06
08
VAR1 rolling
04
06
08
3
2
1
0
-1
-2
94
96
98
00
02
04
VAR2 rolling
04
06
08
3
2
1
0
-1
-2
94
96
98
00
02
04
t-stat of the constant term in the “(MSE_J-MSE_AE)=const” regression. When a curve is
below the -1.69 horizontal line (indicating the 5% p-value), then the corresponding model J
dominates the benchmark.
21
0,1
0
-0,1
-0,2
-0,3
-0,4
-0,5
|
|
|
|
|
|
|
|
IQV
BAL
BAL3
|
CP3
|
CP
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
______________________________________________________________________________________________________________________________
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
0
i=
0,2
0,1
0
-0,1
-0,2
-0,3
-0,4
i=
2
1
4
3
6
5
8
7
9
10 12
11
1
0
3
2
5
4
7
6
8
9 11
10 12
0
2
1
4
3
6
5
8
7
9
10 12
11
1
0
3
2
5
4
7
6
9
8
11
10 12
0
2
1
4
3
6
5
8
7
9
10 12
11
|
|
|
|
|
|
|
|
|
BAL3
|
CP3
CP
|
|
IQV
BAL
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
________________________________________________________________________________________________________________________________
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
0
2
1
4
3
6
5
8
7
10 12
9 11
1
0
3
2
5
4
7
6
8
9 11
10 12
0
2
1
4
3
6
5
8
7
10 12
9 11
1
0
3
2
5
4
7
6
9
8
11
10 12
0
2
1
4
3
Upper Panel: correlation(SNRt-i;MSE_AEt); Lower Panel: correlation(SNRt;MSE_AEt-i)
i=0,…,12. NB: Correlations within (-0.2;+0,2) are statistically zero.
22
6
5
8
7
9
10 12
11
4.
Where Do We Stand?
So far, we have:
• Twelve time series made up by quarterly MSE (85/90Q1-09Q1)
pointing out that the AE model outperforms all the others all
the times. Despite of that:
• Five quarterly signal-to-noise ratios (85Q1-09Q1) point out
that great variety in expectations exists and persists. Then:
• SNR and MSE seems to co-move according to SNR => MSE.
This calls for more formal tests.
Specifically, I estimate a battery of bivariate VAR made up by one
SNR and one MSE (stemming from the best model) in order to
perform Granger, FE variance decomposition and Geweke tests
23
4.1
Granger Causality
24
4.2
No Instantaneous Feedback
25
5
Comments on Results
Granger-causality, Geweke’s instantaneous feedback and VD tests:
• Past, present and future values of MSE are not a determinant of the
dispersion across agents' expectations. Instead,
• Divergent expectations significantly affect the forecasting accuracy of
optimal econometric models. Then, interpreting the MSE as a proxy of
volatility,
• Disagreement across laypeople’s expectations
macroeconomic uncertainty, but not vice versa.
Granger-causes
These outcomes are in line with the literature on the extra information
content of the surveys, and contrast with some of the assumptions behind
i) the adaptive learning, ii) the predictor choice, and iii) the epidemiological
frameworks.
26
5a.
Comments on Results
Negative correlations
A more perturbed signal coming from the surveys leads to:
i) higher model-based MSE; ii) greater macroeconomic uncertainty
No model uncertainty, but high and persistent disagreement
Evidence strongly indicates that a relatively simple AE predictor
outperforms all the other proposed models all the time.
Evidence strongly indicates that laypeople expectations are persistently
heterogeneous.
Results are robust
to several SNR and models, including univariate and multivariate models
even estimated via optimal-size rolling windows.
27
6 Concluding Remarks
Should heterogeneity depend on model uncertainty only, UK citizens’
expectations should likely converge
Should lay consumers’ expectations be best-model-based, then SNR
i) should not significantly help predict optimal model-based MSE and
ii) should not follow univariate processes wrt past information about MSE
Since data show opposite findings, then there must be some additional
explanation behind the formation of disparate expectations
The identification of these disagreement-widening factors is in my research
agenda
28
THANK YOU!
29