Transcript OECD

Impact and timing of revisions for
seasonally adjusted series relative to
those for the corresponding raw series
OECD / Eurostat taskforce on performing
revisions analysis for sub-annual economic
statistics
Richard McKenzie, Zakia Adam & Michela Gamba
OECD
June 25-27 2007
OECD Short-Term Economic Statistics Working Party
Item VI of task-force Terms of Reference
“….. be able to break down the results of a revisions
analysis study to quantify the impact of different
(homogenous groups of) sources of revision. As a
minimum the guidelines must describe how the
effects of seasonal adjustment on revisions can be
separated and quantified in relation to other
reasons.
June 25-27 2007
OECD Short-Term Economic Statistics Working Party
Description of OECD study
• Compare revisions analysis for raw and SA series
for selected countries, IIP & Retail trade
– Size of raw and SA revisions at different intervals
– Occurrence of revisions to raw and SA series
– Bias in revisions to raw series relative to SA series
– Revisions of raw and SA series for year-on-year growth
rates
– Provide a tool which would allow countries to perform
similar studies with their data
June 25-27 2007
OECD Short-Term Economic Statistics Working Party
Countries chosen
• Balance of EU and non-EU countries
• Include some countries where seasonal adjustment
is performed by OECD (forward factor)
• Most importantly, need vintage series for all
monthly snapshots with few missing updates
– Last criteria presented the biggest problem due to lack of
availability of vintages of raw series
June 25-27 2007
OECD Short-Term Economic Statistics Working Party
Index of industrial production
• United States, Japan, Korea, Belgium, Czech
Republic, Finland, France, Netherlands, Poland
• Finland, Belgium, Netherlands – raw data is in fact
working day adjusted
• Belgium, Czech Republic, Poland – SA done by
OECD (forward factor method)
June 25-27 2007
OECD Short-Term Economic Statistics Working Party
Retail Trade Volume
• Canada, Japan, Korea, Mexico, Norway,
Sweden
• SA done by OECD for Mexico
June 25-27 2007
OECD Short-Term Economic Statistics Working Party
Size of raw vs SA revisions for monthon-previous-month growth rates
• Consider relative absolute mean revision (RMAR)
• Review for different revision intervals (is the
relationship stable across time?)
June 25-27 2007
OECD Short-Term Economic Statistics Working Party
0.70
Relative mean absolute revision (RMAR) of IIP for USA
0.60
raw sa
0.50
0.40
0.30
0.20
0.10
0.00
M2_P
M3_P
M3_M2
Y1_P
Y2_P
L_P
Y2_Y1
Y1_M2
Relative mean absolute revision (RMAR) of IIP for Finland
0.90
0.80
raw
sa
0.70
0.60
0.50
0.40
0.30
0.20
0.10
0.00
M2_P
M3_P
M3_M2
Y1_P
Y2_P
L_P
Y2_Y1
Y1_M2
Relative mean absolute revision (RMAR) of IIP for Korea
0.70
0.60
raw
sa
0.50
0.40
0.30
0.20
0.10
0.00
M2_P
M3_P
M3_M2
Y1_P
Y2_P
L_P
Y2_Y1
Y1_M2
RMAR ratio between raw and SA for Y1_P
0.60
Retail trade
Index of Industrial Production
0.50
0.40
0.30
0.20
0.10
SWE
NOR
MEX
KOR
JPN
CAN
USA
POL
NLD
KOR
JPN
FRA
FIN
CZE
BEL
0.00
Size of raw vs SA revisions for monthon-previous-month growth rates
• Revisions of raw consistently smaller than SA by
similar magnitude for different revisions intervals
– Ratio differs across countries, WDA is evident (larger
revisions relative to true raw series)
• Similar picture for Retail Trade, but revisions to raw
series even smaller in proportion of those to SA
• Overwhelming evidence that majority of revisions to
SA series is from process of seasonal adjustment
• ….. But month on month growth rates of raw series
are of little use to users / analysts …..
June 25-27 2007
OECD Short-Term Economic Statistics Working Party
Occurrence of revisions for raw and
SA series
• How many months after first published values do
we expect data to be revised?
– Raw series should only be affected by late data, fixing
errors etc. (especially if direct from one survey with no
models etc.)
– Revisions to SA series will depend on revisions to raw
series and process of seasonal adjustment
• Revisions in longer term may also affect raw series
(e.g. rebases, methodological changes,
benchmarking etc.)
June 25-27 2007
OECD Short-Term Economic Statistics Working Party
Poland total non zero revisions for IIP
80
occurrence_raw
occurrence_sa
70
60
50
40
30
20
10
0
M1_P
M2_M1
M3_M2
M4_M3
M5_M4
M6_M5
M9_M6
Y1_M9
Y2_Y1
Y3_Y2
Japan total non zero revision for IIP
70
occurrence_raw
occurrence_sa
60
50
40
30
20
10
0
M1_P
M2_M1
M3_M2
M4_M3
M5_M4
M6_M5
M9_M6
Y1_M9
Y2_Y1
Y3_Y2
Occurrence of revisions for raw and
SA series
• IIP
– occurrence of revisions similar for raw and SA for USA,
Netherlands, Korea, Japan
– But raw less often revised than SA for Belgium, Czech
Republic, Finland, France and Poland
• Retail trade
– Occurrence of revisions similar for Korea, raw a bit less
for Japan and raw much less for Canada, Mexico,
Norway & Sweden
June 25-27 2007
OECD Short-Term Economic Statistics Working Party
Total non zero revision for IIP for WDA countries (RAW)
BEL
FIN
FRA
NLD
100
90
80
70
60
50
40
30
20
10
0
M1_P
M2_M1
M3_M2
M4_M3
M5_M4
M6_M5
Total non zero revision for IIP for NON WDA countries
(RAW)
CZE
JPN
KOR
POL
USA
100
90
80
70
60
50
40
30
20
10
0
M1_P
M2_M1
M3_M2
M4_M3
M5_M4
M6_M5
Total non zero revision for IIP for NON WDA countries
(RAW)
CZE
JPN
KOR
POL
USA
100
90
80
70
60
50
40
30
20
10
0
M1_P
M2_M1
M3_M2
M4_M3
M5_M4
M6_M5
Total non zero revision for Retail trade for all countries
(RAW)
CAN
JPN
KOR
MEX
NOR
SWE
100
90
80
70
60
50
40
30
20
10
0
M1_P
M2_M1
M3_M2
M4_M3
M5_M4
M6_M5
Occurrence of revisions: raw vs SA
• Some evidence that revisions to raw series less
likely the further from first published data than for
SA
• More evident for Retail than IIP and likelihood of
revisions to raw for Retail trade is lower
– Reflecting again that Retail trade data more likely to be
compiled from single survey in most countries
• Further work needed to understand the impact SA
method has on occurrence and size of revisions
• OECD tool now developed to break down revisions
on a month by month basis
June 25-27 2007
OECD Short-Term Economic Statistics Working Party
Are revisions statistically significant?
• Ideally revisions should centre around zero over
time (i.e. equally likely to be + or - with mean
revision close to 0)
– If this is not true then reasons causing this tendency in
the compilation process should be found
• Should not expect a bias to exist in the raw series
but not in the SA and vice versa
June 25-27 2007
OECD Short-Term Economic Statistics Working Party
Mean revision statistically significant
for raw but not SA (& vice versa)
• IIP
– Some evidence found for Belgium and Korea but very
much on the margin
• Retail trade
– Some evidence for Canada but mean revision still very
small
– For Korea, raw series mean revision significant (at 10%
and 5% level) for all revisions intervals but not for SA
> Mean revisions smaller but growing with length of
interval (SA is in same direction and close to SIG)
June 25-27 2007
OECD Short-Term Economic Statistics Working Party
Which short term measures should we
advise users to focus on?
• Are first estimates of month-on-previous-month growth rates
of SA series for IIP and Retail Trade reliable enough to
enable informed decision making?
– Not in most countries! First estimates of MoM growth rates revised by
2/3 initial value on average after one year (95% countries revised by
more than 2/5)
– Shows the value that revisions analysis can provide to users
• Previous study suggested that users should focus on yearon-year growth rates for short-term analysis as these are
more robust with respect to revisions
June 25-27 2007
OECD Short-Term Economic Statistics Working Party
Revisions to year-on-year growth rates
• Which series should year-on-year growth rates be
calculated on?
– OECD Data presentation handbook recommends YoY
growth rates to be calculated from raw or WDA series if
available
• What about robustness of raw / WDA and SA YoY
growth rates with respect to revisions?
June 25-27 2007
OECD Short-Term Economic Statistics Working Party
MAR of M3_P for IIP for WDA countries - YoY
raw SA
2.50
2.00
1.50
1.00
0.50
0.00
BEL
FIN
FRA
NLD
MAR of M3_P for IIP for Non WDA countries
raw
0.60
0.50
0.40
0.30
0.20
0.10
0.00
CZE
JPN
KOR
POL
USA
SA
MAR of M3_P for Retail trade
0.50
raw SA
0.40
0.30
0.20
0.10
0.00
CAN
JPN
KOR
MEX
NOR
SWE
Revisions to year-on-year growth
rates: Conclusion / Future work
• Some weak evidence to suggest that YoY are more
robust with respect to revisions for raw series
compared to SA
– Thus supports recommendations in the handbook
– More empirical research would be interesting for users,
particularly for WDA series
June 25-27 2007
OECD Short-Term Economic Statistics Working Party
Conclusions / Future work
• Relative mean absolute revisions (RMAR) for
month-on-month growth rates are much lower for
raw relative to SA series
– Process of seasonal adjustment accounts for more than
half of RMAR for the SA series, sometimes up to 90%
and indeed still more than 50% in comparison to WDA
series
– …… but mean average revision is still generally higher
for RAW series and month-on-month growth rates for raw
series are not much use for short-term analysis
June 25-27 2007
OECD Short-Term Economic Statistics Working Party
Conclusions / Future work
• Likelihood of revisions to raw series (MoM growth
rates) drops considerably after the first month but
tends to stabilise at lower level in following months
• Likelihood of short-term revisions to SA series
should depend on method used but more work
needed to assess the relative impacts
• New OECD tool allows analysis of revisions to raw
series on a month-on-month basis up to 6 months
from first published value
June 25-27 2007
OECD Short-Term Economic Statistics Working Party
Future work
• Disseminate results on OECD website, prepare
paper and discuss implications for taskforce
recommendations
• Use detailed revisions triangles to try and identify
and quantify other reasons for revisions?
• OECD also conducting a study to look at the impact
of improvements in timeliness on revisions
June 25-27 2007
OECD Short-Term Economic Statistics Working Party
THE END
June 25-27 2007
OECD Short-Term Economic Statistics Working Party