Transcript Scanner Data

Scanner Data – next steps ahead
Berthold Feldmann – Eurostat
Price statistics; purchasing power parities; housing statistics
Structure of the talk
2

Scanner data – a new paradigm

Main lessons of the last two days

Challenges

Time table and conclusion
Scanner Data Workshop – Stockholm June 2012
Chapter 1
A new universe
Scanner data as source for consumer
price indices
Expected advantages
 Better quality of indices
 Low response burden of enterprises
 Productivity gains of National Statistical Institutes
 Data can be used for other purposes
 Purchasing Power Parities
 Average price levels
 Advanced economic analysis (e.g. substitution rates)
 Regional breakdown of CPI/HICP
4
Scanner Data Workshop – Stockholm June 2012
Scanner data as source for consumer
price indices
Expected disadvantages
 Potential biases in CPI / HICP
 High investment costs (IT)
 Some rules of European Implementing Regulations
may no longer be applicable
 Treatment of rebates
 Quality adjustment
 Direct collection of “price observations”
5
Scanner Data Workshop – Stockholm June 2012
Chapter 2
Lessons learnt in the Workshop
Some general conclusions
 We are still at an early stage in the use of scanner
data as a data source
 Several problems still need to be solved
 Scanner data appears quite promising for the collection
of Multipurpose Consumer Price Statistics
 Scanner data cannot be used for all COICOP groups
 Scanner data offers high quality information of actual
transactions
7
Scanner Data Workshop – Stockholm June 2012
Conclusions on discounts
 Discounts are an important stimulus for increased sales
 important for consumers
 Scanner data offers new opportunities for a more
precise treatment of discounts
 compared to traditional price collection
 For HICP – the inclusion of all non-discriminatory price
reductions should be the rule
 Precise recommendations should be drafted a.s.a.p.
 We might need to evaluate the existing regulatory
framework on price reductions
8
Scanner Data Workshop – Stockholm June 2012
Practical experience
 The exchange of views and experiences between NSIs
could save a lot of time
 Avoid reinventing the wheel
 Full support from the top management of retailers is
mandatory
 Good collaboration with data providers is a key point
 More volatility and some short and medium term
differences when changing the price concept
(transaction price versus shelf price)
 Major challenge: high attrition rate (up to 30%)
 Promising results with the use of “internal” classifications
as links between EAN and COICOP
9
Scanner Data Workshop – Stockholm June 2012
Computation methods
 From GEKS to RYGEKS
 From RYGEKS to imputed Tornquist RYGEKS
 Three different methods are tested for the
imputation of prices for new and disappearing items
Conclusion
 at the lowest level of computation we might need to
go beyond Laspeyres ... ...
 Mainly used for benchmarking
10
Scanner Data Workshop – Stockholm June 2012
Chapter 3
Challenges
Link EAN - COICOP
 EAN differ from country to country
 A high proportion of EAN have a life span of less
than a year
 This results in a high turnover rate of product types
 Automating the link of EAN to COICOP level 5 or 6
is not a trivial task
 In the long run a harmonisation of EAN across
Europe would be desirable
12
Scanner Data Workshop – Stockholm June 2012
Use a sample or all data?
 Current practise differs between countries
 A harmonised approach should be achieved
 Advantages and disadvantages of the two
approaches should be discussed carefully
 Using a sample is closer to the traditional
price observations
 Using all data might offer new opportunities
13
Scanner Data Workshop – Stockholm June 2012
Automation of editing EAN data
 Problem of many products disappearing and
others emerging within a year
 Practice of some advanced National Statistical
Offices can give guidelines
14
Scanner Data Workshop – Stockholm June 2012
Weighted or not weighted average?
(product level)
 Should quantity information be used at basic
calculations?
 Quantities may show huge fluctuations
 A Europe-wide recommendation should be
achieved
15
Scanner Data Workshop – Stockholm June 2012
Scanner data as a source for weights
(level of product groups)
 Quantities from scanner data could be used as
weights at the lowest level (COICOP 5 or 6)
 In any case, only annual averages should be
used as weights
 According to Implementing Regulation no
changing weights from month to month
 Volatility of the data?
16
Scanner Data Workshop – Stockholm June 2012
Further information collected
 Scanner data contains more than prices and
quantities
 Product description is important for EAN –
COICOP link
 Information on discounts may be useful for
further data analysis for economic purposes
17
Scanner Data Workshop – Stockholm June 2012
Scanner data versus price collectors
 Different results are reported from the two
alternative sources for consumer price indices
 A thorough analysis is required to explain these
differences
 Algorithms to be found for possible bias in
scanner data
18
Scanner Data Workshop – Stockholm June 2012
Legal obligations
 National law may set different obligations in
different countries
 Are stores obliged to supply scanner data?
 Can European legislation help?
 COUNCIL REGULATION (EC) No 2166/1999
The statistical units … called upon by the Member States to
cooperate in the collection or provision of basic information
are obliged to give honest and complete information at the
time it is requested and allow the organisations and
institutions responsible for compiling official statistics, on
request, to obtain information at the level of detail
necessary ………
19
Scanner Data Workshop – Stockholm June 2012
IT environment
 High investment costs to start the regular usage
of scanner data
 Economies of scale possible if similar solutions are
used in different countries
20
Scanner Data Workshop – Stockholm June 2012
Chapter 4
Provisional time table
A proposal for a timetable
 Continue intensive discussion among experts
 Enhance transparency on current practice in all European
countries
 Eurostat will create an overview of current practice
in summer 2012
 Discussion at the Working Group meeting in
October 2012
 Meeting of the Task Force “Multipurpose price
statistics” shall draft first recommendations in
January 2013
 Incorporation of basic rules on the use of scanner
data in the new HICP Regulation
22
Scanner Data Workshop – Stockholm June 2012
Conclusion
 The use of scanner data offers several advantages
for future price statistics (not only consumer price
indices)
 Some issue connected with the use of scanner data
are not easy to solve and require a harmonised
approach
 The European legal framework might need to be
assessed if some rules need an adaptation for the
use of scanner data
23
Scanner Data Workshop – Stockholm June 2012
Thank you for
your attention!
Any Questions ?