Transcript Guidelines on Statistical Business Registers Draft Chapter 8: Quality of SBR Caterina Viviano, Monica Consalvi ISTAT Meeting of the Group of Experts on Business.

Guidelines on Statistical Business Registers
Draft Chapter 8: Quality of SBR
Caterina Viviano, Monica Consalvi
ISTAT
Meeting of the Group of Experts on Business Registers,
2-4 September 2013, Geneva
Purpose
1. Provide guidance on how to evaluate quality for SBR
2. Describe quality criteria with an eye to the complexity of SBR
3. Provide measurement tools for quality evaluation
4. Suggest a conceptual framework to build up quality indicators,
some practical examples
5. Strategies for improving quality
1
Draft chapter 8: quality of SBR, Caterina Viviano, Monica Consalvi– Geneva, 2-4 September 2013
Overview
1. Quality components
2. Specificities of SBR in comparison with a survey (extensive
use of administrative data; heterogeneity and variability of
inputs, users, relevant units; relevance of technological
aspects; the continuous updating; etc..)
3. Frame errors: an overall description of the main errors and
consequences in statistical outputs
4. Tools for quality evaluation and metadata
5. The conceptual framework for quality indicators
2
Draft chapter 8: quality of SBR, Caterina Viviano, Monica Consalvi– Geneva, 2-4 September 2013
5. Framework for quality indicators (1)
The three dimensions for defining a Quality Indicator:
1. The BR’s phases (Input, Process, Output)
2. The quality components (relevance, accuracy, timeliness,
punctuality, accessibility and clarity, comparability, coherence)
3.The key factors (time, scope, sub-population, variable, criterion)
3
Draft chapter 8: quality of SBR, Caterina Viviano, Monica Consalvi– Geneva, 2-4 September 2013
5. Framework for quality indicators (2)
Example: to evaluate the completeness in 2-digit Nace code when
using a given input source to update economic activity in a given
sub-population (i.e. large ent.) of the SBR
I(t)% = % 2-digit Nace missing codes (out of the total number of
units) in large enterprises at time t in input source Tax register
I(t, t-1) = Percentage variation Var[I(t)%, I(t-1)%]
1. phase: Input (source=tax register)
2. component: Relevance-completeness
3. factors: Variable=economic activity; scope=enterprises; subpop=ent. 100+ employees; criterion=temporal consistency
4
Draft chapter 8: quality of SBR, Caterina Viviano, Monica Consalvi– Geneva, 2-4 September 2013
Issues for discussion (further in-depth analysis)
1. To better address quality components to the SBR peculiarities
2. to develop indicators to measure each single quality
dimension
3. Add quality reports as tool for quality evaluation
4. Due to the huge use of administrative data, is it necessary to
develop more in-depth the assessment of the quality of
administrative sources? To which extent quality of
administrative sources should be described in Ch. 6 or Ch.8 ?
5
Draft chapter 8: quality of SBR, Caterina Viviano, Monica Consalvi– Geneva, 2-4 September 2013
Good practices from countries (1)
SBR Quality Indicators - Italy (some examples taken from the quality declaration)
1. Quality of Input
Component – 1.1 Completeness
1.1.1 ) Address, s=CCIAA: Number of records ( % weight) with missing information
INDICATOR
COMPUTATION
It=2005
VI=It=2005 - It=2004
Records with missing % weight
address (cciaa)
(abs.number of records)
0.49
(37,408)
-0.03
2. Quality of process
Component – 2.1 Coverage
1) Number of records, s=CCIAA, not matched with the base MEF
INDICATOR
COMPUTATION
It=2005
VI=It=2005 - It=2004
Not matched Records % weight
(cciaa)
(abs. number of records)
5.25
(338,304)
0.03
3. Quality of output
Component – 3.2 Timeliness
3.2 Lag, in days, between dissemination time of BR and reference year of data
INDICATOR
COMPUTATION
It=2005
VI=It=2005 - It=2004
Timeliness
of
dissemination
6
BR Days of delay between the 492
dissemination time and the
reference year of data
+24
Good practices from countries (2)
SBR Quality Indicators – Colombia (some examples of proposed indicators)
•
•
•
•
•
•
•
•
Indicator 1
Name: Level of update
Objective: to know the rate of update for each of the economic sectors in
the frame.
Type of Indicator: Quality of the process
Variables used in the calculations are:
A_j: Total records update for sector j
B_j: Total records expected for update in sector j
The formula used for the calculation is:
•
𝐼1𝑗 =
•
•
Calculation Frequency: Annually
Tolerance ranges: Critical <= 70; 70> Fair <= 90; Satisfactory> 90.
𝐴𝑗
𝐵𝑗
∗ 100
Any other suggestions / good practices?
7
Contacts
Caterina Viviano, [email protected]
Monica Consalvi, [email protected]
8
Titolo intervento, nome cognome relatore – Luogo, data