Implementation of Quality Assurance Activities in Eurostat Third CCSA Data Quality Conference
Download ReportTranscript Implementation of Quality Assurance Activities in Eurostat Third CCSA Data Quality Conference
Implementation of Quality Assurance Activities in Eurostat
Third CCSA Data Quality Conference Rome 7-8 July 2008
A. Baigorri, M. Hahn, H. Linden Eurostat
1
Structure of the presentation
Quality assessments in Eurostat: methodology and implementation Redesigned Quality Barometer: short term and long term perspectives Plans to enhance quality assurance in the European Statistical System
2
Quality assessments methodology Each process is matched with an assessment category
Self Assessment Prod. unit: fills in Checklist, produces outputs DDG02: provides support on demand Supported Self-Assessment Peer review Prod. unit: in control of assessment, Same as Supported Self-Assessment participates in all steps with one or more peers (int/ext) DDG02: pre-fills the Checklist, taking part in all steps produces the outputs Rolling review A comprehensive review based on the checklist implemented by ext.contractor, involving partner & user surveys and cost-benefit analysis Process 1 Process 2 Process 3
Characteristics:
- Periodicity - Legal Basis - Output - ESTAT intervention -availability of quality report Process 4 Process 5
3
Quality Assessments Implementation Plan: Principles
Minimize burden for units
– – – Test of the approach in advance.
Support from unit DDG-02 Flexibility
Build on existing information
– – Quality reports Process analysis
Integrate QA with other horizontal activities 4
III. CONFORMITY
Integrating Quality Assessments
User Requirement Standards
Labelling Quality assessments Rolling reviews Evaluation function Cost/benefit 5YP II. EVALUATION
CVD steps
ASSESSMENT CHECKLIST
| | | | | | Quality Barometer
MEASUREMENT description Quality reports SDMX User satisfaction survey
Best practices Quality diagram Improvement actions
AMP Production processess Statistical products User perception Eurostat Institutional/Legal environment 5
Quality Assessments Implementation Plan: Support
Training
- in-house courses on data quality reporting - lunchtime presentations on the ESPAC - peer review results - seminars on quality assessments
Communication
- intranet site on quality - regular information on Cybernews - Eurostat infos - dissemination of good practices best practices
6
Quality Assessments Implementation Plan: Agenda 2008
Sector
Quarterly non-financial accounts for general government Quarterly Institutional Sector Accounts Prevention of illegal migration and stay statistics Asylum statistics Waste and Recycling Statistics International Trade in Services M-3 and M-1 questionnaires Harmonised unemployment rates Household Budget Survey Quarterly financial accounts for general government ICT usage in enterprises Structural Business Statistics-A. 1 to 4 Annual Questionnaires Quantities Farm structure survey Electricity and gas prices Forestry statistics Meat and livestock statistics - livestock Meat and livestock statistics - slaughterings Meat and livestock statistics - production forecasts Meat and livestock statistics - supply balance sheets PRODCOM - Industry Production LUCAS Catch Statistics E-communication. telecommunication Landing Statistics Aquaculture Statistics Fishing Fleet Register (from DG FISH data base) Demographic data Food safety: Statistics on Control and monitoring Activities Food safety: Organic farming statistics System of health accounts (SHA)
Id
C5-4 C2-2 F1-5 F1-3 E3-1 C4-4 G4-3 F2-3 F3-4 C5-3 F6-2 G1-1 G4-1 E1-3 G4-4 E3-13 E2-4 E2-5 E2-6 E2-7 G1-6 E1-2 E2-16 F6-3 E2-17 E2-18 E2-19 F1-1 F5-2 F5-3 F5-4
QR
Y P Y Y Y P P Y Y-5 years Y Y-annual P L Y Y Y P/F P/F P/F P Y Y P N P N P N N N N
A Cat
SSA SSA SA SA SSA SSA SA SA SSA SSA SSA SSA SSA SSA SA SA SA SSA SA SA SSA PR SSA SSA SA SA SA SA SA SA SA
Estimated start
03/03/2008 23/03/2008 01/04/2008 03/04/2008 08/04/2008 14/04/2008 21/04/2008 01/05/2008 15/05/2008 19/05/2008 01/06/2008 01/06/2008 01/06/2008 01/07/2008 01/07/2008 04/08/2008 01/09/2008 01/09/2008 01/09/2008 01/09/2008 01/10/2008 13/10/2008 01/11/2008 01/11/2008 01/12/2008
Sector
Causes of death statistics (COD) ESAW: Accidents at work statistics Railway transport Air transport
Id
F5-7 F5-8 G5-2 G5-3
QR
N N F Y
A Cat
RR RR RR RR
Timing
February - October 2008 February - October 2008 April 2008 - March 2009 April 2008 - March 2009
7
Quality assessments in Eurostat: next steps
Evaluation of the 2008 program of assessments
Need for adjusting the instruments used Indentify recurrent weaknesses in statistical processes to feed improvement actions into the AMP Promote initiatives to be implemented by horizontal units to facilitate quality improvements by production units.
Elaboration of the 2009 plan
focussed on more objective assessments as peer and rolling reviews.
8
Redefined Quality Barometer: characteristics
Purposes: - to measure the quality level at both EU level and country level; - to monitor change; - for comparisons across statistical domains and across countries.
Users: - production side: top-management, departments/units, survey managers.
user’s side: international organisations, policy makers, researchers, etc.
Scope: - cover all types of statistical processes and their statistical outputs; - the quality of key data (or main time-series); - monitor quality on an annual basis.
9
Redefined Quality Barometer: short term perspective
Restricted QB based on quality indicators that are: - applicable to most statistical domains; - covering the main quality dimensions; - well defined and easy to provide; and - easy to interpret and understand.
The restricted set of Quality Indicators prioritised for the QB: - Rate of available statistics (R2) - Coefficient of variations (A1) - Unit response rates (A2) - Average size of revisions (A7) - Time lag reference period and date of first results (T2) - Number of accesses to databases (AC2) - Rate of completeness of metadata (AC3) - Length of comparable time series (C1) 10
Compilation of a restricted Quality Barometer
Including many statistical processes but only covering the restricted set of quality indicators
The Euro- SDMX Metadata Structure should be the main source of information
Continue the MS’s Expert Group and Eurostat network for methodology for indicators and analysis of results
First RQB planed for the end of 2008 11
Redefined Quality Barometer: long term perspective
I. INTRODUCTION IIa. Social statistics Labour Force Statistics
Relevance Accuracy Timeliness Comparability Coherence Summary for Labour Force Statistics
IIb. Business statistics III. OVERALL SUMMARY 12
Enhancing quality assurance in the ESS
Regulation on European Statistics and Article 10 “Statistical quality”:
Quality of the MS’s and Eurostat data shall be assessed according to the ESS quality dimensions.
- Specific quality requirements, such as target values and minimum standards for the statistical production, in sectoral legislation.
Euro-SDMX Metadata Structure (ESMS)
- Harmonizes quality reporting integrating the ESS quality dimensions - improves metadata sharing and information to users on quality of statistics
Development and sharing of “best practices” for statistics production 13
Enhancing quality assurance in the ESS
Revision of the Eurostat/ESS quality reporting guidelines:
- Standard Quality Report - Handbook: How to make a Quality Report
Facilitate the implementation of CoP improvement actions plans by NSIs
- Eurostat 2009 grants on the implementation of ESS standards, handbooks, guidelines and application of tools - ESS Quality Coaching focusing on the implementation of quality management and quality assurance
14
Thank you for your attention!
http://ec.europa.eu/eurostat/quality
15