Transcript The ESS VIP project on Validation - CROS

ESS VIP project on
Validation
Presented by: Pedro Díaz
EUROSTAT
Eurostat
CONTENT
•
•
•
•
•
Scope & objectives
Business case
Link with previous projects & maturity
Proposed approach & milestones
Next steps
Eurostat
Scope & objectives
• Documentation / Standardisation
• Methodological analysis of Data Validation
• Distribution of responsibilities in the production chain
• Preparing for IS/IT solutions and architecture
Eurostat
Scope & objectives
• Coherent validation policy in the different
statistical domains, in cooperation with MS.
• Standard validation language
• Solutions for complex validation actions to
ensure:
• Coherence between data files
• Coherence between Member States
• Integrity of the ESS data
Eurostat
Scope & objectives
More efficient
production
chain
• Clearly attributed
responsibilities
Contribution
to the VISION:
Development
of functional
specifications
Eurostat
Standard
• Description of
validation steps
• Description of
validation
language
Common tools
to be shared
within the ESS
Business case: Need for change
• Two steps circular validation process
• Step 1: Validation in Member States
• Step 2: Validation in Eurostat
• Lack of harmonisation
• In documentation (validation procedures & rules)
• In coordination: "Validation ping pong"
• In software solutions
• Subjective assessment of data quality
• Risk of low quality results
Eurostat
Link with previous projects & maturity
Deliverables of VIP –Validation (until end 2012)
Guidelines for documentation of validation steps
Structured inventory of validation rules used in Eurostat
Template to deal with error messages
Human Understandable Syntax for validation rules
Functional specifications for adapting existing IT tools
Awareness actions
Eurostat
Link with previous projects & maturity
• Favourable reaction of Member States in
• ITDG: Oct 2011
• DIME: March 2012
• ESSC: May 2012
• Working Groups
Eurostat
Proposed approach & milestones
Packages:
1 - VIPV
IMPLEMENTATION
• Implementing
methodological
outcomes of VIPV
Phase I
2 - VIPV DATA
MODEL –
Household Survey
• Data model and
validation for
Household micro
data
3 - VIPV TOOLS
• Validation solutions
and other related IT
services
4 - VIPV EXTENSIONS AND GENERAL COORDINATION
Eurostat
Proposed approach & milestones
Project team
Decentralised
(Working Groups)
Steering Committee
High level monitoring
(ESSC)
Workshops with users
Methodological / IT
Solutions
Assessment and
monitoring per domain
Links with other ESS VIP
Validation
Warehouses
SIMSTAT
Eurostat
Shared
services
…
Next steps: 2013 - 2014
• Data sent to Eurostat already validated (push mode,
in-file validation, micro-macro)
• Enablers:
•
•
•
•
•
Descriptive syntax
Governance mechanism with Member States
Implementation monitoring
Adaptation of IT Tools
Micro data specificities
Eurostat
Next steps: 2014 - 2020
1.
Extended validation (across files, time series)
–
–
2.
Mirror checks: One country checking after countries' data
–
–
–
3.
Need for shared databases
Need of shared tools
Confidentiality treatment
Enhancement of transmission tools
Change roles (Member States, Eurostat)
Pull and shared modes
–
–
–
Common warehouses
Census hub technologies
Change roles (Member States, Eurostat)
Eurostat
Next steps
Why an ESS VIP?
Role of NSI's in project phases
Methodological Task Force
• Validate phase-1 deliverables
• Advise on implementation/monitoring
• Definition of ESSNet work
1 - Implementation
2 – Data model HHS
3 - Tools
ESSNet
4 – Extensions and
General coordination
Eurostat
Next steps
Why an ESS VIP?
Role of NSI's in project phases
Methodological Task Force
• Validate phase-1 deliverables
• Advise on implementation /
monitoring
• Definition of ESSNet work
1 - Implementation
2 – Data model HHS
3 - Tools
ESSNet
4 – Extensions and
General coordination
Eurostat