Transcript SDMX Implementation in the European Statistical System - by Eurostat

SDMX Implementation in the
European Statistical System
P. Everaers, A. Götzfried
Eurostat
What is SDMX ?
• An information model delivering standardised
data/metadata structure definitions;
• Statistical guidelines standardising statistical
concepts and code lists, and establishing general
governance;
• An IT infrastructure supporting the SDMX-based
data and metadata exchanges.
SDMX = set of standards to be used for
data/metadata exchange and dissemination
Mr. Louis Marc DUCHARME
Ms. Mariana KOTZEVA
Mr. Bruno TISSOT
Mr. Stefan SCHWEINFEST
Ms. Haishan FU
Ms. Martine DURAND
Mr. Werner BIER
Costs/benefits of SDMX
The national/international data/metadata exchange processes
are improved through the use of internationally agreed
standards and the reuse of SDMX objects and IT infrastructure.
Costs: initial investment costs by statistical organisations;
maintenance costs of the SDMX standards/infrastructure carried
by the SDMX Sponsors.
Benefits: Reuse of the technical and statistical standards and of
the IT infrastructure decreases the costs for data/metadata
exchange and increases the data quality.
Main SDMX achievements in 2013/2014
• More and more internationally agreed Data Structure
Definitions (DSDs) for global use (e.g. 4 Data Structure
Definitions for National accounts);
• Various guidelines for the creation/maintenance/governance
of SDMX DSDs and other objects;
• SDMX Global Registry, storing these DSDs and other objects
opened in April 2014 (https://registry.sdmx.org);
• New SDMX statistical guidelines/codelists published on
sdmx.org;
• SDMX Reference Infrastructure more and more used by
countries as basic IT infrastructure.
SDMX deliverables planned for 2014/2015
• Internationally agreed validation language VTL as new SDMX
standard;
• Publication of the SDMX-JSON Data Exchange Format (easier
to use) (currently under public review);
• Redesign of the SDMX website;
• New SDMX implementations in international statistical
domains such as demography/population statistics;
• SDMX Global Conference increasing training/capacity
building actions (Bangkok, autumn 2015).
SDMX implementation
in the ESS
For data exchange
GNI
INT TRADE *
CENSUS
AVIATION
WATER
R&D*
FISHERIES *
JVS / LCI
MARITIME
ORCHARDS
/ PESTICIDES.
Gross National Income
POPULATION *
Job Vacancy Statistics
Labour Cost Index
PESTICIDES USE
EGR IS
TIC
Eurogroup register
Trade in currency
ESSPROS
TEC
WASTE
social protection statistics
Trade by Enterprise
Characteristic
NATIONAL A/Cs *
STS
Short-term Statistics
BOP *
EDUCATION *
FDI *
* Denotes that other International Organisations are involved.
Over 20 Eurostat domains are in various phases of SDMX implementation.
This concerns about 25% of all eligible Eurostat processes.
Coordinator not
yet agreed upon
Coordinated by
Eurostat
Coordinated by
IMF
Coordinated by
OECD
SDMX implementation
in the ESS
For reference metadata exchange
SDMX-based reference Metadata Structure Definitions (MSDs):
• Euro SDMX Metadata Structure (ESMS), and
• ESS Structure for Standard Quality reports (ESQRS).
Both ESS standards are implemented for more and more ESS
reference metadata flows; about 50 % of all national and
Eurostat files are already standardised.
International exchange of reference metadata still to
be put in place.
SDMX implementation
in the ESS
SDMX-based international data sharing
Objectives:
• reduction of the reporting burden on NSIs (data exchange only
with one international organisation);
• more efficient use of resources in the International
Organisations (IOs) concerned;
• improvement of the quality and consistency of official statistics
disseminated by the IOs involved (identical data value
disseminated by all IOs involved);
• improvement of the global data exchange and dissemination
processes thanks to the use of international standards.
International Data
Sharing
The international data sharing model (1)
Eurostat
Data Validation
EU Member
state
EU Member
state
EU Member
state
EU Member
state
European statistics: From national to Eurostat
International Data
Sharing
The international data sharing model (2)
U
S
E
R
S
IMF, UN, WB, ILO,
BIS, other IOs
OECD
Eurostat - ECB
EU countries
OECD countries
(non-EU countries only)
Other countries
(non-OECD countries only)
Eurostat as international hub for European statistics
International Data
Sharing
Necessary pre-conditions (1)
•
Internationally agreed statistical standards
SNA 2008, ESA 2010,
international manuals, etc.
•
BPM6,
UOE
questionnaire,
SDMX data structure definitions for global use
to be agreed upon
Integrated international data structure definitions need to
be agreed between the International Organisation and
national statistical organisations concerned (i.e. the
different use needs from International Organisations need
to be reconciled); a governance procedure is needed.
SDMX implementation
in the ESS
Necessary pre-conditions (2)
• Internationally agreed data validation
Data validation rules and data validation standards need to
be agreed upon; data validation should not be repeated by
several international organisations; the upcoming SDMX
standards should be used.
• Streamlined data exchange and dissemination
processes
National and international organisations need to agree upon
the frequency and timeliness for data exchange and data
dissemination, data quality management etc.; SDMX IT
infrastructure should be used.
• Formal
agreements
between
organisations necessary
international
SDMX co-operation/capacity building provided
by the ESS (1)
Creating and maintaining common SDMX data structures e.g.
with
• The SDMX Sponsors (IMF, ECB, OECD etc.)
• UN organisations such as UNSD, UNESCO, UNCTAD etc.
• ………………
Capacity building (training services) provided for
• The enlargement countries on SDMX in national accounts, July
2014, Skopje;
• The ENP South countries on SDMX, June 2014, Luxembourg
• The ENP East countries on SDMX, November 2014
Eurostat
SDMX co-operation/capacity building provided
by the ESS (2)
Capacity building (training services) provided for
• ECLAC/South American Countries, in particular Chile;
• The Statistical Centre for the Cooperation Council for the Arab
Countries of the Gulf (GCC- Stat)
• Asian countries within the OECD Expert Group meeting in
09/2014;
• Ecuador, Colombia on SDMX, autumn 2014
Attention: regular e-training provided by Eurostat
which is accessible globally
Eurostat
Useful links
Eurostat :
http://epp.eurostat.ec.europa.eu/portal/page/portal/eurostat/home/
European Statistical System :
http://epp.eurostat.ec.europa.eu/portal/page/portal/pgp_ess/ess/ess_news
SDMX Main Site :
http://sdmx.org/
SDMX Global Registry :
https://registry.sdmx.org/
Eurostat
Conclusions
SDMX is the outcome of a fruitful and long-standing
international co-operation delivering standards and an IT
infrastructure for data/metadata exchange and dissemination.
The uptake of SDMX accelerated in the last years as more
SDMX standards got available and as larger statistical domains
will use these standards.
The implementation of SDMX if official statistics needs to
progress to use its full potential.
Eurostat
Questions
Are you aware of these data sharing activities in
your organisation ?
What can be done from central data/metadata
services or from central IT services to facilitate data
sharing ?
Eurostat