Transcript RECENT DEVELOPMENT OF SORS METADATA REPOSITORIES FOR FASTER AND MORE TRANSPARENT PRODUCTION PROCESS Work Session on Statistical Metadata 9-11 February 2004 Andreja Arnič and Julija Kutin Statistical.

RECENT DEVELOPMENT OF SORS
METADATA REPOSITORIES FOR
FASTER AND MORE TRANSPARENT
PRODUCTION PROCESS
Work Session on Statistical Metadata
9-11 February 2004
Andreja Arnič and Julija Kutin
Statistical Office of the Republic of Slovenia
Overview
 Introduction
 Recent development of SORS metadata repositories
 Classification server
 METIS repository
 Findings
INTRODUCTION








metadata within the statistical production process
centralized repositories
SWOT analysis
E-CoRE
further development of the classification server (KLASJE)
notifications
e-government: searching within classifications
E-KLASJE: feasibility study
RECENT DEVELOPMENT OF
METADATA REPOSITORIES
 The main goal: develop an efficient and effective,
standardized and integrated system for collecting and
editing metadata.
 From that system metadata could be quickly and easy
exported and used in other applications and programs.
RECENT DEVELOPMENT OF
METADATA REPOSITORIES
Whole life cycle of a statistical survey to be covered by
metadata or a statistical information system:
 including design
 implementation
 operation
 monitoring
 maintenance
 evaluation
CLASSIFICATION SERVER
 KLASJE enables various contents and time
comparisons between classification versions.
 KLASJE represents the basic metadata infrastructure
 enabled the standardization of processes
 directly influenced the quality of statistical data
 at the end of January 2004: 730 classifications and 75
concordances
CLASSIFICATION SERVER
Further development of the Classification server and egovernment initiative
 automatic load of dimensional tables in data warehouse
 via the Internet:
 additional search facilities
 concordances available
 e-government initiative: feasibility study
 notifications revealing changes in classifications
METIS REPOSITORY
 In production from 2003 and we are working on
 Questionnaires and Methodology (variables)
 Application functionality:
 import, preparation and export metadata for special use.
 Standard solutions: technology, content
 Standardized statistical process (standard tools for each
sub process)
 Metadata play a major role in dissemination of statistics
(standard tools and approaches)
 The other functions of metadata in statistical process
METIS REPOSITORY
We started with detailed analysis of data collection sub
process and revision of this module:
 Eliminations,
 The others (IQML, Blaise ) have some metadata we don’t
have.
We analyzed 7 typical SORS’s questionnaires from
methodology definition to the FOR – micro database.
1. Methodology
Contents
resources
legal basis
Steps in survey design and implementation
- from general framework
to edited (clean) microdata
5. Final observation register (FOR)
creation
a) File systems
update
b) Microdata
bases loading
Directory of folders
2. Survey design
Questionnaire
Variables,
other definitions
Sampling
design
3. Survey processing
4.1. Data editing
4..2. Data analysis
Questionnaires
Editing
and validation
rules
Estimated
aggregations –
requirements
Automation
of editing rules
Aggregations –
IT support
Data
validation
Imputation
Auxiliary
information
Questionnaire
dispatch
Raw data
collection
Raw data
entry
Data
editing
Weighting
Data analysis
Data
quality
assessment
Specific processing in
each survey instance
Data matrix
definition
Database
creation
Documentation
Files update
Database
loading
METIS REPOSITORY
 Some possibilities were offered:
 electronic form as option next to paper form,
 reduce periodicity,
 reduce frequency,
 improve/introduce explanations/instructions,
 simplifications,
 integrations,
 adapting forms to source administrations / common
practices of enterprises.
METIS REPOSITORY
 Then we found out a lot of problems:
 layout of questionnaires,
 general data on the questionnaire,
 lack of documentation available,
 difficult or impossible to map survey data and respondents
list for the same reference period without personal contact.
FINDINGS
 The statistical process needs to be constantly analyzed and




modernized.
Regular training and cooperation in international exchange of
knowledge and experience needs to be organized.
Documents processes and procedures need to be standardized.
Deadlines need to be set and monitored.
Users and producers satisfaction needs to be measured.
We are working on the pilot from methodology definition to the
FOR for one survey in witch we will try to define the metadata
in each sub process.