Transcript Slide 1

Workshop on Metadata Standards and Best Practices
November 19-20th, 2007
Session 5
International Household Survey Network
Microdata Management Toolkit
Pascal Heus
Open Data Foundation
[email protected]
http://www.opendatafoundation.org
Outline
• Toolkit overview
• Demo
• Conclusions / Q&A
http://www.opendatafoundation.org
Open Data Foundation – IZA 2007/11
International Household Survey Network
• Partnership of international organizations seeking to
improve the availability, quality and use of survey
data in developing countries
• United Kingdom Department for International
Development (DfID), * International Labor
Organization (ILO), Partnership for Statistics in the
21st Century (PARIS21), United Nations Children
Fund (UNICEF), United Nations Statistics Division
(UNSD), World Health Organization and the Health
Metrics Network (WHO/HMN), World Bank
• Plays a major role in the adoption of DDI around the
globe, active in many developing countries
• Developer of the Microdata Management Toolkit
• http://www.surveynetwork.org
http://www.opendatafoundation.org
Open Data Foundation – IZA 2007/11
IHSN Activities
• Coordinating survey programs
–  Web based information on planned surveys
• Harmonizing concepts & methods
–  Planning, sampling, questionnaire design,
processing, data disclosure & confidentiality, etc.
• Maintaining a survey catalog
–  Web based central catalog
• Developing data dissemination tools
–  Microdata Management Toolkit
http://www.opendatafoundation.org
Open Data Foundation – IZA 2007/11
Toolkit Requirements
• User friendly software for microdata
• Facilitate metadata exchange (DDI, Dublin Core)
• Facilitate archiving (metadata and data, quality
control)
• Facilitate preservation/dissemination: network, CD /
DVD, web sites
• Works with common data formats
• Multilingual support
• Free or Inexpensive
• Availability of technical support and training
• Supported by national, international and research
communities
http://www.opendatafoundation.org
Open Data Foundation – IZA 2007/11
IHSN Toolkit
1
Import data and compile metadata
3
2
Import metadata and prepare CD-ROM
http://www.opendatafoundation.org
Open Data Foundation – IZA 2007/11
Generate HTML based CD-ROM
Toolkit Components
• Archiving: Metadata Editor (World Bank / Nesstar
Ltd.)
– To compile survey data, documentation and metadata in a
standard format (Nesstar/DDI). Free data reader for users.
– Built on Nesstar Publisher
• Dissemination: CD Builder (World Bank / Mark
Diggory)
– To facilitate the publication of survey data, documentation
and metadata on CD-ROM and on the web (transforms DDI
into HTML based navigation)
– Based on Eclipse Platform, open source
• Nesstar Explorer
– Free software
– Access metadata & data and export to common statistical
formats
• IHSN Tools
– Reporting, diagnostics and other utilities
http://www.opendatafoundation.org
Open Data Foundation – IZA 2007/11
What is the Metadata Editor?
•
•
•
•
•
•
•
•
DDI / Dublin Core specialized editor
Template driven
Enhanced version of the Nesstar Publisher
Import/Export common data files
Integrated interface, multilingual support
Metadata and data in single file
Export to DDI / DC
Licensing agreement
for developing countries
and IHSN members
http://www.opendatafoundation.org
Open Data Foundation – IZA 2007/11
What is the Metadata Editor?
http://www.opendatafoundation.org
Open Data Foundation – IZA 2007/11
What is the CD Builder?
• Publish survey
metadata, documents and data
on a CD-Rom or web site)
• Transforms DDI into an HTML
based interface
• User can customize the layout
(branding) and content of the CD (single or multisurveys)
• Open source application
• Build on the Eclipse Framework
• Based on DDI / Dublin Core
• Integrates with Metadata Editor
http://www.opendatafoundation.org
Open Data Foundation – IZA 2007/11
CD Builder Process
1
Create new CD-ROM Project
2
3
Add a survey to the project and
select its type and branding
Click the “Save” button to
generate the HTML interface
4
http://www.opendatafoundation.org
After a few minutes, your CD
Project is ready for publishing!
Open Data Foundation – IZA 2007/11
• Selecting a survey
consist in opening the
DDI-XML or Nesstar
file
• The survey “branding”
determines the overall
look and feel of the CD
• The survey “type”
determines the default
metadata content
Sample output
http://www.opendatafoundation.org
Open Data Foundation – IZA 2007/11
Nesstar Explorer
•
•
•
•
Free software
PDF philosophy
Access to survey metadata
Access to data (no need for
specialized software)
• Export to common formats
• Single file holds data and
metadata
http://www.opendatafoundation.org
Open Data Foundation – IZA 2007/11
IHSN Tools
• Free / Open Source
• Utilities to complement
existing software
• Integrates in Editor
• Diagnostics
• Reporting
• Metadata sharing
http://www.opendatafoundation.org
Open Data Foundation – IZA 2007/11
Demo / Conclusions
• DEMO
• Improves and facilitates documentation
preservation, cataloguing, metadata exchange,
dissemination, quality, etc.
• Users: survey producers, survey sponsors, data
archives
• Benefits: data producers, national & international
survey sponsors, survey data repositories, data
analysts, policy makers, DDI Community
• Provides a good example of what can be
accomplished with XML metadata
• Based on DDI 1.2.2 (not 3.0)
• Requires a license of Nesstar Publisher
• Rolled out in developing countries national statistical
agencies
• Good to get started with DDI
http://www.opendatafoundation.org
Open Data Foundation – IZA 2007/11