Joint UNECE/Eurostat Meeting on Population and Housing Censuses Geneva, 13-15 May 2008 Census quality evaluation: Considerations from an international perspective Bernard Baffour and Paolo Valente UNECE Statistical.
Download ReportTranscript Joint UNECE/Eurostat Meeting on Population and Housing Censuses Geneva, 13-15 May 2008 Census quality evaluation: Considerations from an international perspective Bernard Baffour and Paolo Valente UNECE Statistical.
Joint UNECE/Eurostat Meeting on Population and Housing Censuses Geneva, 13-15 May 2008
Census quality evaluation: Considerations from an international perspective
Bernard Baffour and Paolo Valente UNECE Statistical Division
Presentation Outline
• Introduction • Quality definitions and dimensions • Assessment of dimensions of quality • Quality evaluation issues for different census methodologies • International perspective: What role for the International Organisations?
Definition of Quality and Census Quality
• Quality is defined as the ‘fitness for use’ of data • Census quality translates into assessment of information produced by the census • Census quality characterized in terms of six dimensions: • Relevance, Accuracy, Timeliness, Accessibility, Interpretability, Coherence,
Dimensions of Quality
• • • • • •
Relevance
: concerns the user requirements
Accuracy
: concerns the reliability and precision
Timeliness
: concerns the time frame
Accessibility
: concerns the dissemination strategy employed
Interpretability
: concerns how easy to understand the data is
Coherence
: concerns the conceptual integrity of the census data
Assessment of the Dimensions of Quality
• Accuracy: quantification of the different errors Coverage, content, operational errors: Coverage Assessment: Post-Enumeration Survey, Demographic Analysis, Comparison to Alternative Sources Content Assessment: Edit and Imputation Operational Assessment: Rigorous review of all census processes
Assessment of Coherence and Interpretability
• Internal Coherence – Validation and Verification – Checks for inconsistencies, incongruities • External Coherence – Comparison to other data sources • Interpretability aided by comprehensive metadata provision – Concepts, Methodology, Coding
Assessment of Accessibility, Relevance, and Timeliness
• Dissemination program – mix of free and tailor-made census products and services • Availability of (non-disclosive) microdata • Management of user requirements and expectations • Clear, realistic targets • User and public consultations and feedback
Quality Evaluation Issues for Different Census Methodologies
• Census methodology adopted influences the assessment of census quality • The same quality dimension may be measured in different ways for different types of censuses • Important implications of census methodology on quality evaluation
Quality Evaluation Issues for Different Census Methodologies
• Census methodologies: – Traditional census – Register-based census – Combination of registers + full enumeration – Combination of registers + sample data – Traditional enumeration with yearly updates of characteristics – Rolling census
Selected Quality Evaluation Issues: 1. Traditional Census
• Relevance: – Content of the census questionnaire – Selection of topics: users needs vs. constraints – Post census consultations • Accuracy: – Coverage and content errors evaluated with PE
S
, Demographic analysis, comparison with other sources • Timeliness: – Respect of planned calendar of output releases – May be difficult to achieve due to size of operations
Selected Quality Evaluation Issues: 2. Register-Based Census
• Relevance: – No questionnaire – Users needs vs. data available from registers – Implications of using data collected for admin. uses – Consultation and feedback from users • Accuracy: – Assess coverage and quality of data in registers – Assess data integration: Matching rates – Consistency of values from different sources – Comparison with data from last traditional census • Timeliness: – Different updating schedules for different registers
Considerations from an International Perspective
• Some countries have long tradition and adopt several evaluation methods • Others have limited experience and need to develop capacity • Challenges in evaluating quality of non-traditional censuses
Activities at international level
• CES Census Recommendations • UNSD Principles and Recommendations • Eurostat: Work on census quality criteria • UNSD: Plans for Handbook on census evaluation/PES
What should International Organisations do?
• Produce and disseminate guidelines, manuals, methodological material?
• Promote methodological work?
• Are standards needed on census quality evaluation?
Thank you!