Joint UNECE/Eurostat Meeting on Population and Housing Censuses Geneva, 13-15 May 2008 Census quality evaluation: Considerations from an international perspective Bernard Baffour and Paolo Valente UNECE Statistical.

Download Report

Transcript Joint UNECE/Eurostat Meeting on Population and Housing Censuses Geneva, 13-15 May 2008 Census quality evaluation: Considerations from an international perspective Bernard Baffour and Paolo Valente UNECE Statistical.

Joint UNECE/Eurostat Meeting on Population and Housing Censuses Geneva, 13-15 May 2008

Census quality evaluation: Considerations from an international perspective

Bernard Baffour and Paolo Valente UNECE Statistical Division

Presentation Outline

• Introduction • Quality definitions and dimensions • Assessment of dimensions of quality • Quality evaluation issues for different census methodologies • International perspective: What role for the International Organisations?

Definition of Quality and Census Quality

• Quality is defined as the ‘fitness for use’ of data • Census quality translates into assessment of information produced by the census • Census quality characterized in terms of six dimensions: • Relevance, Accuracy, Timeliness, Accessibility, Interpretability, Coherence,

Dimensions of Quality

• • • • • •

Relevance

: concerns the user requirements

Accuracy

: concerns the reliability and precision

Timeliness

: concerns the time frame

Accessibility

: concerns the dissemination strategy employed

Interpretability

: concerns how easy to understand the data is

Coherence

: concerns the conceptual integrity of the census data

Assessment of the Dimensions of Quality

• Accuracy: quantification of the different errors Coverage, content, operational errors:  Coverage Assessment: Post-Enumeration Survey, Demographic Analysis, Comparison to Alternative Sources  Content Assessment: Edit and Imputation  Operational Assessment: Rigorous review of all census processes

Assessment of Coherence and Interpretability

• Internal Coherence – Validation and Verification – Checks for inconsistencies, incongruities • External Coherence – Comparison to other data sources • Interpretability aided by comprehensive metadata provision – Concepts, Methodology, Coding

Assessment of Accessibility, Relevance, and Timeliness

• Dissemination program – mix of free and tailor-made census products and services • Availability of (non-disclosive) microdata • Management of user requirements and expectations • Clear, realistic targets • User and public consultations and feedback

Quality Evaluation Issues for Different Census Methodologies

• Census methodology adopted influences the assessment of census quality • The same quality dimension may be measured in different ways for different types of censuses • Important implications of census methodology on quality evaluation

Quality Evaluation Issues for Different Census Methodologies

• Census methodologies: – Traditional census – Register-based census – Combination of registers + full enumeration – Combination of registers + sample data – Traditional enumeration with yearly updates of characteristics – Rolling census

Selected Quality Evaluation Issues: 1. Traditional Census

• Relevance: – Content of the census questionnaire – Selection of topics: users needs vs. constraints – Post census consultations • Accuracy: – Coverage and content errors evaluated with PE

S

, Demographic analysis, comparison with other sources • Timeliness: – Respect of planned calendar of output releases – May be difficult to achieve due to size of operations

Selected Quality Evaluation Issues: 2. Register-Based Census

• Relevance: – No questionnaire – Users needs vs. data available from registers – Implications of using data collected for admin. uses – Consultation and feedback from users • Accuracy: – Assess coverage and quality of data in registers – Assess data integration: Matching rates – Consistency of values from different sources – Comparison with data from last traditional census • Timeliness: – Different updating schedules for different registers

Considerations from an International Perspective

• Some countries have long tradition and adopt several evaluation methods • Others have limited experience and need to develop capacity • Challenges in evaluating quality of non-traditional censuses

Activities at international level

• CES Census Recommendations • UNSD Principles and Recommendations • Eurostat: Work on census quality criteria • UNSD: Plans for Handbook on census evaluation/PES

What should International Organisations do?

• Produce and disseminate guidelines, manuals, methodological material?

• Promote methodological work?

• Are standards needed on census quality evaluation?

Thank you!