Overview of Census Evaluation and Selected Methods Pres. 2 United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census.

Download Report

Transcript Overview of Census Evaluation and Selected Methods Pres. 2 United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses: Census.

Overview of Census Evaluation
and Selected Methods
Pres. 2
United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses:
Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, 14-18 September 2009
Why evaluate ?
 Because the census is a huge operation (size, number of
persons involved) prone to errors
 To provide users with some measures of quality of census
data to help them interpret the results
 To identify types and sources of error in order to assist the
planning of future censuses
 To serve as a basis for constructing a best estimate of
census aggregates, such as total population, or to provide
adjustment of census results
 But…not to criticize the census takers !!
United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses:
Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, 14-18 September 2009
What errors in a census?
 Coverage errors:
 Errors in the count of persons or housing units resulting from
cases having been “missed” or counted erroneously
 Content errors:
 Errors in the recorded characteristics of persons or housing
units enumerated in the census (e.g. wrong age...)
United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses:
Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, 14-18 September 2009
Coverage errors
Omissions
 Missing housing units, households and/or persons during
census enumeration
 If the whole housing unit is missed, all households and
persons living in the housing unit will also be missed
 Major causes of omissions are: failure to cover whole land
area of a country in creating EAs;
 Mistakes made by enumerators in canvassing assigned areas
 Ambiguous definitions of EAs, unclear boundaries of EAs,
faulty maps or coverage error during the pre-census listing
exercise
United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses:
Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, 14-18 September 2009
Coverage errors
Omissions contd.
 In addition, omissions within EAs can result because all
or some of the members of the household were not
present at the time of enumeration
 Proxy respondents can inadvertently or deliberately omit
some members of a household
United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses:
Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, 14-18 September 2009
Coverage errors
Duplications
 Occur when persons, households or housing units are
counted more than once
 Reasons for duplications include:
 Overlapping of enumerator’s assignments owing to errors
done during pre-census listing and delineation
 Failure by enumerators to clearly identify boundaries
 In practice, the number of omissions usually exceeds the
number of duplicates (net under-counts)
United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses:
Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, 14-18 September 2009
Coverage errors
Erroneous Inclusions
 This includes:
 Housing units, households and persons enumerated while
they should have not been (e.g. babies born after the census
reference date)
 Or enumerated in a wrong place
United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses:
Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, 14-18 September 2009
Coverage errors
Gross error
 This is the sum of duplications, erroneous inclusions and
omissions
Net error
 This is the difference between over-counts and undercounts
 Net census under-count exists when number of omissions
exceeds the number of duplicates and erroneous
enumerations
 Net census over-count is the converse
United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses:
Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, 14-18 September 2009
Methods for evaluation of census errors
Single Source of Data
 Demographic analysis of the census
 Interpenetration studies
Multiple Sources of Data
 Non-matching studies
 Demographic analysis using previous censuses
 Comparison with administrative sources or existing surveys
 Matching studies
 Post Enumeration Surveys
 Record checks
United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses:
Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, 14-18 September 2009
Single Source of Data
Demographic Analysis of the Census
 Average number of persons per household
 Sex- and age- ratios
 Tabulations...
 For an overall assessment of quality:
 an age pyramid is a standard method
 stable population analysis can be undertaken as long as
assumptions pertaining to constant fertility and mortality
and no migration are met, for countries with declining
mortality a quasi-stable model may be appropriate
United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses:
Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, 14-18 September 2009
Single Source of Data
Interpenetrating studies
 Method involves drawing subsamples, selected in an
identical manner, from the census frame
 Each subsample should be capable of providing valid
estimates of population parameters
 Assignment of personnel (i.e. enumerators, coders, data
entry staff, etc.) is done randomly
 The method helps to provide an appraisal of the quality of
census information and procedures
United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses:
Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, 14-18 September 2009
Multiple Sources of Data – Non matching studies
Demographic Analysis
 Results from a census may be compared with data from
other demographic systems such as vital registration
systems
 For example, the cohort component method of
demographic analysis uses successive censuses including
 life-table survival rates
 age-specific rates
 age-specific fertility rates and
 estimates of international migration
United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses:
Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, 14-18 September 2009
Multiple Sources of Data – Non matching studies
Demographic Analysis contd.
 Population can be projected forward to the reference date of
the second census based on estimated levels of and age
schedules of fertility, mortality
 The expected population is then compared to the
enumerated population in the current census
 Yet another method is the comparison of age distributions of
successive censuses
United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses:
Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, 14-18 September 2009
Multiple Sources of Data – Non matching studies
Demographic Analysis contd.
 Also the cohort survival method which is a regression
method can be used, thus, population counts by age from
two censuses and deaths by age during the inter-censal
period are used to estimate coverage rate
United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses:
Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, 14-18 September 2009
Multiple Sources of Data – Non matching studies
Comparison with existing household surveys
 In theory any probability sample of households or persons
can be used to evaluate coverage and content error in a
census if:
 They have identical items with same concepts and
definitions
 They are independent from the census
 Must have been conducted close to the census date
 There should be sufficient identification information to
facilitate accurate matching
United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses:
Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, 14-18 September 2009
Multiple Sources of Data – Matching studies
Record checks
 Census records are matched with a sample of records
from identification systems such as the vital registration
system
 The relevant respondents to the census questionnaire are
traced to the time synchronized with the census
 Sources include:





Previous census
Birth registrations
School enrolment
Citizen registration card
Immigration registers etc.
United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses:
Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, 14-18 September 2009
Multiple Sources of Data – Matching studies
Record checks contd.
 Both coverage and content errors could be measured
through the above comparisons
To evaluate coverage efficiently the following
preconditions are essential:
 A large proportion of census population should be covered
in record system
 The census and record system should be independent
from each other
 There should be sufficient information in records
United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses:
Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, 14-18 September 2009
Multiple Sources of Data – Matching studies
To evaluate content efficiently the following
preconditions are essential:
 The record system should contain some relevant items
covered in the census such as age, sex, education,
relationship, marital status etc.
 Definitions of items should be identical between the
census and the record system
Countries that have used record checks include:
Demark, Finland, Norway, Sweden, Taiwan and Canada
United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses:
Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, 14-18 September 2009
Multiple Sources of Data – Matching studies
Post Enumeration Survey (PES)
 Consists of two separate coverage studies :
 A survey conducted using a sample frame independent of
the census. Persons from this survey are matched to the
census to estimate the number of persons missed in the
census
 A survey conducted using a sample drawn from persons
enumerated in the census. This sample is re-enumerated to
determine if the sample person or unit was erroneously
enumerated (inc. erroneously located)
United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses:
Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, 14-18 September 2009
Multiple Sources of Data – Matching studies
Post Enumeration Survey (PES) contd.
 Results can be used to evaluate the reliability of some
characteristics such as sex, age, marital status, relationship to
reference person or head of the household.
 For some countries the results of PES can be used to
adjust some census results
 Facilitates better interpretation of census results
 More discussion of PES is the focus of this workshop
United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses:
Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, 14-18 September 2009
Strengths and weaknesses of evaluation methods
 Single source:
 Methods that depend on a single data source provide less
insight into the magnitude and types of errors in the
census data
 The merit is that the methods using such sources do not
require additional data to be collected
 No need for sophisticated matching although this is also a
limitation
 They provide a general impression of quality of the census
data
United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses:
Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, 14-18 September 2009
Strengths and weaknesses of evaluation methods (contd.)
Single Source - Interpenetrating studies
 Gives good idea of different contribution of component
errors to total census error
 Helps to identify operational stages that contribute to
census error, thus identifying procedural limitations in a
census
 Demerits include:
 That it is an expensive operation demanding many field
staff, intensive training and close supervision
 Relatively complex in designing and implementation
United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses:
Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, 14-18 September 2009
Strengths and weaknesses of evaluation methods (contd.)
Multiple sources - Non-matching studies:
 Review census results at aggregate rather than unit level
i.e. provides only estimates of net census error
 Evaluates very limited characteristics such as sex and age
distributions
Merit
 They are relatively cheap compared to matching studies
United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses:
Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, 14-18 September 2009
Strengths and weaknesses of evaluation methods (contd.)
Non matching methods - Demographic Analysis:
 Advantage – no additional data is needed to be collected
to perform the analysis
 Less costly
 In statistical offices with sufficient numbers of
demographers there is no need for additional staff to do
the technical analysis
 On the negative side these methods provide less insight
into the different contributions of component errors to
total error in the census
 Quality of sources (Vital Statistics…)
United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses:
Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, 14-18 September 2009
Strengths and weaknesses of evaluation methods (contd.)
Matching methods:
 It provide separate estimates of coverage and content error
 Prospects of evaluating more characteristics compared to
what can be done with non-matching studies
Challenges
 Calls for high level technical skills including managerial
 Matching is expensive
United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses:
Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, 14-18 September 2009
Strengths and weaknesses of evaluation methods (contd.)
Post enumeration survey
Merits:
 Its results can be used to independently evaluate census
coverage and content error, including reliability of
selected characteristics collected in a census
 Incorporates matching of individuals or units between the
census and PES
 Its results are generally more reliable than those of the
census i.e. it justification for evaluation
United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses:
Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, 14-18 September 2009
Strengths and weaknesses of evaluation methods (contd.)
Post enumeration survey
Challenges:
 Requires highly skilled field and professional staff
 Matching is complex
 As it is supposed to be carried out immediately after the
census at times there is lack of adequate funds to
implement the PES exercise
United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses:
Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, 14-18 September 2009
Thank You!
United Nations Regional Workshop on the 2010 World Programme on Population and Housing Censuses:
Census Evaluation and Post Enumeration Surveys, Addis Ababa, Ethiopia, 14-18 September 2009