Transcript MICS WS1

Multiple Indicator Cluster Surveys
Survey Design Workshop
Sampling:
Overview
MICS Survey Design Workshop
Introduction
• MICS :
– Household survey program
implemented across various countries
and at multiple point in time within a
country
– Data are consistent and comparable
across countries and over time
Contents
• Importance of a correct sample design
• Major steps in designing MICS sample
• Key principles of MICS sample
• Sampling options
• Sampling tools available for countries
• Group activity: sample size calculation
Sample Design
• Sample design involves determining:
– Sample size: number of units of analysis to be
selected for the survey
– Sampling structure: how those units are to be
selected
– Estimation procedures: how the results from the
sample are to be used to draw inferences about
the entire population of interest from which the
sample was selected
Importance of a correct sample design
• Sample size and structure affect:
– Validity of inferences about the entire
population
– Magnitude of:
•Sampling error
•Non sampling error
Importance of a correct sample design
• Sampling error: due to sampling of a small
number of units from the population instead
of complete enumeration
• Non sampling error: due to problems during
data collection and data processing (e.g.,
failure to locate and interview the correct
sample household, misunderstanding of the
questions, data entry errors)
Importance of a correct sample design
• Link with other aspects of the survey
– Dispersion of sample affects travel cost and
time
– Sample size affects the number of teams,
interviewer workload, and cost of
household listing and interviewing
– Sample size affects timeliness of results
– Large sample size can affect data quality
Major steps in designing MICS sample
• Define objectives
• Identify a suitable pre-existing sampling
frame
• Determine sample size and allocation
Major steps in designing MICS sample
• Define objectives:
– Key indicators
– Reporting domains
– Desired level of precision for survey
results
 Critical for sample size determination
Major steps in designing MICS sample
• Identify a suitable pre-existing sampling
frame:
– Most recent census of population and
housing
– Master sample or sample for another
survey conducted recently which is
large enough to support the MICS
sample design
Major steps in designing MICS sample
• Identify a suitable pre-existing sampling
frame:
– The availability of a suitable sampling
frame is a major determinant of the
feasibility of conducting a MICS survey.
– This issue should be addressed in the
earliest stages of planning for a survey.
Major steps in designing MICS sample
• Identify a suitable pre-existing sampling
frame:
– Regardless of source, evaluate the quality of the
frame before drawing the sample
• Characteristics of a good sampling frame
– Complete coverage of the target population
– No duplicates
– Up to date
Major steps in designing MICS sample
• Characteristics of a good sampling frame
– Area units: Boundaries well defined and
good maps are available
– Identification codes
– Measure of size (household or population)
– Auxiliary information available for
stratification
Major steps in designing MICS sample
• Determine sample size and allocation
– Survey objectives (key indicators, desired
level of precision and need for sub-national
results)
– Sampling parameters from previous MICS
or DHS (e.g., response rates, design effect)
– Survey budget and resource constraints
– Distribution of target population
Key principles in MICS sampling
• Probability sample at every stage of
selection (units are selected randomly
with known and nonzero probabilities)
• Latest census as sampling frame when
available
• Adequate sample size
Key principles in MICS sampling
• Simple design
• Sampling in two or three stages
• Separate household listing
• Clusters of moderate size: 20-25
households
• No replacement of primary sampling
units or households
Key principles in MICS sampling
• Implement the sample exactly as
designed
• Proper sampling weights
–
–
–
–
Extrapolate survey results to the population
Used in all analyses to prevent biased results
Calculation depends on the exact sample design
Weights: households, women, men and children
Key principles in MICS sampling
• Sampling error calculation
– Possible only when probability sampling is used
• Good sample documentation
Key principles in MICS sampling
• Report on sample design describes:
– Sampling frame
– Sampling methodology
– Sample size calculation and sample
allocation
– Survey domains and stratification
– Probabilities of selection at each stage
MICS Sampling Option 1 –
new sample with household listing
• Design new MICS sample
• Two stages with census as frame
• Selection of census EAs with PPS at
first stage
• Carry out household listing in
selected EAs/segments
MICS Sampling Option 1 –
new sample with household listing
• Select households systematically
from listing
• Interview selected households, no
replacement will be allowed
Sampling Option 1 - continued
• Advantages of option 1
- simple design
- probability-based
Sampling Option 1 - continued
• Limitations of option 1
- expense of listing households
- time necessary to list households
[Example, sample size of 5000
households may require 25000 to
50000 households to be listed]
MICS Sampling Option 2 –
use an existing sample
• Design MICS as a rider to another
survey if timely and feasible
• Use sample from a previous survey
and re-interview households for
MICS
• Use old survey sample EAs and
construct new listing of households
to select for MICS
MICS Sampling Option 2 –
use an existing sample
• Old sample must be probabilitybased, national in scope
• Possibilities – DHS, other national
health survey, recent labour force
survey
• Important: design parameters must
be known (such as selection
probability, stratification, etc.)
Sampling option 2 - continued
• Use of existing master sampling frame
• Some countries use master sample design for
intercensal national household surveys
• Master samples generally sufficiently large for
MICS; subsample of PSUs can be selected
• Advantage – updated maps may be available
for master sample of PSUs, and perhaps
updated listing
Sampling option 2 - continued
• Advantages of using previous sample
- cost savings
- maps available for interviewers
- appropriate sampling plan available
- simplicity
Sampling option 2 - continued
• Limitations of using old sample
- burden on respondents
- sample design may need modification
* sample size
* sub-national coverage
* number of PSUs or clusters
• Balance between loss and gain
Sampling strategy for low fertility
countries
• In MICS 4 and 5, some low fertility countries
are using second-stage stratification of listing
by households with and without children
under 5
• Higher sampling rate used for households
with children
• Increases number of households with children
in MICS sample, and therefore number of
sample children
Sampling strategy for low fertility
countries (continued)
• Improves the reliability of the child indicators
without increasing the sample size to a very high
level
• This procedure also increases the variability in the
weights and the design effects for the overall sample
• Important to avoid very large variability in the
weights for households with and without children
– Differential weights between households with and without
children generally should not exceed a factor of about 4
MICS Sampling Tools
• Household listing manual and listing forms
• Template for sample size calculation
• Template for calculation of weights
• Template for household selection
• SPSS program for sampling error estimation
SAMPLE SIZE DETERMINATION
Selection of key indicators
• Choose an important indicator that will yield
the largest sample size
• Step 1: Select 2 or 3 target populations
representing each a small percentage of the
total population (pb); typically
– Children 12-23 months: 2-4% or
– Children under 5 years: 7%-20%
Selection of key indicators
• Step 2: Review important indicators for these
target groups but ignore indicators with very
low or very high prevalence (less 10% or over
40%, respectively)
• Do not choose from the desirably low
coverage indicators an indicator that is
already acceptably low
• Do no choose childhood and maternal
mortality ratios
• n is the required sample size (number of
households)
• 4 is a factor to achieve the 95 percent level of
confidence
• r is the predicted or estimated value of the
indicator in target population
• deff is the design effect
• RR is the response rate
• pb is the proportion of the target
subpopulation in total population (upon
which the indicator, r, is based)
• AveSize is the average household size (that is,
average number of persons per household)
• e is the margin of error to be tolerated at the
95% level of confidence
• Currently, note that e = 0.12r [defined as 12%
of r, in this case the relative standard error of
r is 6% because e = 2 standard error (r)]
Previously in MICS2
• 2 different values for margin of error
– Margin of error was 5 percentage points for high
values of r (over 25%)
– Margin of error was 3 percentage points for low
values of r (25% or less)
• Difficulty for users in deciding on the sample
size for their surveys.
MICS template for sample size
calculation - EXCEL FILE