Transcript Slide 1
Questionnaire design
as related to analysis
Intermediate Training in
Quantitative Analysis
Bangkok 19-23 November 2007
LEARNING PROGRAMME
Objectives
Understand the implications of questionnaire
design on the analysis
Illustrate examples and detect shortcomings
of questions in different questionnaires
Share experience by participants in their
surveys
LEARNING PROGRAMME - 2
General remarks
First think about the objective of the study and the
analysis ( what and how do you want to know) –
then the questionnaire
Difference between kind of survey CFSVA and
EFSA
CFSVA- provides baseline information that can feed into
monitoring systems, not emergency related information,
has more time to be collected and analysed;
EFS(N)A- needs results within short period, 10-15 days vs.
4 weeks plus.
LEARNING PROGRAMME - 3
General remarks
Partner involvement
Partners are important to have a wide buy-in into
the results, synergy effects, cost sharing etc. but
also might add or change type of information that
is collected beyond the need of WFP.
Quality of collected data:
Length of the questionnaire - shorter is usually
better- if you don’t sacrifice important details.
Sacrifice details that are not analysed to avoid
response fatigue
LEARNING PROGRAMME - 4
General remarks cont.
Language and translation
Differences between original and translated version (timing)
Do the questions in the original language mean the same thing
as the working language? If not- the analyst can mis-interpret
the results.
Number of categories for responses used in
the questions.
Recode later or maintain the same categories?
Present graph/table with many categories
Homogeneity of Numbering of questions
(letters or number)
LEARNING PROGRAMME - 5
Open vs closed question
Open question
can be answered with either a single word or a short
phrase.
Closed question
Can be answered with one of the categories/options
included in the question
ex. What is the major material of the roof?
Observe and record. Do not ask question! Circle one
1Straw / thatch
2Earth / mud
3Concrete
4Tiles
5CGI sheet
6Other, specify ____________________
LEARNING PROGRAMME - 6
Open question
When to use an open question
Names (household head, village, unit measure)
Other (when the category is not included in the question)
Community / focus groups questionnaire
Small survey (max 50 households )
When not to use an open question
When we have an exhaustive list of categories (crops,
livelihoods)
‘Other’ should not be used as alternative to a category
(ex. in Sudan 23% of the pop answer ‘other’ to livelihood
activities and we were not able to specify what ‘other’
meant)
LEARNING PROGRAMME - 7
Closed questions
When to use a closed question
When we know the categories - especially for
question related with materials (roof, floor) and
questions related with the context (ex. crops,
livelihood activities etc.).
When we are not interested in a continuous
variable (ex. age) and we want to collect it in
categorical variable.
LEARNING PROGRAMME - 8
Example
LEARNING PROGRAMME - 9
LEARNING PROGRAMME - 10
Continuous vs categorical
When to recode a continuous variable to
minimize errors
Demography of the households
Land size
Stocks and agricultural production
Other?
LEARNING PROGRAMME - 11
When to use a continuous
variable
FCS (1 to 7)
Number of animals
Proportions (proportional piling)
Expenditure
Measurement (height, weight, muac, child
age in months)
LEARNING PROGRAMME - 12
Yes/no questions
Coding of a yes/ no question
It is helpful to code yes=1 and no =0, this allows
the analysis to check the % of yes or no running a
simple mean.
Hav e electricity-- for wealth index
Valid
Mi ssing
Total
No
Yes
Total
System
Frequency
18976
4702
23678
849
24527
Percent
77.4
19.2
96.5
3.5
100.0
Valid Percent
80.1
19.9
100.0
Cumulative
Percent
80.1
100.0
Descriptive Statistics
N
Have electricity-for wealth index
Valid N (listwise)
23678
Mean
.1986
23678
LEARNING PROGRAMME - 13
Skip rule
It is important that the skips for the questions
are correct, if not the analyst will have
problem in deciding which is the right variable
and in the majority of the case he can not use
both of them.
LEARNING PROGRAMME - 14
Skip rule- example
LEARNING PROGRAMME - 15
Missing values
How to recode missing values
Difference between missing and not applicable.
Be sure that you know the difference in the analysis!
Negative coding
For values as expenditure or income, the value 999 or
888 can be a real value. In these cases might be
better to code the missing or not applicable as a
negative number -999
LEARNING PROGRAMME - 16
Household ID
Importance of HHID in linking one module
with different modules of the questionnaire
and with different questionnaires
ex. household /child / mother, household / village
Importance of the coding (village, cluster,
state, community)
It always has to be unique!
LEARNING PROGRAMME - 17
ID-Section – Darfur
LEARNING PROGRAMME - 18
LEARNING PROGRAMME - 19
Household ID - Exercise
What are the important elements?
How can we ensure this part is done correctly?
LEARNING PROGRAMME - 20
Modules
Now we are going to see some examples of
modules of the questionnaire and how they are
linked with the analysis
LEARNING PROGRAMME - 21
Demographics – example of
indicators
Average size of household
Number of educated people in an household
Incidence of absenteeism amongst school-going
children; enrolment ratio, drop-out
Literacy of household heads
Percentage of male, female and children-headed
households
Percentage of disabled/chronically ill in the
households
Dependency rate
LEARNING PROGRAMME - 22
LEARNING PROGRAMME - 23
Demographic - issues
Age as continuous or categorical variables?
Age categories should be related to standards
School age, productive members, children, etc.
Polygamy / number of wives
Household size (1.1 & 1.7 should be the same)
Number of categories in the education level
Education of the mother of children as opposed to
simply spouse’s education
LEARNING PROGRAMME - 24
Housing – example of
indicators
Crowding (how many people sleep in the
house)
Most common building materials used in
housing (of floors, roofs and walls)
Availability of toilet facilities and type
Source of lighting, cooking fuel and water
Wealth index
LEARNING PROGRAMME - 25
Housing example
W hat is the maj or construction material of the outside walls - other specified
Frequency
Valid
W hat is the maj or material of theroof - other specified
Frequency
Valid
Percent
Valid Percent
Cumulative
Percent
Percent
Valid Percent
Cumulative
Percent
2958
99.0
99.0
99.0
BRICKS
1
.0
.0
99.1
BURNED B
2
.1
.1
99.1
CEMENT W
1
.0
.0
99.1
GRASS
2
.1
.1
99.2
GRASS HU
2
.1
.1
99.3
GRASS SH
2
.1
.1
99.4
grass, p
2
.1
.1
99.5
IRON SHE
0
.0
.0
99.5
MUD
4
.1
.1
99.6
MUD, STI
1
.0
.0
99.6
2983
99.9
99.9
99.9
OLD IRON
1
.0
.0
99.9
PAPYRUS
2
.1
.1
100.0
MUD/BURN
1
.0
.0
99.7
TURPULIN
1
.0
.0
100.0
NOT BURN
0
.0
.0
99.7
2987
100.0
100.0
PAPYRUS
1
.0
.0
99.7
STONES
0
.0
.0
99.7
STONES &
0
.0
.0
99.8
STRAW
2
.1
.1
99.8
STRAW ON
1
.0
.0
99.9
STRW ONL
1
.0
.0
99.9
TURPULIN
1
.0
.0
99.9
UN BURNE
1
.0
.0
99.9
UNBURNED
1
.0
.0
100.0
100.0
Total
WOOD
Total
1
.0
.0
2987
100.0
100.0
From Uganda database
LEARNING PROGRAMME - 26
Housing - issues
Pilot testing the questionnaire – it’s useful to
explore possible answers to a question
After the pilot the possible answers are included
with codes, so that the ‘other’ will not be as
necessary
recode the meaning of “other” when you have a
lot of them (when the enumerator has entered in a
string response)
Exclude the possibility of other for material
questions (ex. Housing)
LEARNING PROGRAMME - 27
Housing - issues
The number of digits should be limited for any
figure through boxes |_|
(helps in data entry and cleaning)
Distance in km or minutes?
To water source, market, school, health centre HH vs. community?
One way vs round trip, waiting time and means of
transport
LEARNING PROGRAMME - 28
Agricultural – example of
indicators
Percentage of households having access to
land
Most common types / methods of land
access
Common crops cultivated and amount
Source of seeds
People involved in agricultural activities
Stocks and agriculture production
LEARNING PROGRAMME - 29
Agriculture - issues
Units of land measurement
Acres, hectares, parcels, etc.
Land size in absolute value or in categories?
What is more relevant in the analysis: the
mean land size or the division in categories?
Mis-leading cash crop definition
LEARNING PROGRAMME - 30
Income – example of
indicators
Income diversification
The most common activities
Average contribution of each of the income
generating activities to a household’s income
LEARNING PROGRAMME - 31
Expenditure – example of
indicators
The most common expenditure items- food &
non-food
The average monthly expenditure of a
household or per capita for each of the above
items
Food /non food expenditure quintiles
Proportion of food expenditure versus non
food expenditure
LEARNING PROGRAMME - 32
Income and expenditure – issues
Proportional piling (100%)
Income in absolute real value or express in
categories
The number of digits should be limited for any figure
through boxes |_| for data cleaning and entry
Different recall period for expenditures are often
used- so this means it’s necessary to carefully
calculate the monthly expenditure values in the
analysis.
LEARNING PROGRAMME - 33
Food consumption – example of
indicators
Average number of meals an adult and a
child ate the previous day
Diet Diversity and Food Frequency
Food consumption profiles
Source of foods
LEARNING PROGRAMME - 34
Food consumption – issues
Collection of gender disaggregated data
(meals per day)
Specify the child age range (infant vs
children)
Don’t consider 0 if the household has no
children
Rank the sources of food (main and second)
LEARNING PROGRAMME - 35
Maternal health and nutrition –
example of indicators
Percentage of households with children aged between 6 – 59
months
Malnutrition indicators for:
children (waz, haz, whz)
Mother ( bmi)
Incidence of miscarriages / still-births (averaged for the sample)
Percentage of mothers who breast-fed their children
Information on prenatal and antenatal care available and used by
mothers
Information on incidence and treatment of diseases such as
malaria, diarrhea, fever, cholera, measles, cough etc
Information on prevalent hygienic practices followed
LEARNING PROGRAMME - 36
Maternal and child - issues
Link mother with child database
Date of birth – local calendar
Fever and diarrhoea separate question
Child size at birth, continuous or
categorical? (subjective)
Mosquito net only for the mother or even
for the child?
LEARNING PROGRAMME - 37
Conclusions
Data analysts must participate in the design of the
questionnaire to avoid difficulties or missing
information in the analysis
Even if PDAs are used, the analyst should carefully
examine all the skip rules to be sure the correct information
will be collected
The questionnaire designers and enumerator
trainers should be involved in the analysis (if the
analyst him/herself was not) to be sure the
questions are understood by the analyst.
Information should be collected in order to calculate
key indicators during analysis- questions that are not
necessary in the analysis should not be included.
LEARNING PROGRAMME - 38