Transcript Slide 1

Questionnaire design
as related to analysis
Intermediate Training in
Quantitative Analysis
Bangkok 19-23 November 2007
LEARNING PROGRAMME
Objectives
 Understand the implications of questionnaire
design on the analysis
 Illustrate examples and detect shortcomings
of questions in different questionnaires
 Share experience by participants in their
surveys
LEARNING PROGRAMME - 2
General remarks
 First think about the objective of the study and the
analysis ( what and how do you want to know) –
then the questionnaire
 Difference between kind of survey CFSVA and
EFSA
 CFSVA- provides baseline information that can feed into
monitoring systems, not emergency related information,
has more time to be collected and analysed;
 EFS(N)A- needs results within short period, 10-15 days vs.
4 weeks plus.
LEARNING PROGRAMME - 3
General remarks
 Partner involvement
 Partners are important to have a wide buy-in into
the results, synergy effects, cost sharing etc. but
also might add or change type of information that
is collected beyond the need of WFP.
 Quality of collected data:
 Length of the questionnaire - shorter is usually
better- if you don’t sacrifice important details.
 Sacrifice details that are not analysed to avoid
response fatigue
LEARNING PROGRAMME - 4
General remarks cont.
 Language and translation
 Differences between original and translated version (timing)
 Do the questions in the original language mean the same thing
as the working language? If not- the analyst can mis-interpret
the results.
 Number of categories for responses used in
the questions.
 Recode later or maintain the same categories?
 Present graph/table with many categories
 Homogeneity of Numbering of questions
(letters or number)
LEARNING PROGRAMME - 5
Open vs closed question
 Open question
 can be answered with either a single word or a short
phrase.
 Closed question
Can be answered with one of the categories/options
included in the question
ex. What is the major material of the roof?
Observe and record. Do not ask question! Circle one
1Straw / thatch
2Earth / mud
3Concrete
4Tiles
5CGI sheet
6Other, specify ____________________
LEARNING PROGRAMME - 6
Open question
 When to use an open question




Names (household head, village, unit measure)
Other (when the category is not included in the question)
Community / focus groups questionnaire
Small survey (max 50 households )
 When not to use an open question
 When we have an exhaustive list of categories (crops,
livelihoods)
 ‘Other’ should not be used as alternative to a category
(ex. in Sudan 23% of the pop answer ‘other’ to livelihood
activities and we were not able to specify what ‘other’
meant)
LEARNING PROGRAMME - 7
Closed questions
 When to use a closed question
 When we know the categories - especially for
question related with materials (roof, floor) and
questions related with the context (ex. crops,
livelihood activities etc.).
 When we are not interested in a continuous
variable (ex. age) and we want to collect it in
categorical variable.
LEARNING PROGRAMME - 8
Example
LEARNING PROGRAMME - 9
LEARNING PROGRAMME - 10
Continuous vs categorical
When to recode a continuous variable to
minimize errors
 Demography of the households
 Land size
 Stocks and agricultural production
 Other?
LEARNING PROGRAMME - 11
When to use a continuous
variable
 FCS (1 to 7)
 Number of animals
 Proportions (proportional piling)
 Expenditure
 Measurement (height, weight, muac, child
age in months)
LEARNING PROGRAMME - 12
Yes/no questions
 Coding of a yes/ no question
 It is helpful to code yes=1 and no =0, this allows
the analysis to check the % of yes or no running a
simple mean.
Hav e electricity-- for wealth index
Valid
Mi ssing
Total
No
Yes
Total
System
Frequency
18976
4702
23678
849
24527
Percent
77.4
19.2
96.5
3.5
100.0
Valid Percent
80.1
19.9
100.0
Cumulative
Percent
80.1
100.0
Descriptive Statistics
N
Have electricity-for wealth index
Valid N (listwise)
23678
Mean
.1986
23678
LEARNING PROGRAMME - 13
Skip rule
 It is important that the skips for the questions
are correct, if not the analyst will have
problem in deciding which is the right variable
and in the majority of the case he can not use
both of them.
LEARNING PROGRAMME - 14
Skip rule- example
LEARNING PROGRAMME - 15
Missing values
 How to recode missing values
 Difference between missing and not applicable.
 Be sure that you know the difference in the analysis!
 Negative coding
 For values as expenditure or income, the value 999 or
888 can be a real value. In these cases might be
better to code the missing or not applicable as a
negative number -999
LEARNING PROGRAMME - 16
Household ID
 Importance of HHID in linking one module
with different modules of the questionnaire
and with different questionnaires
 ex. household /child / mother, household / village
 Importance of the coding (village, cluster,
state, community)
 It always has to be unique!
LEARNING PROGRAMME - 17
ID-Section – Darfur
LEARNING PROGRAMME - 18
LEARNING PROGRAMME - 19
Household ID - Exercise
 What are the important elements?
 How can we ensure this part is done correctly?
LEARNING PROGRAMME - 20
Modules
Now we are going to see some examples of
modules of the questionnaire and how they are
linked with the analysis
LEARNING PROGRAMME - 21
Demographics – example of
indicators
 Average size of household
 Number of educated people in an household
 Incidence of absenteeism amongst school-going
children; enrolment ratio, drop-out
 Literacy of household heads
 Percentage of male, female and children-headed
households
 Percentage of disabled/chronically ill in the
households
 Dependency rate
LEARNING PROGRAMME - 22
LEARNING PROGRAMME - 23
Demographic - issues
 Age as continuous or categorical variables?
 Age categories should be related to standards
 School age, productive members, children, etc.
 Polygamy / number of wives
 Household size (1.1 & 1.7 should be the same)
 Number of categories in the education level
 Education of the mother of children as opposed to
simply spouse’s education
LEARNING PROGRAMME - 24
Housing – example of
indicators
 Crowding (how many people sleep in the
house)
 Most common building materials used in
housing (of floors, roofs and walls)
 Availability of toilet facilities and type
 Source of lighting, cooking fuel and water
 Wealth index
LEARNING PROGRAMME - 25
Housing example
W hat is the maj or construction material of the outside walls - other specified
Frequency
Valid
W hat is the maj or material of theroof - other specified
Frequency
Valid
Percent
Valid Percent
Cumulative
Percent
Percent
Valid Percent
Cumulative
Percent
2958
99.0
99.0
99.0
BRICKS
1
.0
.0
99.1
BURNED B
2
.1
.1
99.1
CEMENT W
1
.0
.0
99.1
GRASS
2
.1
.1
99.2
GRASS HU
2
.1
.1
99.3
GRASS SH
2
.1
.1
99.4
grass, p
2
.1
.1
99.5
IRON SHE
0
.0
.0
99.5
MUD
4
.1
.1
99.6
MUD, STI
1
.0
.0
99.6
2983
99.9
99.9
99.9
OLD IRON
1
.0
.0
99.9
PAPYRUS
2
.1
.1
100.0
MUD/BURN
1
.0
.0
99.7
TURPULIN
1
.0
.0
100.0
NOT BURN
0
.0
.0
99.7
2987
100.0
100.0
PAPYRUS
1
.0
.0
99.7
STONES
0
.0
.0
99.7
STONES &
0
.0
.0
99.8
STRAW
2
.1
.1
99.8
STRAW ON
1
.0
.0
99.9
STRW ONL
1
.0
.0
99.9
TURPULIN
1
.0
.0
99.9
UN BURNE
1
.0
.0
99.9
UNBURNED
1
.0
.0
100.0
100.0
Total
WOOD
Total
1
.0
.0
2987
100.0
100.0
From Uganda database
LEARNING PROGRAMME - 26
Housing - issues
 Pilot testing the questionnaire – it’s useful to
explore possible answers to a question
 After the pilot the possible answers are included
with codes, so that the ‘other’ will not be as
necessary
 recode the meaning of “other” when you have a
lot of them (when the enumerator has entered in a
string response)
 Exclude the possibility of other for material
questions (ex. Housing)
LEARNING PROGRAMME - 27
Housing - issues
 The number of digits should be limited for any
figure through boxes |_|
(helps in data entry and cleaning)
 Distance in km or minutes?
 To water source, market, school, health centre HH vs. community?
 One way vs round trip, waiting time and means of
transport
LEARNING PROGRAMME - 28
Agricultural – example of
indicators
 Percentage of households having access to
land
 Most common types / methods of land
access
 Common crops cultivated and amount
 Source of seeds
 People involved in agricultural activities
 Stocks and agriculture production
LEARNING PROGRAMME - 29
Agriculture - issues
 Units of land measurement
 Acres, hectares, parcels, etc.
 Land size in absolute value or in categories?
What is more relevant in the analysis: the
mean land size or the division in categories?
 Mis-leading cash crop definition
LEARNING PROGRAMME - 30
Income – example of
indicators
 Income diversification
 The most common activities
 Average contribution of each of the income
generating activities to a household’s income
LEARNING PROGRAMME - 31
Expenditure – example of
indicators
 The most common expenditure items- food &
non-food
 The average monthly expenditure of a
household or per capita for each of the above
items
 Food /non food expenditure quintiles
 Proportion of food expenditure versus non
food expenditure
LEARNING PROGRAMME - 32
Income and expenditure – issues
 Proportional piling (100%)
 Income in absolute real value or express in
categories
 The number of digits should be limited for any figure
through boxes |_| for data cleaning and entry
 Different recall period for expenditures are often
used- so this means it’s necessary to carefully
calculate the monthly expenditure values in the
analysis.
LEARNING PROGRAMME - 33
Food consumption – example of
indicators
 Average number of meals an adult and a
child ate the previous day
 Diet Diversity and Food Frequency
 Food consumption profiles
 Source of foods
LEARNING PROGRAMME - 34
Food consumption – issues
 Collection of gender disaggregated data
(meals per day)
 Specify the child age range (infant vs
children)
 Don’t consider 0 if the household has no
children
 Rank the sources of food (main and second)
LEARNING PROGRAMME - 35
Maternal health and nutrition –
example of indicators
 Percentage of households with children aged between 6 – 59
months
 Malnutrition indicators for:
 children (waz, haz, whz)
 Mother ( bmi)
 Incidence of miscarriages / still-births (averaged for the sample)
 Percentage of mothers who breast-fed their children
 Information on prenatal and antenatal care available and used by
mothers
 Information on incidence and treatment of diseases such as
malaria, diarrhea, fever, cholera, measles, cough etc
 Information on prevalent hygienic practices followed
LEARNING PROGRAMME - 36
Maternal and child - issues
 Link mother with child database
 Date of birth – local calendar
 Fever and diarrhoea separate question
 Child size at birth, continuous or
categorical? (subjective)
 Mosquito net only for the mother or even
for the child?
LEARNING PROGRAMME - 37
Conclusions
 Data analysts must participate in the design of the
questionnaire to avoid difficulties or missing
information in the analysis
 Even if PDAs are used, the analyst should carefully
examine all the skip rules to be sure the correct information
will be collected
 The questionnaire designers and enumerator
trainers should be involved in the analysis (if the
analyst him/herself was not) to be sure the
questions are understood by the analyst.
 Information should be collected in order to calculate
key indicators during analysis- questions that are not
necessary in the analysis should not be included.
LEARNING PROGRAMME - 38