Transcript Innovations in LSMS data & implementation Kathleen Beegle Workshop 17, Session 3 Designing and Implementing Household Surveys March 31, 2009

Innovations in LSMS data &
implementation
Kathleen Beegle
Workshop 17, Session 3
Designing and Implementing Household Surveys
March 31, 2009
1
Overview





Testing: measuring ability through tests
Computer Assisted Personal Interviewing
(CAPI)
New health measures
Impact evaluation
Other new areas….
2
Testing (D. Filmer and N. Schady work in
Cambodia)

Students



Early Child Development




Mathematics test
Peabody Picture Vocabulary Test
Test de Vocabulario en Imagenes Peabody (TVIP)
Woodcock-Johnson III Memory for Names
Ages & Stages test for child development
Control measures of caregiver cognitive ability

Progressive Matrices Test
3
Testing, cont.

Appropriateness vs. Benchmarking
 For example:



Translation



Familiarity with / mastery of materials
Rapport with interviewees (esp. children for child development tests)
Implementation


Even more important to be precise
Training


Developing math test (need to ensure compatibility with
expectations/curriculum)
Adapting vocabulary test (at what point is it no longer comparable?)
Quiet space
Cost

Internationally comparable copyrighted tests expensive (TVIP/WJ)
4
Implementing tests
5
What is CAPI?



Interviewing and data entry are brought together. No
paper questionnaires
PDAs, mini-PCs (ultra mobile PCs)
Software




Off-the-shelf products designed for linear questionnaires
On-going development of software for complicated
questionnaire structures (like LSMS)
Hardware can come with built in cameras, sound
recording, CPS, external keyboard, finger
scanner……etc…..
Models to allow internet access through cell phone
network
6
Why CAPI?

Interviewing and data entry are brought
together



Faster data: data are available as soon as
transferred (same day)
More accurate data: quality improved through
validation checks during/immediately after interview
Other advantages


Flexible questionnaire design
Can embed listing, sampling, past data (panels), multilanguages, instructions from manual
7
An example: UMPCs in Tanzania
• Ultra Mobile Personal Computer
• Large touch Screen
• Runs standard desktop operating systems
• Multiple ports just like a desktop
Large screen allows you to see the
whole picture
Whole sections are
displayed and flow
maintained
Rapid click navigation to other
sections
You are about to finish an interview when a
child walks in that the respondent had not
mentioned before. With a UMPC layout you
can get to the census/ roster section in one
click to add the details and continue your
interview
10
Display of picture options for greater
accuracy
Respondent says “a bunch of bananas”. With a large
UMPC screen and storage you can display different
sizes to choose from, calculate the correct energy
value and therefore obtain accurate data (food,
medication, amount of production, size of asset...)
CAPI Experience




Adapting DE software is complicated.
Through 3 surveys, gaining experience on
practicalities of this technology: security,
remote support, battery life, etc…
Challenges of developing questionnaire
UMPCs have some advantages over PDA:

Maintaining the best features of a well-laid out
questionnaire: screen size and segueing to the next
screen.
12
Collecting New Health Measures in
LSMS/IS Surveys
Health measures collected multi-topic surveys to
measure outcomes that are not reliably self-reported.





Anthropometrics
BP/pulse
Observed physical ability
Arm circumference






Hemoglobin (anemia)
HIV/AIDS
Malaria
Avian flu
STDs
In the latter group, testing in a number of ways:
 laboratory testing of blood samples
 in-home testing (small blood sample from finger pin-prick
in case of Hemoglobin, saliva for HIV)
 physical examination.
13
Collecting New Health Measures:
Challenges


Staffing. May require a trained nurse (VCT) to be
added to the field team. Cost/Management
implications
Human Subjects Protocols. Internal Review Board
(IRB) process: both within country and with other
institutions



Consent processes: verbal consent replaced with signed
forms.
Data security: both within field teams and once electronic.
Informing respondents of the test result. Obligations for
follow-up (ability to provide, implications for research).
14
Impact evaluation


LSMS is (usually) not designed for specific
evaluation work, but can potentially be used
for this
Usually requires coordination of the
evaluation team with the national statistics
office either before or after, since additional
may be needed
15
Impact evaluation, cont.

Nicaragua FISE evaluation (Social Fund)



Booster sample of FISE communities. 1998 LSMS
provided adequate non-FISE communites (piggy-backing)
Oversample FISE households and use the LSMS survey
data (i) as a backdrop for comparison for assessing poverty
targeting and (ii) to construct a control group to assess
impact
Malawi Fertilizer Voucher Evaluation (2006/07)

Constructed a panel from the Malawi LSMS in 2004/05.
Re-interviewed households after the implementation of a
new fertilizer voucher scheme.
16
Pros and Cons of “Piggy Backing”
on an LSMS
Pros



matching method can allow for
use of data collected during
one time period to conduct an
evaluation -- more rapid than
method with baseline
good measures of targeting
and impact (consumptionbased poverty estimates;
variety of outcomes including
anthropometric measures,
etc.)
more cost-effective than
separate, in-depth survey
Cons



somewhat lengthy
evaluation process
use of LSMS
questionnaire limits ability
to tailor questions to FISE
Evaluation
cross-sectional surveys
may not be intended to
become panels
(availability of re-contact
information, names, etc..)
17
Extending an LSMS to other new areas

New methods to extend the range of topics in an
LSMS and improve the quality of data.

Expectations of the future.



Using stones/beans to illicit expected probabilities of
respondents
E.g. Pastoralists with little formal education asked about
expectations of rainfall in the coming season, allocating
stones across three different piles for “above normal”,
“normal” and “below normal”
Validity discussed in “Measuring Subjective Expectations in
Developing Countries.” 2009. Delavande et al. World Bank
Policy Research Working Paper 4824.
18
Extending an LSMS to other new areas

Subjective reporting of health, welfare,
satisfaction of life, etc…



Problem of different scales across people. A
person’s standard of good health may be different
from another person’s standard.
Hypothetical vignettes: Respondents score a set
of vignettes describing different scenarios related
to the topic (such as health status) for a
hypothetical person. These vignettes can help
reveal the respondents own scale.
King at al (2004) and King and Wand (2007).
19
Extending an LSMS to other new areas

These new areas imply new costs:



Piloting to ensure they work in different context
Time to administer questionnaire
Training interviewers appropriately
20