Transcript SURVEYS © LOUIS COHEN, LAWRENCE MANION & KEITH MORRISON STRUCTURE OF THE CHAPTER • • • • • • • • • • What is a survey? Some preliminary considerations Planning a survey Low response and.

SURVEYS
© LOUIS COHEN, LAWRENCE
MANION & KEITH MORRISON
STRUCTURE OF THE CHAPTER
•
•
•
•
•
•
•
•
•
•
What is a survey?
Some preliminary considerations
Planning a survey
Low response and non-response, and how to
reduce them
Survey sampling
Longitudinal, cross-sectional and trend studies
Strengths and weaknesses of longitudinal,
cohort and cross-sectional studies
Postal, interview and telephone surveys
Internet-based surveys
Comparing methods of data collection in surveys
KEY FEATURES OF SURVEYS
•
•
•
•
•
•
•
•
•
•
•
•
•
Gather data on a one-shot basis;
Represent a wide target population
Generate numerical, often large scale data;
Provide descriptive, inferential and explanatory data;
Manipulate key factors and variables;
Gather standardized information;
Ascertain correlations;
Remove contextual data;
Gather data from multiple choice, closed questions, test
scores or observation schedules;
Support or refute hypotheses about the target population;
They are piloted;
Make generalizations about, and observe patterns of
response in, the targets of focus;
Gather data which can be processed statistically.
TYPES OF SURVEY
LONGITUDINAL
SAME SAMPLE /
COHORT OVER
TIME
CROSSSECTIONAL
SNAPSHOTS OF
DIFFERENT
SAMPLES AT ONE
OR MORE TIMES
TREND/
PREDICTION
STUDIES
SELECTED
FACTORS
CONTINUOUSLY
OVER TIME
PLANNING A SURVEY
• Specify the exact purpose of the enquiry:
– Primary topics
– Subsidiary topics
– Specific information requirements
• Specify the population on which the survey is
to focus;
• Specify the resources that are available:
– Human (e.g. for administering and processing
survey)
– Material
– Financial
– Administrative
– Temporal
– Geographical
– Software
PLANNING A SURVEY
•
•
Data collection
– Researcher-administered
– Self-administered
– Postal survey
– Telephone survey
– Internet survey
– Face-to-face interviews
– Email
Self-reporting
– May lead to biased reporting
PLANNING A SURVEY
•
•
•
•
Problem definition
Sample selection
Design of measurements
Concern for participants
STAGES IN SURVEY DESIGN
DEFINE OBJECTIVES
DECIDE INFORMATION
SAMPLING
INSTRUMENTATION
PILOT
TRAINING
DATA COLLECTION
DATA ANALYSIS
REPORTING
STAGES IN CONDUCTING A SURVEY
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
Define the objectives;
Decide the kind of survey required;
Formulate research questions or hypotheses;
Decide the issues on which to focus;
Decide the information that is needed to address the
issues;
Decide the sampling required;
Decide the instrumentation and the metrics required;
Generate the data collection instruments;
Decide how the data will be collected;
Pilot the instruments and refine them;
Train the interviewers (if appropriate);
Collect the data
Analyze the data;
Report the results.
PROBLEMS IN SURVEYS
• Poor sampling
• Poor question design and wording (failure to
operationalize)
• Incorrect or biased responses
• Low response or non-response
HOW TO IMPROVE RESPONSE RATES
• Identify where the problem lies:
– The survey never reached the intended people;
– People refuse to answer ;
– People may not be available;
– People may not be able to answer the questions;
– People may not actually have the information
requested;
– People may overlook some items in error;
– The survey was completed and posted but failed to
return.
– The pressure of competing activities on the time of
the respondent;
– Potential embarrassment at their own ignorance if
respondents feel unable to answer a question;
HOW TO IMPROVE RESPONSE RATES
– Ignorance of the topic/no background in the topic;
– Dislike of the contents or subject matter of the
interview;
– Fear of possible consequences of the survey to
himself/herself or others;
– Lack of clarity in the instructions;
– Fear or dislike of being interviewed;
– Sensitivity of the topic, or potentially insulting or
threatening topic;
– Betrayal of confidences;
– Losing the return envelope or return address;
– The wrong person may open the mail, and fail to
pass it on to the most appropriate person.
HOW TO IMPROVE RESPONSE RATES
•
•
•
•
•
•
•
•
•
•
•
Ensure that the topic is interesting and motivating;
Ensure that it is easy to complete and layout is attractive;
Include a covering letter that explains the survey;
Choose a good time to conduct the survey;
Follow-ups and polite reminders;
Pre-paid return stamped addressed envelopes;
Institutional affiliation, survey sponsorship or support from
a high status agent;
Financial incentives and rewards for return;
Making instructions about responses and return very
clear;
Avoid open-ended questions unless these are really
important;
Avoid placing open-ended questions at the start of a
questionnaire;
HOW TO IMPROVE RESPONSE RATES
• Have easy-to-follow instructions and spacing of
the text;
• Flatter the participants without being seen to
flatter them;
• Providing information about the research through
a covering letter and/or advance notification;
• Making the survey look very unlike junk mail;
• Consider asking the respondents for an interview
to complete the survey questionnaire;
• Deliver the questionnaire personally rather than
through mail;
• Ensure that the questions or items are nonjudgemental;
• Assure confidentiality and anonymity.
HOW TO IMPROVE RESPONSE RATES
• Follow a sequence:
– Send non-respondents a friendly reminder
after ten days, stressing the importance of
the research;
– Send a further friendly reminder ten days
after the initial reminder, stressing the
importance of the research;
– Make a telephone call to the respondents
shortly after the second reminder,
indicating the importance of the research.
CONSIDERATIONS IN SAMPLING
•
•
•
•
•
•
•
•
A probability and non-probability sample;
Confidence level and interval;
The desire to generalize, and to whom;
The sampling frame;
The sample size;
The representativeness of the sample;
Access to the sample;
Anticipated response rate.
LONGITUDINAL DATA ARE
AFFECTED BY . . .
• History (events occurring may change the
observations of a group under study);
• Maturation (participants mature at different speeds
and in different ways);
• Testing (test sensitization may occur – participants
learn from exposure to repeated testing/interviews);
• The timing of cause and effect: some causes may
produce virtually instantaneous effects and others
may take a long time for the effects to show;
• The direction of causality not always being clear or
singular.
• Comparability of data over time;
• Attrition (dropout);
• Respondents’ memory and recall capabilities.
MEMORY IS AFFECTED BY . . .
• The time that has elapsed since the event took
place;
• The significance of the event for the participant;
• The amount of information required for the study
– the greater the amount, the harder it is to
provide;
• The contamination/interference effect of other
memories of a similar event (i.e. the inability to
separate similar events);
• The emotional content or the social desirability of
the content;
• The psychological condition of the participant at
interview.
• Hindsight.
ADVANTAGES OF POSTAL SURVEYS
• Reach many people
• Comparatively cheap
• Can be completed at respondents’ preferred
time
• No risk of interviewer bias
• Can reach scattered populations
• Can gather sensitive data (as nobody else
present)
DISADVANTAGES OF POSTAL
SURVEYS
•
•
•
•
•
Low response rate
No check on understanding
No check whether all items are completed
Need to have a very simple format
Presentation matters affect completion
ADVANTAGES OF INTERVIEW
SURVEYS
• Improves response rates
• Can clarify queries from respondents
• Can stimulate the respondent to give full
answers
• Can be flexible (e.g. in item sequence)
• Benefit from non-verbal communication
• Can build in trust and rapport
• Ensures that only the respondent answers
the questions
DISADVANTAGES OF INTERVIEW
SURVEYS
•
•
•
•
Interviewer characteristics affect responses
Conduct of interview affects responses
Flexibility can reduce standardization
Costly in time, travel and training
ADVANTAGES OF TELEPHONE
SURVEYS
• Reduce bias in the researcher or the interviewee
• Reduce costs of time and travel
• Easy to find more people to contact
DISADVANTAGES OF TELEPHONE
SURVEYS
•
•
•
•
•
•
•
•
Not everyone has a telephone
Not everyone is available for interview
People are ex-directory
The person answering the call may not be the
most suitable person
Multiple choice and rating scale questions are
difficult
Order effects can be strong
People may lie or hang up if questions are
unwelcome, sensitive, too long, too many
Lack of non-verbal cues
ADVANTAGES OF INTERNET-BASED
SURVEYS
•
•
•
•
•
•
•
Many people can be reached
Cheap to administer
Quick to process results
Can reach difficult populations
Anonymity and non-traceability
Novelty value
Respondents can complete the questionnaire from
home;
• Respondents can complete it at a time to suit
themselves;
• Respondents can complete the survey over time;
• Complex skip-patterns can be created and
organized by the computer, so that participants do
not have to understand complicated instructions;
ADVANTAGES OF INTERNET-BASED
SURVEYS
• The software can prompt respondents to complete
missed items or to correct errors;
• Computer can check incomplete or inconsistent
replies;
• For each screen, the computer can provide an onscreen indication of how much of the questionnaire
has been completed;
• Reduction of researcher effects;
• Human error is reduced in entering and processing
data;
• Additional features may make the survey attractive;
• Greater generalizability may be obtained as Internet
users come from a wide and diverse population;
• Greater authenticity of responses may be obtained.
DISADVANTAGES OF INTERNETBASED SURVEYS
•
•
•
•
•
•
•
•
•
•
Not everyone has internet access
Limited control over sampling
Ethics (e.g. traceability)
Hardware and software problems
Expertise of respondents
Visual presentation takes on added signficance
Limited number of lines per screen
People give a minimal response
Order effects (early responses affect later ones)
People stop if it is too long or complicated