Transcript Survey Research

Surveys
Respondents
Respondents
are a
representative
sample of
people
Surveys
Surveys ask respondents for
information using verbal or
written questioning
Gathering Information via Surveys
•
•
•
•
•
Quick
Inexpensive
Efficient
Accurate
Flexible
Tree Diagram of Total Survey Error
Random sampling
error
Total
error
Systematic
error (bias)
Random Sampling Error
• A statistical fluctuation that occurs because
of change variation in the elements selected
for the sample
Tree Diagram of Total Survey Error
Random sampling
error
Total
error
Systematic
error (bias)
Common Errors in Survey Research
• Systematic Error
–Also called Nonsampling Error
–Results from
• some imperfect aspect of the
research design or
• from a mistake in the execution of
the research.
Tree Diagram of Total Survey Error
Administrative
error
Systematic
error (bias)
Respondent
error
Respondent Error
• A classification of sample bias resulting
from some respondent action or inaction
• Nonresponse bias
• Response bias
Nonresponse Error
• Statistical difference between
– a survey that includes only those who responded and
– a perfect survey that would also include those who
failed to respond
• Nonrespondent
– Person who is not contacted or who refuses to
cooperate in the research
– No Contact
• Person who is not at home on the 1st or 2nd contact, or
who is otherwise inaccessible.
– Refusal
• Person who is unwilling to participate in a research
project
Response Bias
• Bias that occurs when respondents either consciously
or unconsciously tend to answer questions with a
certain slant that misrepresents the truth.
• Can arise from
– The question format
– The question content
– Some other stimulus
• such as the situation in which the question is asked
• Deliberate falsification may occur when people
misrepresent answers to appear intelligent, to conceal
personal information, to avoid embarrassment, etc.
Tree Diagram of Total Survey Error
Acquiescence bias
Extremity bias
Interviewer bias
Auspices bias
Social desirability bias
Acquiescence Bias
• A category of response bias that results
because some individuals tend to agree with
all questions or to concur with a particular
position.
Extremity Bias
• A category of response bias that results
because response styles vary from person to
person; some individuals tend to use
extremes when responding to questions.
Interviewer Bias
• A response bias that occurs because the
presence of the interviewer influences
answers.
Auspices Bias
• Bias in the responses of subjects caused by
the respondents being influenced by the
organization conducting the study.
Social Desirability Bias
• Bias in responses caused by respondents’
desire, either conscious or unconscious, to
gain prestige or appear in a different social
role.
Tree Diagram of Total Survey Error
Administrative
error
Systematic
error (bias)
Respondent
error
Administrative Errors
• Error caused by the improper administration or
execution of the research task.
• Caused by
–
–
–
–
–
Carelessness
Confusion
Neglect
Omission
Some other blunder
Tree Diagram of Total Survey Error
Data processing error
Sample selection error
Interviewer error
Interviewer cheating
Types of Administrative Errors
• Data-Processing Error
– Occurs because of incorrect data entry, incorrect
computer programming, or some other procedural
errors during the data-processing stage
• Sample Selection Error
– Caused by improper sample design or sampling
procedure execution
– Hoover vs. Roosevelt, 1932 Presidential Election
Types of Administrative Errors
• Interviewer Error
– Mistakes made by interviewers who fail to record
survey responses correctly
– Could also occur if selective perception causes
interviewers to misrecord data that do not support
their own attitudes and opinions
• Interviewer Cheating
– Practice by interviewers of filling in fake answers or
falsifying questionnaires.
– If suspect this, tell interviewers that a small
percentage of respondents will be called back to
confirm whether the initial interview was actually
conducted
Classification of Survey Methods
Survey Methods
Telephone
Personal
Electronic
Mail/Fax
Interview
In-Home
Traditional
Telephone
Mail
E-Mail
Mall Intercept
Computer-Assisted
Telephone Interviewing
Mail Panel
Computer-Assisted
Personal Interviewing
Internet
Personal Interviews
• Form of direct communication in which an
interviewer asks respondents questions
face-to-face.
Advantages of Personal Interviews
• Opportunity for Feedback
• Probing Complex Answers
– Interviewer asks for clarification or expansion of
answers
• Length of Interview
• Completeness of Questionnaires
– Minimizes Item Nonresponse: failure by a respondent
to answer a question on a questionnaire
• Props & Visual Aids
• High Participation
Disadvantages of Personal
Interviews
• Interviewer Influence
• Lack of Respondent Anonymity
• Cost
Types of Personal Interviews
• Door-to-Door (In-Home) Interviews
–Conducted at the respondent’s home
or place of business
• Mall Intercept Interviews
–Personal interview conducted in a
shopping mall or other high-traffic
area
Aspects of Personal Interview
• Speed of data collection
– Slow to Moderate
• Geographical flexibility
– Limited to moderate
• Respondent cooperation
– Excellent
• Versatility of questioning
– Quite versatile
Aspects of Personal Interview
• Questionnaire length
– Long
• Item nonresponse
– Low
• Possibility of respondent misunderstanding
– Lowest
Aspects of Personal Interview
• Degree of interviewer influence of answer
– High
• Supervision of interviewers
– Low to Moderate
• Anonymity of respondent
– Low
Aspects of Personal Interview
• Ease of call back or follow-up
– Difficult
• Cost
– Highest
• Special features
– Visual materials may be shown or
demonstrated; extended probing possible
Telephone Surveys
Telephone Surveys
• Speed of Data Collection
– Very fast
• Geographical Flexibility
– High
• Respondent Cooperation
– Poor
• Versatility of Questioning
– Moderate
Telephone Surveys
• Questionnaire Length
– SHORT!!!!!!!!!!!!!!!!!!!
• Item Nonresponse
– Medium
• Possibility of Respondent Misunderstanding
– Average
• Degree of Interviewer Influence of Answer
– Moderate
Telephone Surveys
• Supervision of interviewers
– High, especially with central location WATS
interviewing
• Anonymity of respondent
– Moderate
• Ease of call back or follow-up
– Easy
Telephone Surveys
• Cost
– Low to Moderate to High
• Special features
– Fieldwork and supervision of data collection
are simplified; quite adaptable to computer
technology
Self-Administered
Questionnaires
SELF-ADMINISTERED
QUESTIONNAIRES
PAPER
QUESTIONNAIRES
MAIL
IN-PERSON
DROP-OFF
INSERTS
ELECTRONIC
QUESTIONNAIRES
FAX
E-MAIL
INTERNET
WEB SITE
KIOSK
Mail Surveys
Mail Surveys
• Speed of data collection
– Researcher has no control over return of
questionnaire; slow
• Geographical flexibility
– High
• Respondent cooperation
– Moderate--poorly designed questionnaire will
have low response rate
Mail Surveys
• Versatility of questioning
– Highly standardized format
• Questionnaire length
– Varies depending on incentive
– Varies depending on whether business or
consumer respondents
• Item nonresponse
– High
Mail Surveys
• Possibility of respondent misunderstanding
– Highest--no interviewer present for clarification
• Degree of interviewer influence of answer
– None--interviewer absent
• Supervision of interviewers
– Not applicable
Mail Surveys
• Anonymity of respondent
– High
• Ease of call back or follow-up
– Easy, but takes time
• Cost
– Moderate
Improving Response Rates
Methods of Improving Response Rates
Prior
Notification
Incentives
Monetary
Prepaid
Follow-up
Nonmonetary
Promised
Other
Facilitators
E-Mail Questionnaire Surveys
• Speed of data collection
– Virtually Instantaneous
• Geographic flexibility
– worldwide
• Cheaper distribution and processing costs
E-Mail Questionnaire Surveys
• Flexible, but
– Extensive differences in the capabilities of
respondents’ computers and e-mail software
limit the types of questions and the layout
• E-mails are not secure and “eavesdropping”
can possibly occur
• Respondent cooperation
– Varies depending if e-mail is seen as “spam”
Internet Surveys
• Self-administered questionnaire posted on a site.
• Respondents answer questions displayed online by
highlighting a phrase, clicking an icon, or keying
in an answer.
• Many of the same advantages & disadvantages as
E-Mail Surveys.
• Key differences between Internet & E-Mail
surveys:
– Must rely on respondents coming to the site (and
deciding to participate)
– Data collection can be much slower
Internet & E-Mail Surveys
Never forget:
• Not all individuals in the general public have
Internet access.
• Many respondents lack powerful computers with
high-speed connections to the Internet.
• Many respondents’ computer skills will be
relatively unsophisticated.
There is no best form
of survey; each has
advantages and
disadvantages.