What Is Asked in a Survey?
Download
Report
Transcript What Is Asked in a Survey?
SURVEY RESEARCH
BUSN 364 – Week 12
Özge Can
Survey Research
Asks a large number of people (respondents) the
same questions about their beliefs, opinions,
characteristics and past and present behaviors
Standard
questionnaire forms
Self-reported beliefs and behaviors
Produce information that is statistical in nature =>
quantitative
Asks many questions at once, thereby measuring many
variables => one can test multiple hypotheses
Alternative explanations are statistically controlled
What Is Asked in a Survey?
Behavior
How frequently do you brush your teeth?
Attitudes/beliefs/opinions
What is the biggest problem facing the youth today?
Characteristics
What is your age? Are you married or not?
What Is Asked in a Survey?
Expectations
Do you plan to buy a car in the next 12 months?
Self-Classifications
Which social class would you put your family into?
Knowledge
Who was elected as the city mayor in the last election?
Steps in Conducting
a Survey (1):
Steps in Conducting
a Survey (2):
Principles of Good Question Writing
Two key principles guide writing good survey
questions:
Avoid
any possible confusion
Keep the respondent’s perspective in mind
We want a survey to provide a valid and reliable
measure:
Respondents
should easily understand the question’s
meaning as you intended (clear, relevant and
meaningful) and answer completely and honestly
Principles of Good Question Writing
1. Avoid jargon, slang, abbreviations or acronyms
When surveying general public, use the language of
popular culture
2. Avoid ambiguity, confusion and vagueness
“What is your income?” “Do you jog regularly?”
3. Avoid emotional language and prestige bias
“What do you think about paying murderous terrorists
who threaten to steal the freedom of peace-loving
people?”
Principles of Good Question Writing
4. Avoid double-barreled questions
“Does your employer offer pension and health insurance
benefits?”
5. Avoid leading questions
“You don’t smoke, do you?”
6. Avoid questions beyond respondents’ capabilities
“How did you feel about your brother when you were 6
years-old?”
“How much was your electricty bill last year?”
Principles of Good Question Writing
7. Avoid asking about distant future intentions
“Suppose 8-years later you become a manager, how
would you behave to your employees?”
8. Avoid double negatives
“I ain’t got no job”
“Do you agree or disagree that students should not be
required to take a comprehensive exam to graduate?”
9. Avoid overlapping or unbalanced response
categories
Make response choices mutually exclusive, exhaustive
and balanced.
Getting Honest Answers
Sensitive Topics => Illness and disability (mental health
problems, cancer), illegal or deviant behavior (drug or
alcohol use, law violations), financial status (income, debts,
occupation)
1.
2.
3.
4.
Threatens people’s presentation of themselves; their
positive image. We should particularly cautious about these
questions and the results. To increase honest answering:
Create comfort and trust
Use enhanced phrasing
Establish a desensitizing context
Use anonymous questioning methods
Getting Honest Answers
Social Desirability Bias => occurs when respondents
distort answers to conform to popular social norms.
They give a socially acceptable answer rather than
an honest answer
Ex:
people tend to overstate being highly cultured,
being outgoing and fun, giving money to charity, having
a good marriage, loving their children
One way to reduce it: phrase questions in ways that
make norm violation appear less objectifiable and
give respondents “face-saving” alternatives
Open-Ended vs. Close-Ended Questions
I. Open-ended question:
A
type of survey inquiry that allows respondents
freedom to offer any answer they wish to the question
II. Close-ended question:
A
type of survey inquiry in which respondents must
choose from a fixed set of answers
Partially
open questions => a set of fixed choices with a
final open choice of “other”
Open-Ended Questions
Open-Ended Questions
Close-Ended Questions
Close-Ended Questions
Valid Responses
Three types of answers from respondents yield
invalid responses:
Swayed opinion => falsely overstating a position
False positive => selecting an attitude position but lacking
any knowledge on the issue
False negative => respondent refuses to answer the
question when he/she actually has information or an
opinion
Neutral
positions: Should we offer respondents who lack
knowledge or have no position a neutral or “no opinion”
choice?
Valid Responses
Response Set (Reponse Bias) => tendency of
respondents to agree with every question in a series
rather than carefully thinking through one’s answer to
each
To avoid it: include different question types; change the
sequence of response categories
Response Set Bias:
Valid Responses
To get valid responses:
Choose
the appropriate response style for the
question: agreement scales, rankings or ratings?
Present the alternatives fairly
Attach numbers to a response scale to assist
respondents and give them a clue for understanding
Use visual presentations, colors, symbols and pictures
Appropriate question format and questionnaire design
Valid Responses
Questionnaire Design Issues
Length of survey or questionnaire
No absolute proper length; it depends on the survey format
(e.g. mail, internet) and respondent characteristics
Short questionnaires are appropriate for the general
population
Question order and sequence
You should sequence questions to minimize respondent
discomfort and confusion
After an introduction explaining the survey, make opening
questions easy to answer and pleasent.
Order effect => answers to earlier questions can influence
later ones
Questionnaire Design Issues
Layout and format
Questionnaire
layout => Appearence of the
questionnaire form should be clear, neat and easy to
follow; sheets, fonts, envelope, cover letter...
Question format => Should respondents circle
responses, check boxes, fill in dots or write in a blank?
Nonresponse
The
failure to get a valid response from every sampled
respondent = nonresponse bias
Can be a major problem if a high proportion of the
sample does not respond; lowers generalizability
Ways to Increase Response Rate
•
•
Address to specific individual
Include a cover letter
–
–
–
–
•
•
•
•
•
•
•
Request cooperation
Guarantee confidentiality
Explain purpose
Provide researcher contact information
Include postage-paid, addressed return envelope
Easy to follow questionnaire
Send follow-up reminders
Avoid conducting study during busy holidays
Back page for general comments
Advertise legitimate sponsors (university, gov’t agency, etc.)
Small monetary incentives
Types of Surveys
Mail surveys
Telephone surveys
Face-to-face surveys
Internet/web surveys
Types of Surveys
Features to consider when choosing a particular
survey type:
Administrative
issues => response rate, cost,
speed, length
Researcher control on questions => explaining the
questions, flexible formats
Success with different questions => open-ended
questions, complex or sensitive questions
Sources of bias => social desirability, interviwer
bias, respondent’s reading skills
Constructing a Survey – Some Resources
Many links for carrying out survey research:
http://managementhelp.org/businessresearch/surveys.htm
Information on designing surveys:
http://www.sciencebuddies.org/mentoring/project_id
eas/Soc_survey.shtml
Online survey websites:
SurveyMonkey (www.surveymonkey.com)
PsychSurveys (www.psychsurveys.org)