Questionnaire Design
Download
Report
Transcript Questionnaire Design
Questionnaire Design
“It is not every question
that deserves an answer.”
Publius Syrus
(Roman, 1st century B.C.)
Ask a Sensitive Question, Get a
Sensitive Answer
• Survey researchers believe
that the responses that
people give are valid.
• Care must be taken with
sensitive questions.
• Researchers must take care
in asking relevant questions
in ways that produce the
most truthful results.
Basic Considerations in
Questionnaire Design
• Questionnaire design is one of the most
critical stages in the survey research
process.
– A questionnaire (survey) is only as good
as the questions it asks — ask a bad
question, get bad results.
– The questions must meet the basic criteria
of relevance and accuracy.
What Should Be Asked?
• Questionnaire Relevancy
– All information collected should address a research question
that helps the decision maker in solving a current marketing
problem.
• Questionnaire Accuracy
– The information is valid; it faithfully represents reality.
• Questionnaires should use simple, understandable, unbiased,
unambiguous, & nonirritating words.
• Questionnaire design should facilitate recall & motivate
respondents to cooperate.
• Proper question wording & sequencing to avoid confusion &
biased answers.
Major Decisions in Questionnaire
Design
What should be asked?
How should each question be phrased?
In what sequence should the questions be
arranged?
What questionnaire layout will best serve
the research objectives?
How should the questionnaire be pretested?
Does the questionnaire need to be revised?
Phrasing Questions
• Open-ended questions
–Aka essay questions, short-answer
questions
• Fixed-alternative questions
–Aka closed or closed-ended questions
Open-Ended Response Questions
– Pose some problem & ask respondents to answer in their
own words.
– Advantages:
• Particularly beneficial in exploratory research, especially when
the range of responses is not known.
• Identify which words & phrases people spontaneously give.
• Valuable at the beginning of an interview.
– Disadvantages:
• High cost of administering open-ended response questions.
• The possibility that interviewer bias will influence the answer.
• Bias introduced by articulate individuals’ longer answers.
Example of Open-Ended Response
Question
Fixed-alternative Questions
– Questions in which respondents are given specific,
limited-alternative responses & asked to choose the
one closest to their own viewpoint.
– Advantages:
•
•
•
•
Require less interviewer skill
Take less time to answer
Are easier for the respondent to answer
Provides comparability of answers
– Disadvantages:
• Researcher may be unaware of all potential responses
• Tendency of respondents to choose more prestigious or
socially acceptable alternative
Example of a Fixed-Alternative
Question
Types of Fixed-Alternative Questions
• Simple-dichotomy (dichotomous) Question
– Requires the respondent to choose one of two alternatives (e.g., yes or no).
– Example:
Did you make any calls with your home (landline) phone during the 7 days?
_____ Yes
_____ No
• Determinant-Choice (multiple-choice) Question
– Requires the respondent to choose one response from among multiple
alternatives (e.g., A, B, or C).
– Example:
Types of Fixed-Alternative Questions
(con’t.)
• Frequency-determination Question
– Asks for an answer about general frequency of occurrence (e.g., often,
occasionally, or never).
• Checklist Question
– Allows the respondent to provide multiple answers to a single question by
checking off items.
• Scale
– Likert, Semantic Differential, Stapel, etc.
Phrasing Questions for Self-Administered,
Telephone, & Personal Interview Surveys
• Influences on Question Phrasing
– Means of data collection — telephone
interview, personal interview, self-administered
questionnaire — will influence question format
& question phrasing.
• Questions for mail, Internet, & telephone surveys
must be less complex than those used in personal
interviews.
• Questionnaires for telephone & personal interviews
should be written in a conversational style.
Best Question Formats Vary by the Interview Medium
Guidelines For Avoiding Mistakes
• Simpler is better
• Avoid leading & loaded questions
– Leading question – directs respondents to an answer you want them to give
– Loaded question – suggests a socially desirable answer or is emotionally
charged.
Guidelines For Avoiding Mistakes
(con’t.)
• “Avoid” ambiguity: Be as specific as possible
– Probably /Definitely; Sometimes/Always, etc.
• Avoid double-barreled items
– Double-barreled question – may induce bias because it covers two or more
issues at once.
– Do you think the President is responsible for the federal government shutdown and the currently rising gasoline prices?
Yes
No
• Avoid making assumptions
– Given Macy’s skill-level at gift wrapping, …….
• All-inclusive response alternatives
• Avoid taxing respondent’s memory
Avoid Common Wording Mistakes in
Questionnaire Design
Order Bias
• Question Sequence
– Order bias
• Bias caused by the influence of earlier questions in a questionnaire or by an
answer’s position in a set of answers.
– Funnel technique
• Asking general questions before specific questions in order to obtain
unbiased responses.
• Randomized Presentations
– Used in electronic questionnaires, but rarely used in printed
questionnaires due to coding difficulties.
• Randomized Response Techniques
– Randomly assigning respondents to answer either the question of
interest (embarrassing) or a mundane & unembarrassing question.
Survey Flow
• Survey flow
– The ordering of questions through a survey.
• Breakoff
– A respondent who stops answering questions before reaching the
end of the survey.
• Filter question
– A question that screens out respondents who are not qualified to
answer a second (or follow-up) question.
• Branching
– Directing respondents to alternative portions of the questionnaire
based on their response to a filter question.
Survey Flow for Eurocar Tour de France Sponsorship
Telephone Questionnaire with Skip Questions
Survey Technology
• Physical Features
– Heat map question
• A graphical question that tracks the parts of an image or advertisement
that most capture a respondent’s attention.
– Status Bar
• A visual indicator that tells the respondent what portion of the survey he
or she has completed.
• Prompting
– Informs the respondent that he or she has skipped an item or
provided implausible information.
• Piping Software
– Allows question answers to be inserted into later questions.
Tracking Points of Interest Using a Heat Map
Illustration of Status Bar & Prompts
Pretesting & Revising Questionnaires
• Pretesting Process
– Seeks to determine whether respondents have
any difficulty understanding the questionnaire
– whether there are any ambiguous or biased
questions.
• Preliminary Tabulation
– A tabulation of results of a pretest to help
determine whether questionnaire will meet the
objectives of the research.
I Give Up!
Large portion of respondents give up
before finishing & abandon the survey
(break-offs).
Guidelines
Visually appealing & easy to read
Fewer questions per page (no more than 20)
4 pages maximum for consumers
6 pages maximum for business leaders
Question order (funnel vs. reverse-funnel)
Sensitive questions & open-ended questions
encourage break-offs &/or item non-response
Sophisticated samples increase response rate
Make information requests legitimate
Pretesting is important (pretest EVERYTHING)
Questionnaire Reproduction
(if mail survey)
Professional appearance
Booklet format for long
questionnaires
Place directions as close to
questions as possible
Expect reproduction errors