Document 7319318
Download
Report
Transcript Document 7319318
Questionnaire
Design
What is a
Questionnaire?
A set of Questions
designed to generate
the statistical
information from a
specific demographic
needed to accomplish
the research objectives
Surveys are more than a collection
of unambiguous questions
How questions are specified and
put together will influence the
respondents’ willingness to
participate & the responses they
provide
Purposes of the Questionnaire
Ensures standardization and comparability of the data
across interviews – everyone is asked the same questions
Increases speed and accuracy of recording
Facilitates data processing
Allows the researcher to collect the relevant information
necessary to address the management decision problem
Improper design can lead to?
Incomplete information
Inaccurate data
Higher costs
Criteria to consider
Does it provide the necessary
information
Does it consider the respondent
Does it meet editing, coding and
data processing requirements
Designing the Questionnaire
Determine survey objectives: Plan what to measure.
Decide on format. E.g. personal interview, telephone, self.
Formulate questions to obtain the needed information
Decide on the wording of questions
Decide on the question sequence and layout of the
questionnaire
Using a sample, test the questionnaire for omissions and
ambiguity
Correct the problems (pretest again, if necessary)
The Major Decisions in
Questionnaire Design
1. Content - What should be asked?
2. Wording - How should each question be phrased?
3. Sequence - In what order should the questions be
presented?
4. Layout - What layout will best serve the research
objectives?
The most difficult step is specifying exactly what
information is to be collected from each respondent
Content: Questions must meet 5 requirements
1. Are the questions relevant. Do they pertain to the
research problem
2. Are the questions accurate. Do they accurately depict
the attitudes, behaviors, etc. intended to investigate
3. Do respondents have the necessary information?
• Qualify respondents
4. Do respondents understand and interpret the question
correctly
5. Will respondents give the information?
How should the questions be asked?
Format: How much freedom do we give respondents in
answering questions
Open-ended questions
– What do you look for most in a job?
– Is there anything else you would like to add about the product
Open Ended Questions: key advantages
Wide range of responses and information can be obtained
Answers based on respondent’s not researcher’s frame of
reference – consumer’s terms
Lack of influence. Don't channel respondents thinking
Can help interpret closed-ended questions - why
Particularly useful as introduction to survey or topic
When it’s important to measure the salience of an issue
When too many possible responses to be listed or
unknown
Open-ended questions: Key disadvantages
Ability and/or willingness of respondent to answer
Interviewer’s ability to record answers quickly or summarize
accurately & probe effectively
Interviewer’s attitude influences response
Time consuming (interview sessions, tabulation, classification,
assignment, validation)
Difficulty in coding
Require respondents to be articulate
Respondents may miss important points
Non-response
Closed-ended questions (Fixed-alternative responses)
What do you look for most in a job?
____ Work that pays well
____ Work that gives a sense of accomplishment
____ Work where you make most decisions by yourself
____ Work that is steady with little chance of being laid off.
Advantages
Ease of understanding
Requires less effort on part of interviewer and respondent
Ease of tabulation & analysis
Less error prone
Less interviewer bias
Less time consuming
Answers directly comparable from respondent to respondent
Closed-ended questions (Fixed-alternative questions)
Disadvantages
Middle/Neutral categories often selected inappropriately
(ignorance, safety)
Less opportunity for self-expression or subtle qualifications
Less involving for respondents
Order of response categories can have major impact on results
Key tradeoff
Want to get respondent to address issues our research is
concerned with (Forced response) and at same time give
respondent opportunity to honestly opt out of question (i.e., Don’t
Know, No Answer, Neither Agree nor Disagree) so as not to dilute
data collected
Dichotomous Questions
Should the Alberta Government give consumers an energy rebate?
1. Agree
2. Disagree
Advantages
Easy to administer and tabulate
Disadvantages
Prone to large amounts of error since polarized responses
prevent gaining information on the range of variation
Fail to communicate any intensity of feeling
Multiple Choice Questions
Are all possible alternatives included?
Too many alternatives
Position Bias
Scaled Response Questions
Closed ended questions where the response choices are
designed to capture an intensity of feeling (Likert, Staple,
Semantic differential)
Easy to code and more powerful statistical tools
Main problem: Respondent misunderstanding
Question Wording
Can have major impact on how respondent interprets question
All respondents should interpret in the same way
Things to avoid
Complexity: use simple, direct, conversational language
leading questions -- that suggest or imply certain answers
loaded questions -- suggest social desirability, or are emotionally
charged. Have you purchased a high quality Sony TV this year
Ambiguity and vagueness: Words such as “often”, “occasionally”,
“usually”, “regularly”, “frequently”, “many”, should be used with
caution. If these words have to be used, their meaning should be
explained properly.
Which province is bigger Manitoba or Alberta? – would your
answer be based on population or area?
More things to avoid
long-worded questions
double-barreled questions. Questions that refer to two
or more issues within the same question. Where
respondent may agree with only 1 part of multipart
statement.
Do you think Nike offers better pricing and variety
than other brands
Making implicit assumptions
Jargon
More things to avoid
burdensome questions - that may tax the respondent’s
memory
How many tubes of toothpaste have you purchased in
the last 3 months
Have you purchased toothpaste in the past week
Embarrassing, sensitive, or threatening questions:
Have you charged more on your credit card than you
should
Use third person do you think most people…..
Question Sequence & Layout Decisions
Initial stages
Screening or qualifying Questions: Have you been to the movie
theatre this month?
Need to gain & maintain respondent’s cooperation
Make questionnaire simple for interviewer to administer
opening questions should be interesting, simple, and easy to
answer.
which theatre did you go to last?
Transition Questions
What aspects of the theatre did you like best
Questions directly related to research objectives which
require more effort and get respondent thinking about topic
Difficult and Complicated Questions
The following 10 questions relate to the characteristics of
theatres
Respondent now committed and can see an end in sight
Classifying and Demographic Questions
What is your average annual household income
May not be answered (sensitive or threatening) but most
questions have been answered
Other Considerations
Questions should flow logically from one to the next
General questions should be asked before more specific ones
Earlier questions should not influence response to later ones
Questions should flow from factual and behavioural questions to
attitudinal and opinion questions
Questions should flow from the least sensitive to the most sensitive.
Questions should flow from unaided (which brand do you prefer) to
aided questions (which brand do you prefer Tide, Cheer, Gain)
Demographic questions should come at the end.
1. initial questions - screening and rapport questions
2. Middle - product specific questions
3. End demographic questions.
Other Considerations
Use multiple questions instead of one
Similar questions together
consistent mindset for respondents
Develop a logical flow
Use transitions between sections (E.g. In this section we ask
questions about X)
Distinguish between instructions, questions and responses
Distinguishing question and responses:
CAPS or BOLD or Underline versus lower case or unformatted
INSTRUCTIONS LAYOUT A:
Do you agree, disagree or have no opinion that this company has:
• A good vacation policy - disagree/not sure/agree.
• Good management feedback - disagree/not sure/agree.
• Good medical insurance - disagree/not sure/agree.
INSTRUCTIONS LAYOUT B:
Does this company have:
___
• A good vacation policy
Disagree Not Sure Agree
1
2
3
• Good management feedback 1
2
3
• Good medical insurance
2
3
Matrix question
1
Filter and pivot questions should be used as necessary.
A FILTER question is one that screens out
respondents who are not qualified to answer a
second question.
A PIVOT question is a type of filter question that is
used to determine what version of a second
question to ask.
Developing a logical flow
If the questionnaire deals with several topics,
complete questions on a single topic before moving
on to a new topic
If topics are related, ask questions on related topics
before asking questions about unrelated topics
If you ask questions about behaviors over some time
period, follow chronological order backward in time
When changing topics, use some transitional phrase
QUESTIONNAIRE LAYOUT
The layout and physical attractiveness of a
questionnaire are important aspects
Questionnaires should be designed to appear as
short as possible
Questionnaires should not appear overcrowded
Leave lots of space for open ended questions
Questionnaires in booklet form are often
recommended
OPENING
Provide first name (at least)
Provide name of company doing research
Provide reason for survey and topic
State that no selling will be involved and no personal data
other than for statistical purposes
Tell respondent approximate time to complete
Reinforce that respondent’s time is appreciated
Invite to participate
AND CLOSING
Thank for time
Ask if they had a positive experience and remind them that
their opinions count
Pretesting and Correcting Problems
Purpose of pretest: To ensure that the questionnaire meets the
expectations in terms of the information that will be obtained
Is question necessary – does it serve a purpose, will info be used
Missing important variables – does it provide the info needed
Match questions to objectives
Pretest Specific Questions For
• Variation
• Meaning
• Task difficulty
• Respondent interest and attention
• Ambiguous, ill-defined, loaded, double-barreled questions
Pretest the Questionnaire
• Flow of the questionnaire
• Skip patterns
•Length
Respondent Interest and Attention
Questionnaire Design Flow Chart
Step 1: Specify what information will be sought
Step 2: Determine type of questionnaire and method of
administration
Step 3: Determine the content of individual questions
Step 4: Determine form of response for each question
Step 5: Determine wording for each question
Step 6: Determine sequence of questions
Step 7: Determine physical characteristics of questionnaire
Step 8: Revise steps 1-7, revise if necessary
Step 9: Pretest questionnaire, revise if necessary
Step 10: implement
A QUESTIONNAIRE
IS ONLY AS GOOD
AS THE QUESTIONS
IT ASKS