Transcript STAT472

STAT 472
Survey Design
Constructing the Questionnaire



A good questionnaire forms an integrated
whole.
The researcher weaves questions together so
they flow smoothly.
He or She includes Introductory Remarks,
Instructions for clarification and measure each
variable with one or more survey questions





There are two keys principles:
- Avoid confusion (Keep it clear and simple)
- Keep the respondent’s perspective in mind
Good survey questions give the researcher
Valid and reliable measures
Help respondents feel that they understand the
question and that their answers are meaningful
Do not mesh with a respondent's view point
A survey researcher must exercise extra care if the
respondents are come from different life situations
Researchers face a dilemma
All respondents hear exactly the same questions,
equally clear, relevant, meaningful and exact wording
If respondents have diverse back grounds and frames
of reference, the exact same wording may not have
the same meaning. Yet, tailoring question wording to
each respondent makes comparisons almost impossible
Question writing is more of an art than a science. It takes skill,
practice, patience, and creativity
(1) Avoid jargon, slang, and abbreviations:
(unless a specialized population is being surveyed)
Jargons and technical terms come in many forms:
Plumbers talk about snakes
Slang: is a kind of jargon
Skiers about a hotdog
Abbreviations: for example NATO means
 North Atlantic Treaty Organization
 North African Tea Office
 National Auto Tourist Organization
 Native Alaskan Trade Orbit
(2) Avoid ambiguity, confusion and vagueness
Ambiguity and vagueness plague most question writers

For example: What is your income?
Could mean, weekly, monthly, or annual; family
or personal; before tax or after; for this year or last
year; from salary or from all sources
"Do you jog regularly? Yes / No”
Use of indefinite word regularly or response categories
“hinges on the meaning of the word regularly. It may define as every day,
others as once a week”
 Good Question: “To reduce respondent confusion and get more information”
Do you jog "about once a day”, “a few times a week”, “once a
week”?
The question wording is an art that may improve
with practice, patience, and pilot testing
Here are three survey questions written by
experienced professional researchers. They
revised the original wording after a pilot test
discover that 15% of respondents asked for
clarification or gave inadequate answers
(e.g., don't know)
Original question
Q1. Do you exercise or play sports regularly?
Problem: What counts as exercise?
Response to question: 48% saying “YES”
% asking for clarification: 5%
Revised question
Q1. Do you do any sports or hobbies, physical activities, or
exercise, including walking, on a regular basis?
Response to question: 60% saying “YES”
% asking for clarification: 0%
Original question
Q2. What is the average number of days each week
you have butter?
Problem: Does margarine count as butter?
Response to question: 33% saying “none”
% asking for clarification: 18%
Revised question
Q2. The next question is just about butter-not including
margarine. How many days a week do you have butter?
Response to question: 55% saying “none”
% asking for clarification: 13%
Original question
Q3. [Following question on eggs] What is the number of servings
in a typical day?
Problem: How many eggs is a serving? What is a typical day?
Response to question: 80% saying “one”
% asking for clarification: 33%
Revised question
Q3. On days when you eat eggs, how many eggs do you usually
have?
Response to question: 33% saying “one”
% asking for clarification: 0%
(3) Avoid emotional language and prestige bias
Example (emotional): What do you think about a
policy to pay murderous terrorists who threaten
to steal the freedom of peace-loving people?
Is full of emotional words (murderous, freedoms, steal, and
peace)
Problem: respondents may react to the emotionally
laden words rather than to the issue
Example 1 (Prestige bias) Most doctors say that
cigarette smoke causes lung disease for those
near a smoker, do you agree?
Problem: tends to provoke "yes" answers because
people trust doctors
Example 2 (Prestige bias) "Do you support the
president's policy on Zimbobutu?
Problem: tends to provokes "yes" answers
because people trust president
(4) Avoid double-barreled questions
Double-barreled questions consists of two or
more questions joined together.
Example: How do you evaluate the work of the
Jordanian parliament and prime minister?
The two variables are: Jordanian parliament
: prime minister
The wrong way is to ask one question:
How do you evaluate the work of the Jordanian
parliament and prime minister?
The correct way is to ask two questions:
1- How do you evaluate the work of the
Jordanian parliament?
2- How do you evaluate the work of the prime
minister?
Example: Tell me whether you like summer and
winter in Irbid or not?
The correct way is to ask two questions:
1- Do you like summer in Irbid?
2- Do you like winter in Irbid?
(5) Avoid leading (Bias) Questions
Make respondents feel that all responses are legitimate. Do not let
them become aware of an answer that the researcher wants.
A leading question is one that leads the
respondent to choose one response over another
by its wording.
Example: The government should force you to pay
higher taxes.
No one likes to be forced, and no one likes higher taxes.
Alternative: The government should increase taxes,
or the government needs to increase taxes.
Example: Don’t you think that suffering terminal
cancer patients should be allowed to be released
from their pain?
Researchers should never try to make one response option look
more suitable than another.
Alternative: The suffering terminal cancer patients should be
allowed to be released from their pain, do you agree?
Example: Wouldn’t you like to receive our free brochure?
Example: You don't smoke, do you?
(6) Avoid asking questions that are beyond
respondents capabilities.
People have cognitive limitations, especially when it comes to memory
of past events
Example: (asking adult) How do you feel about your
brother when you were 6 years old?
is probably useless
It is pointless to ask people about things that are not natural
ways for them to think.
Example: How many gallons of gasoline did you buy
for your car last year?
"Yet, respondents may be able to answer a question about gasoline
purchases for a typical week, which the researcher can multiply by 52 to
estimate annual purchases