Transcript Data Collection-- Issues
Attitude Measurement, Question Scaling and Questionnaire Design
Feb 10, 2011
Today’s Objectives
To accelerate your learning with respect to: 1) the basics of attitude measurement 2) scaling techniques 3) the fundamentals of questionnaire design
Attitude Measurement Overview
•
Attitudes as surrogates for behaviour
•
Useful in diagnosis and explanation
•
Intangible/inferred --more like states or (in some cases) pre dispositions
•
Three general types:
– – –
cognitive affective intentional
Closed-ended Questions Vs Open-ended Questions
25 reasons for switching to another financial institution for investments lack of caring lack of advice have to wait inconvenient don’t change/improve technology not up to date hard to access new channels -- on-line, telephone don’t support the community solid investment advice not there poor fund performance solid investment advice not there poor fund performance don’t spend time with you don’t give you investment help don’t have experienced RSP counsel not the best fund offering returns on GICs not good don’t offer bank/non-bank funds not a wide range of international investments limited range of funds in general RSP literature hard to understand make it hard to contribute website not effective info source internet transactions difficult cannot transact easily by phone
Closed-ended Questions Vs Open-ended Questions
Use open-ends when:
you cannot anticipate the answers answers are complex you need their own words you need to probe responses (e.g. what do you mean by “convenient”) you want to understand what is salient -- what is on their minds you wish to complement closed-ended approaches Use closed-ended questions when:
– – – – – –
you require specific tasks to be done you wish to profile or describe in detail you want reactions to ideas which you present your task is more to precisely measure than to `explore’. You wish to capture attitudes
• • •
affective cognition intention Often large batteries of attributes are used
Four Primary Scales Of Measurement
Scale Nominal Characteristics Examples A scale where the #s serve as labels or tags to identify or classify objects Player #s, brand names, gender Descriptive Statistics Counts, %, mode Ordinal Percentile, median Interval Ratio A scale where the #s indicate the order of objects relative to each other but not the size of differences between them Preference rankings, market position, social class A scale that arranges objects according to their magnitudes in units of equal intervals. Differences between objects can be compared but 0-point is arbitrary Temperature in o C, perceptions, beliefs, attitudes, consumer price index A scale that has absolute rather than relative values
and
an absolute zero where a given attribute is absent. Can classify (N), rank (O), calculate differences (I) & ratios (R) Length, age, income, sales, market share Range, mean, s.d. Geometric mean, harmonic mean
1) Examples Of Nominal Scale Questions
A scale where the numbers serve as labels or tags to identify or
classify
objects
1) A Simple Dichotomous Question
A closed-ended question that requires the respondent to choose one of two alternatives: Yes/No, True/False, Agree/Disagree, Support/Oppose, Male/Female
Do you own a car? Yes No 1 2 Refused/No Answer 3 Please indicate your gender: Male Female 1 2 Refused/No Answer 3 Do you support or oppose the death penalty?
Support Oppose Undecided Refused/No Answer 1 2 3 4
Results reported as %
1) Examples Of Nominal Scale Questions (Cont.)
A scale where the numbers serve as labels or tags to identify or
classify
objects
2) A Determinant-Choice Question
A closed-ended question that requires the respondent to choose one, and only one answer, from among multiple alternatives
Which of the flavours of ice cream listed below is your personal favourite? (Circle one answer) Butter Pecan Chocolate Strawberry Vanilla Other None/Don’t like ice cream 1 2 3 4 5
6 Please specify:
A “free-response” item Results reported as %
1) Examples Of Nominal Scale Questions (Cont.)
3) A Checklist Question
A closed-ended question that allows the respondent to provide multiple answers to a single question by checking-off all items that apply to him/her
Which of the following pets, if any, do you have? (Circle all that apply) Bird Cat 1 2 Dog 3 Fish 4 Other 5
None/Don’t have any pets 6
Results reported as %
Please specify:
A “free-response” item
4) An Open-Ended Question
A nominal scale question in that respondent answers are
classified
into response categories after-the-fact based on answer content Results reported as %
2) Examples Of Ordinal Scale Questions
A scale where the numbers indicate the order of objects relative to each other but not the size of differences between them
1) A Frequency Question
A closed-ended question about the general frequency of occurrence of an event. Response alternatives are mutually exclusive (non-overlapping) and exhaustive (cover the set of all possible choices)
How often do you personally watch the MTV channel? (Circle one answer) Every day 5 – 6 times a week 2 – 4 times a week Once a week Less than once a week Hardly ever/Never Don’t know/Not sure 1 2 3 4 5 6 7
Note that the distance between categories is not equal Results reported as %
2) Examples Of Ordinal Scale Questions (Cont.)
A scale where the numbers indicate the order of objects relative to each other but not the size of differences between them
3) A Ranking Question
A measurement task that requires respondents to rank order a set of objects in terms of overall preference, importance or some characteristic of the object (e.g., size, price, etc.). This task becomes more difficult as the number of alternatives increases [n*(n -1)/2] and may be preceded by a sorting task first. Ranking tasks cannot be done over the telephone
Example:
Shown below is a list of 20 vacation destinations. Please indicate the five places you would most like to visit for a vacation, assuming price were no object. To do this, please write a “1” beside your first choice, a ”2” beside your second choice and so on for each of your top five vacation destinations.
Results reported as %
3) Interval Scale Questions: The Likert Scale
A scale where respondents rate how strongly they agree or disagree with carefully constructed statements. Several scale items may be used to create a summated index or composite score The most popular scale by far for measuring attitudes Generally treated as an interval scale The wording of the answer alternatives and the number of scale intervals (3 to 10) varies for a Likert scale (See examples below) 1) Playing tennis is a great way to exercise
(Circle one answer)
Strongly Disagree Disagree Neither Agree Nor Disagree Agree Strongly Agree 1 2 The most common Likert scale 3 4 5 Results: % in top box + top 2 boxes, or the mean
3) Interval Scale Questions: The Likert Scale (Cont.)
2) Women should put their family before their career Disagree Completely Disagree Somewhat Neither Agree Nor Disagree Agree Somewhat Agree Completely Results: % in top box +/or top 2 boxes or the mean 1 2 3 4 5 3) Using the 10-point rating scale below, please write in the number which best describes how much you agree each of the following statements applies to XYZ Company: Don’t Know 0 Do Not Agree At All Agree Completely 1 2 3 4 5 6 7 8 9 10 Provides innovative products and services………………………………… Is committed to excellent customer service………………………………… Charges reasonable fees for the services it provides……………………... Is a leader in the use of computer technology……………………………... Is a company I would recommend to others……………………………….. Results: % in top 2/3 boxes
or
the mean on a 5 to 10-point scale
3) Interval Scale Questions: The Semantic Differential
A measure of attitudes consisting of a series of seven-point rating scales that use bipolar adjectives to anchor the beginning and end of each scale Weights assigned to each position on the rating scale: 7, 6, 5, 4, 3, 2, 1 or +3, +2, +1, 0, -1, -2, -3.
Please put a “X” at that point on
each
impression of
Labatt’s Blue
rating scale below that best describes your brand of beer: Something special ___ : ___ : ___ : ___ : ___ : ___ : ___ Just another beer Weak ___ : ___ : ___ : ___ : ___ : ___ : ___ Strong Really refreshing ___ : ___ : ___ : ___ : ___ : ___ : ___ Not really refreshing Light feeling ___ : ___ : ___ : ___ : ___ : ___ : ___ Heavy feeling Aged a long time ___ : ___ : ___ : ___ : ___ : ___ : ___ Not aged a long time Results: Usually the mean where the “positive” end of the scale = 7 and the “negative” end = 1
3) Other Common Interval Scales
Behavioural Intention
Results: % in top box + top 2/3 boxes, +/or the mean Definitely will buy Probably will buy Might or might not buy Probably will not buy Definitely will not buy 5 4 3 2 1 Extremely likely Very likely Somewhat likely About 50-50 chance Somewhat unlikely Very unlikely Extremely unlikely 7 6 5 4 3 2 1
Liking
Like a lot Like somewhat Indifferent Dislike somewhat Dislike a lot 5 4 3 2 1 Very interested Quite interested Somewhat interested Not very interested Not at all interested 5 4 3 2 1 Results: % in top box + top 2 boxes, +/or the mean
3) Other Common Interval Scales (Cont.)
Satisfaction
Completely satisfied Somewhat satisfied Neither satisfied nor dissatisfied Somewhat dissatisfied Completely dissatisfied 5 4 3 2 1 Very satisfied Quite satisfied Somewhat satisfied Not at all satisfied 4 3 2 1 Very satisfied Mostly satisfied Partly satisfied/Partly dissatisfied Mostly dissatisfied Very dissatisfied 5 4 3 2 1 Results: % in top box + top 2 boxes +/or the mean
Steps in Questionnaire Construction
•
1) Planning what to measure:
• •
how will the information be used does it directly address the research objectives
•
2) Formatting the questionnaire:
•
open-ended vs closed ended questions
•
3) Question wording:
• • • • • •
simple, easily understood language lack of ambiguity interpretability loaded questions complexity double-barreled questions
Steps in Questionnaire Construction (Cont’d)
•
4) Question Sequencing-- some guidelines:
• • • • • •
natural, logical interview flow start with the simple things watch for order bias sequencing open-ended questions and closed-ended ones reducing ‘demand’ characteristics (the respondent tells you what you want to hear) treating levels of awareness of an object
Types of Scaling Used In Marketing Research – Single item • itemized category scale • comparative scale • rank-order scale • Q -sort • Constant sum • Paired Comparison – Multiple Item • Likert scale • semantic differential scale • associative scale
Some Examples Of Lifestyle Statements
Conservative/Liberal
• • • • • I have somewhat old-fashioned tastes and habits Everything is changing too fast today People today are more patriotic than they were a few years ago Couples should live together before getting married Government should not attempt to control big corporations
Contentment
• • • • • I am a happy person I always find something good in everyone I often feel lonely Others get more out of life than I do I wish I could change my role in life and do something entirely different • • • •
Family Values
Women should put their family before their career The needs of the children should come first in a family Our family eats dinner together six out of seven times a week I prefer a marriage where husband and wife share housework equally
Some Examples Of Frozen Meal Psychographics
Please circle the answer which best describes how much you agree or disagree with each statement shown below.
• • • • • • • • • • • • •
Frozen meals are a lot better now than they were a few years ago Frozen meals are too expensive to serve regularly Frozen meals should come in larger size portions All brands of frozen meals taste the same Frozen mea!s provide good value for the money Frozen meals are nutritious I tend to stock-up on frozen meals when they are on "special" I'm not buying as many "low calorie" frozen meals as I did a year ago Frozen meals taste bland Preparing a frozen mea! is easier than preparing a meal from scratch I'm starting to replace frozen meals with more fresh foods I prefer frozen complete meals over frozen entrees or single dishes Frozen meals are good to serve to children
What’s Wrong with These Questions (If anything)?
a) Typical Poll Question:
“Do you object to the fact that many of Canada’s youth are dying needlessly in Iraq?”
b) A typical Lou Dobbs on-line poll:
“Would you agree that the export of American jobs to Asian countries is an affront to many hard-working blue collar Americans who cannot get jobs?” c) d) Social opinion poll: “Do you agree that torture is always wrong in principle?” Surveying potential product improvements “Do you think Tim Horton’s should improve on the speed of its service in drive thrus?”
a)
How We Might Correct Them
Corrected Poll Question:
“Some people have said that many of Canada’s youth are dying needlessly in Afghanistan and others have said that, while deaths in the military are always unfortunate, the importance of the cause being defended is what first must be considered? Which do you agree with?”
b) c)
d)
Corrected typical Lou Dobbs on-line poll:
For each of the statements below, please tell me if you agree or disagree with that statement: “Buying imported goods such as cars penalizes blue collar Americans who need work.” “Free trade is about competition and efficiency: that is, letting jobs go to those who can perform them most effectively”
Corrected social opinion poll:
“Do you agree or disagree that torture is wrong in principle?” Corrected potential product improvements question How important is it that Tim Horton’s improve on each of the following areas: Very Fairly the speed of service in drive-thrus the speed of its service in the restaurant the accuracy of its order taking etc Not very (Even better: How important is it to you personally services
that Tim Horton’s improve on each of the following OR AN OPEN ENDED QUESTION: What areas of improvement would you want to see in Tim Horton’s drive thrus, if any?