Primary Data and Measurement

Jeremy Kees, Ph.D.

Stages in the Research Process Formulate Problem Determine Research Design Design Data Collection Method and Forms Design Sample and Collect Data Analyze and Interpret the Data Prepare the Research Report

Primary Data: Overview

• Types of Primary Data – Demographic / Socioeconomic Characteristics – Psychological / Lifestyle Characteristics – Attitudes / Opinions – Awareness / Knowledge – Intentions – Motivation – Behavior • What, how much, where, when, how, who • Purchase behavior vs. use behavior • E.g., --- basic hierarchy of effects models

Primary Data: Overview

• Qualities of Primary Data – Versatility – Can be obtained quickly – Quality • Objectivity • Accuracy

Primary Data: Overview

• Two Classes of Primary Data – Observation – Communication

Decisions, Decisions

Degree of Structure Structured Unstructured Communication Observation Degree of Disguise Method of Administration Degree of Structure Degree of Disguise Setting Method of Administration Disguised Undisguised Personal Interview Telephone Interview Mail Questionnaire Structured Unstructured Disguised Undisguised Natural Contrived Human Mechanical

Structure and Disguise Decisions

Structured Unstructured Undisguised Typical Questionnaire (very frequently used) Interviews Open-Ended Questions Disguised least used Motivation Research Word Association Sentence Completion Story Telling

Trade-Offs Among Methods of Survey Administration LIMITED phone mail fax email HI COST personal phone web HIGHLY VERSATILE personal (e.g., mall, home) LOW COST fax mail web email SLOW mail personal fax phone web email FAST TURN AROUND

Mall Intercepts vs. E-Panels

household size average age employed white male college mall tests

2.8

40.5

71% 86% 20% 40%

internet panel tests members

2.9

39.2

3.0

37.2

72% 88% 21% 43% 69% 89% 15% 46%

Correlation between Responses: purchase intent frequency liking price / value mall vs. internet internet test/retest reliability

.86

.94

.94

.97

.85

.90

.91

.99

E-Panels vs. Phone

Time survey took to administer Upon completion, would respondent participate in future studies?

More experienced Internet Users Used rating scale extreme “endpoints” more frequently Internet 12.5

35% yes x x Phone 19.4 minutes 26% yes

Jeff Miller and Alan Hogg “Internet vs. Telephone Data Collection” (www.burke.com). Also see Ashok Ranchhod and Fan Zhou “Comparing Respondents of E-Mail and Mail Surveys,”

Marketing Intelligence & Planning

19 (2001), 254.

Burke White Paper

series 2 (4)

Basic Question Formats

• Open-ended question

•

Unprobed format:

information seeks no additional •

Probed format:

includes a response probe instructing the interviewer to ask for additional information

Basic Question Formats

• Close-ended question

•

Dichotomous:

has only two response options, such as “yes” or “no” •

Multiple response:

has more than two options for the response •

Scaled-response:

utilizes a scale developed by the researcher to measure the attributes of some construct under study • Labeled vs. Unlabeled

Other Considerations…

• The nature of the property being measured • Previous research studies • The data collection mode • The ability of the respondent • The scale level desired…

Types of Scales

• • • •

Nominal scales:

those that use only labels

Ordinal scales:

those with which the researcher can rank-order the respondents or responses

Interval scales:

equal those in which the distance between each descriptor is

Ratio scales:

ones in which a true zero exists

Examples…

Nominal Which of the soft drinks in the following list do you like? (Check ALL that apply): ___Coke ___Dr. Pepper ___Mountain Dew ___Pepsi ___Seven Up ___Sprite Interval Please indicate how much you like each soft drink by checking the appropriate position on the scale: dislike like Coke Dr. Pepper Mountain Dew Pepsi a lot dislike like a lot ____ ____ ____ ___ ____ ____ ____ ___ ____ ____ ____ ___ ____ ____ ____ ___ Seven Up Sprite ____ ____ ____ ___ ____ ____ ____ ___ Ordinal Rank the soft drinks according to how much you like each (most preferred drink = 1, and least preferred drink = 6): ___Coke ___Dr. Pepper ___Mountain Dew ___Pepsi ___Seven Up ___Sprite Ratio Please divide 100 points among these soft drinks To represent how much you like each: ___Coke ___Dr. Pepper ___Mountain Dew ___Pepsi ___Seven Up ___Sprite 100

15

Thinking Ahead…

What analysis technique is used when … 1.

2.

3.

4.

5.

There are 2 nominally scaled variables There are 2 intervally (or ratio) scaled variables There is one DV that your trying to predict or explain with multiple interval variables (the independent variables) You are checking for differences between 2 groups (nominal) for an interval scaled variable You are checking for differences between 4 stores on total $ sales across 4 categories?

Potential Answers --- t-test; multiple regression, oneway ANOVA, factorial ANOVA, correlation, crosstabs, multivariate analysis of variance

Survey (Questionnaire) Design

Procedure for Developing a Questionnaire Step 1: Specify What Information Will be Sought Step 2: Determine Type of Questionnaire and Method of Administration Step 3: Determine Content of Individual Questions Step 4: Determine Form of Response to Each Question

Procedure for Developing a Questionnaire Step 5: Wording of Each Question Step 6: Sequence of Questions Step 7: Physical Characteristics of Questionnaire Step 8: Re-examine Steps 1-7 and Revise if Necessary Step 9: PRE-TEST the Survey, Revise where Needed

Desirable Characteristics

• Brief • Objective • Specific • Relevant

“Other” Important Issues

• Respondent qualification • Frame of reference • Question loading

QUESTION SEQUENCING General Guidelines

• Use (more) simple, interesting opening questions • Use the

funnel approach

specific questions , asking broad questions first, and follow with more • Carefully design branching questions • Ask for classification information last • Place more difficult or sensitive questions near the end

QUESTION SEQUENCING General Guidelines Question ordering #1

1 – EVALUATION OF FAT LEVEL OF A PRODUCT 2 – EVALUATION OF OVERALL PROD. NUTRITIOUSNESS 3 – EVALUATION OF OVERALL PRODUCT ATTITUDE AND INTENTIONS TO PURCHASE

Question ordering #2

1 – EVALUATION OF OVERALL PRODUCT ATTITUDE AND INTENTIONS TO PURCHASE 2 – EVALUATION OF LEVEL OF PROD. NUTRITIOUSNESS 3 – EVALUATION OF FAT LEVEL OF PRODUCT

QUESTION WORDING - General Guidelines

• Use simple words and questions • Avoid ambiguous words and questions • Avoid leading questions---be objective • Avoid implicit alternatives • Avoid generalizations and estimates ---Be specific • Avoid double-barreled questions

1. What is your income?

$10,000 or less………………….1

$10,000 to $25,000……………..2

$25,000 to $50,000……………..3

$50,000 to $75,000………..…….4

$75,000 to $100,000..……..…….5

$100,000 or more…………..……6

What is the problem and how would you revise the question?

2. Do you agree with many experts that the United States will run out of oil by the year 2020?

Yes………………….1

No……………...……2

3. How important was price compared to service when your company decided to buy its current computer network?

Very important…………….4

Somewhat important……….3

Somewhat unimportant…….2

Very unimportant...…...……1

Please rate the importance of the following factors in making the choice among vendors.

Selling price: Not Very Important ___ ___ ___ ___ ___ ___ ___ Important Vendor’s reputation for service: Not Very Important ___ ___ ___ ___ ___ ___ ___ Important

4. Is the speed and efficiency of the drive-in teller services at your regular bank…..(READ CATEGORIES) Very Satisfactory………………………4 Somewhat satisfactory…………………3 Somewhat unsatisfactory………………2 Very unsatisfactory……………….……1 PROBLEMS a. Speed and efficiency (double-barreled) b. Do they use drive-in teller service; Do R’s know what drive in tellers are? What about ATM drive-ins?

c. What are regular banks?

Itemized Rating Scales

• The respondents are provided with a scale that has a number or brief description associated with each category. • The categories are ordered in terms of scale position, and the respondents are required to select the specified category that best describes the object being rated. • The commonly used itemized rating scales are the Likert and semantic differential – Others include Thurstone and Guttman

Itemized Rating Scales

• Likert Scales – requires the respondents to indicate a degree of agreement or disagreement with each of a series of statements about the stimulus objects

Strongly disagree Disagree Neither agree nor disagree Agree 1. Wal-Mart sells high quality merchandise. 1 2X 3 4 2. Wal-Mart has poor in-store service.

3.

I like to shop at Wal-Mart .

4. Wal-Mart has low prices .

1 1 1 2X 2 2 3 3X 3X 4 4 4 Strongly agree 5 5 5 5

Itemized Rating Scales

• Semantic Differential Scales – End points associated with bipolar labels that have semantic meaning SEARS IS: Powerful --:--:--:--:-X-:--:--: Weak Unreliable --:--:--:--:--:-X-:--: Reliable Modern fashioned --:--:--:--:--:--:-X-: Old-

Key Decisions for Itemized Scales

• • • • • Number of categories • Although there is no single, optimal number, traditional guidelines suggest that there should be between five and nine categories Balanced vs. unbalanced • In general, the scale should be balanced to obtain objective data Odd/even no. of categories • If a neutral or indifferent scale response is possible for at least some respondents, an odd number of categories should be used Forced vs. non-forced • In situations where the respondents are expected to have no opinion, the accuracy of the data may be improved by a non-forced scale( DK/NR) Verbal description • An argument can be made for labeling all or any scale categories. The category descriptions should be located as close to the response categories as possible

Measurement Theory

• Construct Validity

– Criterion (Face) Validity – Predictive Validity – Nominological Validity

• Reliability

Construct Validity

Construct Abstract Concept

“Unobservable” “Latent” “Psychological”

We often use Multi-item measures (scales) to assess unobservable constructs (WHY???)

Reliability

• • • – True Score Test Theory All measures have 1. “True” Score 2. “Error” (Random and Systematic) Good measures minimize the systematic error component of the score – – – Types of Reliability Inter-Rater Test-Retest Internal Consistency (Cronbach’s Alpha)

Developing Sound Measures

Step 1: Specify Domain of the Construct Step 2: Generate Sample of Items Step 3: Collect Data Step 4: Purify Measure Step 5: Assess Validity

• • • • • • •

Tips for Maximizing Participation

Personal communication Asking a favor Importance of the research project and its purpose Importance of the recipient, how recipient was selected How the recipient may benefit from this research Completing the questionnaire will take only a short time The questionnaire can be answered easily

• • • • • • • •

Tips for Maximizing Participation

A stamped reply envelope is enclosed Answers are anonymous or confidential Offer to send report on results of survey Appreciation of sender Importance of sender or sender’s organization Description and purpose of incentive Style, format and appearance Brevity

Tips for Maximizing Participation

Professional Mail Surveys Company 7432 East Court Avenue Elveron, CA 90101 (213) 991-5550 Dear Mr. Smythe: Will you do us a favor?

We are conducting a nationwide survey among executives and managers in the metalworking industry. The purpose of this research is to find out the opinion of experts like you on the advantages and disadvantages of using three new steel products. Your answers will enable steel manufacturers to be aware of the requirements of the users and the opinions of the nonusers of these products, and this in turn will help them design the products you need.

Your name appeared in a scientifically selected random sample. Your answers are very important to the accuracy of our research, whether or not your company is a user of one or more of the products described.

It will take only a short time to answer the simple questions on the enclosed questionnaire and to return it in the stamped reply envelope.

Of course all answers are confidential and will be used only in combination with those of other metalworking executives and managers.

If you are interested in receiving a report on the findings of this research, just write your name and address at the end of this questionnaire, or if you prefer, request the results of the Survey on Steel Products in a separate letter. We will be glad to send you a complimentary report when it is ready.

Please return the completed questionnaire at your earliest convenience. Thank you for your help!

Sincerely,

James B. Jones

James B. Jones Director P.S. The enclosed dollar bill is just a token of appreciation.

And finally, remember the golden rule….

Do unto your respondents as you would have them do unto you!!

Team Assignment #2

** Don’t start on this assignment until you’ve read Fowler (CH 6-7) • Based on your research design 1. Write a paragraph about what your measurement instrument is supposed to accomplish 2. Make a list of what should be measured to accomplish the goals of the study 3. Develop your measurement instrument

Team Assignment #2

• Deliverables include: 1. A very clean, polished version that you could use to actually collect data • This means you will need to carefully think through all of the issues we covered tonight (e.g., set-up, ordering, length, multi-item scales, etc.) 2. Intro paragraph and variable list (see previous slide) (Note: Don’t worry about defining your sample--you’ll have your chance to do that next week)