Transcript Journalism 658: Communication Research Methods

Measurement and
Observation
Choices During Operationalization
 Researchers make a number of key decisions when
deciding how to measure a concept
 Dimensions and sub-dimensions
 Range of variation within dimensions
 Categories to represent range
 Levels of measurement
 Nominal, ordinal, interval, ratio
Operationalization:
A deliberative process
 Not a simple, linear process
 Complicated and fraught with trade-offs
 Iterative process with cycles of consideration
 Debate over proper measurement is key
Dimensions of the Concept
 Creating operational measures forces realization about
lack of conceptual clarity
 List of possible dimensions may be long
 Need to decide which ones are most relevant
 Ask which ones are central to the inquiry
 Reflect on research hypotheses or theories
Range of Variation
 Sense of the upper and lower limits
 How much are you willing to combine different people into
the same category?
 Extremely high and Extremely low may be collapsed
 Eg. Income, age, height, etc.
 Opposition and support for attitudes
 Agreement and disagreement
Variation Between Extremes
 Degree of precision
 How detailed you need to be in measurement
 Eg. Age breaks or Exact age?
 Related to purpose of study
 Eg. Political Party ID:
 Dichotomy: Democrat or Republican
 Continuum: 7-point scale w/ “independent-leaner”
Levels of Measurement
 Nominal Measures
 Ordinal Measures
 Interval Measures
 Ratio Measures
Nominal Measures
 Names for characteristics
 Do not Exist along an Explicit continuum
 Exhaustive
 Mutually Exclusive
 Eg. Religious Affiliation
 Eg. Place of Birth
Ordinal Measures
 Can be logically rank-ordered
 Represent relatively more of less of variable
 No consistent distance between points of measurement
 Not just different from one another
 More of less of some attribute
 Eg. “Not very important,” “fairly important,” “very
important” “Extremely important”
Interval Measures
 Consistent distance separating attribute
 We can say how much more of an attribute
 Logical distance between attributes can be Expressed in
meaningful standard intervals
 Eg. Temperature
 90 degrees vs. 80 degrees = 10 degree difference
 50 degrees vs. 40 degrees = 10 degree difference
 Zero-point is arbitrary
Ratio Measures
 In addition to all the properties of nominal, ordinal, and
interval measures, ratio measures have a true zero point
 Eg. Length of time
 Eg. Number of times
 Eg. Number of affiliations
 Can actually state ratio of one to another
 X has twice as many affiliations as Y
What’s that scale?
 Style of music in a music video
 Number of violent acts in a music video
 Whether a music video has violence or not?
 High, Medium or Low violence in a music video
 Hair color
 Number of hairs on your head
 Sat scores
 Social Security Number
What’s that scale?
 A baseball player's batting average
 A baseball player's field position
 A baseball player's position in the batting order
 A baseball player's uniform number
 College football rankings
 IQ
Types of questions












Multiple choice questions
Agree/disagree questions
Likert questions
Frequency scales
Semantic differential scales
Forced-choice statement pairs
Thermometer feeling scales
Nominal checklists
Ordinal categories
Rank-order questions
Filter questions
Open-ended
Multiple Choice Question
Multiple Choice with Range
Options
Agree/Disagree Questions
Likert Scale
Frequency Scale
Semantic Differential Scales
Forced-choice Statement Pairs
Thermometer Feeling Scales
Nominal Checklist
Ordinal Categories
Rank-order Preference Questions
Rank-order Evaluation Questions
Filter Questions
Open-ended Questions
Tips on Question Construction
 1. Make questions clear using simple language
 2. Keep questions concise
 3. Provide instructions for answering questions
 Don’t assume respondent knows question style
 4. Keep research purpose in mind
 Make sure items can answer research question
 5. Don’t ask double-barreled questions
 E.g., “How well do you think the current Presidential
Administration is handling foreign policy and the war on
terrorism?”
More Tips
 6. Avoid leading questions
 E.g., “Like most Americans, do you read a newspaper every
day?”
 7. Avoid negative questions
 E.g., “The U.S. should not invade Iraq” Agree or disagree?
 8. Do not ask questions that require complicated
mental calculus
 E.g., “In the past 30 days, how many hours have you spent
watching television with your family?”
 9. Keep ordering of questions in mind
Using Pre-Existing Measures
 It is okay to borrow measures
 Cite source of questions to give credit
 Benefits of using Existing measures:
 Saves work
 Pre-tested for reliability/validity
 Research becomes cumulative
Pretesting
 Clarity in question wording
 Are categories:
 Exhaustive?
 Mutually Exclusive?
 Realistic time estimate
 Preliminary empirical analysis