Transcript Slide 1
Chapter 4
Measurement Issues in Research Sum 2014
Dawn M. McBride
The Process of Research in Psychology
Second Edition
Steps in the Research Process
2 Dawn M. McBride The Process of Research in Psychology, 2nd Edition © 2013 SAGE Publications, Inc.
Variables: Operational Definition
3 Way to
measure
or
manipulate
(construct) in a research study a concept Example: intelligence -
numerical measure
Q test score; GPA -- problem-solving accuracy Example: treatments
Groups - Nominal
Manual for Cognitive Behavior and for Interpersonal treatment Violent videogames and non violent-videogames Dawn M. McBride - The Process of Research in Psychology, 2 nd Edition © 2013 SAGE Publications, Inc.
Scales of Measurement
4 Dawn M. McBride - The Process of Research in Psychology, 2 nd Edition © 2013 SAGE Publications, Inc.
5
Scales of Measurement
Nominal
: non-ordered category responses Example: Groups formed based on : #1 How do you feel today? (Circle one)
happy sad anxious excited
#2 Condition: Type of Condition
Violent
or
non-violent
videogame Dawn M. McBride The Process of Research in Psychology, 2nd Edition © 2013 SAGE Publications, Inc.
6
Scales of Measurement
Ordinal:
ordered category responses; no equal space between categories;
not numerical
Example: How do you feel today? (Circle one)
Scales of Measurement
Interval
: equally spaced, numerical responses, values are not ratio relations Example: Rate your pain (
Likert Scale
) 7 A rating of 10 twice the pain of a 5
Scales of Measurement
8
Ratio:
equally spaced, numerical responses, values are ratio relations (difference between 4 and 8 = difference between 6 and 12) Examples: Time Length Accuracy Distance Height Weight Dawn M. McBride - The Process of Research in Psychology, 2 nd Edition © 2013 SAGE Publications, Inc.
9
Validity of Scales
Construct validity
:
scale measures the construct it was designed to measure
(e.g. behavior, attitude, belief, value, feeling, state)
Face validity
Do the scale (items, drawings) reflect the construct of interest ?
Dawn M. McBride The Process of Research in Psychology, 2nd Edition © 2013 SAGE Publications, Inc.
Construct Validity
Concurrent Validity
A new measure of depression
correlates highly
with the Beck Depression Inventory 10
Discriminant Validity
A measure of parenting
does not correlate
highly with a measure of Social Desirability Dawn M. McBride - The Process of Research in Psychology, 2 nd Edition © 2013 SAGE Publications, Inc.
Construct Validity
Predictive Validity
SAT score (taken now) predicts College GPA (some time later) 11
In sum
Validity
of numerical scales is determined by examining the correlation of the scores with scores in other numerical scales Dawn M. McBride The Process of Research in Psychology, 2nd Edition © 2013 SAGE Publications, Inc.
12
Name of the Scale: Type of Scale:
Nominal, Ordinal, Interval, Ratio ?
Dawn M. McBride - The Process of Research in Psychology, 2 nd Edition © 2013 SAGE Publications, Inc.
13
Name of the Scale: Distress Disclosure Index Type of Scale:
Nominal, Ordinal,
Interval
, Ratio
Likert
14
Reliability:
Consistency of Measurement
15
Reliability: Measures
Test-retest reliability
Internal reliability
Inter-rater reliability - Observations
16
Reliability: Measures
Test-Retest
Responses do not change across time
With no intervention, participants respond to scale similarly across two points in time
Internal
Responses are consistent across items in the survey
Similar score with
different forms
Similar scores in two half's of the test
17
Reliability: Measures
Inter-rater reliability - Observations
Several observers rate behaviors in a similar way
18
Independent Variables
:
Experimental Research
Manipulation Independent Variables: Type and Amount
19
Type
Compares different that is manipulated
versions
of the “something” drugs or therapies versions of an ad or product different types of instructions All groups include the factor being compared Each group
receives a different type or version
of that factor (between groups designs) Dawn M. McBride - The Process of Research in Psychology, 2 nd Edition © 2013 SAGE Publications, Inc.
20
IV: Type of Video Game DV:
Manipulation: Independent Variables Type and Amount
Amount
manipulates amount (quantity, intensity, length of time) of a factor in each group Quantity of drug that is optimal for relieving symptoms Off time between the study and test of material memory research Intensity of noise 21 Dawn M. McBride - The Process of Research in Psychology, 2 nd Edition © 2013 SAGE Publications, Inc.
22
IV: Amount of Sleep DV: Reaction Time
Designs: Independent Variables
Bivalent design:
an independent variable with
two levels
or groups:
Multivalent design:
an independent variable that includes
three or more levels
or groups 23 Dawn M. McBride - The Process of Research in Psychology, 2 nd Edition © 2013 SAGE Publications, Inc.
24
IV: Type : consequence observed in the model DV: Extent child imitated model ’s behavior
25
Quasi- Independent Variables
:
Quasi-Independent Subject in Experimental Designs : Variables
Characteristic of the participant s that is used to group participants
in the analysis of the data
– e.g. Gender – Age Effect of IV on DV is observed separagtelty for the various groups 26 Caution in reaching conclusions, since randomization is not possible .
27
IV(s): Consequence observed & Gender (ANOVA) DV: Extent child imitated model’s behavior
28
Quasi-Experiment Design
Participants are assigned to different treatments based
on already existing groups
5 th grade Homerooms A and B and C- Bandura study Researcher is unable to manipulate participants assignment to treatments, therefore there is no
Randomization
at all Groups are compared in the same way as in experimental designs Dawn M. McBride - The Process of Research in Psychology, 2 nd Edition © 2013 SAGE Publications, Inc.
29
HRA- Reward Model ; HRB Punish Model ; HRC- No consequence Model (Bias)
30
Bias: Experimental Designs
Bias can lower internal validity of experiment A study with
good internal validity
provides a good test of
a causal hypothesis
controls for
confounding factors
that could affect results, but are not of interest in the study. In other words, a study with good internal validity will remove
alternative explanations for the findings
(besides the independent variable). Dawn M. McBride - The Process of Research in Psychology, 2 nd Edition © 2013 SAGE Publications, Inc.
31
Sources of Bias in Research
Treatment Group differences
(randomization does not always work)
Experimenter bias
(e.g. allegiance effect ) Single and double blind studies (deception)
Testing effects
(order, practice, fatigue) Survey Research: vary order of measures across participants Dawn M. McBride The Process of Research in Psychology, 2nd Edition © 2013 SAGE Publications, Inc.
Sources of Bias in Experiments
32
Regression towards the mean
Extreme scores are not likely to recur Difficult Bias to Remove Random assignment Use several repetitions of the test that measures DV for each condition Taking outliers out of the data Large number of participants (dilutes extreme scores) Dawn M. McBride - The Process of Research in Psychology, 2 nd Edition © 2013 SAGE Publications, Inc.
33
External Validity: Experiment
External validity is the degree to which a study: measures realistic behaviors and results can be generalized to every day life If participants behave in a research study the way they would in their everyday lives, then the study has good external validity. Dawn M. McBride The Process of Research in Psychology, 2nd Edition © 2013 SAGE Publications, Inc.
34
External Validity
External Validity can be affected by:
Hawthorne effect Demand Characteristics Participants
comply with or contradict
what they perceive is expected from them Deception – does not always work Dawn M. McBride - The Process of Research in Psychology, 2 nd Edition © 2013 SAGE Publications, Inc.
ZERO-SUM Game
35
Internal Validity External Validity
Between-Within Subjects Designs
Between Subject
Different participants receive different treatments
Within Subjects
All participants receive the same treatments some times in different order 36 Dawn M. McBride - The Process of Research in Psychology, 2 nd Edition © 2013 SAGE Publications, Inc.
Smith & Moynan (2008)
Within Subject Manipulation
IV #1 = type of word list – Categories: neutral, emotional/memorable, and emotional/not memorable
Between Subject
IV #2 = filler activity (control and “forget” tasks ).
. 37 Dawn M. McBride The Process of Research in Psychology, 2nd Edition © 2013 SAGE Publications, Inc.
Smith & Monyan (2008)
Within Subjects
#1 type of list (neutral, emotional and memorable, and emotional but not memorable);
Between Subjects
# 2 filler activity (control and “forget” tasks).
38 Dawn M. McBride - The Process of Research in Psychology, 2 nd Edition © 2013 SAGE Publications, Inc.