Transcript Analysis of Experiments, continued

Research Methods in Psychology
Independent Groups Designs
Why Psychologists Conduct Experiments
 To test
• hypotheses derived from theories
• effectiveness of treatments and programs
 Third goal of psychological research
• explanation
 examine the causes of behavior
Experimental Research
 An experiment must include
• independent variable (IV)
• dependent variable (DV)
 An independent variable
• manipulated (controlled) by experimenter
• at least two conditions (levels)
 “treatment” and “control”
Experimental Research
 dependent variables
• measured by experimenter
• used to determine effect of IV
 In most experiments, researchers measure several
dependent variables to assess effect of IV
Example:
Body Image Among Young Girls
 Dittmar, Halliwell, and Ives (2006)
• Research question
 Does exposure to very thin body images cause
young girls to experience negative feelings about
their own body?
• Independent Variable
 version of picture book with three levels
• Barbie (very thin body image)
• Emme (realistic body image)
• Neutral (no body images)
Body Image Among Young Girls,
continued
• Dependent Variables
 Several measured body image and body
dissatisfaction, including:
 Child Figure Rating Scale
• rate perceived actual body shape
• rate ideal body shape
• obtain difference score:
score of zero: no body shape dissatisfaction
positive score: a desire to be bigger
negative score: desire to be thinner (body dissatisfaction)
Body Image Among Young Girls,
continued
 Dittmar et al.’s hypothesis
• Young girls who are exposed to the very thin
body image (Barbie) will experience greater
body dissatisfaction than young girls who are
exposed to realistic body images (Emme) or
neutral images.
Experimental Control and
Internal Validity
 Internal Validity
• An experiment has internal validity when we
can state confidently that the independent
variable caused differences between groups
on the dependent variable
 a causal inference
• alternative explanations for a study’s findings
are ruled out
Control and Internal Validity, continued
 Example:
• Suppose young girls who view the Barbie
images are more overweight or own more
Barbie dolls than girls in the other conditions
• How do we know viewing the Barbie images
in the experiment caused them to experience
greater body dissatisfaction?
 What are some alternative explanations?
Causal Inferences
 Three conditions for causal inference
• Covariation
 relationship between IV and DV
 example: young girls’ body dissatisfaction covaried
with experimental condition
 correlation does not imply causation
Causal Inferences, continued
• Time-order relationship
 presumed cause precedes the effect
 example: version of images (cause) was
manipulated prior to measuring body
dissatisfaction (effect)
 How can we be sure girls in Barbie condition didn’t
have greater body dissatisfaction than the other
girls before the manipulation (effect precedes
cause)?
Causal Inferences, continued
• Elimination of plausible alternative causes
 use control techniques to eliminate other
explanations
 example: if the three groups differ in ways other
than the type of images they viewed, these
differences are alternative explanations for the
study’s findings
Causal Inferences, continued
 Confoundings
• when the IV is allowed to covary with a
different, potential independent variable
• confoundings represent alternative
explanations for a study’s findings
• an experiment that is free of confoundings
has internal validity
Causal Inferences, continued
 Example of confounding
• suppose that after viewing the Barbie images,
young girls in this condition are interviewed by
a counselor to make sure they’re okay after
exposure to the very thin images; as part of
this interview, they’re asked specifically about
feelings toward their body
• suppose, too, that young girls in the Emme
and Neutral conditions are not interviewed
Causal Inferences, continued
• What is the confounding?
 version of images (IV of interest) covaries with
interview (present, absent)
• viewing Barbie images is always paired with interview
• viewing Emme or neutral images is always paired with no
interview
 alternative explanation for findings cannot be ruled
out
• greater body dissatisfaction in Barbie condition could be
explained by interview, not viewing Barbie images
 Note: This confounding was not present in the
Dittmar et al. study
Control Techniques
 Two control techniques to eliminate
alternative explanations
• holding conditions constant
• balancing
 With proper use of control techniques, an
experiment has internal validity
Control Techniques, continued
 Holding conditions constant
• Independent variable: groups in the different
conditions have different experiences
 example: view Barbie, or Emme, or neutral images
• Experiences should differ only in terms of the
independent variable
• The only thing we allow to vary across groups
are the IV conditions—everything else should
be the same for the groups of the experiment
Control Techniques, continued
 Example of holding conditions constant
• Dittmar et al. (2006) held constant
 all the young girls listened to the same story
 all were given the same instructions
 all completed the same questions after the story
• What if only girls in the Barbie condition
listened to the story and girls in the other two
conditions sat quietly?
 alternative explanation: listening to a story caused
the different outcomes
Control Techniques, continued
 Balancing
• some alternative explanations for a study’s
findings concern characteristics of participants
• example
 What if girls in Barbie condition were more
overweight, owned more Barbie dolls, or greater
body dissatisfaction even before they viewed the
picture books?
Control Techniques, continued
• Some variables cannot be held constant
 subjects’ characteristics cannot be held constant
•
•
•
•
participants all have the same body weight
same number of Barbie dolls
same preexisting levels of body dissatisfaction
same everything
• Balancing controls for alternative explanations
due to subject characteristics
 Goal: make sure that on average, participants (as
a group) in each condition are essentially
equivalent
Control Techniques, continued
 How to balance subject characteristics
across the levels of the experiment:
• Participants are assigned to conditions using
some random procedure (e.g., two conditions:
flip a coin)
• Random assignment creates, on average,
equivalent groups of participants in the
experimental conditions
• Rule out alternative explanations due to
subject characteristics
Independent Groups Designs
 Independent groups design
• different individuals participate in each
condition of the experiment (i.e., no overlap of
participants across conditions)
• three types
 random groups design
 matched groups design
 natural groups design
Random Groups Designs
 Individuals are randomly assigned to
conditions of the IV
• Groups of participants are equivalent, on
average, before the IV manipulation
• Any differences between groups on
dependent variable are caused by
independent variable (if conditions are held
constant)
• Dittmar et al. (2006) study used a random
groups design
Random Groups Designs, continued
 Block randomization
• A “block” is a random order of all conditions in
the experiment
 Example: a random order of conditions A, B, C
could be B C A
• 1st participant assigned to condition B
• 2nd participant—condition C
• 3rd participant—condition A
 Generate random orders until goal for number of
participants in each condition is met (e.g., 10 in
each condition)
Random Groups Designs, continued
• Advantages of block randomization
 creates groups of equal size for each condition
 controls for time-related events that occur during
course of experiment
• natural changes in experimental conditions,
experimenters, participants that occur over time are
balanced across the experimental conditions
 as with all random assignment, block
randomization balances subject characteristics
across conditions of the experiment
Threats to Internal Validity
 Ability to make causal inferences is
jeopardized when
•
•
•
•
intact groups are used
extraneous variables are not controlled
selective subject loss occurs
demand characteristics and experimenter
effects are not controlled
Threats to Internal Validity, continued
 Intact groups
• these groups exist before experiment
• examples
 children in different classrooms, departments within an
organization, sections of Introductory Psychology course
• individuals are not randomly assigned to intact groups
• when intact groups (not individuals) are randomly
assigned to conditions, subject characteristics are not
balanced
• do not use intact groups
Threats to Internal Validity, continued
 Extraneous variables
• practical considerations when conducting an
experiment may create confoundings
• examples of extraneous variables
 number of participants in each session
 different experimenters
 different rooms where experiment is conducted
Threats to Internal Validity, continued
• Example
Suppose two experimenters help to conduct an
experiment. One experimenter tests all of the
participants in the treatment condition and the
second experimenter tests all of the participants in
the control condition.
• This experiment is confounded because any
differences on the DV may be due to the IV
(treatment, control) or to the two
experimenters.
Threats to Internal Validity, continued
 How to control extraneous variables
• Balancing
 randomly assign extraneous variables across the
conditions of the experiment
• example: Each experimenter conducts both treatment
and control sessions, and are randomly assigned to
administer a condition at any particular time
• Holding conditions constant
 hold extraneous variables constant across the
conditions of the experiment
• example: one experimenter conducts both treatment and
control sessions
Threats to Internal Validity, continued
 Subject loss (attrition)
• occurs when participants fail to complete an
experiment
• equivalent groups formed at beginning of an
experiment through random assignment may
no longer be equivalent
• two types of attrition
 mechanical subject loss
 selective subject loss
Threats to Internal Validity, continued
• Mechanical subject loss
 when equipment failure or experimenter error
results in participant’s inability to complete
experiment
 often due to chance factors
 likely to occur equally across conditions of
experiment
 because mechanical subject loss is due to chance
events, it does not threaten internal validity of
experiment
Threats to Internal Validity, continued
• Selective subject loss occurs when
 participants are lost differentially across conditions
 some characteristic of participant is responsible for
the loss
 the subject characteristic is related to the
dependent variable
 example:
• suppose a treatment for depression is compared to a notreatment control condition
• selective subject loss might occur if people drop out of
the control condition more than the treatment condition
Threats to Internal Validity, continued
 Placebo control and double-blind
experiments
• demand characteristics are cues participants
use to guide their behavior in a study
• example:
 in drug treatment research, demand characteristics
suggest to participants they will improve as a result
of the drug
• participants may expect to improve
• expectations may cause improvement, not the drug
Threats to Internal Validity, continued
• Placebo control group
 used to assess whether participants’ expectancies
contribute to outcome of experiment
 participants in placebo control group receive a
placebo (inert substance), but believe they may be
receiving an effective treatment
 if participants who receive the actual drug improve
more than participants who receive the placebo,
we gain confidence that the drug produced the
beneficial outcome, rather than expectancies
Threats to Internal Validity, continued
• Experimenter effects
 potential biases that occur when experimenter’s
expectancies regarding the outcome of the
experiment influence their behavior toward
participants
 control by keeping experimenters and observers
“blind” or unaware of the expected results
Threats to Internal Validity, continued
• Double-blind experiment
 procedures in which both participants and
experimenters/observers are unaware of the
condition being administered
 controls both
• demand characteristics
• experimenter effects
 allows researchers to rule out participants’ and
experimenters’ expectancies as alternative
explanations for a study’s outcome
Analysis and Interpretation
of Experimental Findings
 We rely on statistical analysis to
• claim an independent variable produced an effect on
a dependent variable
• rule out the alternative explanation that chance
produced differences among the groups in an
experiment
 Replication
• best way to determine whether findings are reliable
• repeat experiment and see if same results are
obtained
Analysis of Experimental Designs
 Three steps
• Check the data
 errors? outliers?
• Describe the results
 descriptive statistics such as means, standard
deviations
• Confirm what the data reveal
 inferential statistics
Analysis of Experiments, continued
 Descriptive Statistics
• Mean (central tendency)
 average score on a DV, computed for each group
 not interested in each individual score, but how
people responded on average in a condition
• Standard deviation (variability)
 average distance of each score from the mean of a
group
 not everyone responds the same way to an
experimental condition
Analysis of Experiments, continued
• Effect size
 measure of the strength of the relationship
between the IV and DV
 Cohen’s d
difference between treatment and control means
average variability for all participants’ scores
Guidelines for interpreting Cohen’s d:
small effect of IV:
d = .20
medium effect of IV:
d = .50
large effect of IV:
d = .80
Analysis of Experiments, continued
• Meta-analysis
 summarize the effect sizes across many
experiments that investigate the same IV or DV
 select experiments to include based on their
internal validity and other criteria
 allows researchers to gain confidence in general
psychological principles
Analysis of Experiments, continued
 Confirm what the data reveal
• use inferential statistics to determine whether
the IV had a reliable effect on the DV
• rule out whether findings are due to chance
(error variation)
• two types of inferential statistics
 Null Hypothesis Significance Testing
 Confidence intervals
Analysis of Experiments, continued
 Null Hypothesis Significance Testing
• statistical procedure to determine whether
mean difference between conditions is greater
than what might be expected due to chance
or error variation
• the effect of an IV on the DV is statistically
significant when the probability of the results
being due to chance is low
Analysis of Experiments, continued
 Steps for Null Hypothesis Testing
(1) Assume the null hypothesis is true
 The null hypothesis assumes the population
means for groups in the experiment are equal.
 example:
• the population mean for body dissatisfaction following
Barbie images is equal to the population mean for Emme
images or neutral images
Analysis of Experiments, continued
(2) Use sample means to estimate population
means.
 example:
mean body dissatisfaction for Barbie = -.76
mean body dissatisfaction for Emme = 0.00
mean body dissatisfaction for neutral = 0.00
difference between Barbie and Emme/neutral = -.76
Is the observed mean difference (-.76) greater than
what is expected when we assume the null
hypothesis is true (zero)?
Analysis of Experiments, continued
(3) Compute the appropriate inferential statistic.
 t-test: test the difference between two sample
means
 F-test (ANOVA): test the difference among three or
more sample means
(4) Identify the probability associated with the
inferential statistic
 p value is printed in computer output or can be
found in statistical tables
Analysis of Experiments, continued
(5) Compare the observed probability with the
predetermine level of significance (alpha),
which is usually p < .05
 If the observed p value is greater than .05, do not
reject the null hypothesis of no difference
• conclude IV did not produce a reliable effect
 If the observed p value is less than .05, reject the
null hypothesis of no difference.
• conclude IV did produce a reliable effect
• version of picture books (Barbie, Emme, neutral) caused
differences in young girls’ body dissatisfaction
Analysis of Experiments, continued
 Confidence intervals
• sample means estimate population means
• Confidence intervals provide the range of
values that contains the true population mean
 with some probability, usually .95
Analysis of Experiments, continued
• we typically want to conclude that
performance in one experimental condition
differs from performance in a second
condition
• compute the confidence interval around the
sample mean in each condition
 if the confidence intervals do not overlap, we gain
confidence that the population means for the
conditions are different
 —that is, there is a difference among conditions
Analysis of Experiments, continued
• Example of confidence intervals
 suppose the confidence interval for mean body
dissatisfaction in the Barbie condition is
–1.16 -- –.36
• This interval contains the true population mean for body
dissatisfaction following Barbie images (remember the
sample mean is –.76).
 suppose the confidence interval for mean body
dissatisfaction in the neutral image condition is
–.25 -- +.25
• this interval contains the true population mean for body
dissatisfaction following neutral images (the sample
mean is 0.00)
Analysis of Experiments, continued
Barbie: –1.16 -- –.36
Neutral: –.25 -- +.25
 because the confidence intervals do not overlap,
we can be confidence that the population means
for the two groups differ
 viewing Barbie images, compared to neutral
images, produces greater body dissatisfaction in
the population of young girls
Analysis of Experiments, continued
• suppose instead that the confidence intervals
overlap:
Barbie
–1.56 -- +.04
Neutral
–.82 -- +.82
 even though the sample means differ (–.76 and
0.00), we cannot conclude that the population
means differ because the confidence intervals
overlap
 the difference between the sample means could be
attributed to chance
External Validity
 External validity
• the extent to which findings from an
experiment can be generalized to describe
individuals, settings, and conditions beyond
the scope of a specific experiment
 any single experiment has limited external validity
 external validity of findings increase when findings
are replicated in a new experiment
External Validity, continued
 Questions of external validity
• would the same findings occur
 in different settings?
 in different conditions?
 for different participants?
• example:
 research with college students is often criticized
because of low external validity
• sample often doesn’t matter when testing a theory
• on what dimensions do college students differ?
External Validity, continued
• Increasing external validity
 include characteristics of situations, settings, and
population to which researchers wish to generalize
 partial replications
 field experiments
 conceptual replications
Additional Independent Groups Designs
 Matched Groups Design
• random assignment requires large samples to
balance subject characteristics
• sometimes only small samples are available
• in matched groups design,
 researchers select one or two individual
differences variables for matching
Matched Groups Design
 Procedure
• select matching variable
 individual differences variables are characteristics of people
that differ, or vary
 choose matching variable related to outcome or dependent
variable
• measure variable and order individuals’ scores
• match pairs (or triples, quadruples, etc. depending on
number of conditions) of identical or similar scores
• randomly assign participants within each match to the
different conditions
Matched Groups Design, continued
 Important points about matching
• participants are matched only on the matching
variable
• participants across conditions may differ on
other important variables
• these differences may be alternative
explanations for study’s results (confounding)
• the more characteristics a researcher tries to
match, the harder if will be to match
Natural Groups Designs
 Natural Groups Designs
• psychologists’ questions often ask about how
individuals differ, and how these individual
differences are related to important outcomes.
• examples:
 Do men and women differ in what they seek in
intimate relationships?
 Are extraverted individuals, compared to
introverted individuals, more likely to succeed in
business?
Natural Groups Designs, continued
 Individual differences (subject) variables
• characteristics or traits that vary across
individuals
 physical characteristics
• sex, race
 social (demographic) characteristics
• ethnicity, religious affiliation, marital status
 personality characteristics
• extraversion, emotional stability, intelligence
 mental health characteristics
• depression, anxiety, substance abuse
Natural Groups Designs, continued
 Researchers can’t randomly assign
participants to these groups
• random assignment to male/female groups?
 When a researcher investigates an
independent variable in which the groups
(conditions) are formed naturally, we say a
“natural groups design” is used
Natural Groups Designs, continued
 Example:
• Suppose we want to compare occupational
functioning of schizophrenics and normal
(nonschizophrenic) controls?
• Independent variable
 natural groups variable: schizophrenic vs. normal
• Dependent variable
 measure of occupational functioning
• Result
 suppose schizophrenics have poorer occupational
functioning than normal participants
Natural Groups Designs, continued
 Causal inferences and natural groups
design
• Researchers can’t make a causal inference
when a natural groups design is used
 example: can we say that schizophrenia causes
poorer occupational functioning?
 No. The two groups likely differ in other ways that
may cause poorer occupational functioning among
schizophrenics (confoundings)
• education level, drugs, nutritional status, tardive
dyskinesia, etc.
Natural Groups Designs, continued
 Natural groups designs
• correlational research
• allow researchers to describe and predict
relationships among
 individual differences variables and
 outcomes
• do not allow researchers to make causal
inferences about individual differences
variables