Transcript Choosing your design

Choosing your design
One-way designs
• Simplest possible experimental design
• Also known as the “two-groups” design
• There is only one experimental variable
(IV)
• The IV only has two levels
– Drink v. No drink
– Drug v. no drug
– Dose 1 v. Dose 2
– Therapy v no therapy
• Control group gets one level; Experimental
group gets the other level
One-way with multiple groups
• The independent variable takes on more
than one value
• Still only one variable being manipulated
though
• Dose 1; Dose 2; Dose 3; Dose 4 ……..
• Social Judgments on unattractive vs.
moderately attractive vs. very attractive
people
One-way design with multiple
levels
• One variable with multiple levels
– Manipulate level of intoxication
• None
level 1
.04 BAC
level 2
.08 BAC
level 3
– Manipulate level of water given to bean plants
• None
10 dl per day
100 dl per day
– Manipulate using one level as the control group
• 10 dl
100 dl
1000 dl
Factorial Designs
• Limitation of one-way design is that they
only look at one independent variable
• Factorial designs look at multiple
independent variables
• Every level of one independent variable is
“crossed” with every level of the other
independent variables.
• Get to look at how multiple independent
variable interact
Factorial Designs
• The labeling of a factorial design specifies
– The number of independent variables in the
design
– The number of levels of each variable
• A 2x3 design (two by three) has two
independent variables.
– The first independent variable has 2 levels
– The second independent variable has 3 levels
Factorial Designs
• A 3x3 design (three by three) has two
independent variables.
– The first independent variable has 3 levels
– The second independent variable has 3 levels
• A 2x2x3 design has three independent
variables.
– The first independent variable has 2 levels
– The second independent variable 2 levels
– The third independent variable has 3 levels
Factorial Designs
• A 2x3x4 design has X independent
variables.
– How many levels does each independent
variable have?
• A 2x2x3x3 design has X independent
variables.
– How many levels does each independent
variable have?
Factorial Designs
• The description of the design tells the researcher
how many different cells are needed.
• A 2x2 design has 4 cells.
• Must randomly assign subjects to each of the 4
cells
Low Light
(1hr/day)
High Light
(12 hr/day)
Low water
(10 dl)
High Water
(1000 dl)
Cell 1
Cell 2
Cell 3
Cell 4
Factorial Designs
• Can measure more variables and interactions. A
3x3 design has how many cells?
Low water
(10dl)
No light
moderate
light
Heavy light
Medium
High water
water (100dl) (1000dl)
Factorial Designs
• Can even measure more variables…
• 2x2x2 design. How many cells?
– Low light v heavy light
– Low water v heavy water
– High temperature v low temperature
Factorial Designs
Low temperature
High temperature
Low High
water water
Low High
water water
Low
light
Low
light
High
light
High
light
2 by 2 by 2 design
Factorial Designs
• A 2 x 2 x 3 design has how many cells?
• A 2 x 2 x 3 x 3 design has how many
cells?
• A 2 x 2 x 2 x 2 x 2 x 2 design has how
many cells?
Analysis of Factorial Designs
• A one-way design uses a one-way ANOVA
• A 2 x 2 design uses a two way ANOVA
• A 2 x 2 x 2 x 2 x 2 x 2 needs a 6-way
ANOVA
• As the design gets more complicated so
does the analysis
Factorial Designs
• Be reasonable:
– The more cells you have, the more people
you need in your experiment.
– If you need 20 people per cell to have enough
subjects to detect effects then in a 2x2 design
you already need 80 subjects.
– In a 2x2x3 design, you now need 240
subjects
– One more level – a 2x2x3x3 design and we
are up to 720 subjects
Factorial Designs
• Most designs used by researchers are simple (2
x 2 or 2 x 3)
• Simple design allow for qualification of theories
– that the theory is true under some conditions
but not others
• Rephrased: That the effect of one independent
variable on the dependent variable DEPENDS
on the level of another independent variable
• The two independent variables INTERACT
Factorial Designs and Main effects
• Factorial designs first allow you to test for main
effects. A main effect is the effect of one of the
independent variables.
• In the water and sunlight example. A main effect
of water is just that. What effect does water have
on the dependent variable. Does water affect
growth by itself. A main effect of light is what
effect light has on the dependent variable
(growth)
Factorial Designs and Main effects
0.5
0.45
0.4
0.35
0.3
self-esteem tape
memory tape
0.25
0.2
0.15
0.1
0.05
0
self-esteem label
memory label
Factorial Designs and main effects
In the previous example there was a main effect of
Label but not of the actual tape. Red was different
from Blue
Memory
label
Memory
tape
.45
Self-esteem .46
tape
Self-esteem
label
.125
.2
Factorial Designs and main effects
In the previous example there was a main effect of
Label but not of the actual tape. Green was not different
from Yellow
Memory
label
Memory
tape
.45
Self-esteem .46
tape
Self-esteem
label
.125
.2
Factorial Designs and Interactions
• Factorial designs first allow you to test for
main effects. A main effect is the effect of
one of the independent variables.
• Interactions exist when the effect of one
independent variable depends on the level
of the second independent variable
Interactions
• Question: when are people likely to take social context
into consideration when they are judging other
people’s personalities
• Context:
– Imagine you saw a woman behaving anxiously while talking
to someone
– Find out it is a stranger: How to you evaluate woman?
– Find out she’s supposed to talk about anxiety provoking
topics (sexual fantasies, being publicly humiliated, etc).
Reevaluate.
– Find out she is supposed to be talking about relaxing topics
(ideal vacations, hobbies, etc.) Reevaluate
Interactions
• One-way design – show two groups the different
images of the woman and see how they evaluate her.
Find out if context is important in evaluating her
personality based on her behavior
• Two factor design – evaluate effect of the cognitive
load of the participant on making the personality
judgment.
– Half do as above and try to figure out her personality
– Half try to figure out her personality while engaging in a
taxing secondary task
Interactions
Low cognitive
load
Anxiety
provoking
discussion
Relaxing
discussion
2 x 2 design
High cognitive
load
Judged level of trait anxiety
Interactions
11
10
9
Anxious topics
relaxing topics
8
7
6
Low load
high load
When people were not taxed they did a good job of using
context to evaluate the person
When taxed they were less likely to take context into account
Interactions
• IV 1 = sunlight; IV 2 = water; DV = growth
Low Light
High Light
Low Water
22.4
25.6
High Water
10.8
37.2
16.6
31.4
24.0
24.0
Interactions
Judged level of trait anxiety
11
10
9
Anxious topics
relaxing topics
8
7
6
Low load
high load
Interactions show up as nonparallel lines when graphed
Interactions
When an IV has an effect at one level of the
second IV, but not at the other level of the
second IV it is known as an ordinal or
spreading interaction.
When neither IV has a main effect and when
an IV has the opposite effect at each level
of the second IV it is known as a disordinal
or crossover interaction
Ordinal Interactions
Judged level of trait anxiety
11
10
9
Anxious topics
relaxing topics
8
7
6
Low load
high load
Crossover Interactions
Judged level of trait anxiety
11
10
9
Anxious topics
relaxing topics
8
7
6
Low load
high load
• Remember there are no Main effects in crossover
interactions, but there are another type of effect known
Crossover interactions
11
10
T-test
9
Anxious topics
relaxing topics
8
7
6
Low load
high load
• Simple main effects look to see if there is a difference in
low load people who received the different types of
priming and if there is a difference in high load people
who received the different priming. If both are significant
it is definitely a crossover interaction
Between-Subjects Designs
• Between-subjects designs are designs where
each participant serves in one and only one
condition of the experiment
20
individuals
20
individuals
20
individuals
20
individuals
• Fill the cells with 20 individuals each. 80 total
individuals in the experiment
Within-Subjects Designs
• In Within-subjects designs (also known as
repeated measures designs) the subjects
will be in more than one condition of the
experiment (perhaps all conditions)
• Most of the design considerations are the
same as between subjects designs
• The statistics will be slightly different since
the groups & means are no longer
independent of each other
Within-Subjects Designs
• Within subjects designs require fewer subjects
• Consider a 3x3x2 design with 50 subjects per
cell. 18x50 = 900 subjects. A within subjects
design would mean only 50 subjects total.
• The goal of many subjects and random
assignment into each cell was to equalize
person confounds – within-subjects designs
eliminate person confounds.
• May even need fewer subjects because there is
less “noise” due to individual differences.
Within-Subjects Designs
• Disadvantages of within-subjects designs
– Sequence effects: the passage of time between conditions
affects the outcome (people get bored or tired or even
excited they are almost done)
– Carryover effects: people’s responses to one stimulus may
affect their responses to a second stimulus.
• Order effects: get different results depending on which
question you ask first:
• What is your general life satisfaction?
• What is your dating frequency?
• Correlation between the two depends on which question
you ask first.
Within-Subjects Designs
• Disadvantages of within-subjects designs
– Practice effects:
• If asked to perform a task, people improve with time.
• Performing one task can also interfere with performing a
second task (interference effect)
•
•
•
•
What is a three letter synonym for police officer?
What do you do with an ax?
What’s the opposite of bottom?
What do you do when you come to a green light?
Within-Subjects Designs
• Solutions to disadvantages:
– Counterbalancing: the order in which the
participants receive the conditions vary.
– Complete counterbalancing is when you complete
the cells with all possible orders
– Complete counterbalancing gets very difficult as
soon as you get more than a couple of factors
– For two conditions you have 2! (two factorial)
conditions. 2x1 = 2. This means some get A first,
then B (AB) other get B first, then A (BA)
Within-Subjects Designs
• Solutions to disadvantages:
– For three conditions (1x3 design) you have 3! (three
factorial) condition. 3x2x1 = 6. This means some
get A first, then B (ABC) other get B first, then A,
then C (BAC) then BCA, ACB, CAB, CBA.
– For a 2x2 design already have 4 conditions 4!
Equals 24 different orders to present the conditions
– ugh……
– Researchers will often resort to incomplete
counterbalancing
Incomplete counterbalancing
• Reverse counterbalancing ABCDE 
EDCBA
– Good for some – but C is always in the same
spot
• Partial Counterbalancing: Choosing
random orders from all possible orders
• Latin Squares: a mechanism for choosing
the order without resorting to random
order
Counterbalancing
• Counterbalancing will get rid of some
problems. People may still get tired or
bored, but it will happen in various
conditions rather than always in one or two
of the conditions
• Counterbalancing can also determine if
there are order (carryover) effects. It can’t
eliminate them, but if we know they are
there we can control for them.
Mixed-Model Designs
• Mixed-model designs are studies where at
least one variable is manipulated on a
between-subjects basis and at least one
variable is manipulated on a withinsubjects basis
• Why? Sometimes no matter the efforts of
counterbalancing, within-subjects tests are
not useful because the subjects cannot be
exposed to one condition without
influencing another condition