Transcript 8_Bestiary_of_designs_2013

A Bestiary of
Experimental and
Sampling Designs
REMINDERS
The goal of experimental design is to minimize the
potential “sources of confusion” (Hurlbert 1984):
1.
2.
3.
4.
5.
6.
Temporal (and spatial) variability
Procedural effects
Experimenter bias
Experimenter-generated variability (“random error”)
Inherent variability among experimental units
Non-demonic intrusion
“…it is the elementary principles of experimental
design, not advanced or esoteric ones, which are most
frequently and severely violated by ecologists...”
The design of an experiment
The details of:
• Replication
• Randomization
• Independence
… are these
always obvious in
biological
research? Are
they systemdependent?
We cannot draw blood from a stone
• Even the most sophisticated analysis
CANNOT rescue a poor design!!
Categorical variables
• They are classified into one or more
unique categories
– Sex (male, female)
– Trophic status (producer, herbivore, carnivore)
– Habitat type (shade, sun)
– Species
Continuous variables
• They are measured on a continuous
numerical scale (real or integer
values)
– Size
– Species richness
– Habitat coverage
– Population density
NOTE: Discrete random variables such as counts are still
considered continuous variables because they represent a
numerical scale and not a category…
Dependent and independent
variables
• The assignment of dependent and
independent variables implies a
hypothesis of cause and effect that you
are trying to test.
• The dependent variable is the response
variable…
• The independent variable is the predictor
variable…
Ordinate (vertical y-axis)
Ln (lambda)
2.000
Y=26.06X-2.99
r2=0.355
Fstat=25.84
1.000
0.000
-1.000
0
5
10
15
20
25
30
Time since fire (years)
Abscissa (horizontal x-axis)
By convention independent variables are plotted in the x-axis and
dependent variables in the y-axis… in this example we are
implying that lambda (population growth) depends or is affected
directly by time since fire…
Four classes of experimental design
Dependent
(response)
variable
Independent (predictor) variable
Continuous
Categorical
Continuous
Regression
ANOVA
Categorical
Logistic regression
Tabular
The Analysis of Covariance
(ANCOVA)
• It is used when there are two independent
variables, one of which is categorical and
one of which is continuous (the covariate)
Four classes of experimental design
Dependent
variable
Independent variable
Continuous
Categorical
Continuous
Regression
ANOVA
Categorical
Logistic regression
Tabular
Regression designs
• Single-factor regression
• Multiple regression
Single-factor regression
• Collect data on a set of independent
replicates.
• For each replicate, measure both the
predictor and the response variables.
• e.g. Hypothesis: seed density (the
predictor variable) is responsible for rodent
density (the response variable).
Plot #
Rodents/m2
Seeds
1
50
3.2
2
12
11.7
.
.
.
n
300
5.3
Variables
Plots
Single-factor regression
• You assume that the predictor variable is a
causal variable: changes in the value of
the predictor would cause a change in the
value of the response.
• This is very different from a study in which
you would examine the correlation
(statistical covariation) between two
variables.
In regression (Model I)
• You are assuming that the value of the independent
variable is known exactly and is not subject to
measurement error…
Assumptions and caveats
• Adequate replication.
• Independence of the data.
• Ensure that the range of values sampled
for the predictor variable is large enough
to capture the full range of responses by
the response variable.
• Ensure that the distribution of predictor
values is approximately uniform within the
sample range.
A
What is different between
these two designs?
B
Would the conclusions be
different?
A
What is different between
these two designs?
B
Would the conclusions
be different?
Multiple regression
• Two or more continuous predictor variables
are measured for each replicate, along with
the single response variable…
Assumptions and caveats
• Adequate replication.
• Independence of the data.
• Ensure that the range of values sampled for the predictor
variables is large enough to capture the full range of
responses by the response variable.
• Ensure that the distribution of predictor values is
approximately uniform within the sample range.
These are the same assumptions as for the
single-factor regression BUT additionally…
Multiple regression
• Ideally, the different predictor variables
should be independent of one another;
however in reality, many predictor
variables are correlated (e.g., height and
weight).
• This collinearity makes it difficult to
estimate accurately regression parameters
and to tease apart how much variation in
the response variable is associated with
each of the predictor variables.
Multiple regression
• As always, replication becomes important
as we add more predictor variables to the
analysis.
• In many cases it is easier to collect
additional predictor variables on the same
replicates than to obtain additional
independent replicates.
• Avoid the temptation to measure
everything that you can just because it is
possible.
• Think about measuring variables that are
meaningful for you study system!
Multiple regression
• It is a mistake to
think that a model
selection algorithm
can reliably identify
the correct set of
predictor variables...
Four classes of experimental design
Dependent
variable
Independent variable
Continuous
Categorical
Continuous
Regression
ANOVA
Categorical
Logistic regression
Tabular
ANOVA designs
• Analysis of Variance
• Treatments: refers to the different
categories of the predictor variables.
• Replicates: each of the observations
made.
ANOVA designs
•
•
•
•
•
•
•
Single-factor designs
Randomized block designs
Nested designs
Multifactor designs
Split-plot designs
Repeated measurements designs
BACI designs (before-after-control-impact)
Single-factor designs
• It is one of the simplest, but most powerful,
experimental designs.
• Can readily accommodate studies in which
the number of replicates per treatment is
not identical (unequal sample size).
Single-factor designs
• In a single-factor design, each of the
treatments represent variation in a single
predictor variable or factor
• Each value of the factor that represents a
particular treatment is called a treatment
level
Id #
Treatment
Replicate
Number of flowers
1
Watered
1
9
2
Not watered
1
4
.
.
.
.
11
Watered
6
10
12
Not watered
6
2
Good news, bad news:
• This design does not explicitly accommodate
environmental heterogeneity, so we need to
sample the entire array of background
conditions.
• This means the results can potentially be
generalized across all environments, BUT…
• If the background noise is much stronger than
the signal of the treatments, the experiment may
have low power, and therefore the analysis may
not reveal treatment differences unless there are
many replicates.
Randomized block designs
• An effective way to incorporate
environmental heterogeneity into a design.
• A block is a delineated area or time period
within which environmental conditions are
relatively homogeneous.
• Blocks can be placed randomly or
systematically in the study area, but
should be arranged so that the
environmental conditions are more similar
within blocks than between them.
Randomized block designs
Valid blocking
Invalid blocking
Randomized block designs
• Once blocks are
established, replicates
will still be assigned
randomly to treatments,
but a single replicate
from each of the
treatments is assigned to
each block.
Id # Treatment
Block Number of flowers
1
Watered
1
9
2
Not watered
1
4
.
.
11
Watered
6
10
12
Not watered
6
2
.
Caveats
• Blocks should have enough room to
accommodate a single replicate of each of
the treatments, and enough spacing
between replicates to ensure their
independence.
• The blocks themselves also have to be far
enough apart from each other to ensure
independence of replicates among blocks.
Advantages
• It can be used to control for environmental
gradients and patchy habitats.
• It is useful when your replication is
constrained by space or time.
• Can be adapted for a matched pair layout.
Disadvantages
• If the sample size is small and the block effect
weak, the randomized block design is less
powerful than the simple one-way layout.
• If blocks are too small, you may introduce nonindependence by physically crowding the
treatments together (e.g., nectar-removal and
control plots on p. 152 of Gotelli & Ellison).
• If any of the replicates are lost, the data from the
block cannot be used unless the missing values
can be estimated indirectly.
Disadvantages
• It assumes that there is no interaction between
the blocks and the treatments.
• BUT, replication within blocks will indeed tease
apart main effects, block effects, and the
interaction between blocks and treatments. It will
also address the problem of missing data from
within a block.
Nested designs
• It is any design in which there is
subsampling within each of the replicates..
• In this design the subsamples are not
independent of one another (if we
analyze them assuming independence is it
an example of pseudoreplication)
• The rational of this design is to increase
the precision with which we estimate the
response of each replicate.
Id #
Treatment
Subsample
Replicate
Number of flowers
1
Watered
1
1
9
2
Watered
2
1
4
3
Watered
3
1
7
.
.
.
.
.
19
Not watered
1
7
16
20
Not watered
2
7
10
21
Not watered
3
7
2
Advantages
•
•
Subsampling increases the precision of
the estimate for each replicate in the
design.
Allows to test two hypothesis:
1. First: Is there variation among treatments?
2. Second: Is there variation among replicates
within treatments?
•
Can be extended to a hierarchical
sampling design.
Disadvantages
• They are often analyzed incorrectly!
• It is difficult or even impossible to analyze
properly if the sample sizes are not equal.
• It often represents a case of misplaced
sampling effort.
Subsampling is not a solution to
inadequate replication
Randomized block designs
• Strictly speaking, the randomized block
and the nested ANOVA are two-factor
designs, but the second factor (i.e., the
blocks or subsamples) is included only to
control for sampling variation and is not of
primary interest.
Multifactor designs
• In a multifactor design, the treatments
cover two (or more) different factors, and
each factor is applied in combination in
different treatments.
• In a multifactor design, there are different
levels of the treatment for each factor.
Multifactor designs
• Why not just run two separate
experiments?
• Efficiency. It is often more cost effective to
run a single experiment than to run two
separate experiments.
• A multifactor design allows you to test for
both main effects and for interaction
effects.
Multifactor designs
• the main effects are the additive effects of
each level of one treatment averaged over
all levels of the other treatment.
• the interaction effects represent unique
responses to particular treatment
combinations that cannot be predicted
simply from knowing the main effects.
Interactions
60
60
50
50
40
40
West
North
30
20
20
10
10
0
1st Qtr
2nd Qtr
3rd Qtr
West
North
30
0
4th Qtr
1st Qtr
60
60
50
50
40
40
West
North
30
20
10
10
0
0
2nd Qtr
3rd Qtr
4th Qtr
3rd Qtr
4th Qtr
West
North
30
20
1st Qtr
2nd Qtr
1st Qtr
2nd Qtr
3rd Qtr
4th Qtr
Which of these graphs are showing interactions between
direction (west or north) and quarter (1st to 4th)?
Orthogonal
• The key element of a proper multifactorial
design is that the treatments are fully
crossed or orthogonal : every treatment
level of the first factor must be represented
with every treatment level of the second
factor and so on…
• If some of the treatment combinations are
missing we end with a confounded design.
Two-factor design
Substrate treatment
Granite
Predator
treatment
Unmanipulated
Cage Control
Predator
exclusion
Predator
intrusion
Slate
Cement
Advantages
•
The key advantage is the ability to tease
apart main effects and interactions
between factors. The interaction
measures the extent to which different
treatment combinations act additively,
synergistically, or antagonistically.
Disadvantages
•
•
•
The number of treatment combinations
can quickly become too large for
adequate replication!
It does not account for spatial
heterogeneity. This can be handled by a
simple randomized block design, in
which each block contains exactly one of
the treatment combinations.
It may not be possible to establish all
orthogonal treatment combinations.
Split-plot designs
• It is an extension of the randomized block
design to two treatments.
• What distinguishes a split plot design from
a randomized block design is that a
second treatment factor is also applied,
this time at the level of the entire plot.
Split plot design
Substrate treatment
The subplot factor
Granite
Unmanipulated
Predator
treatment
The whole- Control
plot factor
Predator exclusion
Predator intrusion
Slate
Cement
Advantages
•
•
The chief advantage is the efficient use
of blocks for the application of two
treatments.
This is a simple layout that controls for
environmental heterogeneity.
Disadvantages
•
As with nested
designs, a very
common mistake is
for investigators to
analyze a split-plot
design as a two
factor ANOVA…
Repeated measurements designs
• It is used whenever multiple observations
on the same replicate are collected at
different times (it can be thought of as a
split-plot in which a single replicate serves
as a block, and the subplot factor is time).
Repeated measurements designs
• The between-subjects factor corresponds
to the whole-plot factor.
• The within-subjects factor corresponds to
the different times.
• The multiple observations on a single
individual are not independent of one
another… why do you think this is?
Advantages
•
•
•
Efficiency.
It allows each replicate to serve as its
own block or control.
It allows us to test for interactions
between treatments and time.
Circularity
• Both the randomized block and the
repeated measures designs make a
special assumption of circularity for the
within-subjects factor.
• It means that the variance of the difference
between any two treatment levels in the
subplots is always the same… i.e. there is
the same variance between t1 and t2, as
between t2 and t3, etc..
For repeated measures design it means that the
variance of the difference of observations between
any pair of times is the same
This assumption is unlikely to be met in biological
systems because of their temporal memory!
Disadvantages
•
In many cases the assumption of
circularity is unlikely to be met for
repeated measures.
•
The best way to meet the circularity
assumption is to use evenly spaced
sampling times along with knowledge of
the natural history of your organisms
to select the appropriate sampling
interval.
Alternatives
1. To set enough replicates so that a different set
is sampled at each time period. With this
design, time can be treated as a simple factor
in a two-factor analysis of variance.
2. Use the repeated measures layout but
collapse the correlated repeated measures into
a single response variable for each individual,
and then use a simple one-factor analysis of
variance i.e. instead of height at age 0 and
height at age 1 use growth…
Think outside the ANOVA Box
• Many ecological
experiments test a
continuous predictor
at only a few values
so they can be
“shoehorned” into an
ANOVA design…
• One Alternative:
Experimental
regression design!
Four classes of experimental design
Dependent
variable
Independent variable
Continuous
Categorical
Continuous
Regression
ANOVA
Categorical
Logistic regression
Tabular
Tabular designs
• The measurements of these designs are
counts.
• A contingency table analysis is used to
test hypotheses.
… we will cover this later on