Transcript ANALYSIS OF VARIANCE - University of Melbourne

ANALYSIS OF VARIANCE
One factor
 Gerry Quinn & Mick Keough, 1998
Do not copy or distribute without
permission of authors.
Barnacle recruitment
• Effects of 4 surface types on barnacle
recruitment on rocky shore
• Surface type is independent variable:
– alga sp.1, alga sp.2, bare rock, scraped rock
• Independent variable:
– categorical with 2 or more levels
– factor
– levels termed groups (“treatments”)
• Five replicate plots for each surface
type
• Dependent (response) variable:
– number barnacles after 4 weeks
Alg 2
Bare
Alg 2
Scraped
Bare
Scraped
Alg 1
etc.
Data
Treatment group:
Alga sp.1 Alga sp.2
Number of
barnacles
per plot
27
19
18
23
25
24
33
27
26
32
Bare
Scraped
9
13
17
14
22
12
8
15
20
11
ANOVA vs regression
• One factor ANOVA:
– 1 continuous dependent variable and 1
categorical independent variable (factor)
• Compare with regression:
– 1 continuous dependent variable and 1
continuous independent variable
Aims
• Measure relative contribution of different
sources of variation (factors or
combination of factors) to total variation
in dependent variable
• Test hypotheses about group
(treatment) population means for
dependent variable
Terminology
• Factor (independent variable):
– usually designated factor A
– number of levels/groups/treatments = a
• Number of replicates within each group
–n
• Each observation:
–y
Data layout
1
2
…
i
Replicates
y11
y1j
...
y1n
y21
y2j
...
y2n
...
...
...
...
yi1
yij
...
yin
Sample means
y1
y2
yi
Population means
m1
m2
mi
Factor level (group)
Grand mean y estimates m
Alg sp1
Alg sp2
Bare
Scraped
y11=27
y12=19
y13=18
y14=23
y15=25
y21=24
y22=33
y23=27
y24=26
y25=32
y31= 9
y32=13
y33=17
y34=14
y35=22
y41=12
y42= 8
y43=15
y44=20
y45=11
28.4
15.0
13.2
Means: 22.4
Overall mean: 19.75
Linear model for 1 factor
ANOVA
The linear model for 1 factor ANOVA:
yij = m + ai + eij
where
m = overall population mean
ai = effect of ith treatment or group (m - mi)
eij = random or unexplained error (i.e.
variation not explained by treatment
effects)
Compare with regression model
yi = b0 + b1x1 + ei
• intercept is replaced by m
• slope is replaced by ai (treatment effect):
– independent variable is categorical rather than
continuous
– still measures “effect” of independent variable
Types of factor
• Fixed factor:
– all levels or groups of interest are used in
study
– conclusions are restricted to those groups
• Random factor:
– random sample of all groups of interest are
used in study
– conclusions extrapolate to all possible
groups
Null hypothesis
• Ho: m1 = m2 = mi = m
• No difference between population group
(treatment) means
• Mean number of barnacle recruits is
same on four substrata
HO - fixed factor
Treatment or group effects:
a1 = (m1 - m), a2 = (m2 - m), ai = (mi - m)
where ai = the effect of group or treatment i
HO: a1 = a2 = ai = 0
• No group (treatment) effects
• No effect of any surface type on barnacle
recruitment
HO - random factor
HO: m1 = m2 = mi = ma = m
• i.e. all possible group means the same
HO: sa2 = 0
• i.e. no variance between groups
Basic assumption of ANOVA
s12 = s22 = si2 = s2
where si2 = population variance of
dependent variable (y) in each group
Each group (or treatment) population has
same variance
– homogeneity of variance assumption
Partitioning variation
• Variation in DV partitioned into:
– variation explained by difference between
groups (or treatments)
– variation not explained (residual variation)


SS Total
  ( yij  y )
SS Between groups +
n  ( yi  y )
2
2
SS Within groups
(Residual)
  ( yij  yi )
2
SS Total
2
  ( yij  y )
= (27 - 19.75)2 + (19 - 19.75)2 + ... + (11 - 19.75)2
= 1033.75
Total variation in dependent variable (y)
across all groups
SS Between groups
n  ( yi  y )
2
= 5*[(22.4 - 19.75)2 + (28.4 - 19.75)2 + etc.
= 736.55
Variation between group means =
treatment variation
SS Residual = Within groups
  ( yij  yi )
2
= (27 - 22.4)2 + (19 - 22.4)2+ ... + (11 - 13.2)2
= 297.20
Pooled variation between replicates
within each group
Mean squares
• Average sum-of-squared deviations
• Degrees of freedom:
– number of components minus 1
– df total [an-1] = df groups [a-1] + df residual
[a(n-1)]
• Mean square is a variance:
– SS divided by df
ANOVA table
Source
Groups
SS
n ( yi  y ) 2
Residual   ( y  y ) 2
ij
i
Total
2
  ( yij  y )
df
MS
a-1
n  ( yi  y ) 2
a 1
a(n-1)
2
  ( yij  yi )
an-1
a (n  1)
Worked example
Source
SS
df
MS
Groups
736.55
3
245.22
Residual 297.20 16
18.58
Total
1033.75 19
Treatments (= groups) explain nothing, ie.
SSGroups equals zero
Replicate
1
2
3
4
Mean
Group1 Group2 Group3
Group4
16.0
15.0
17.0
16.0
15.0
17.0
16.0
16.0
16.0
16.0
17.0
15.0
17.0
16.0
15.0
16.0
16.0
16.0
16.0
16.0
Grand mean = 16.0
Treatments (= groups) explain everything, ie.
SSResidual equals zero
Replicate
1
2
3
4
Mean
Group1 Group2 Group3 Group4
19.5
19.5
19.5
19.5
15.0
15.0
15.0
15.0
16.5
16.5
16.5
16.5
13.0
13.0
13.0
13.0
19.5
15.0
16.5
13.0
Grand mean = 16.0
Testing ANOVA HO
• All population group means the
same
m 1 = m2 = mi = ma = m
• No population group or treatment
effects
a1 = a2 = ai = 0
F-ratio statistic
• F-ratio statistic is ratio of 2 sample
variances (i.e. 2 mean squares)
• Probability distribution of F-ratio known
– different distributions depending on df of 2
variances
• If homogeneity of variances holds, Fratio follows F-distribution
F-ratio distribution
3, 16 df
P(F)
0
1
2
3
F
4
5
Expected mean squares
• If fixed factor and if homogeneity of
variance assumption holds:
• MSGroups estimates s2 + n(ai)2/a-1
• MSResidual estimates s2
• If HO is true:
– all ai’s = 0
– MSGroups and MSResidual both estimate s2
– so F-ratio = 1
• If HO is false:
– at least one ai  0
– MSGroups estimates s2 + treatment effects
– so F-ratio > 1
Expected mean squares
• If random factor and homogeneity of
variance assumption holds:
• MSGroups estimates s2 + nsa2
• MSResidual estimates s2
• If HO is true:
– s a2 = 0
– MSGroups and MSResidual both estimate s2
– so F-ratio = 1
• If HO is false:
– s a2 > 0
– MSGroups estimates s2 plus added variance
due to groups or treatments
– so F-ratio > 1
Worked example
• MSGroups = 245.52
• MSResidual = 18.58
• F-ratio = 245.52/18.58 = 13.22
• Probability of getting F-ratio of 13.22 (or
larger) if HO true (and F-ratio should be
1)?
Testing HO
• Compare sample F-ratio (13.22) to
probability distribution of F-ratio:
– distribution of F when HO is true (sampling
distribution of F-ratio)
• Degrees of freedom:
– dfGroups = 3
– dfResidual =16
3, 16 df
P(F)
a = 0.05
0
1
2
3
F
4
5
F = 3.24
• Any F-ratio > 3.24 has < 0.05 (5%)
chance of occurring if HO is true
• F = 13.22 >> 3.24
• Much less than 0.05 chance of
occurring if HO is true
• We reject HO
– statistically significant result
Assumptions
ANOVA assumptions apply to dependent
variable.
• Observations within each group come
from normally distributed populations
• ANOVA robust:
– use boxplots to check for skewness and
outliers
Assumptions (cont.)
• Variances of group populations are the
same - homogeneity of variance
assumption
– skewed populations produce unequal
group variances
– ANOVA reliable if group n’s are equal and
variances not too different:
• ratio of largest to smallest variance  3:1
Residuals in ANOVA
Residual:
• difference between observed and
predicted value of dependent variable
• in ANOVA, residual is difference
between each y-value and group mean
( yij  yi )
• Even spread of
residuals
• Assumptions OK
Residual
Residual plots - residuals vs
group means
Mean
• Wedge-shaped
spread of residuals
• Indicates unequal
variances and skewed
dependent variable
• Transformation will
help
Residual
Residual plots
Mean
Variance vs mean plot
• Plot group variances against group
means
• In skewed distributions (lognormal and
Poisson), variance is +vely related to
mean
• In normal distributions, variance is
independent of mean
• No relationship
between variance
and mean
• Distribution(s)
probably normal
Variance
Variance vs mean
Mean
• Positive relationship
between variance
and mean
• Distribution(s)
probably skewed
• Transformation
required
Variance
Variance vs mean
Mean
• No pattern in
residuals
• Normality &
homogeneity of
variances OK
Residual
Barnacle example
8
6
4
2
0
-2
-4
-6
-8
10
15
20
25
Group mean
30
• No relationship
between variance
and mean
• Suggests nonskewed
distribution
Group variance
Barnacle example
30
25
20
15
10
5
0
10
15
20
25
Group mean
30
Asssumptions (cont.)
• Data should be independent within and
between groups
– no replicate used more than once
– must be considered at design stage
ANOVA with 2 groups
• Null hypothesis:
– no difference between 2 population means
• ANOVA F-test or t-test
• F = t2
• P-values identical
Specific comparisons of groups
Type I error
• Probability of rejecting HO when true
– probability of false significant result
• Set by significance level (e.g. 0.05)
– 5% chance of falsely rejecting HO
• Probability of Type I error for each
separate test
Specific comparisons of means
Which groups are significantly different
from which?
• Multiple pairwise t-tests:
– each test with a = 0.05
• Increasing Type I error rate:
– probability of at least one Type I error
among all comparisons (family-wise Type I
error rate) increases
No. of
groups
3
5
10
No. of
comparisons
3
10
45
Familywise
probability
Type I error
0.14
0.40
0.90
Unplanned pairwise comparisons
Unplanned comparisons
• Comparisons done after a significant
ANOVA F-test
• Usually comparing each group to each
other group:
– which are significantly different from
which?
• Lots of comparisons:
– not independent
Unplanned comparisons
• Control familywise Type I error rate to
0.05:
– significance level for each comparison
must be below 0.05
• Many different tests that try to achieve
this
• Called unplanned (pairwise) multiple
comparisons
Tukey’s test
• Tests every pair of group means:
– adjusts a (significance level) so probability
of Type I error among all tests < 0.05
• Uses Q distribution (studentized range
distribution)
• Uses SE = (MSResidual/n)
• Compares difference between each pair
of means to Q*SE:
– differences larger than Q*SE significant
– differences less than Q*SE non-significant
• Available in SYSTAT and SPSS
Barnacle example
• SE = (MSResidual/n) = (18.58/5) = 1.93
• Q with 16df for 4 groups (from Q table)
= 4.05
• Q*SE = 7.82
• Compare difference between each pair
of group means with 7.82
• Pairwise comparisons:
– alg2 vs scraped = 15.2 (significant)
– alg2 vs bare = 13.4 (significant)
– alg1 vs scraped = 9.2 (significant)
– alg2 vs alg1 = 6.0 (not significant)
– alg1 vs bare = 7.4 (not significant)
– bare vs scraped = 1.8 (not significant)
Underlines join means not significantly
different:
Scraped
Bare
Alg1
Alg2
Pairwise t-tests
• Use SE = (MSResidual/n) in denominator of
test
• Adjust significance levels of each test with
Bonferroni adjustment:
– 0.05 / no. tests
• 6 t-tests to compare all pairs of 4 groups:
– use a = 0.05/6 = 0.0083
• Available in SYSTAT and SPSS
Planned comparisons
Planned comparisons
• Also called contrasts
• Interesting and logical comparisons of
means or combinations of means
• Planned before data analysis
• Ideally independent:
– therefore only small number of
comparisons allowed
• Number of independent comparisons <
df Groups
– e.g. 4 groups, 3 df, maximum 3
independent contrasts
• Each test can be done at 0.05
– no correction for increased family-wise
error rate ????
Methods for planned comparisons
t - tests
• Usual t-tests to compare 2 means
• Use standard error based on whole data
set, not just two groups being compared
– SE = (MSResidual/n)
Partition variance - ANOVA
• Partition SSGroups:
– SS for each comparison
– 1 df
– test with F-test as part of ANOVA
• F-test vs t-test:
– F = t2 because each comparison compares
2 groups
Barnacle example
• Specific comparisons planned as part of
barnacle experiment
HO:
• No difference in recruitment between 2
algal species
• HO: m1 = m2
HO: m1 = m2 or HO: m1 - m2 = 0
Linear combination of means using
coefficients (ci’s):
c1 y1  c2 y2  ...  ci yi
where ci = 0
HO: m1 = m2
(1) y1  (1) y2  (0) y3  (0) y4
(10) y1  (10) y2  (0) y3  (0) y4
Note ci = 0:
(+1) + (-1) + (0) + (0) = 0
(+10) + (-10) +(0) + (0) = 0
• HO:
• no difference in recruitment between algal
and bare surfaces
• (m1+m2)/2 = (m3+m4)/2
• 0.5m1+ 0.5m2 = 0.5m3 + 0.5m4
(0.5) y1  (0.5) y2  (0.5) y3  (0.5) y4
(1) y1  (1) y2  (1) y3  (1) y4
Note ci = 0
• SS for each contrast determined
• df = 1:
– 2 groups or combinations of groups being
compared
• SSContrast = MSContrast
• Test with F-test:
MSContrast
MSResidual
Partitioning SSGroups
Source
SS
Groups
736.55
3
90.00
1
90.00
638.50
1
638.50
297.20 16
18.58
Alg1 vs alg2
Alg&alg2 vs
bare&scraped
Residual
Total
df
MS
1033.75 19
F
P
245.52 13.22 <0.001
4.84
0.043
34.45 <0.001
• Reject HO:
– significant difference between algal types
in barnacle recruitment.
• Reject HO:
– significant difference between algal types
(1 & 2) and bare substrata (bare &
scraped).
Planned comparisons in the
literature
Newman (1994)
• Ecology 75:1085-1096
• Effects of changing food levels on size
and age at metamorphosis of tadpoles
• Four treatments used:
– low food (n=5), medium food (n=8), high
food (n=6), food decreasing from high to
low (n=7)
• HO: no effect of food levels on size of
toads at metamorphosis.
Planned comparison of decreasing food vs
constant high food:
HO: no difference between decreasing food and
high food on size of toads at metamorphosis.
Source
df
Food
3
0.0448
17.41
<0.001
1
0.0345
40.27
<0.001
High vs
decreasing
Residual
22
SS
0.0189
F
P
2. Pairwise t-tests with adjustment to a
• uses Bonferroni adjustment (a/c, where c is
number of tests), eg. 4 groups, 6 pairwise
comparisons, Bonferroni a - 0.05/6 = 0.0083
• more conservative than Tukey’s, ie. fewer
significant results
One factor ANOVAs in the
literature
Shapiro et al. (1994)
• Ecology 75:1334 - 1344
• Spawning time (DV) of female coral reef
fish (Thalassoma bifasciatum) of 3 size
classes.
• Null hypothesis:
– no difference in mean spawning time
between 3 size classes of T. bifasciatum
Female size
small
medium
large
n
13
16
14
Time (mean ± SE)
97+8
68+13
35+10
ANOVA F = 6.98, P = 0.003. Reject HO.
Cushman et al. (1994)
• Ecology 75:1031 -1042
• Effect of different diets on survival of ants:
– plant only, plant+butterfly larvae,
plant+artificial ant food
– three groups (treatments), 10 replicate vials
of ants per treatment.
• Null hypothesis:
– no difference in survival between groups.
Survival
100
80
60
40
20
0
Plant only
Plant+larva
Plant+food
Diet
Source
df
Diet
2
5594.9
27
206.2
Residual
Reject HO
MS
F
27.13
P
0.001
Independence of comparisons
• All pairwise comparisons not independent
of each other:
– e.g. pairwise comparisons of 3 means
– Test 1: mean 1 significantly less than mean 2
– Test 2: mean 2 significantly less than mean 3
– Test 3: compare mean 1 to mean 3
• Ho much less likely to be true given tests 1 and 2
• P-value difficult to interpret