Transcript Chapter 2. Random Variables

Chapter 11
The Analysis of Variance
11.1 One Factor Analysis of Variance
11.2 Randomized Block Designs (not for this course)
NIPRL
11.1 One Factor Analysis of Variance
11.1.1 One Factor Layouts (1/4)
• Suppose that an experimenter is interested in k populations
with unknown population means 1 , 2 , , k
• The one factor analysis of variance methodology is
appropriate for comparing three of more populations.
• The observation xij represents the j th observation from
the i th population.
• The sample from population i consists of the ni observations
xi1 ,
, xini
• If the sample sizes n1 , , nk are all equal, then the data set
is balanced, and if the sample sizes are unequal, then the
data set is unbalanced.
NIPRL
2
11.1.1 One Factor Layouts (2/4)
• The total sample size of the data set is nT  n1 
 nk
• A data set of this kind is called a one-way or one factor layout.
• The single factor is said to have k levels corresponding to the k
populations under consideration.
• Completely randomized designs : the experiment is performed by
randomly allocating a total of nT “units” among the k
populations.
• Modeling assumption xij  i   ij
where the error terms
iid
 ij ~ N (0,  2 )
iid
• Equivalently, xij ~ N ( i ,  2 )
NIPRL
3
11.1.1 One Factor Layouts (3/4)
• Point estimates of the unknown population means
i  xi 
•
H 0 : 1 
xi1 
 xini
ni
,
1 i  k
 k
H A : i   j , for some i and j
• Acceptance of the null hypothesis indicates that there is no
evidence that any of the population means are unequal.
• Rejection of the null hypothesis implies that there is evidence
that at least some of the population means are unequal.
NIPRL
4
11.1.1 One Factor Layouts (4/4)
• Example 60 : Collapse of Blocked Arteries
– level 1 : stenosis = 0.78
level 2 : stenosis = 0.71
level 3 : stenosis = 0.65
NIPRL
10.6   8.3
 11.209
11
11.7   17.6
 2  x2 
 15.086
14
19.6   16.6
 3  x3 
 17.330
10
–
 1  x1 
–
H0 : 1  2  3
5
11.1.2 Partitioning the Total Sum of Squares (1/5)
• Treatment Sum of Squares
–
n1 x1   nk x k
x 
nT
– SSTr 

x11 
 xknk
nT
k
2
n
(
x

x
)
i
i
i 1
– A measure of the variability between the factor levels.
– SSTr 
k
 n (x
i 1
k
i
k
i
k
k
i 1
i 1
 x )   ni x i  2 ni x i x  ni x
2
2
i 1
k
  ni x i  2nT x  nT x   ni x i  nT x
i 1
NIPRL
2
2
2
2
2
2
i 1
6
11.1.2 Partitioning the Total Sum of Squares (2/5)
• Error sum of squares
– SSE 
k
ni
2
(
x

x
)
i
 ij
i 1 j 1
– A measure of the variability within the factor levels.
– SSE 
k
ni
 ( x
ij
i 1 j 1
k
ni
k
ni
ni
k
k
ni
 x i )   x 2 xij x i   x i
2
i 1 j 1
k
k
2
ij
i 1 j 1
k
2
i 1 j 1
ni
k
  x 2 ni x i   ni x i   x  ni x i
i 1 j 1
NIPRL
2
ij
i 1
2
i 1
2
i 1 j 1
2
ij
2
i 1
7
11.1.2 Partitioning the Total Sum of Squares (3/5)
• Total sum of squares
– SST 
k
ni
2
(
x

x
)
 ij
i 1 j 1
– A measure of the total variability in the data set
– SST 
k
ni
 ( x
ij
i 1 j 1
k
k
ni
k
ni
ni
i 1 j 1
2
ij
i 1 j 1
k
ni
  x 2nT x  nT x   x  nT x
i 1 j 1
k
ni
 x )   x  2 xij x   x
2
2
ij
2
2
i 1 j 1
2
ij
2
i 1 j 1
2
– SST = SSTr + SSE
NIPRL
8
11.1.2 Partitioning the Total Sum of Squares (4/5)
• P-value considerations
– The plausibility of the null hypothesis that the factor level
means are all equal depends upon the relative size of the
sum of squares for treatments, SSTr, to the sum of squares
for error, SSE.
NIPRL
9
11.1.2 Partitioning the Total Sum of Squares (5/5)
• Example 60 : Collapse of Blocked Arteries
– x  11.209, x  15.086, x  17.330
1
2
x 
k
3
10.6  16.6
 14.509
35
ni
 x
i 1 j 1
2
ij
 10.62 
 16.62  7710.39
ni
k
SST   xij2  nT x  7710.39  (35 14.509 2 )  342.5
2
i 1 j 1
k
SSTr   ni x i  nT x
2
2
i 1
 (1111.2092 )  (14 15.086 2 )  (10 17.3302 )  (35 14.509 2 )
 204.0
– SSE=SST-SSTr=342.5-204.0=138.5
NIPRL
10
11.1.3 The Analysis of Variance Table (1/5)
2
(
x

x
)
 j 1 ij i
ni
si2 
ni  1
k
SSE   (ni  1) si2
i 1
•
Mean square error
–
MSE =
SSE
SSE
=
d.f.
nT  k
n2 k
–
MSE ~  2
–
E(MSE)   2
T
nT  k
since E ( n2T k )  nT  k
2
– So, MSE is an unbiased point estimate of the error variance 
NIPRL
11
11.1.3 The Analysis of Variance Table (2/5)
• Mean squares for treatments
–
MSTr =
– E (MSTr)   2 
SSTr SSTr
=
d.f.
k -1
2
n
(



)
 i1 i i
k
k 1
( why ?)
where
n11  nk k
nT
– If the factor level means i are all equal,
2
then
2
2  k 1
E (MSTr)   and MSTr ~ 
( why ?)
k 1
• These results can be used to develop a method for calculating
the p-value of the null hypothesis
– When this null hypothesis is true, then F -statistic
 k21 /(k  1)
MSTr
F
 2
~ Fk 1,nT k ( why ?)
MSE  nT k /(nT  k )
NIPRL
12
11.1.3 The Analysis of Variance Table (3/5)
•
p-value  P( X  F )
where X ~ Fk 1,nT k
Fk 1,nT k distribution
p  value
F
NIPRL
MSTr
MSE
13
11.1.3 The Analysis of Variance Table (4/5)
• Analysis of variance table for one factor layout
Source
Sum of
squares
Mean squares
k 1
SSTr
MSTr 
Error
nT  k
SSE
MSE 
total
nT  1
SST
Treatments
NIPRL
d.f.
SSTr
k 1
F-statistic
F
MSTr
MSE
P-value
P( Fk 1,nT k  F )
SSE
nT  k
14
11.1.3 The Analysis of Variance Table (5/5)
• Example 60 : Collapse of Blocked Arteries
– The degrees of freedom for treatments is k  1  3  1  2
SSTr 204.0
MSTr 

 102.0
2
2
– The degrees of freedom for error is nT  k  35  3  32
SSE 138.5
MSE 

 4.33
32
32
– F  MSTr  102.0  23.6
MSE
4.33
p  value  P( X  23.6) 0 where X ~ Fk 1,nT k
– Consequently, the null hypothesis that the average flowrate
at collapse is the same for all three amounts of stenosis is
not plausible.
NIPRL
15
11.1.4 Pairwise Comparisons of the Factor Level Means
• When the null hypothesis is rejected, the experimenter can
follow up the analysis with pairwise comparisons of the factor
level means to discover which ones have been shown to be
different and by how much.
• With k factor levels there are k(k-1)/2 pairwise differences
i  i ,
1
NIPRL
2
1  i1  i2  k
16
•
A set of 1   confidence level simultaneous confidence intervals for
these pairwise differences are

q ,k ,v
i1  i2  x i1  x i2  s

2

where
–
NIPRL
1
1
 ,
ni1 ni2
x i1  x i2  s
q ,k ,v
2
1
1 


ni1 ni2 
s    MSE
q ,k ,v (see pp. 888-889) is a critical point that is the upper  point
of the Studentized range distribution with parameter k and degrees
of freedom v  nT  k .
17
• These confidence intervals are similar to the t-intervals
– Difference : q , , / 2 is used instead of t / 2,
– T-intervals have an individual confidence level whereas this
set of simultaneous confidence intervals have an overall
confidence level
– All of the k(k-1)/2 confidence intervals contain their
respective parameter value i1  i2
– q , , / 2 is larger than t / 2,
• If the confidence interval for the difference i1  i2 contains
zero, then there is no evidence that the means at factor levels
i1 and i2 are different
NIPRL
18
• Example 60 : Collapse of Blocked Arteries
– s    MSE  4.33  2.080
– With 32 degrees of freedom for error, the critical pt is
q0.05,3,32  3.48
– the overall confidence level is 1-0.05=0.95
– Individual confidence intervals have confidence levels of
1-0.0196 0.98
– The confidence interval for 1  2

2.080  3.48 1 1
2.080  3.48 1 1 
1  2  11.209  15.086 
 ,11.209  15.086 
 , 
11 14
11 14 
2
2

 (3.877  2.062, 3.877  2.062)  (5.939, 1.814)
NIPRL
19

2.080  3.48 1 1
2.080  3.48 1 1 
1  3  11.209  17.330 
 ,11.209  17.330 
 , 
11
10
11
10 
2
2

 (6.121  2.236, 6.121  2.236)  (8.357, 3.884)

2.080  3.48 1 1
2.080  3.48 1 1 
2  3  15.086  17.330 
 ,15.086  17.330 
 , 
14
10
14
10 
2
2

 (2.244  2.119, 2.244  2.119)  (4.364, 0.125)
– None of these three confidence intervals contains zero, and
so the experiment has established that each of the three
stenosis levels results in a different average flow rate at
collapse.
NIPRL
20
11.1.5 Sample Size Determination
• The sensitivity afforded by a one factor analysis of variance
depends upon the k sample sizes n1 , , nk
• The power of the test of the null hypothesis that the factor level
means are all equal increase as the sample sizes increase.
• An increase in the sample size results in a decrease in the
lengths of the pairwise confidence intervals.
• If the sample sizes ni are unequal,
1
1
L  2 sq , ,

ni1 ni2
• If the sample sizes are all equal to n,
L  2sq , , / n
NIPRL
21
• If prior to experimentation, an experimenter decides that a
confidence interval length no large than L is required, then the
sample size is
4s 2 q 2  , ,
n
L2
– The experimenter needs to estimate the value of s  
– The critical point q , , gets larger as the number of factor
levels k increases, which results in a larger sample size
required.
NIPRL
22
11.1.6 Model Assumptions
• Modeling assumption of the analysis of variance
– Observations are distributed independently with normal
distribution that has a common variance
– The independence of the data observations can be judged
from the manner in which a data set is collected.
– The ANOVA is fairly robust to the distribution of data, so that
it provides fairly accurate results as long as the distribution
is not very far from a normal distribution.
– The equality of the variances for each of the k factor levels
can be judged from a comparison of the sample variances
or from a comparison of the lengths of boxplots of the
observations at each factor level.
NIPRL
23
Summary problems
1. Can the equality of two population means be tested by ANOVA?
2. When does the F-statistic follow an F-distribution?
3. Why do you use the q-values instead of the t-quantiles in
pairwise comparisons of multiple means?
4. Does SSE /  follow a chi-square distribution under both of
a null and its alternative hypothesis?
2
NIPRL
24