Transcript Orthogonal Linear Contrasts - Department of Mathematics

Orthogonal Linear Contrasts
This is a technique for partitioning
ANOVA sum of squares into
individual degrees of freedom
Definition
Let x1, x2, ... , xp denote p numerical quantities
computed from the data.
These could be statistics or the raw observations.
A linear combination of x1, x2, ... , xp is defined to be
a quantity ,L ,computed in the following manner:
L = c1x1+ c2x2+ ... + cpxp
where the coefficients c1, c2, ... , cp are predetermined
numerical values:
Definition
If the coefficients c1, c2, ... , cp satisfy:
c1+ c2 + ... + cp = 0,
Then the linear combination
L = c1x1+ c2x2+ ... + cpxp
is called a linear contrast.
Examples
1. L  x 
x1  x2    x p
p
1
1
1




   x1    x2      x p
 p
 p
 p
2.
A linear
combination
A linear
x1  x2  x3 x4  x5
L

contrast
3
2
1
1
1
 1
 1
   x1    x2    x3     x4     x5
 3
 3
 3
 2
 2
3. L = x1 - 4 x2+ 6x3 - 4 x4 + x5
A linear
contrast
= (1)x1+ (-4)x2+ (6)x3 + (-4)x4 + (1)x5
Definition
Let A = a1x1+ a2x2+ ... + apxp and B= b1x1+ b2x2+ ...
+ bpxp be two linear contrasts of the quantities x1, x2,
... , xp. Then A and B are c called Orthogonal
Linear Contrasts if in addition to:
a1+ a2+ ... + ap = 0 and
b1+ b2+ ... + bp = 0,
it is also true that:
a1b1+ a2b2+ ... + apbp = 0.
.
Example
Let
x1  x2  x3 x4  x5
A

3
2
1
1
1
 1
 1
   x1    x2    x3     x4     x5
 3
 3
 3
 2
 2
 x1  x2

B
 x3   x4  x5 
 2

1
1
   x1    x2   1x3  1x4   1x5
2
2
Note:
11 11 1
1
1
            - 1    1    - 1  0
 3  2  3  2  3
2
2
Definition
Let
A = a1x1+ a2x2+ ... + apxp,
B= b1x1+ b2x2+ ... + bpxp ,
..., and
L= l1x1+ l2x2+ ... + lpxp
be a set linear contrasts of the quantities x1, x2, ... , xp.
Then the set is called a set of Mutually Orthogonal
Linear Contrasts if each linear contrast in the set is
orthogonal to any other linear contrast..
Theorem:
The maximum number of linear contrasts in a
set of Mutually Orthogonal Linear Contrasts
of the quantities x1, x2, ... , xp is p - 1.
p - 1 is called the degrees of freedom (d.f.)
for comparing quantities x1, x2, ... , xp .
Comments
1. Linear contrasts are making comparisons
amongst the p values x1, x2, ... , xp
2. Orthogonal Linear Contrasts are making
independent comparisons amongst the p
values x1, x2, ... , xp .
3. The number of independent comparisons
amongst the p values x1, x2, ... , xp is p – 1.
Definition
L  a1x1  a2 x2    ap xp
denotes a linear contrast of the p means
x1 , x2 ,  , x p
If each mean, xi , is calculated from n
observations then:
The Sum of Squares for testing the Linear
Contrast L, is defined to be:
2
nL
SS L = 2 2
2
a1 +a2+...+ap
the degrees of freedom (df) for testing the
Linear Contrast L, is defined to be
dfL  1
the F-ratio for testing the Linear Contrast
L, is defined to be:
SSL
F
1
MSError
Theorem:
Let L1, L2, ... , Lp-1 denote p-1 mutually
orthogonal Linear contrasts for comparing the
p means . Then the Sum of Squares for
comparing the p means based on p – 1 degrees
of freedom , SSBetween, satisfies:
SSBetween = SSL1 + SSL2 + ... + SSLp -1
Comment
Defining a set of Orthogonal Linear Contrasts for
comparing the p means
x1 , x2 ,  , x p
allows the researcher to "break apart" the Sum of
Squares for comparing the p means, SSBetween, and
make individual tests of each the Linear Contrast.
The Diet-Weight Gain example
x1  100.0, x2  85.9, x3  99.5,
x4  79.2, x5  83.9, x6  78.7
The sum of Squares for comparing the 6
means is given in the Anova Table:
Five mutually orthogonal contrasts are given below
(together with a description of the purpose of these
contrasts) :
1
1
L1  x1  x2  x3   x4  x5  x6 
3
3
(A comparison of the High protein diets with Low
protein diets)
1
1
L2  x1  x 4   x3  x6 
2
2
(A comparison of the Beef source of protein with the
Pork source of protein)
1
1
L3  x1  x3  x 4  x6   x 2  x5 
4
2
(A comparison of the Meat (Beef - Pork) source of
protein with the Cereal source of protein)
1
1
L4  x1  x6   x3  x 4 
2
2
(A comparison representing interaction between
Level of protein and Source of protein for the Meat
source of Protein)
1
1
L5  x1  x3  2 x5   x 4  x6  2 x 2 
4
4
(A comparison representing interaction between
Level of protein with the Cereal source of Protein)
The Anova Table for Testing these contrasts is given
below:
Source:
DF:
Sum Squares:
Mean Square:
F-test:
Contrast L1
Contrast L2
Contrast L3
Contrast L4
Contrast L5
Error
1
1
1
1
1
54
3168.267
2.500
264.033
0.000
1178.133
11586.000
3168.267
2.500
264.033
0.000
1178.133
214.556
14.767
0.012
1.231
0.000
5.491
The Mutually Orthogonal contrasts that are
eventually selected should be determine prior to
observing the data and should be determined by the
objectives of the experiment
Another Five mutually orthogonal contrasts are
given below (together with a description of the
purpose of these contrasts) :
1
1
L1  x1  x3  x4  x6   x2  x5 
4
2
(A comparison of the Meat (Beef - Pork) source of
protein with the Cereal source of protein)
1
1
L2  x1  x 4   x3  x6 
2
2
(A comparison of the Beef source of protein with the
Pork source of protein)
L3  x1  x4
(A comparison of the high and low protein diets for
the Beef source of protein)
L4  x2  x5
(A comparison of the high and low protein diets for
the Cereal source of protein)
L5  x3  x6
(A comparison of the high and low protein diets for
the Pork source of protein)
The Anova Table for Testing these contrasts is given
below:
Source:
DF:
Sum Squares:
Mean Square:
F-test:
Beef vs Pork ( L1)
Meat vs Cereal ( L2)
High vs Low for Beef ( L3)
High vs Low for Cereal ( L4)
High vs Low for Pork ( L5)
Error
1
1
1
1
1
54
2.500
264.033
2163.200
20.000
2163.200
11586.000
2.500
264.033
2163.200
20.000
2163.200
214.556
0.012
1.231
10.082
0.093
10.082
Orthogonal Linear Contrasts
Polynomial Regression
Orthogonal Linear Contrasts for Polynomial Regression
Orthogonal Linear Contrasts for Polynomial Regression
Example
In this example we are measuring the “Life”
of an electronic component and how it
depends on the temperature on activation
Table
Activation
Temperature
0
53
50
47
Ti.
150
Mean
50
yij2 = 56545
25
50
75
100
60
67
65
58
62
70
68
62
58
73
62
60
T..
180
210
195
180
915
60
70
65
60
Ti.2/n = 56475
T..2/nt = 55815
The Anova Table
L = 25.00 Q2 = -45.00 C = 0.00 Q4 = 30.00
Source
Treat
Linear
Quadratic
Cubic
Quartic
Error
Total
SS
df
660
4
187.50
1
433.93
1
0.00
1
38.57
1
70
10
730
14
MS
165.0
187.50
433.93
0.00
38.57
7.00
F
23.57
26.79
61.99
0.00
5.51
The Anova Tables
for Determining degree of polynomial
Testing for effect of the factor
Source
Treat
Error
Total
SS
660
70
730
df
4
10
14
MS
165
7
F
23.57
Testing for departure from Linear
Testing for departure from Quadratic
y = 49.751 + 0.61429 x -0.0051429 x^2
70
65
Life
60
55
50
45
40
0
20
40
60
Act. Temp
80
100
120
Multiple Testing
•Tukey’s Multiple comparison procedure
•Scheffe’s multiple comparison procedure
Multiple Testing – a Simple Example
Suppose we are interested in testing to see if
two parameters (q1 and q2) are equal to zero.
There are two approaches
1. We could test each parameter separately
a) H0: q1 = 0 against HA: q1 ≠ 0 , then
b) H0: q2 = 0 against HA: q2 ≠ 0
2. We could develop an overall test
H0: q1 = 0, q2= 0 against HA: q1 ≠ 0 or q2 ≠ 0
1. To test each parameter separately
a) H0(1) :q1  0 against H A(1) :q1  0
then
b) H ( 2) : q  0 against H ( 2) : q  0
0
2
A
2
We might use the following test:
Reject H
(1)
0
if qˆ1  K
then
Reject H 0( 2) if qˆ2  K
K
is chosen so that the probability of a
Type I errorof each test is .
2. To perform an overall test
H0: q1 = 0, q2= 0 against HA: q1 ≠ 0 or q2 ≠ 0
we might use the test
Reject H0 if qˆ12  qˆ22  K(overall)
(overall )
K
is chosen so that the probability of a
Type I error is .
qˆ2
qˆ1  K
qˆ1
qˆ2
qˆ2  K
qˆ1
qˆ2
qˆ1  K
qˆ2  K
qˆ1
qˆ2
( multiple)
ˆ
q1  K
qˆ2  K
( multiple)
qˆ1
qˆ2
2
2
( overall )
ˆ
ˆ
q q  K
1
2

qˆ1
qˆ2
qˆ2  K( multiple)
qˆ1  K( multiple)
2
2
( overall )
ˆ
ˆ
q q  K
1
2

qˆ1
qˆ2
c1qˆ1  c2qˆ2  K( Scheffe)
2
2
( overall )
ˆ
ˆ
q q  K
1
c1qˆ1  c2qˆ2  K( Scheffe)
2

qˆ1
Post-hoc Tests
Multiple Comparison Tests
Post-hoc Tests
Multiple Comparison Tests
Suppose we have p means
x1 , x2 ,  , x p
An F-test has revealed that there are significant
differences amongst the p means
We want to perform an analysis to determine
precisely where the differences exist.
Tukey’s Multiple Comparison
Test
Let
MSError
s

n
n
denote the standard error of each xi
Tukey's Critical Differences
D  q
s
n
 q
MS Error
n
Two means are declared significant if they
differ by more than this amount.
q = the tabled value for Tukey’s studentized
range p = no. of means, n = df for Error
Scheffe’s Multiple Comparison
Test
Scheffe's Critical Differences (for Linear
contrasts)
s
2
2
2
S   p  1F  p  1,n 
a1  a 2    a p
n

 p  1F  p  1,n 
MS Error
n
a12  a22    a 2p
A linear contrast is declared significant if it
exceeds this amount.
F  p 1,n  = the tabled value for F distribution
(p -1 = df for comparing p means,
n = df for Error)
Scheffe's Critical Differences
(for comparing two means)
L  xi  x j
S 
 p 1F  p 1,n 
MSError
n
2
Two means are declared significant if they
differ by more than this amount.
Table 5: Critical Values for
the multiple range Test , and the F-distribution
Length
Temp,Thickness,Dry
q.05
q.01
F.05
F.01
3.84
4.60
4.80
5.54
2.92
2.33
4.51
3.30
Table 6: Tukey's and Scheffe's Critical Differences
Tukeys
Scheffés
 = .05
 = .01
 = .05
 = .01
Length
1.59
1.99
2.05
2.16
Temp, Thickness, Dry 3.81
4.59
4.74
5.64
Table of differences in means
4.25
4.25
4.44
5.66
15.45
17.43
28.76
29.95
4.44
0.19
5.66
1.41
1.22
15.45
11.2
11.01
9.79
17.43
13.18
12.99
11.77
1.98
28.76
24.51
24.32
23.1
13.31
11.33
29.95
25.7
25.51
24.29
14.5
12.52
1.19
Underlined groups have no significant differences
There are many multiple (post hoc) comparison
procedures
1. Tukey’s
2. Scheffe’,
3. Duncan’s Multiple Range
4. Neumann-Keuls
etc
Considerable controversy:
“I have not included the multiple comparison methods of
D.B. Duncan because I have been unable to understand
their justification” H. Scheffe, Analysis of Variance