Transcript Lecture 3: Resemblance Between Relatives

Lecture 10A:
Matrix Algebra
Matrices: An array of elements
Vectors
0
1written in bold lowercase
Usually
12
a = @ 13 A
(3 x 1)
47
b = (2 0 5
21 )
Column vector Row vector (1 x 4)
0
1
0 bold1 uppercase
General Matrices Usually
written
in
3 1 2
0 1
C = @2
1
(3 x 3)
5 4A
1 2
Square matrix
D = @3 4 A
2 9
(3 x 2)
Dimensionality of a matrix: r x c (rows x columns)
think of Railroad Car
A matrix is defined by a list of its elements.
B has ij-th element Bij -- the element in row i
and column j
Addition and Subtraction of Matrices
If two matrices have the same dimension (same number
of rows and columns), then matrix addition and subtraction
simply follows by adding (or subtracting) on an element by
element basis
Matrix addition: (A+B)ij = A ij + B
ij
Matrix subtraction: (A-B)ij = A ij - B ij
Examples:
µ
A =
Ž
( )
( )
3
1
0
2
µ
C= A+ B =
4 2
3 3
µ
and
Ž
B =
1 2
2 1
Ž
( )
(- )
µ
and
D= A° B =
2
°1
°2
1
Ž
Partitioned Matrices
It will often prove useful to divide (or partition) the
elements of a matrix into a matrix whose elements are
itself matrices.
0
1
..
0
1
3
.
1
2 C µ
B …
3 1 2
…
…
B ¢¢¢ ¢¢¢ ¢¢¢ ¢¢¢C
C = a
C = @2 5 4A = B
.
B
C
..
d
5
4 A
@ 2
1 1 2
..
1
.
1
2
Ž
( )
µ
a = (3);
b = ( 1 2);
d=
Ž
()
2
1
µ
;
B =
b
B
Ž
( )
5 4
1 2
• One
A row
vector
whose
elements
are column
vectors
partition
a whose
matrix
in many
ways
A can
column
vector
elements
are row
vectors
0 1
0 1
0 1
1 3
1
2
b
1
@
A
@
A
@
where
c1 = 2 ; c2 =
5 ; c3 =
4A
C = @b 2 A 1
1
2
b3
0
C = ( c1
c2
c3 )
b1 = ( 3 1
2);
b2 = ( 2 5
4);
b3 = ( 1 1
2)
Towards Matrix Multiplication: dot products
The dot (or inner) product of two vectors (both of
length n) is defined as follows:
. =
a ¢b
Xn
a i bi
i= 1
Example:
.
0 11
B 2C
a = @ A
3
4
and
b = (4 5
7 9)
a b = 1*4 + 2*5 + 3*7 + 4*9 = 60
Towards Matrix Multiplication: Systems of Equations
The
least-squares
solution
for the
Matrix
multiplication
is defined
in linear
such amodel
way so as
…solve linear systems of
to compactly
and¢¢¢Ø
y = represent
š + Ø1 x 1 +
n xn
equations.
yields the following system of equations for the bi
æ(y; z1 ) = Ø1 æ2 (z1 )
+ Ø2 æ(z1 ; z2 ) + ¢¢¢+ Øn æ(z1 ; zn )
æ(y; z2 ) = Ø1 æ(z1 ; z2 ) + Ø2 æ2 (z2 ) +
..
..
..
.
.
.
æ(y; zn )= Ø1 æ(z1 ; zn ) + Ø2 æ(z2 ; zn ) +
¢¢¢+ Øn æ(z2 ; zn )
..
..
.
.
¢¢¢ + Øn æ2 (zn )
This can be more compactly written in matrix form as
0
æ2 (z1 )
B æ(z1 ; z2 )
B
..
@
.
æ(z1 ; z2 )
æ2 (z2 )
..
.
æ(z1 ; zn ) æ(z2 ; zn )
: : : æ(z1 ; zn ) 1 0 Ø1
: : : æ(z2 ; zn ) C B Ø2
CB .
..
..
A @ ..
.
.
: : : æ2 (zn )
Øn
1
0 æ(y; z ) 1
1
C
B æ(y; z2 ) C
C = B
C
..
A @
A
.
æ(y; zn )
Matrix Multiplication: Quick overview
The order in which matrices are multiplied effects
the matrix product, e.g. AB = BA
For the product of two matrices to exist, the matrices
must conform. For AB, the number of columns of A must
equal the number of rows of B.
The matrix C = AB has the same number of rows as A
and the same number of columns as B.
C(rxc) = A(rxk) B(kxc)
Elements
in the
Elements
in the
ith
ij-th element of C is given by
column A
of B
rowjth
of matrix
kOuter indices given dimensions of
X
Inner indices must match
C i j columns
= resulting
Ai lof
B l jA =matrix,
with
rows of
B r rows (A)
l = and
1
c columns (B)
More formally, consider the product L = MN
Express the matrix M as a column vector of row vectors
0
1
m1
B m2 C
C
M = B
@ ... A
where
m i = ( Mi1
Mi2
… Mi c )
¢¢¢
mr
Likewise express N as a row vector of
column vectors
… nb )
N = ( n 1 n 2 ¢¢¢
where
0 N
1j
B N2j
nj = B
@ ..
.
The ij-th element of L is the inner product
of M’s row i with N’s column j
0
… m .¢n 1
m 1 .¢n 1 m 1 .¢n 2 ¢¢¢
1
b
… m 2 .¢n b C
B m 2 .¢n 1 m 2 .¢n 2 ¢¢¢
C
L= B
.
.
.
.
@
A
..
..
..
..
… m r ¢n
. b
m r .¢n 1 m r .¢n 2 ¢¢¢
N cj
1
C
C
A
Examples
µ
AB =
(
a
c
µ
BA =
(
b
d
Žµ
)(
ae + cf
ga + ch
e
g
f
h
Ž
µ
)= (
eb + df
gd + dh
Ž
)
ae + bg
ce + dg
af + bh
cf + dh
Ž
)
The Transpose of a Matrix
The transpose of a matrix exchanges the
rows and columns, ATij = Aji
Useful identities
(AB)T = BT AT
(ABC)T = CT BT AT
Inner product = abT = a(n X 1) bT(1 X n)
0 b 1
1
Xn
.
T
@
A
..
¢¢¢ a n )
= a b =
a i bi
…
0
1
a1
.
a = @ .. A
an
0
1
b1
.
b = @ .. A
bn
Note that bTa = (aTb)T = aTb
i= 1
bn
Indices
Dimension
match,
of resulting
matricesproduct
conform
T
T
Outer product = a b =isa 1 (n
b (1 Xa n)scalar)
X0X11)(i.e.
1
( a1
0
1
a1
@ ... A ( b1
an
a 1 b1 a 1 b2
B a 2 b1 a 2 b2
T
…
¢¢¢ bn ) = ab = B
@ ... is ...an
Resulting product
an b1 a n b2
¢¢¢ a1 bn
¢¢¢ a2 bn C
C
.
.nxn
..
.. A
matrix
¢¢¢ a n bn
The Identity Matrix, I
The identity matrix serves the role of the
number 1 in matrix multiplication: AI =A, IA = A
I is a diagonal matrix, with all diagonal elements
being one, all off-diagonal elements zero.
Iij =
1 for i = j
0 otherwise
0
I3x 3
1 0
= @0 1
0 0
1
0
0A
1
-1
A
The Inverse Matrix,
For a square matrix A, define the Inverse of A, A-1, as
the matrix satisfying
-1
A A = AA
µ
For
A =
(
a
c
b
d
Ž
)
A° 1
-1
= I
1
=
ad ° bc
µ
(
d
°c
°b
a
If det(A) is not zero, A-1 exists and A is said to be
non-singular. IfIf
det(A)
= 0, A is (the
singular,
and no
this quantity
determinant)
unique inverse exists
(generalized
do)
is zero,
the inverseinverses
does not
exist.
Ž
)
Useful identities
(AB)-1 = B-1 A-1
(AT)-1 = (A-1)T
Key use of inverses --- solving systems of equations
Ax = c
Vector
of known
constants
Matrix
of known
constants
wish
to=solve
for
-1
A Ax
Ix = x
Hence, x = A-1 c
unknowns
we
-1 c.
A-1Vector
Ax = Aof
Note that
0
æ2 (z1 )
B æ(z1 ; z2 )
B
..
@
.
æ(z1 ; z2 )
æ2 (z2 )
..
.
æ(z1 ; zn ) æ(z2 ; zn )
: : : æ(z1 ; zn ) 1 0 Ø1
: : : æ(z2 ; zn ) C B Ø2
CB .
..
..
A @ ..
.
.
: : : æ2 (zn )
Øn
Vb = c ---> b = V-1 c
1
0 æ(y; z ) 1
1
C
B æ(y; z2 ) C
C = B
C
.
A @
A
..
æ(y; zn )
Example: solve the OLS for b in y = a + b1x1 + b2x2 + e
0
b=
V-1
c
c= @
æ(y; z1 )
1
0
A
V = @
æ(y; z2 )
2
æ (z1 )
æ(z1 ; z2 )
æ(z1 ; z2 )
æ2 (z2 )
1
A
It is more compact to use æ(z1 ; z2 ) = ž 12 æ(z1 )æ(z2 )
0
V
0
1
°- 1
æ2 (z
2)
1
@
= 2
2
2
æ (z1 )æ (z2 ) (1 °- ž 12 ) °- æ(z ; z )
1
2
0
-° æ(z1 ; z2 )
1
A
æ2 (z1 )
1 0
1
æ(y; z1 )
-° æ(z1 ; z2 )
2)
1
@ A =
@
A@
A
2
2
2
æ (z1 )æ (z2 ) (1 °- ž 12 ) Ø2
° æ(z1 ; z2 )
æ2 (z1 )
æ(y; z2 )
Ø1
æ2 (z
•
•
æ(y; z1 )
æ(y; z2 )
°- ž 12
æ2 (z1 )
æ(z1 )æ(z2 )
)
1
Ø1 =
1 °- ž 212
(
1
Ø2 =
1 °- ž 212
•
•
æ(y; z2 )
æ(y; z1 )
°- ž 12
æ2 (z2 )
æ(z1 )æ(z2 )
)
(
If r12 = 0, these reduce to the two univariate slopes,
æ(y; z1 )
Ø1 = 2
æ (z1 )
and
æ(y; z2 )
Ø2 = 2
æ (z2 )
Quadratic and Bilinear
Forms
Quadratic product: for An x n and xn x 1
xT A x =
Xn Xn
a i j x i xj
Scalar (1 x 1)
i= 1 j = 1
Bilinear Form (generalization of quadratic product)
for Am x n, an x 1, bm x1 their bilinear form is bT1 x m Am x n an x 1
Note that bTA a = aTAT b
Covariance Matrices for
Transformed Variables
What is the variance of the linear combination,
c1x1 + c2x2 + … + cnxn ? (note this is a scalar)
æ2
°
¦
¢
(c x) =
T
æ2
!
Xn
(
ci x i
i= 1
0
1
Xn
Xn
= æ@
ci x i ;
cj x j A
i= 1
Xn Xn
=
j=1
Xn Xn
æ(ci x i ; cj x j ) =
i= 1 j = 1
T
ci cj æ (x i ; x j )
i= 1 j = 1
= c Vc
Likewise, the covariance between two linear combinations
can be expressed as a bilinear form,
æ(a T x ; b T x ) = a T V b
Now suppose we transform one vector of random
variables into another vector of random variables
Transform x into
(i) yk x 1 = Ak x n xn x 1
(ii) zm x 1 = Bm x n xn x 1
The covariance between the elements of these
two transformed vectors of the origin is a
k x m covariance matrix = AVBT
For example, the covariance between yi and yj
is given by the ij-th element of AVAT
Likewise, the covariance between yi and zj
is given by the ij-th element of AVBT
The Multivariate Normal
Distribution (MVN)
Consider the pdf for n independent normal
random variables, the ith of which has mean
mi and variance s2i
Ž
2
(x
°
š
)
i
i
p(x) =
(2º ) °- 1 =2 æ°i 1 exp °2 æi2
i= 1
¦ n
! °- 1
¦
!
n
2
Y
X (xi °- š i )
°- n = 2
= (2º )
æi
exp -°
2
2
æ
i
i
i
µ
Yn
(
(
=1
)
(
=1
This can be expressed more compactly in matrix form
Define the covariance matrix V for the vector x of
the n normal random variable by
0 æ2
1
B 0
V = B
@ ..
.
0
…
¢¢¢
¢¢¢
..
.
1
Fun fact! YnFor a
C diagonal
jV j = matrix
æi2 D, the
C
A Det = | D | = product
i= 1
¢¢¢… ¢¢¢ æn2
of the diagonal elements
0
æ22
..
.
0
0
..
.
Define the mean vector m by
0
1
š1
B š2 C
C
š = B
@ .. A
.
šn
Xn (x i °- š i )2
T
°- 1
=
(x
°
š
)
V
(x °- š )
2
æi
i
=1
Hence in matrix from the MVN pdf becomes
-
-
-
(
-
-
-
)
Now notice that this holds for any vector m and symmetric
(positive-definite) matrix V (pd -> aTVa = Var(aTx) > 0 for all nonzero a)
Properties of the MVN - I
1) If x is MVN, any subset of the variables in x is also MVN
2) If x is MVN, any linear combination of the
elements of x is also MVN. If x is MVN(m,V)
for
y = x + a;
for y = a T x =
y is MVNn (š + a; V )
Xn
a i xi ;
y is N(a T š ; a T V a )
k= 1
for
y = A x;
•
¥
T
y is MVNm ( A š ; A V A )
Properties of the MVN - II
3) Conditional distributions are also MVN. Partition x
into two components, x1 (m dimensional column vector)
and x2 ( n-m dimensional column vector)
µ
x=
x1
x2
Ž
( )
( )
x1 | x2 is MVN with m-dimensional mean vector
š x 1 j x 2 = š 1 + V x 1 x 2 V x°- 21x 2 (x 2 °- š 2 )
and m x m covariance matrix
V x 1 j x 2 = V x 1 x 1 °- V x1 x 2 V x°- 21x 2 V xT 1 x 2
Properties of the MVN - III
4) If x is MVN, the regression of any subset of
X on another subset is linear
linear and homoscedastic
homoscedastic
x1 = š x1 jx2 + e
= š 1 + V x 1 x 2 V x°-21x 2 (x 2 °- š 2 ) + e
Where e is MVN with mean vector 0 and
Variance-covariance matrix V
x1 jx2
The regression
regression is
is linear
homoscedastic
because
thefunction
variance
The
because it
is a linear
is ax2constant that does not change with the particular
of
value of the random vector x2
Example: Regression of Offspring value on Parental values
0
1
20
1
0
1 3
Assumezo the vector ofš ooffspring1value and the values of
A is
4@
zs parents
ª MVNis
š s A ; Then
æ2z @ h 2from
=2
1the correlations
0 A5
both @
its
MVN.
2 =2
z
š
h
0
1
d
d
among (outbred) relatives,
µ Ž
Let x 1 = ( zo ) ; x 2 = zs
h 2 =2
(z )
d
V x 1 ; x 1 = æ2z ;
Hence,
V x 1 ;x 2 =
zo
h 2 æz2
2
( 1 1);
h 2 =2
µ
Ž
)
(
µ
Žµ
Ž
1 0
z °- š
( 0 1 ) (z °- š ) + e
V x 2 ; x 2 = æz2
1 0
0 1
h 2 æz2
s
= šo +
( 1 1 ) æz°- 2
2
d
h2
h2
= šo +
(zs °- š s ) +
(zd °- š d ) + e
2
2
s
d
Where e is normal with mean zero and variance
Ž- 2 2 µ Ž
°- 1µ
2
2
=
š
+
V
V
(x
h æz x 1 x 2 °-x2 2 x 12 02 °h šæ2z ) +1 e
2
2 - 1
æe = æz °
( 1 1 ) æz
µ
= æz2
(
2
1 -°
h4
2
Ž
)
(0 1 )
2
(1)
Hence, the regression of offspring trait value given
the trait values of its parents is
zo = mo + h2/2(zs- ms) + h2/2(zd- md) + e
where the residual e is normal with mean zero and
Var(e) = sz2(1-h4/2)
Similar logic gives the regression of offspring breeding
value on parental breeding value as
Ao = mo + (As- ms)/2+ (Ad- md)/2 + e
= As/2+ Ad/2 + e
where the residual e is normal with mean zero and
Var(e) = sA2/2
Lecture 10b:
Linear Models
Quick Review of the Major Points
The general linear model can be written as
y = Xb + e
Vector
Vector
of
of
observed
residuals.
dependent
Usually
assumed
values
MVN with
mean
Design
Vector
matrix
of
--unknown
observations
parents
of
to
the
estimate
variables
in
the
If the
residuals
are
homoscedastic
and
uncorrelated,
zero linear model
assumed
so that we can write the cov matrix of e as Cov(e) = s2I
then use the OLS estimate, OLS(b) = (XTX)-1 XTy
If the residuals are heteroscedastic and/or dependent,
Cov(e) = V and we must use GLS solution for b,
GLS(b) = (XT V-1 X)-1 V-1 XTy
Linear Models
One tries to explain a dependent variable y as a linear
function of a number of independent (or predictor)
variables.
A multiple regression is a typical linear model,
…
y = š + Ø1 x 1 + Ø2 x2 + ¢¢¢+
Øn x x + e
The (partial) regression coefficients are interpreted
Here e is the residual, or deviation between the true
as follows. A unit change in xi while holding all
value observed and the value predicted
by the linear
other variables constant results in a change of bi in y
model.
As with a univariate regression (y =a + bx + e), the model
parameters are typically chosen by least squares,
wherein they are chosen to minimize the sum of
squared residuals, S ei2
Predictor and Indicator Variables
In a regression, the predictor variables are
typically continuous, although they need not be.
Suppose we measuring the offspring of p sires. One
linear model would be
yij = m + si + eij
Trait
valuefor
of
offspring
jincluded
fromof
sire
igive
Overall
mean.
This
term
invariables
to
the
sisterms
The
effect
sire
i
(the
mean
its
offspring).
Recall
Note
The
that
deviation
the
predictor
of
the
jth
offspring
here
from
are
the
the
, something
ifamily
We can write this in linear model form by using indicator
a that
mean
value
of
zero,
i.e.,
they
are of
expressed
as deviations
that
variance
in
the
s
estimates
Cov(half
sibs)
= VA/2the
i
mean
we
are
of
trying
sire
i.
to
The
estimate
variance
the
e’s
estimates
variables,
ž
from the
mean
within-family variance.
1 if sire k = i
xi k =
0
ot herwise
Giving the above model as
yi j = š +
Xp
s k x i k + ei j
k= 1
Models consisting entirely of indicator variables
are typically called ANOVA, or analysis of variance
models
Models that contain no indicator variables (other than
for the mean), but rather consist of observed value of
continuous or discrete values are typically called
regression models
Both are special cases of the General Linear Model
(or GLM)
yijk = m + si + dij + bxijk + eijk
Regression
model
ANOVA
model
Example: Nested
half sib/full
sib design with an
age correction b on the trait
Linear Models in Matrix Form
Suppose we have 3 variables in a multiple regression,
with four (y,x) vectors of observations.
yi = š + Ø1 xi 1 + Ø2 x i 2 + Ø2 x i 2 + ei
In matrix form,
0 y 1
1
By C
y = @ 2A
y3
y4
y = XØ+ e
0 1
0 š 1
B 1
B Ø1 C
Ø= @ A X = @
Ø2
1
Ø3
1
x 11
x 21
x 31
x 41
x 12
x 22
x 32
x 42
0 e 1
x 13
1
B e2 C
x 23 C
e
=
@e A
x 33 A
3
x 43
e4
1
Vector of
parameters
The
design matrix X. Details of both
to estimate
the experimental design and the
observed values of the predictor
variables all reside solely in X
The Sire Model in Matrix Form
The model here is yij = m + si + eij
Consider three sires. Sire 1 has 2 offspring, sire 2 has
one and sire 3 has three. The GLM form is
0
1
y11
B y12 C
B
C
B y21 C
y= B
C;
B y31 C
@
A
y32
y33
0
1
B1
B
B1
X = B
B1
@
1
1
1
1
0
0
0
0
0
0
1
0
0
0
1
0
0C
C
0C
C
1C
A
1
1
0
0
1
š
B s1 C
Ø= @ A ;
s2
s3
1
e11
B e12 C
B
C
B e21 C
and e = B
C
B e31 C
@e A
32
e33
Ordinary Least Squares (OLS)
When the covariance structure of the residuals has a
certain form, we solve for the vector b using OLS
IfIfresiduals
followare
a MVN
distribution,
OLS
= ML solution
the residuals
homoscedastic
and
uncorrelated,
s2(ei) = se2, s(ei,ej) = 0. Hence, each residual is equally
weighted,
Xn
Sum of squared
ei2 = b
b
e T be = (y °- X b ) T (y °- X b )
residuals can
i= 1
be written as
Predicted value of the y’s
-
T
° 1 T
X) X
bTaking
= (X
y
(matrix) derivatives shows this is minimized by
Thisvariance-covariance
is the OLS estimateestimate
of the vector
b sample
Xn
1
The
for
the
estimates
2
c2 =
æ
e
b
e
i
is
T
°- 1 2
n °- ran k(X ) i
=1
V b = (X X ) æe
Example: Regression Through the Origin
yi = bxi + ei
0 x
1
B x2
X = B
@ ..
.
Here
1
C
C
A
xn
XT X =
Xn
x 2i
XT y =
P
•
¥° 1
xi yi
b = XT X
XT y = P 2
xi
)
1
æ2 (b) =
n °- 1
Ø = ( Ø)
Xn
x i yi
i= 1
i= 1
(
0 y 1
1
B y2 C
C
y= B
@ .. A
.
yn
P
(yi °- bx i ) 2
P 2
xi
•
¥-° 1
æe2
T
2
æ (b) = X X
æe = P 2
xi
2
æe2
(
)
1 X
=
(yi °- bx i ) 2
n °- 1
Polynomial Regressions and
Interaction Effects
GLM
can easily
handle any function of the observed
Quadratic
regression:
predictor variables, provided the parameters to estimate
2 + e …
y
=
Æ+
Ø
x
+
Ø
x
are still linear,
i e.g. Y = 1a +b
i 1f(x)2+ bi1g(x) +
i +e
0
1
Æ
Ø = @ Ø1 A
Ø2
0 1
B1
X = B
@ ..
.
x1
x2
..
.
x 21
x 22
..
.
1
xn
x 2n
1
C
C
A
Interaction terms (e.g. sex x age) are handled similarly
yi = Æ+ Ø1 x i 1 + Ø2 x i 2 + Ø3 x i 1 xi 2 + ei
1
1
x
x
x
x
Æx1 changes
1 1 2y
With
x1 held
constant,
change in
x1 12 bchanges
Likewise,
a unit
change ainunit
y by
11 2+ b31 x
2
B 1 x 2 1 x2 2 x2 1 x 2 2 C
B Ø1 C
C
by b2 + b3x1
Ø= @ A
X = B
..
..
..
@ ...
A
Ø2
.
.
.
0
1
Ø3
0
1
xn 1
xn 2
xn 1 xn 2
Fixed vs. Random Effects
In linear models are are trying to accomplish two goals:
estimation the values of model parameters and estimate
any appropriate variances.
For example in the simplest regression model,
y = a + bx + e, we estimate the values for a and b and
also the variance of e. We, of course, can also
estimate the ei = yi - (a + bxi )
Note that a/b are fixed constants are we trying to
estimate (fixed factors or fixed effects), while the
ei values are drawn from some probability distribution
(typically Normal with mean 0, variance s2e). The
ei are random effects.
This distinction between fixed and random effects is
extremely important in terms of how we analyzed a model.
If a parameter is a fixed constant we wish to estimate,
it is a fixed effect. If a parameter are drawn from
some probability distribution and we are trying to make
inference on either the distribution and/or specific
realizations from this distribution, it is a random effect.
We generally speak of estimating fixed factors and
predicting random effects.
“Mixed” models contain both fixed and random factors
Example: Sire model
yij = m + si + eij
It
Conversely,
depends.
Is the sire
if
Ifwe
effect
we
Fixed
are
have
not
fixed
effect
(say)
Random
only
or10
interested
random?
sires,
effect
if we
in these
are ONLY
interested
10 particular
in the
sires
values
but also
of these
wish to
particular
make some
10 sires and
don’t
inference
care to
about
make
the
any
population
other inferences
from which
about
they
the
population
were drawn
from
(such
which
as the
theadditive
sires are
variance,
drawn, then
sincewe can
treat
fixedthe
effects.
In the effects.
case, the In
model
s2A =them
4s2s, as
), then
si are random
thisis
fully
covariance
for the residuals.
case specified
we wish tothe
estimate
m andstructure
the variances
2 . Since
Thus,
s2s and
wesneed
to estimate
2si also m,
estimates
s1 to s10 (or
andpredicts)
s2e, and we
w
2(e
2 .
the breeding
value
wish
to
write
the model
as for
yij =sire
m + si,i we
+ ealso
,
s
)
=
s
ij
ij
e
estimate (predict) these as well. Under a
Random-effects interpretation, we write the model as
yij = m + si + eij, s2(eij) = s2e, s2(si) = s2s
Generalized Least Squares (GLS)
Suppose the residuals no longer have the same
variance (i.e., display heteroscedasticity). Clearly
we do not wish to minimize the unweighted sum
of squared residuals, because those residuals with
smaller variance should receive more weight.
Likewise in the event the residuals are correlated,
we also wish to take this into account (i.e., perform
a suitable transformation to remove the correlations)
before minimizing the sum of squares.
Either of the above settings leads to a GLS solution
in place of an OLS solution.
In the GLS setting, the covariance matrix for the
vector e of residuals is written as R where
Rij = s(ei,ej)
The linear model becomes y = Xb + e, cov(e) = R
The GLS solution for b is
•
¥ °- 1
T
°
T
°
1
b = (X R X) X R 1 y
The variance-covariance of the estimated model
parameters is given by
•
¥°-1
V b = X T R °- 1 X
æe2
(
)
Example
One common setting is where the residuals are uncorrelated
but heteroscedastic, with s2(ei) = s2e/wi.
For example, sample i is the mean value of ni individuals,
with s2(ei) = se2/ni. Here wi = ni
R = Diag(w°1-1 ; w2° -1 ; : : : ; w°n- 1 )
Consider the model yi = a + bxi
µ
Ø=
Ž
Æ
Ø
( )
0
1
1 x1
.. A
X = @ ...
.
1 xn
Here
R
°- 1
= Diag(w1 ; w2 ; : : : ; wn )
giving
0
X
T
-
R ° 1y
= w@
yw
0
1
X T R °- 1 X = w @
A
1
xw
xy w
where
Xn
w=
wi ;
xw
i= 1
Xn wi xi
=
;
w
x2 w
i= 1
Xn wi yi
yw =
;
w
i= 1
Xn wi x2
i
=
w
i= 1
Xn wi xi yi
xyw =
w
i= 1
xw
x2 w
1
A
This gives the GLS estimators of a and b as
a = y w °- bx w
b=
x yw -° x w y w
x 2 w °- x w2
Likewise, the resulting variances and covariance for
these estimators is
2
æ (a) =
æ2e ¢x 2 w
w ( x 2 w °- x w2 )
æe2
æ (b) =
w ( x 2 w °- x w2 )
2
æ(a; b) =
-° æ2e x w
w ( x 2 w -° x w2 )