Transcript Model B: Properties of the Regression Coefficients

Type
author name/s here
Dougherty
Introduction to Econometrics,
5th edition
Chapter heading
Chapter 8: Stochastic Regressors
and Measurement Errors
© Christopher Dougherty, 2016. All rights reserved.
MODEL B: PROPERTIES OF THE REGRESSION COEFFICIENTS
Yi   1   2 X i  ui


ˆ
2 
X i  X Yi  Y 
 X
i
 X
2
  2   ai ui
We will now look at the properties of the OLS regression estimators with the assumptions of
Model B. We will do this within the context of the simple regression model. We will start by
demonstrating unbiasedness.
1
MODEL B: PROPERTIES OF THE REGRESSION COEFFICIENTS
Yi   1   2 X i  ui
ˆ2
 X  X Y  Y 


   a u
 X  X 
i
i
2
i
2
i
i
Xi  X
ai 
2


X

X
 j
We saw in Chapter 2 that the slope coefficient can be decomposed into the true value plus a
weighted linear combination of the values of the disturbance term in the sample, where the
weights depend on the observations on X.
2
MODEL B: PROPERTIES OF THE REGRESSION COEFFICIENTS
Yi   1   2 X i  ui
ˆ2
 X  X Y  Y 


   a u
 X  X 
i
i
2
2
i
i
i
Xi  X
ai 
2


X

X
 j
 
E ˆ2   2  E   ai ui    2   E  ai ui 
We now take expectations. 2 is just a constant, so it is unaffected.
3
MODEL B: PROPERTIES OF THE REGRESSION COEFFICIENTS
Yi   1   2 X i  ui
ˆ2
 X  X Y  Y 


   a u
 X  X 
i
i
2
2
i
i
i
Xi  X
ai 
2


X

X
 j
 
E ˆ2   2  E   ai ui    2   E  ai ui 
E   a i ui   E a1 u1  ...  a n un   E a1 u1   ...  E a n un    E a i ui 
We have now used the first expectation rule to rewrite the expectation of the linear
combination as the sum of the expectations of its components.
4
MODEL B: PROPERTIES OF THE REGRESSION COEFFICIENTS
Yi   1   2 X i  ui
ˆ2
 X  X Y  Y 


   a u
 X  X 
i
i
2
2
i
i
i
Xi  X
ai 
2


X

X
 j
 
E ˆ2   2  E   ai ui    2   E  ai ui 
Model A
E ai ui   ai E ui   0
 
E ˆ2   2   ai E  ui    2
In Model A, the values of X were nonstochastic. This meant that the ai terms were also
nonstochastic and could therefore be taken out of the expectations as factors. E(ui) = 0 for
all i, and hence we proved unbiasedness.
5
MODEL B: PROPERTIES OF THE REGRESSION COEFFICIENTS
Yi   1   2 X i  ui
ˆ2
 X  X Y  Y 


   a u
 X  X 
i
i
2
2
i
i
i
Xi  X
ai 
2


X

X
 j
 
E ˆ2   2  E   ai ui    2   E  ai ui 
Model A
E ai ui   ai E ui   0
 
E ˆ2   2   ai E  ui    2
We cannot do this with Model B because we are assuming that the values of X are
generated randomly (from a defined population).
6
MODEL B: PROPERTIES OF THE REGRESSION COEFFICIENTS
Yi   1   2 X i  ui
ˆ2
 X  X Y  Y 


   a u
 X  X 
i
i
2
2
i
i
i
Xi  X
ai 
2


X

X
 j
 
E ˆ2   2  E   ai ui    2   E  ai ui 
Model B
E a i ui   E a i E ui   0
E  f  X  g Y   E  f  X  E  g Y 
f  X   ai , g Y   ui
Instead we appeal to Assumption B.7. We saw in the Review chapter that if X and Y are two
independent random variables, the expectation of the product of functions of them can be
decomposed as the product of the expectations of the functions.
7
MODEL B: PROPERTIES OF THE REGRESSION COEFFICIENTS
Yi   1   2 X i  ui
ˆ2
 X  X Y  Y 


   a u
 X  X 
i
i
2
2
i
i
i
Xi  X
ai 
2


X

X
 j
 
E ˆ2   2  E   ai ui    2   E  ai ui 
Model B
E a i ui   E a i E ui   0
E  f  X  g Y   E  f  X  E  g Y 
f  X   ai , g Y   ui
Under Assumption B.7, ui is distributed independently of every value of X in the sample. It
is therefore distributed independently of ai. So if X and u are independent, we can make use
of the decomposition.
8
MODEL B: PROPERTIES OF THE REGRESSION COEFFICIENTS
Yi   1   2 X i  ui
ˆ2
 X  X Y  Y 


   a u
 X  X 
i
i
2
2
i
i
i
Xi  X
ai 
2


X

X
 j
 
E ˆ2   2  E   ai ui    2   E  ai ui 
Model B
E a i ui   E a i E ui   0
 
E  f  X  g Y   E  f  X  E  g Y 
f  X   ai , g Y   ui
E ˆ2   2   E  ai  E  ui    2
Since E(ui) = 0 for all i, under Assumption B.4, we have proved unbiasedness, assuming
E(ai) exists. For this to be the case, there must be some variation in X in the sample
(Assumption B.3). Otherwise the denominator of the expression for ai would be zero.
9
MODEL B: PROPERTIES OF THE REGRESSION COEFFICIENTS
Yi   1   2 X i  ui


ˆ
2 
X i  X Yi  Y 
 X
i
 X
2
 2
 X  X  u  u 


 X  X 
i
i
2
i
The next property, efficiency, we will take for granted. The Gauss–Markov theorem assures
that the OLS estimators are BLUE (best linear unbiased estimators), provided that the
regression model assumptions are valid.
10
MODEL B: PROPERTIES OF THE REGRESSION COEFFICIENTS
Yi   1   2 X i  ui


ˆ
2 
X i  X Yi  Y 
 X
i
 X
2
 2
 X  X  u  u 


 X  X 
i
i
2
i
   X i  X  ui  u  
plim ˆ2   2  plim 

2

  X i  X  

1

 n   X i  X  ui  u  
cov  X , u 
  2  plim 
 2 
 2

1
2
var  X 
X

X




 i


n
We will prove consistency. We have decomposed the limiting value of the estimator of the
slope coefficient into the true value and the limiting value of the error term.
11
MODEL B: PROPERTIES OF THE REGRESSION COEFFICIENTS
 A  plim A
plim   
 B  plim B
if A and B have probability limits
and plim B is not 0.
   X i  X  ui  u  
plim ˆ2   2  plim 

2

  X i  X  

1

 n   X i  X  ui  u  
cov  X , u 
  2  plim 
 2 
 2

1
2
var  X 
X

X




 i


n
We would now like to use the plim quotient rule. The plim of a quotient is the plim of the
numerator divided by the plim of the denominator, provided that both of these limits exist.
12
MODEL B: PROPERTIES OF THE REGRESSION COEFFICIENTS
 A  plim A
plim   
 B  plim B
if A and B have probability limits
and plim B is not 0.
   X i  X  ui  u  
plim ˆ2   2  plim 

2

  X i  X  

1

 n   X i  X  ui  u  
cov  X , u 
  2  plim 
 2 
 2

1
2
var  X 
X

X




 i


n
However, as the expression stands, the numerator and the denominator of the error term do
not have limits. The denominator increases indefinitely and the numerator does not
converge on a limit.
13
MODEL B: PROPERTIES OF THE REGRESSION COEFFICIENTS
 A  plim A
plim   
 B  plim B
if A and B have probability limits
and plim B is not 0.
   X i  X  ui  u  
plim ˆ2   2  plim 

2

  X i  X  

1

 n   X i  X  ui  u  
cov  X , u 
  2  plim 
 2 
 2

1
2
var  X 
X

X




 i


n
To deal with this problem, we divide both the numerator and the denominator by n.
14
MODEL B: PROPERTIES OF THE REGRESSION COEFFICIENTS
1
plim   X i  X ui  u   cov X , u   0
n
1
2
plim   X i  X   var  X 
n
   X i  X  ui  u  
plim ˆ2   2  plim 

2

  X i  X  

1

 n   X i  X  ui  u  
cov  X , u 
  2  plim 
 2 
 2

1
2
var  X 
X

X




 i


n
It can be shown that the limit of the numerator is the covariance of X and u and the limit of
the denominator is the variance of X.
15
MODEL B: PROPERTIES OF THE REGRESSION COEFFICIENTS
1
plim   X i  X ui  u   cov X , u   0
n
1
2
plim   X i  X   var  X 
n
   X i  X  ui  u  
plim ˆ2   2  plim 

2

  X i  X  

1

 n   X i  X  ui  u  
cov  X , u 
  2  plim 
 2 
 2

1
2
var  X 
X

X




 i


n
Under Assumption B.7, X and u are independent. Hence the covariance of X and u is zero
(see the Review chapter).
16
MODEL B: PROPERTIES OF THE REGRESSION COEFFICIENTS
Yi   1   2 X i  ui


ˆ
2 
X i  X Yi  Y 
 X
i
 X
2
 2
 X  X  u  u 


 X  X 
i
i
2
i
   X i  X  ui  u  
plim ˆ2   2  plim 

2

  X i  X  

1

X

X
u

u
 i  
 n  i
cov  X , u 
  2  plim 
 2 
 2

1
2
var  X 
X

X




 i


n
Thus we demonstrate that ̂ 2 is a consistent estimator of 2, provided that the regression
model assumptions are valid.
17
MODEL B: PROPERTIES OF THE REGRESSION COEFFICIENTS
B.1
The model is linear in parameters and correctly specified.
Y = 1 + 2X2 + … + kXk + u
B.2
The values of the regressors are drawn randomly from fixed
populations.
B.3
There does not exist an exact linear relationship among the regressors.
B.4
The disturbance term has zero expectation.
B.5
The disturbance term is homoskedastic.
B.6
The values of the disturbance term have independent distributions.
B.7
The disturbance term is distributed independently of the regressors.
B.8
The disturbance term has a normal distribution.
Finally, a note on Assumption B.8, that the disturbance term has a normal distribution. The
justification is that it is reasonable to suppose that the disturbance term is jointly generated
by a number of minor random factors.
18
MODEL B: PROPERTIES OF THE REGRESSION COEFFICIENTS
Yi   1   2 X i  ui


ˆ
2 
X i  X Yi  Y 
 X
B.8
i
 X
2
  2   ai ui
The disturbance term has a normal distribution.
Xi  X
ai 
2


X

X
 j
A central limit theorem states that the combination of these factors should approximately
have a normal distribution, even if the individual factors do not.
19
MODEL B: PROPERTIES OF THE REGRESSION COEFFICIENTS
Yi   1   2 X i  ui


ˆ
2 
X i  X Yi  Y 
 X
B.8
i
 X
2
  2   ai ui
The disturbance term has a normal distribution.
Xi  X
ai 
2


X

X
 j
If the disturbance term has a normal distribution, the regression coefficients also have
normal distributions. This follows from the fact that a linear combination of normal
distributions is also normal.
20
MODEL B: PROPERTIES OF THE REGRESSION COEFFICIENTS
Yi   1   2 X i  ui


ˆ
2 
X i  X Yi  Y 
 X
B.8
i
 X
2
  2   ai ui
The disturbance term has a normal distribution.
Xi  X
ai 
2


X

X
 j
What happens if we have reason to believe that the assumption is not valid? The central
limit theorem comes into the frame a second time.
21
MODEL B: PROPERTIES OF THE REGRESSION COEFFICIENTS
Yi   1   2 X i  ui


ˆ
2 
X i  X Yi  Y 
 X
B.8
i
 X
2
  2   ai ui
The disturbance term has a normal distribution.
Xi  X
ai 
2


X

X
 j
The random component of a regression coefficient is a linear combination of the values of
the disturbance term in the sample.
22
MODEL B: PROPERTIES OF THE REGRESSION COEFFICIENTS
Yi   1   2 X i  ui


ˆ
2 
X i  X Yi  Y 
 X
B.8
i
 X
2
  2   ai ui
The disturbance term has a normal distribution.
Xi  X
ai 
2


X

X
 j
By a central limit theorem, it follows that the combination will have an approximately normal
distribution, even if the individual values of the disturbance term do not, provided that the
sample is large enough.
23
MODEL B: PROPERTIES OF THE REGRESSION COEFFICIENTS
Yi   1   2 X i  ui


ˆ
2 
X i  X Yi  Y 
 X
B.8
i
 X
2
  2   ai ui
The disturbance term has a normal distribution.
Xi  X
ai 
2


X

X
 j
Hence asymptotically (in large samples) it ought to be safe to assume that the regression
coefficients have normal distributions, even if Assumption B.8 is invalid, provided that the
other regression model assumptions are satisfied.
24
Copyright Christopher Dougherty 2016.
These slideshows may be downloaded by anyone, anywhere for personal use.
Subject to respect for copyright and, where appropriate, attribution, they may be
used as a resource for teaching an econometrics course. There is no need to
refer to the author.
The content of this slideshow comes from Section 8.3 of C. Dougherty,
Introduction to Econometrics, fifth edition 2016, Oxford University Press.
Additional (free) resources for both students and instructors may be
downloaded from the OUP Online Resource Centre
http://www.oxfordtextbooks.co.uk/orc/dougherty5e/.
Individuals studying econometrics on their own who feel that they might benefit
from participation in a formal course should consider the London School of
Economics summer school course
EC212 Introduction to Econometrics
http://www2.lse.ac.uk/study/summerSchools/summerSchool/Home.aspx
or the University of London International Programmes distance learning course
EC2020 Elements of Econometrics
www.londoninternational.ac.uk/lse.
2016.05.07