Transcript SM SF --------+

POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY
u
2

(1)
2
b2

  X 2i
 X2
2

1
1 r
2
X 2 ,X 3
u
2

n MSD ( X 2 )

1
1  rX 2 , X 3
2
Combine the correlated variables.
In this sequence, we look at four possible indirect methods for alleviating a problem of
multicollinearity.
1
POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY
u
2

(1)
2
b2

  X 2i
 X2
2

1
1 r
2
X 2 ,X 3
u
2

n MSD ( X 2 )

1
1  rX 2 , X 3
2
Combine the correlated variables.
First, if the correlated variables are similar conceptually, it may be reasonable to combine
them into some overall index.
2
POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY
. reg EARNINGS S EXP EXPSQ MALE ASVABC
Source |
SS
df
MS
-------------+-----------------------------Model | 28957.3532
5 5791.47063
Residual | 83052.8779
534 155.529734
-------------+-----------------------------Total | 112010.231
539 207.811189
Number of obs
F( 5,
534)
Prob > F
R-squared
Adj R-squared
Root MSE
=
=
=
=
=
=
540
37.24
0.0000
0.2585
0.2516
12.471
-----------------------------------------------------------------------------EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------S |
2.031419
.296218
6.86
0.000
1.449524
2.613315
EXP | -.0816828
.6441767
-0.13
0.899
-1.347114
1.183748
EXPSQ |
.0130223
.021334
0.61
0.542
-.0288866
.0549311
MALE |
5.762358
1.104734
5.22
0.000
3.592201
7.932515
ASVABC |
.2447687
.0714294
3.43
0.001
.1044516
.3850858
_cons | -26.18541
5.452032
-4.80
0.000
-36.89547
-15.47535
------------------------------------------------------------------------------
That is precisely what has been done with the three cognitive ASVAB variables. ASVABC
has been calculated as a weighted average of scores on subtests: ASVAB02 (arithmetic
reasoning), ASVAB03 (word knowledge), and ASVAB04 (paragraph comprehension).
3
POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY
. reg EARNINGS S EXP EXPSQ MALE ASVABC
Source |
SS
df
MS
-------------+-----------------------------Model | 28957.3532
5 5791.47063
Residual | 83052.8779
534 155.529734
-------------+-----------------------------Total | 112010.231
539 207.811189
Number of obs
F( 5,
534)
Prob > F
R-squared
Adj R-squared
Root MSE
=
=
=
=
=
=
540
37.24
0.0000
0.2585
0.2516
12.471
-----------------------------------------------------------------------------EARNINGS |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------S |
2.031419
.296218
6.86
0.000
1.449524
2.613315
EXP | -.0816828
.6441767
-0.13
0.899
-1.347114
1.183748
EXPSQ |
.0130223
.021334
0.61
0.542
-.0288866
.0549311
MALE |
5.762358
1.104734
5.22
0.000
3.592201
7.932515
ASVABC |
.2447687
.0714294
3.43
0.001
.1044516
.3850858
_cons | -26.18541
5.452032
-4.80
0.000
-36.89547
-15.47535
------------------------------------------------------------------------------
The three components are highly correlated and by combining them as a weighted average,
rather than using them individually, one avoids a potential problem of multicollinearity.
4
POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY
u
2

(2)
2
b2

  X 2i
 X2
2

1
1 r
2
X 2 ,X 3
u
2

n MSD ( X 2 )

1
1  rX 2 , X 3
2
Drop some of the correlated variables.
Dropping some of the correlated variables, if they have insignificant coefficients, may
alleviate multicollinearity.
5
POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY
u
2

(2)
2
b2

  X 2i
 X2
2

1
1 r
2
X 2 ,X 3
u
2

n MSD ( X 2 )

1
1  rX 2 , X 3
2
Drop some of the correlated variables.
However, this approach to multicollinearity is dangerous. It is possible that some of the
variables with insignificant coefficients really do belong in the model and that the only
reason their coefficients are insignificant is because there is a problem of multicollinearity.
6
POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY
u
2

(2)
2
b2

  X 2i
 X2
2

1
1 r
2
X 2 ,X 3
u
2

n MSD ( X 2 )

1
1  rX 2 , X 3
2
Drop some of the correlated variables.
If that is the case, their omission may cause omitted variable bias, to be discussed in
Chapter 6.
7
POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY
u
2

(3)
2
b2

  X 2i
 X2
2

1
1 r
2
X 2 ,X 3
u
2

n MSD ( X 2 )

1
1  rX 2 , X 3
2
Empirical restriction
Y  1  2 X  3P  u
A further way of dealing with the problem of multicollinearity is to use extraneous
information, if available, concerning the coefficient of one of the variables.
8
POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY
u
2

(3)
2
b2

  X 2i
 X2
2

1
1 r
2
X 2 ,X 3
u
2

n MSD ( X 2 )

1
1  rX 2 , X 3
2
Empirical restriction
Y  1  2 X  3P  u
For example, suppose that Y in the equation above is the demand for a category of
consumer expenditure, X is aggregate disposable personal income, and P is a price index
for the category.
9
POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY
u
2

(3)
2
b2

  X 2i
 X2
2

1
1 r
2
X 2 ,X 3
u
2

n MSD ( X 2 )

1
1  rX 2 , X 3
2
Empirical restriction
Y  1  2 X  3P  u
To fit a model of this type you would use time series data. If X and P are highly correlated,
which is often the case with time series variables, the problem of multicollinearity might be
eliminated in the following way.
10
POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY
u
2

(3)
2
b2

  X 2i
 X2
2

1
1 r
2
X 2 ,X 3
u
2


n MSD ( X 2 )
1
1  rX 2 , X 3
2
Empirical restriction
Y  1  2 X  3P  u
Y  1  2X  u
'
'
'
'
'
'
'
'
Yˆ  b1  b 2 X
Obtain data on income and expenditure on the category from a household survey and
regress Y' on X'. (The ' marks are to indicate that the data are household data, not
aggregate data.)
11
POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY
u
2

(3)
2
b2

  X 2i
 X2
2

1
1 r
2
X 2 ,X 3
u
2


n MSD ( X 2 )
1
1  rX 2 , X 3
2
Empirical restriction
Y  1  2 X  3P  u
Y  1  2X  u
'
'
'
'
'
'
'
'
Yˆ  b1  b 2 X
This is a simple regression because there will be relatively little variation in the price paid
by the households.
12
POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY
u
2

(3)
2
b2

  X 2i
 X2
2

1
1 r
2
X 2 ,X 3
u
2


n MSD ( X 2 )
1
1  rX 2 , X 3
2
Empirical restriction
Y  1  2 X  3P  u
Y  1  b X  3P  u
'
2
Y  1  2X  u
'
'
'
'
'
'
'
'
Yˆ  b1  b 2 X
Z  Y  b2 X   1   2 P  u
'
Now substitute b'2 for 2 in the time series model. Subtract b'2 X from both sides, and regress
Z = Y – b'2 X on price. This is a simple regression, so multicollinearity has been eliminated.
13
POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY
u
2

(3)
2
b2

  X 2i
 X2
2

1
1 r
2
X 2 ,X 3
u
2


n MSD ( X 2 )
1
1  rX 2 , X 3
2
Empirical restriction
Y  1  2 X  3P  u
Y  1  b X  3P  u
'
2
Y  1  2X  u
'
'
'
'
'
'
'
'
Yˆ  b1  b 2 X
Z  Y  b2 X   1   2 P  u
'
There are some problems with this technique. First, the 2 coefficients may be conceptually
different in time series and cross-section contexts.
14
POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY
u
2

(3)
2
b2

  X 2i
 X2
2

1
1 r
2
X 2 ,X 3
u
2


n MSD ( X 2 )
1
1  rX 2 , X 3
2
Empirical restriction
Y  1  2 X  3P  u
Y  1  b X  3P  u
'
2
Y  1  2X  u
'
'
'
'
'
'
'
'
Yˆ  b1  b 2 X
Z  Y  b2 X   1   2 P  u
'
Second, since we subtract the estimated income component b'2 X, not the true income
component  2X, from Y when constructing Z, we have introduced an element of
measurement error in the dependent variable.
15
POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY
u
2

(4)
2
b2

  X 2i
 X2
2

1
1 r
2
X 2 ,X 3
u
2

n MSD ( X 2 )

1
1  rX 2 , X 3
2
Theoretical restriction
The last, but by no means least, indirect method for alleviating multicollinearity is the use of
a theoretical restriction, which is defined as a hypothetical relationship among the
parameters of a regression model.
16
POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY
u
2

(4)
2
b2

  X 2i
 X2
2

1 r
u
2
1
2
X 2 ,X 3

n MSD ( X 2 )

1
1  rX 2 , X 3
2
Theoretical restriction
S   1   2 ASVABC
  3 SM   4 SF  u
It will be explained using an educational attainment model as an example. Suppose that we
hypothesize that highest grade completed, S, depends on ASVABC, and highest grade
completed by the respondent's mother and father, SM and SF, respectively.
17
POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY
. reg S ASVABC SM SF
Source |
SS
df
MS
-------------+-----------------------------Model | 1181.36981
3 393.789935
Residual | 2023.61353
536 3.77539837
-------------+-----------------------------Total | 3204.98333
539 5.94616574
Number of obs
F( 3,
536)
Prob > F
R-squared
Adj R-squared
Root MSE
=
=
=
=
=
=
540
104.30
0.0000
0.3686
0.3651
1.943
-----------------------------------------------------------------------------S |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------ASVABC |
.1257087
.0098533
12.76
0.000
.1063528
.1450646
SM |
.0492424
.0390901
1.26
0.208
-.027546
.1260309
SF |
.1076825
.0309522
3.48
0.001
.04688
.1684851
_cons |
5.370631
.4882155
11.00
0.000
4.41158
6.329681
------------------------------------------------------------------------------
A one-point increase in ASVABC increases S by 0.13 years.
18
POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY
. reg S ASVABC SM SF
Source |
SS
df
MS
-------------+-----------------------------Model | 1181.36981
3 393.789935
Residual | 2023.61353
536 3.77539837
-------------+-----------------------------Total | 3204.98333
539 5.94616574
Number of obs
F( 3,
536)
Prob > F
R-squared
Adj R-squared
Root MSE
=
=
=
=
=
=
540
104.30
0.0000
0.3686
0.3651
1.943
-----------------------------------------------------------------------------S |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------ASVABC |
.1257087
.0098533
12.76
0.000
.1063528
.1450646
SM |
.0492424
.0390901
1.26
0.208
-.027546
.1260309
SF |
.1076825
.0309522
3.48
0.001
.04688
.1684851
_cons |
5.370631
.4882155
11.00
0.000
4.41158
6.329681
------------------------------------------------------------------------------
S increases by 0.05 years for every extra year of schooling of the mother and 0.11 years for
every extra year of schooling of the father.
19
POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY
. reg S ASVABC SM SF
Source |
SS
df
MS
-------------+-----------------------------Model | 1181.36981
3 393.789935
Residual | 2023.61353
536 3.77539837
-------------+-----------------------------Total | 3204.98333
539 5.94616574
Number of obs
F( 3,
536)
Prob > F
R-squared
Adj R-squared
Root MSE
=
=
=
=
=
=
540
104.30
0.0000
0.3686
0.3651
1.943
-----------------------------------------------------------------------------S |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------ASVABC |
.1257087
.0098533
12.76
0.000
.1063528
.1450646
SM |
.0492424
.0390901
1.26
0.208
-.027546
.1260309
SF |
.1076825
.0309522
3.48
0.001
.04688
.1684851
_cons |
5.370631
.4882155
11.00
0.000
4.41158
6.329681
------------------------------------------------------------------------------
Mother's education is generally held to be at least, if not more, important than father's
education for educational attainment, so this outcome is unexpected.
20
POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY
. reg S ASVABC SM SF
Source |
SS
df
MS
-------------+-----------------------------Model | 1181.36981
3 393.789935
Residual | 2023.61353
536 3.77539837
-------------+-----------------------------Total | 3204.98333
539 5.94616574
Number of obs
F( 3,
536)
Prob > F
R-squared
Adj R-squared
Root MSE
=
=
=
=
=
=
540
104.30
0.0000
0.3686
0.3651
1.943
-----------------------------------------------------------------------------S |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------ASVABC |
.1257087
.0098533
12.76
0.000
.1063528
.1450646
SM |
.0492424
.0390901
1.26
0.208
-.027546
.1260309
SF |
.1076825
.0309522
3.48
0.001
.04688
.1684851
_cons |
5.370631
.4882155
11.00
0.000
4.41158
6.329681
------------------------------------------------------------------------------
It is also surprising that the coefficient of SM is not significant, even at the 5 percent level,
using a one-sided test.
21
POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY
. reg S ASVABC SM SF
Source |
SS
df
MS
-------------+-----------------------------Model | 1181.36981
3 393.789935
Residual | 2023.61353
536 3.77539837
-------------+-----------------------------Total | 3204.98333
539 5.94616574
Number of obs
F( 3,
536)
Prob > F
R-squared
Adj R-squared
Root MSE
=
=
=
=
=
=
540
104.30
0.0000
0.3686
0.3651
1.943
-----------------------------------------------------------------------------S |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------ASVABC |
.1257087
.0098533
12.76
0.000
.1063528
.1450646
SM |
.0492424
.0390901
1.26
0.208
-.027546
.1260309
SF |
.1076825
.0309522
3.48
0.001
.04688
.1684851
_cons |
5.370631
.4882155
11.00
0.000
4.41158
6.329681
. cor SM SF
-----------------------------------------------------------------------------(obs=540)
|
SM
SF
--------+-----------------SM |
1.0000
SF |
0.6241
1.0000
However assortive mating leads to correlation between SM and SF and the regression
appears to be suffering from multicollinearity.
22
POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY
u
2

(4)
2
b2

  X 2i
 X2
2

1 r
u
2
1
2
X 2 ,X 3

n MSD ( X 2 )

1
1  rX 2 , X 3
2
Theoretical restriction
S   1   2 ASVABC
  3 SM   4 SF  u
3  4
Suppose that we hypothesize that mother's and father's education are equally important.
We can then impose the restriction 3 = 4.
23
POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY
u
2

(4)
2
b2

  X 2i
 X2
2

1 r
u
2
1
2
X 2 ,X 3

n MSD ( X 2 )

1
1  rX 2 , X 3
2
Theoretical restriction
S   1   2 ASVABC
  3 SM   4 SF  u
3  4
S   1   2 ASVABC
  1   2 ASVABC
  3 ( SM  SF )  u
  3 SP  u
This allows us to rewrite the equation as shown.
24
POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY
u
2

(4)
2
b2

  X 2i
 X2
2

1 r
u
2
1
2
X 2 ,X 3

n MSD ( X 2 )

1
1  rX 2 , X 3
2
Theoretical restriction
S   1   2 ASVABC
  3 SM   4 SF  u
3  4
S   1   2 ASVABC
  1   2 ASVABC
  3 ( SM  SF )  u
  3 SP  u
Defining SP to be the sum of SM and SF, the equation may be rewritten as shown. The
problem caused by the correlation between SM and SF has been eliminated.
25
POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY
. g SP=SM+SF
. reg S ASVABC SP
Source |
SS
df
MS
-------------+-----------------------------Model | 1177.98338
2 588.991689
Residual | 2026.99996
537 3.77467403
-------------+-----------------------------Total | 3204.98333
539 5.94616574
Number of obs
F( 2,
537)
Prob > F
R-squared
Adj R-squared
Root MSE
=
=
=
=
=
=
540
156.04
0.0000
0.3675
0.3652
1.9429
-----------------------------------------------------------------------------S |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------ASVABC |
.1253106
.0098434
12.73
0.000
.1059743
.1446469
SP |
.0828368
.0164247
5.04
0.000
.0505722
.1151014
_cons |
5.29617
.4817972
10.99
0.000
4.349731
6.242608
------------------------------------------------------------------------------
The estimate of 3 is now 0.083.
26
POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY
. g SP=SM+SF
. reg S ASVABC SP
-----------------------------------------------------------------------------S |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------ASVABC |
.1253106
.0098434
12.73
0.000
.1059743
.1446469
SP |
.0828368
.0164247
5.04
0.000
.0505722
.1151014
_cons |
5.29617
.4817972
10.99
0.000
4.349731
6.242608
-----------------------------------------------------------------------------. reg S ASVABC SM SF
-----------------------------------------------------------------------------S |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------ASVABC |
.1257087
.0098533
12.76
0.000
.1063528
.1450646
SM |
.0492424
.0390901
1.26
0.208
-.027546
.1260309
SF |
.1076825
.0309522
3.48
0.001
.04688
.1684851
_cons |
5.370631
.4882155
11.00
0.000
4.41158
6.329681
------------------------------------------------------------------------------
Not surprisingly, this is a compromise between the coefficients of SM and SF in the
previous specification.
27
POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY
. g SP=SM+SF
. reg S ASVABC SP
-----------------------------------------------------------------------------S |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------ASVABC |
.1253106
.0098434
12.73
0.000
.1059743
.1446469
SP |
.0828368
.0164247
5.04
0.000
.0505722
.1151014
_cons |
5.29617
.4817972
10.99
0.000
4.349731
6.242608
-----------------------------------------------------------------------------. reg S ASVABC SM SF
-----------------------------------------------------------------------------S |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------ASVABC |
.1257087
.0098533
12.76
0.000
.1063528
.1450646
SM |
.0492424
.0390901
1.26
0.208
-.027546
.1260309
SF |
.1076825
.0309522
3.48
0.001
.04688
.1684851
_cons |
5.370631
.4882155
11.00
0.000
4.41158
6.329681
------------------------------------------------------------------------------
The standard error of SP is much smaller than those of SM and SF. The use of the
restriction has led to a large gain in efficiency and the problem of multicollinearity has been
eliminated.
28
POSSIBLE INDIRECT MEASURES FOR ALLEVIATING MULTICOLLINEARITY
. g SP=SM+SF
. reg S ASVABC SP
-----------------------------------------------------------------------------S |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------ASVABC |
.1253106
.0098434
12.73
0.000
.1059743
.1446469
SP |
.0828368
.0164247
5.04
0.000
.0505722
.1151014
_cons |
5.29617
.4817972
10.99
0.000
4.349731
6.242608
-----------------------------------------------------------------------------. reg S ASVABC SM SF
-----------------------------------------------------------------------------S |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------ASVABC |
.1257087
.0098533
12.76
0.000
.1063528
.1450646
SM |
.0492424
.0390901
1.26
0.208
-.027546
.1260309
SF |
.1076825
.0309522
3.48
0.001
.04688
.1684851
_cons |
5.370631
.4882155
11.00
0.000
4.41158
6.329681
------------------------------------------------------------------------------
The t statistic is very high. Thus it would appear that imposing the restriction has improved
the regression results. However, the restriction may not be valid. We should test it. Testing
theoretical restrictions is one of the topics in Chapter 6.
29
Copyright Christopher Dougherty 2012.
These slideshows may be downloaded by anyone, anywhere for personal use.
Subject to respect for copyright and, where appropriate, attribution, they may be
used as a resource for teaching an econometrics course. There is no need to
refer to the author.
The content of this slideshow comes from Section 3.4 of C. Dougherty,
Introduction to Econometrics, fourth edition 2011, Oxford University Press.
Additional (free) resources for both students and instructors may be
downloaded from the OUP Online Resource Centre
http://www.oup.com/uk/orc/bin/9780199567089/.
Individuals studying econometrics on their own who feel that they might benefit
from participation in a formal course should consider the London School of
Economics summer school course
EC212 Introduction to Econometrics
http://www2.lse.ac.uk/study/summerSchools/summerSchool/Home.aspx
or the University of London International Programmes distance learning course
EC2020 Elements of Econometrics
www.londoninternational.ac.uk/lse.
2012.10.28