Transcript Ch.6 Simple Linear Regression: Continued

6.1
Ch.6 Simple Linear Regression: Continued
To complete the analysis of the simple linear regression model, in this
chapter we will consider
• how to measure the variation in yt, that is explained by the model
• how to report the results of a regression analysis
• how changes in the units of measurement affect the estimates
• some alternative functional forms that may be used to represent
possible relationships between yt and xt.
6.2
The Coefficient of Determination (R2)
Two major reasons for analyzing the model
y =  1 + 2 x + e
are
• To explain how the dependent varaible (yt) changes as the independent
variable (xt) changes
• To predict yo given xo.
We want the independent variable (xt) to explain as much of the variation
in the dependent variable (yt) as possible.
We introduced the independent variable (xt) in hope that its variation will
explain the variation in y
A measure of goodness of fit will measure how much of the variation in
the dependent variable (yt) has been explained by variation in the
independent variable (xt).
6.3
Separate yt into its explainable and unexplainable components:
yt  E( yt )  et
where
E( yt )  1   2 xt
is explainable.
The error term et is unexplainable. Using our estimates for 1 and
2, we get estimates of E(yt) and our residuals give us estimates of
the error terms.
yˆt  b1  b2 xt
eˆt  yt  yˆt
yt  yˆt  eˆt
Residual is defined as the difference between
the actual and the predicted values of y.
6.4
The total variation in yt is measured as the sum of
the squared deviations from the mean:
2
(
y

y
)
 t
Also known as SST (Total Sum of Squares)
A single deviation of yt from its mean can be split into two parts:
yt  y  yˆt  eˆt  y
The sum of squared deviations from the mean is:
2
ˆ
ˆ
 ( yt  y )   ( yt  et  y )
2
  (( yˆ t  y )  eˆt )
This term is
zero
2
  ( yˆ t  y )   eˆ  2 ( yˆ t  y )eˆt
2
2
t
  ( yˆ t  y ) 2   eˆt2
6.5
Graphically, a single y deviation from mean can be split into the two parts:
Unexplained
Total
Variation
yt
yˆt
eˆt  yt  yˆt

yt  y
Explained
yˆt  y
y
xt
ˆy  b1  b2 x
6.6
Analysis of Variance (ANOVA):
2
2
2
ˆ
ˆ
(
y

y
)

(
y

y
)

e
 t
 t
t
SST
=
SSR
+ SSE
Where:
SST: Total Sum of Squares with T-1 degrees of freedom. It
measures the total variation in the actual yt values about its
mean.
SSR: Regression Sum of Squares with 1 degree of freedom. It
measures the variation in the predicted values of yt about
their mean. It is the part of the total variation that is
explained by the model.
SSE: Error Sum of Squares with T-2 degrees of freedom. It
measures the variation in the actual yt values about the
predicted yt values. It is the part of the total variation that is
left unexplained.
6.7
R2 = SSR/SST = 1 – SSE/SST
SST
Multiple R
R Square
Adjusted R Square
Standard Error
Observations
SSR
SSE
0.563132517
0.317118231
0.299147658
37.80536423
40
ANOVA
df
Regression
Residual
Total
Intercept
x
1
38
39
SS
MS
F
Significance F
25221.22299 25221.22299 17.64652878
0.00015495
54311.33145 1429.245564
79532.55444
Coefficients Standard Error
t Stat
P-value
40.76755647
22.13865442 1.841464964 0.073369453
0.128288601
0.030539254 4.200777164 0.00015495
Lower 95%
Upper 95%
-4.049807902 85.58492083
0.066465111 0.190112091
Coefficient of Determination: R2
2
ˆ
e
SSR
SSE
t
R2 
 1
 1
2
SST
SST
(
y

y
)
 t
• R2 is the proportion of the total variation (SST)
that is explained by the model. We can also think
of it as one minus the proportion of the total
variation that is unexplained (left in the
residuals).
• 0  R2  1
• The closer R2 is to 1.0, the better the fit of the
model and the greater is the predictive ability of
the model over the sample.
• If R2 =1  the model has explained everything.
All the data points lie on the regression lie (very
unlikely). There are no residuals.
• If R2 = 0  the model has explained nothing.
6.8
y
6.9
Graph A
R2 appears to be 1.0. All data
Points lie on a line.
x
Graph B
y
R2 appears to be 0. The best line thru these
points appears to have a slope of zero.
x
y
6.10
Graph C
R2 appears to be close to 1.0.
x
Graph D
y
R2 appears to be greater than 0 but
less than R2 in graph C.
x
• In the food expenditure example, = 0.317  “31.7%
of the total variation in food expenditures has been
explained by variation in household income.”
R2
• More Examples:
6.11
6.12
Correlation Analysis
• Correlation coefficient
between x and y is:
• The Sample Correlation
between x and y is:
• It is always true that
-1  r  1
• It measures the strength
of a linear relationship
between x and y.
Cov( x, y)

Var( x) Var ( y)
Coˆv( x, y )
r
Vaˆr ( x) Vaˆr ( y )
1
( xt  x )( yt  y )

T 1

1
1
2
2
(
x

x
)
(
y

y
)


t
t
T 1
T 1
( xt  x )( yt  y )


2
2
(
x

x
)
(
y

y
)
 t
 t
6.13
Correlation and R2
• It can be shown that the square
of the sample correlation
coefficient for x and y is equal to
R2.
• R2 can also be computed as the
square of the sample correlation
coefficient for the y values and
the y
ˆ values.
• It can also be shown that
b2  r
sy
sx
6.14
Reporting Regression Results
yˆt  40.768 0.1283xt
(s.e.)
(22.139)
(0.0305)
R2 = 0.317
• The numbers in parentheses are the standard errors of the coefficients
estimates. These can be used to construct the necessary t-statistics to
ascertain the significance of the estimates.
• Sometimes, authors will report the t-statistic instead of the standard
error. This would be the t-statistic for the Ho:  = 0
yˆt  40.768 0.1283xt
(t-stat) (1.841)
(4.201)
R2 = 0.317
6.15
Units of Measurement
b1  y  b2 x
b2 
 ( x t  x )( yt  y )
 ( xt  x )
2
b1 is measured in “y units”
b2 is measured in “y units over x units”
Example 3.15 from Chapter 3 Exercises
y = number of sodas sold
x = temperature in degrees (oF)
yˆt  240 6x
If xo = 0o then the model predicts:
So b1 is measured in y units (# of sodas).
yˆ o  240
b2 = 6 where 6 is in (# of sodas / degrees).
If x increases by 10 degrees  ^
y increases
by 60 sodas
yˆ o  6x
6.16
Let newx = x/100. We have no change to b1 because b1 is in
Y units. b2 increases by 100, because it is in y units/x units.
If newx increases by 1 unit (weekly income increases by $100),
the model predicts food spending to rise by $12.83.
Note this isn’t a new result. It still predicts that if income increases
by $1, food spending will increase by $0.1283
SUMMARY OUTPUT
Regression Statistics
Multiple R
0.563132517
R Square
0.317118231
Adjusted R Square
0.299147658
Standard Error
37.80536423
Observations
40
ANOVA
df
Regression
Residual
Total
Intercept
newx
1
38
39
SS
MS
F
Significance F
25221.22299 25221.22299 17.64652878
0.00015495
54311.33145 1429.245564
79532.55444
Coefficients Standard Error
t Stat
P-value
40.76755647
22.13865442 1.841464964 0.073369453
12.82886011
3.053925406 4.200777164 0.00015495
Lower 95%
Upper 95%
-4.049807902 85.58492083
6.646511122 19.01120909
6.17
Functional Forms
A linear model is one that is linear in the parameters with an additive error
term.
y =  1 + 2 x + e
The coefficient 2 measures the effect of a one unit change in x on y. As the
model is written above, this effect is assumed to be constant:
However, we want to have the ability to model relationships among economic
variables where the effect of x on y is not constant.
Example: our food expenditure example assumes that the increase in food
spending from an additional dollar of income was the same whether the
family had a high or low income. We can capture these effects using logs,
powers and reciprocals yet still maintain a model that is linear in the
parameters with an additive error term.
6.18
The Natural Logarithm
• We will use the derivative property
often:
• Let y be the log of X:
y = ln(x)  dy/dx = 1/x or dy = dx/x
• This means that the absolute change in
the log of X is equivalent to the relative
change in the level of X.
Let x=50
 ln(x) = 3.912
Let x=52
 ln(x) = 3.951
 dln(x) = 3.951 – 3.912
= 0.039
The absolute change in ln(x) is 0.039,
which can be interpreted as a relative
change in X (X increases from 50 to 52,
which, in relative terms, is 3.9%)
6.19
Linear Log
yt  1   2 ln(x)  et
Ex: food expenditures
Diagram
What does 2 measure?
6.20
Example: Y: food $, X: Weekly Income
yt  1   2 ln(x)  et
SUMMARY OUTPUT
Regression Statistics
Multiple R
0.571864156
R Square
0.327028613
Adjusted R Square
0.30931884
Standard Error
37.5300348
Observations
40
xbar=698
ybar=130
ANOVA
df
Regression
Residual
Total
Intercept
lnx
1
38
39
SS
26009.42098
53523.13346
79532.55444
MS
F
26009.42098 18.46599654
1408.503512
Coefficients Standard Error
t Stat
P-value
-415.5556981
127.1672145 -3.267789578 0.002303753
83.91235619
19.52718051 4.297207994 0.000115804
Log-Linear
ln(yt )  1  2 xt  et
Ex: wages
Diagram
What does 2 measure?
6.21
Double-Log
ln(yt )  1   2 ln(xt )  et
Ex: demand model
Diagram
What does 2 measure?
6.22
SUMMARY OUTPUT
Linear Model
Regression Statistics
Multiple R
0.959804589
R Square
0.921224848
Adjusted R Square
0.913347333
Standard Error
51.41714671
Observations
12
ANOVA
df
Regression
Residual
Total
Intercept
p
1
10
11
SS
309166.4369
26437.22976
335603.6667
MS
F
309166.4369 116.9435829
2643.722976
Coefficients Standard Error
t Stat
2813.319917
175.3238285 16.04642073
-1577.581002
145.8825916 -10.81404563
P-value
1.82583E-08
7.72349E-07
SUMMARY OUTPUT
Double Log Model
Regression Statistics
Multiple R
0.938565661
R Square
0.880905499
Adjusted R Square
0.868996049
Standard Error
0.068144835
Observations
12
ANOVA
df
Regression
Residual
Total
Intercept
lnp
1
10
11
SS
0.343481619
0.046437185
0.389918805
MS
F
0.343481619 73.96693327
0.004643719
Coefficients Standard Error
t Stat
7.152753536
0.044168328 161.9430444
-1.927315081
0.224095901 -8.600403088
P-value
1.98024E-18
6.21314E-06
6.23