Document 7753096
Download
Report
Transcript Document 7753096
Review of Chapter 2
Some Basic Concepts:
Sample center
Sample deviation
Sample standard deviation
Sample spread---sample variance
Standardization of a variable
Measure of the direction of the linear relationship between Y and X
Measure of the strength of the linear relationship between Y and X
5/26/2016
ST3131, Review of Chapter 2
1
Simple Linear Regression Model:
Y 0 1 X
ˆ 0 y x ˆ1 ,
LS Estimators
ˆ1 Cov( X , Y ) / Var ( X )
where
n
n
Var(X)
5/26/2016
(x
i 1
i
x)2
n 1
Cov( X , Y )
ST3131, Review of Chapter 2
(x
i 1
i
x ) yi
n -1
2
yˆ i ˆ 0 ˆ1 xi
Fitted values:
Residuals:
ˆi yi yˆ i
Noise variance estimator:
ˆ 2 ˆi2 /( n 2) SSE /( n 2) MSE
Decompositions
( yi y ) ( yˆ i y ) ( yi yˆ i )
Observation
Squares
Degrees of Freedom
5/26/2016
( yi y) 2 ( yˆ i y) 2 ( yi yˆ i ) 2
SST
=
df(SST)
=
ST3131, Review of Chapter 2
SSR
+
SSE
df(SSR) + df(SSE)
3
Interpretation of the Coefficients
0 intercept E (Y ) at X 0,
1 expected change in Y correspond ing to a unit change of X.
Another Interpretation:
1 effect of X on Y contributi on of X to Y
Assumption:
5/26/2016
~ N (0, 2 ), 2 noise variance
ST3131, Review of Chapter 2
4
Properties:
1). Linearity
ˆ1 ai yi a T y
ˆ 0 bi yi b T y
2). Unbiased
E ( ˆ j ) j , j 0,1.
3). BLUE estimators (Best Linear Unbiased Estimators)
ˆ j has the smallest v ariance among all
linear unbiased estimators of j .
4) Normality
1
ˆ 0 ~ N ( 0, (
n
x2
n
(x
i 1
5/26/2016
i
x)2
n
) ), ˆ1 ~ N ( 1 , 2 / ( xi x ) 2 )
2
ST3131, Review of Chapter 2
i 1
5
n
Var ( ˆ1 ) / ( xi x ) 2
2
Variances of the Estimates
i 1
n
1
2
Var( ˆ 0 ) ( x / ( xi x ) 2 ) 2 ,
n
i 1
x
Cov( ˆ1 , ˆ 0 ) n
2
2
(
x
x
)
i
Noise variance estimator and Standard Errors
i 1
2
ˆ
(
y
y
)
SSE
i
i
ˆ 2
, W SSE / 2 ~ n2 2
n2
n2
ˆ 2 , ˆ 0 , and ˆ are independen t
Thus
s.e.( ˆ 0 )
1
n
x2
n
(x
i 1
5/26/2016
i
ˆ ,
x)2
s.e.( ˆ j ) ˆ /
ST3131, Review of Chapter 2
n
2
(
x
x
)
i
i 1
6
Inferences for Individual Coefficients/T-test
Hypothesis Testing
H0 : j j
0
vs
H1 : j j
0
H1 : j j
0
H1 : j j
0
Test Statistics :
T ( ˆ j 0j ) / s.e.( ˆ j ) ~ t ( n2) , j 0,1
To check if there is some effect of X on Y, we set the target 10 0,
and use the coefficien t table provided by Minitab.
Confidence Interval
100(1 - )% CI for j , j 0,1,
ˆ j t (n -2,/2)s.e.( ˆ j )
5/26/2016
ST3131, Review of Chapter 2
7
Special Case: ANOVA Table (Analysis of Variance)
RM H 0 : Y 0
FM H1 : Y 0 1 X
F
( SSE R SSE F ) / 1 MSR F
~ F (1, n 2)
SSE F /( n 2)
MSE F
RF2 / 1
F
~ F(1,n2) .
(1 RF2 ) /( n 2)
F T 2 , T ˆ / s.e.( ˆ )
1
Source
1
Sum of df
Squares
1
1
Mean Square
F-test
F=MSR/MSE
Regression SSR
1
MSR=SSR/1
Residuals
SSE
n-2
MSE=SSE/(n-2)
Total
SST
n-1
5/26/2016
ST3131, Review of Chapter 2
P-value
8
Coefficient Table
variable
coef
ˆ
constant
X
0
ˆ1
s.e.
t - test
s.e.( ˆ 0 )
Tˆ0
s.e.( ˆ1 )
Tˆ1
p - value
P(| T || Tˆ0 |)
P(| T || Tˆ1 |)
T - test is for H 0 : j 0 vs H1 : j 0, j 0,1.
A Special Test
RM H 0 : Y
FM H 1 : Y 0 1 X
( SSR F ny 2 ) / 2
F
~ F (2, n 2)
SSE F /( n 2)
5/26/2016
ST3131, Review of Chapter 2
9
yi 0 1 xi i ,
Prediction: for SLR Model
i ~ N (0, 2 )
For any given NEW x0 , both the Prediction of the response y x0
and the Estimation of the expected response x 0 are yˆ x0 ˆ 0 ˆ1 x0
Standard Errors
s.e.( yˆ x0 y x0 ) 1
(x x)
1
n 0
ˆ
n
( xi x ) 2
2
s.e.( yˆ x0 x0 )
i 1
i 1
a 100( 1-)% PI for y x0 is
yˆ x0 t ( n2, / 2) s.e.( yˆ x0 -y x0 )
5/26/2016
( x0 x ) 2
1
n
ˆ
n
( xi x ) 2
a 100( 1-α) % CI for x0 is
ˆ x t ( n2, / 2) s.e.( ˆ x )
0
ST3131, Review of Chapter 2
0
10
Methods for Assessing the Linear Relationship/Quality of Linear Fit
Method 1: Use the Correlation Cor(X,Y)
Method 2 : Use Cor(Y, Ŷ) or Examine the scatter plot of Y against Ŷ.
Cor(Y, Ŷ) sgn( ˆ1 )Cor( X , Y ) | Cor( X , Y ) |
(1). Cor(Y, Ŷ) can NOT be negative.
(2). 0 Cor(Y, Ŷ) 1 .
Method 3 : Use Coefficien t of Determinat ion R 2
(1). 0 R 2 1;
SSR
SSE
1
.
SST
SST
(2). R 2 Cor 2 (Y, Ŷ);
(3). For SLR model, R 2 Cor 2 ( X , Y ).
Method 4: Use Hypothesis Test
5/26/2016
H 0 : 1 0 vs H1 : 1 0
ST3131, Review of Chapter 2
11
Special SLR Models
General SLR Model :
No Intercept Model :
No Slope Model :
Trivial Model :
5/26/2016
yi 0 1 xi i ,
yi 1 xi i ,
yi 0 i ,
yi i ,
i ~ N (0, 2 )
i ~ N (0, 2 )
i ~ N (0, 2 )
i ~ N (0, 2 )
ST3131, Review of Chapter 2
12
Test H 0 : Y 1 X , vs H1 : Y 0 1 X
Test H 0 : Y 0 , vs H1 : Y 0 1 X
Test H 0 : Y , vs H1 : Y 0
Test H 0 : Y , vs H1 : Y 0 1 X
5/26/2016
ST3131, Review of Chapter 2
13