Transcript Document 7753096

Review of Chapter 2
Some Basic Concepts:
Sample center
Sample deviation
Sample standard deviation
Sample spread---sample variance
Standardization of a variable
Measure of the direction of the linear relationship between Y and X
Measure of the strength of the linear relationship between Y and X
5/26/2016
ST3131, Review of Chapter 2
1
Simple Linear Regression Model:
Y   0  1 X  
ˆ 0  y  x ˆ1 ,
LS Estimators
ˆ1  Cov( X , Y ) / Var ( X )
where
n
n
Var(X) 
5/26/2016
 (x
i 1
i
 x)2
n 1
Cov( X , Y ) 
ST3131, Review of Chapter 2
 (x
i 1
i
 x ) yi
n -1
2
yˆ i  ˆ 0  ˆ1 xi
Fitted values:
Residuals:
ˆi  yi  yˆ i
Noise variance estimator:
ˆ 2  ˆi2 /( n  2)  SSE /( n  2)  MSE
Decompositions
( yi  y )  ( yˆ i  y )  ( yi  yˆ i )
Observation
Squares
Degrees of Freedom
5/26/2016

( yi  y) 2   ( yˆ i  y) 2   ( yi  yˆ i ) 2
SST
=
df(SST)
=
ST3131, Review of Chapter 2
SSR
+
SSE
df(SSR) + df(SSE)
3
Interpretation of the Coefficients
 0  intercept  E (Y ) at X  0,
1  expected change in Y correspond ing to a unit change of X.
Another Interpretation:
1  effect of X on Y  contributi on of X to Y
Assumption:
5/26/2016
 ~ N (0,  2 ),  2 noise variance
ST3131, Review of Chapter 2
4
Properties:
1). Linearity
ˆ1   ai yi  a T y
ˆ 0   bi yi  b T y
2). Unbiased
E ( ˆ j )   j , j  0,1.
3). BLUE estimators (Best Linear Unbiased Estimators)
ˆ j has the smallest v ariance among all
linear unbiased estimators of  j .
4) Normality
1
ˆ 0 ~ N (  0, ( 
n
x2
n
 (x
i 1
5/26/2016
i
 x)2
n
) ), ˆ1 ~ N (  1 ,  2 /  ( xi  x ) 2 )
2
ST3131, Review of Chapter 2
i 1
5
n
Var ( ˆ1 )   /  ( xi  x ) 2
2
Variances of the Estimates
i 1
n
1
2
Var( ˆ 0 )  (  x /  ( xi  x ) 2 ) 2 ,
n
i 1
x
Cov( ˆ1 , ˆ 0 )   n
2
2
(
x

x
)
 i
Noise variance estimator and Standard Errors
i 1
2
ˆ
(
y

y
)
SSE

i
i
ˆ 2 

, W  SSE /  2 ~  n2 2
n2
n2
ˆ 2 , ˆ 0 , and ˆ are independen t
Thus
s.e.( ˆ 0 ) 
1

n
x2
n
 (x
i 1
5/26/2016
i
ˆ ,
 x)2
s.e.( ˆ j )  ˆ /
ST3131, Review of Chapter 2
n
2
(
x

x
)
 i
i 1
6
Inferences for Individual Coefficients/T-test
Hypothesis Testing
H0 :  j   j
0
vs
H1 :  j   j
0
H1 :  j   j
0
H1 :  j   j
0
Test Statistics :
T  ( ˆ j   0j ) / s.e.( ˆ j ) ~ t ( n2) , j  0,1
To check if there is some effect of X on Y, we set the target  10  0,
and use the coefficien t table provided by Minitab.
Confidence Interval
100(1 -  )% CI for  j , j  0,1,
ˆ j  t (n -2,/2)s.e.( ˆ j )
5/26/2016
ST3131, Review of Chapter 2
7
Special Case: ANOVA Table (Analysis of Variance)
RM H 0 : Y   0  
FM H1 : Y   0   1 X  
F
( SSE R  SSE F ) / 1 MSR F

~ F (1, n  2)
SSE F /( n  2)
MSE F
RF2 / 1
F
~ F(1,n2) .
(1  RF2 ) /( n  2)
F  T 2 , T  ˆ / s.e.( ˆ )
1
Source
1
Sum of df
Squares
1
1
Mean Square
F-test
F=MSR/MSE
Regression SSR
1
MSR=SSR/1
Residuals
SSE
n-2
MSE=SSE/(n-2)
Total
SST
n-1
5/26/2016
ST3131, Review of Chapter 2
P-value
8
Coefficient Table
variable
coef
ˆ
constant
X
0
ˆ1
s.e.
t - test
s.e.( ˆ 0 )
Tˆ0
s.e.( ˆ1 )
Tˆ1
p - value
P(| T || Tˆ0 |)
P(| T || Tˆ1 |)
T - test is for H 0 :  j  0 vs H1 :  j  0, j  0,1.
A Special Test
RM H 0 : Y  
FM H 1 : Y   0   1 X  
( SSR F  ny 2 ) / 2
F
~ F (2, n  2)
SSE F /( n  2)
5/26/2016
ST3131, Review of Chapter 2
9
yi  0  1 xi   i ,
Prediction: for SLR Model
 i ~ N (0, 2 )
For any given NEW x0 , both the Prediction of the response y x0
and the Estimation of the expected response  x 0 are yˆ x0  ˆ 0  ˆ1 x0
Standard Errors
s.e.( yˆ x0  y x0 )  1 
(x  x)
1
 n 0
ˆ
n
 ( xi  x ) 2
2
s.e.( yˆ x0   x0 ) 
i 1
i 1
a 100( 1-)% PI for y x0 is
yˆ x0  t ( n2, / 2) s.e.( yˆ x0 -y x0 )
5/26/2016
( x0  x ) 2
1
 n
ˆ
n
 ( xi  x ) 2
a 100( 1-α) % CI for  x0 is
ˆ x  t ( n2, / 2) s.e.( ˆ x )
0
ST3131, Review of Chapter 2
0
10
Methods for Assessing the Linear Relationship/Quality of Linear Fit
Method 1: Use the Correlation Cor(X,Y)
Method 2 : Use Cor(Y, Ŷ) or Examine the scatter plot of Y against Ŷ.
Cor(Y, Ŷ)  sgn( ˆ1 )Cor( X , Y ) | Cor( X , Y ) |
(1). Cor(Y, Ŷ) can NOT be negative.
(2). 0  Cor(Y, Ŷ)  1 .
Method 3 : Use Coefficien t of Determinat ion R 2 
(1). 0  R 2  1;
SSR
SSE
 1
.
SST
SST
(2). R 2  Cor 2 (Y, Ŷ);
(3). For SLR model, R 2  Cor 2 ( X , Y ).
Method 4: Use Hypothesis Test
5/26/2016
H 0 : 1  0 vs H1 : 1  0
ST3131, Review of Chapter 2
11
Special SLR Models
General SLR Model :
No Intercept Model :
No Slope Model :
Trivial Model :
5/26/2016
yi  0  1 xi   i ,
yi  1 xi   i ,
yi   0   i ,
yi   i ,
 i ~ N (0, 2 )
 i ~ N (0,  2 )
 i ~ N (0, 2 )
 i ~ N (0,  2 )
ST3131, Review of Chapter 2
12
Test H 0 : Y  1 X   , vs H1 : Y   0  1 X  
Test H 0 : Y   0   , vs H1 : Y   0  1 X  
Test H 0 : Y   , vs H1 : Y   0  
Test H 0 : Y   , vs H1 : Y   0  1 X  
5/26/2016
ST3131, Review of Chapter 2
13