Transcript Lecture (11,12) Parameter Estimation of PDF and Fitting a Distribution Function How can we specify a distribution from the data? Two steps procedure: 1.

Lecture (11,12)
Parameter Estimation
of PDF and
Fitting a Distribution
Function
How can we specify a distribution from the
data?
Two steps procedure:
1. Decide which family to use (Normal, Log-normal,
exponential …, etc.
This step is done by:
- Guess the family by looking at the observations.
- Use the Chi-square goodness-of-fit test to test our
guess.
2. Decide which member of the chosen family to use.
This means specify the values of the parameters.
This is done by producing estimates of the parameters
based on the observations in the sample.
Estimation
Estimation has to do with the second step.
2. Decide which member of the chosen family to use.
This means specify the values of the parameters.
General Concept of Modelling
Point Estimates
A point estimate of an unknown parameter is a number
which to the best of our knowledge represents
the parameter-value.
Each random sample can give an estimator.
So, the estimator is regarded as a random variable.
A good estimator has the following:
1. It gives a good result. Not always too big or always
too small.
1. Unbiased. The expected value of the estimator
should be equal to the true value of the parameter.
3. The variance is small.
Unbiased Estimators
x is unbiased estimator for  if,
E{x }  
s is unbiased estimator for  if,
2
2
x
E{s }  
2
2
x
Method of Moments
If the random variable x has a Normal distribution
with unknown parameters  and  x2 ,
the method of moments is simple,
  first moment of x ,
n
 =  f rj x j
j 1
  second central moment of x ,
2
x
   f rj  x j   
n
2
x
j 1
2
Method of Moments (Cont.)
We use the following notation:
ˆ  x
ˆ  s
2
x
2
the "hat" indicates that
the given value is only an estimate.
2
x and s are random variables.
ˆ and ˆ are also random variables.
2
x
Mean of the Means
1
E{x }   E{x1}  E{x2 }  E{x3}  ...  E{xn }
n
1
E{x }         ...     
n
Standard Deviation of the Means
x 1  X 1  X 1, x 2  X 2  X

2
X 1 X 2
X
1

2
X1

2
X2
X2 0
 2X X   2X   2X
by induction,
1
2
1
2
 2 X   2

 2 X  n 2



X
X
 n

n
  
n 

n
n
n
2
X

2
n
1
X2
Confidence Intervals
When we estimate a parameter from a sample the estimation
can be different from different samples.
It would be better to indicate reliability of the estimate.
This can be done by giving the confidence of the result.
Random variable x in the population has
E{x}   and  2 (x)   2
Irrespective of the sample-size n and
the distribution of x holds:
E{x }   and  (x ) 
2

2
n
, E{s }  
2
2
Confidence Intervals for the mean
How to constract a confidence-interval for  ?
1. Estimate  .
2. For 90%-confidence-interval we can derive
from the table of N(0,1) by cutting from both ends 5%:
P (1.645  u  1.645)  0.90

x 
2
3.
u
  


 n

2
μ - zασ/ n
μ
μ  zασ/ n
2
2
  
  E
P (1.645 
  x    1.645 
)  0.90
 n
 n
  
  
P ( x  1.645 
    x  1.645 
)  0.90
 n
 n
E
Confidence Interval For the Mean (cont.)
• A general expression for a 100(1-)%
confidence interval for the mean is given by:

2

2
μ
μ - zασ/ n
μ  zασ/ n
2
2
E
E  z 
n
2




E




  
x z 

n 
2 
Confidence Interval For the Mean (Cont.)
• According to the above formula we have
• 90% x 1.645 / n
•
• 95% x 1.96  / n
• 98%
x  2.33  / n
• 99% x  2.576 / n
• These formulae apply for any population as long as
the sample size is sufficiently large for the central
limit theorem to hold
Statistical Inference for The population
Variance
• For normal populations statistical inference
procedures are available for the population variance
• The sample variance S2 is an unbiased estimator of
2
• We assume we have a random sample of n
observations from a normal population with unknown
variance 2 .
The Chi Square Distribution
• If the population is Normal with variance 2 then
the statistic
 
2
(n  1) S
2
2
• Has a Chi Square distribution with (n-1) degrees of
freedom
Confidence Region For The Variance
• Using this result a confidence interval for 2 is
given by the interval:


2 
 (n  1) S 2
(n  1) S


,
2
2
  
 

 1 ;( n 1) 
 2 ;( n 1)
 2

Confidence Region For The Variance
(n-1) d.f.
 /2
 /2
1 ;( n1)
2
2
 ;( n1)
2
2
Estimation of the Confidence Intervals of
the variance
How to constract a confidence-interval for  2 ?
1. Estimate  2 .
2. For 90%-confidence-interval we use the  2 -distribution:  2 (n  1)
the degree of freedom: df  n -1
2
2
P(  0.95
  2   0.05
)  0.90
P(a   2  b)  0.90
3.we determine a and b from the table of  2
in such a way that we cut of both tails 5%. This gives (for n  30),
a  17.7 and b  42.6
P(17.7   2  42.6)  0.90
P(17.7 
(n  1) s 2

2
 42.6)  0.90
(n  1) s 2
(n  1) s 2
2
P(
 
)  0.90
42.6
17.7
Fitting a Distribution Function
Goodness-of-Fit Test
A goodness-of-fit test is an inferential
procedure used to determine whether a
frequency distribution follows a claimed
distribution.
Hypothesis Testing
• Hypothesis:
– A statement which can be proven false
• Null hypothesis Ho:
– “There is no difference”
• Alternative hypothesis (H1):
– “There is a difference…”
• In statistical testing, we try to “reject the null hypothesis”
– If the null hypothesis is false, it is likely that our
alternative hypothesis is true
– “False” – there is only a small probability that the results
we observed could have occurred by chance
Application of Testing hypothesis on
Goodness of Fit
Testing Hypothesis:
Ho: the null hypothesis is defined as the distribution
function is a good fit to the empirical distribution.
H1: the alternative hypothesis is defined as the distribution
function is not a good fit to the empirical distribution.
Testing of hypothesis is a procedure for deciding whether
to accept or reject the hypothesis.
The Chi-squared test can be used to test if the
fit is satisfactory.
Testing Goodness of Fit of a Distribution
Function to an Empirical Distribution
Unknown real situation
Decision
Accept Ho
Reject Ho
Ho is True
Correct
Decision
Type I Error
Probability 
Ho is False
Type II Error
Probability 
Correct
Decision
Common Values for Significant Levels
Significant-level of the test (Risk level): 
P(type I error)  
Common values for  are 1%, 2.5%, 5% or 10%
P(type II error)  
Common values for  are 1%, 2.5%
The Chi-Square Distribution
1.
It is not symmetric.
2. The shape of the chi-square
distribution depends upon the
degrees of freedom.
3. As the number of degrees of
freedom increases, the chi-square
distribution becomes more
symmetric as is illustrated in the
figure.
4. The values are non-negative. That
is, the values of are greater than
or equal to 0.
Chi2 Degrees of Freedom
• All statistical tests require the compotation of
degrees of freedom
• Chi2 df = (No. classes -1)
Critical Values of Chi2
Significance Level
df
0.10
0.05
0.25
0.01
0.005
1
2.7055
3.8415
5.0239
6.6349
7.8794
2
4.6062
5.9915
7.3778
9.2104
10.5965
3
6.2514
7.8147
9.3484
11.3449
12.8381
Chi-Square Table
Procedure for Chi Square Test
Step 1: A claim is made regarding a
distribution. The claim is used to
determine the null and alternative
hypothesis.
Ho: the random variable follows the
claimed distribution
H1: the random variable does not follow
the claimed distribution
Procedure for Chi Square Test (cont.)
Step 2: Calculate the expected frequencies for
each of the k classes. The expected frequencies
are Ei , i = 1, 2, …, k assuming the null hypothesis
is true.
Procedure for Chi Square Test (cont.)
Step 3: Verify the requirements fort he
goodness-of-fit test are satisfied.
(1) all expected frequencies are greater
than or equal to 1 (all Ei > 1)
(2) no more than 20% of the expected
frequencies are less than 5.
Procedure for Chi Square Test (cont.)
Procedure for Chi Square Test (cont.)
Procedure for Chi Square Test (cont.)
Example 1 (Discrete Variable)
Example 1 (cont.)
• Observed Frequency
– The obtained frequency for each category.
fo
Example 1 (cont.)
• State the research hypothesis.
– Is the rat’s behavior random?
• State the statistical hypotheses.
Example 1 (cont.)
.25
.25
.25
.25
If picked by
chance.
H 0 : PA  .25, PB  .25, PC  .25, PD  .25
H A : H 0 is false.
Example 1 (cont.)
• Expected Frequency
– The hypothesized frequency for each
distribution, given the null hypothesis is true.
– Expected proportion multiplied by number of
observations.
f e  .25(32)  8
Example 1 (cont.)
• Set the decision rule.
Example 1 (cont.)
• Set the decision rule.
– Degrees of Freedom
• Number of Categories -1
• (C) –1
df  4  1  3
Example 1 (cont.)
• Set the decision rule.
  .05
df  3
2
 crit
 7.81
Example 1 (cont.)
• Calculate the test statistic.
2

(
f

f
)
2
e
    o
fe




Example 1 (cont.)
• Calculate the test statistic.
Example 1 (cont.)
Do Not Reject H0
Reject H0
2
7.81
• Decide if your result is significant.
– Reject H0, 9.25>7.81
• Interpret your results.
– The rat’s behavior was not random.
Example 2 (Continuous Variable)
Observed frequency of 10 size classes of shale thicknesses,
Class
number
Class
interval
1
Observed
frequency
Expected
frequency
0.00-1.00 0.5
14
20
2
1.00-2.00 1.5
18
20
3
2.00-3.00 2.5
26
20
4
3.00-4.00 3.5
18
20
5
4.00-5.00 4.5
20
20
6
5.00-6.00 5.5
18
20
7
6.00-7.00 6.5
24
20
8
7.00-8.00 7.5
22
20
9
8.00-9.00 8.5
16
20
10
9.00-10.0 9.5
24
20
1
1
1


b  a 10  0 10
1
E f j  f ( x).N .x  .200.1  20
10
f ( x) 
Class
Mark
Example (Cont.)
k
( E f j  O f j )2
j 1
Efj
 
2
2
2
2
2
(20

14)
(20

18)
(20

26)
(20

18)
2 



20
20
20
20
2
2
2
2
(20  20) (20  18) (20  24) (20  22)





20
20
20
20
2
2
(20  16) (20  24)


 6.8
20
20
Example (Cont.)
For significant level (risk level)   0.05
degrees of freedom=(n -1)  10 -1  9
 table gives, 
2
 <
2
2
critical
2
critical
=16.92
we accept the hypotesis
that the distribution is uniform
because we do have sufficient evidence
to reject it at the chosen level of significant.
Chi2 Graph
16.92
Exercise
In exercise 1 test the hypothesises that the distribution is Normal
For =0.05.
 ( x  ) 
f ( x; , ) 
exp 

2
2
2 
2

E f j  f ( x).N .x
1
where,
N = Number of observations.
x  class interval.
2
Other Statistical Tests
• The Chi2 and Independent T-test are very useful.
• A Variety of other tests are available for other
research designs.
• Parametric Examples Follow
• T-tests are used to compare 2 groups
• F tests (Analyses of Variance tests )are used to
compare more than 2 groups.
Common Statistical Tests
Question
Does a single observation belong to a population of
values?
Are two (or more populations) of number
different?
Test
Z-test (Standard
Normal Dist)
T-test
F-test (ANOVA)
Is there a relationship between x and y
Regression
Is there a trend in the data (special case of above
Regression
SPSS and Computer Applications
• Most actual analysis is done by computer
• Simple test are easily done in Excel
• Sophisticated programs (such as SPSS) are used
for more complicated designs