Transcript Use of moment generating functions
Use of moment generating functions
Definition
Let
X
denote a random variable with probability density function
f
(
x
) if continuous (probability mass function
p
(
x
) if discrete) Then
m X
(
t
) = the moment generating function of
X
x tx
tx
The distribution of a random variable
X
is described by either 1.
2.
3.
The density function
f
(
x
) if
X
continuous (probability mass function
p
(
x
) if
X
discrete), or The cumulative distribution function
F
(
x
), or The moment generating function
m X
(
t
)
Properties
1.
m X
(0) = 1 2.
m X
3.
k
m X
k th
k
derivative of
m X
at
t
0.
k
k
1
t
2!
2
t
2
X
continuous
X
discrete 3!
3
t
3
k k
!
t k
.
4. Let
X
be a random variable with moment generating function
m X
(
t
). Let
Y
=
bX
+
a
Then
m Y
(
t
) =
m bX
+
a
(
t
) =
E
(
e
[
bX
+
a
]
t
) =
e at m X
(
bt
) 5. Let
X
and
Y
be two independent random variables with moment generating function
m X
(
t
) and
m Y
(
t
) . Then
m X+Y
(
t
) =
m X
(
t
)
m Y
(
t
)
6. Let
X
and
Y
be two random variables with moment generating function
m X
(
t
) and
m Y
(
t
) and two distribution functions
F X
(
x
) and
F Y
(
y
) respectively. Let
m X
(
t
) =
m Y
(
t
) then
F X
(
x
) =
F Y
(
x
). This ensures that the distribution of a random variable can be identified by its moment generating function
M. G. F.’s - Continuous distributions
Name Continuous Uniform Exponential Gamma 2 d.f. Normal Moment generating function M X (t) e bt -e at [b-a]t t for t < t for t < 1 1-2t /2 for t < 1/2
e t
+(1/2)
t
2 2
M. G. F.’s - Discrete distributions
Name Discrete Uniform Bernoulli Binomial Geometric Negative Binomial Poisson Moment generating function M X (t) e t N e tN -1 e t -1 q + pe t (q + pe t ) N pe t 1-qe t pe t 1-qe t e (e t -1) k
Moment generating function of the gamma distribution
m X
tx
where
x
1
e
x
0
x
0
x
0
m X
or using
tx
0
e tx
0
x
x
1
e
0
b a
0
x a x
1
e a
1
e bx
dx bx
dx
b a
1
e
x dx dx
then
m X
0
x
1
e
dx
t
t
t
Moment generating function of the Standard Normal distribution
m X
tx
where 1 2
e
x
2 2 thus
m X
e tx
1 2
e
x
2 2
dx
1 2
e
x
2 2
tx dx
We will use 0 1 2
b e
2
b
2 2
dx
1
m X
1 2 1 2 1 2
e
x
2 2
tx dx e
x
2 2
tx
2
dx e
x
2 2 2 2
t
2
t
2
e
2
t
2
e
2 1 2
e
2 2
dx
Note:
m X e x t
2
e
2
x x
2 2!
x
3 3!
2
t x
4 4!
t
3
t
2 2 2!
3!
t
2 2
t
4 2 2 2!
t
6 3 2 3!
t
2
m
2
m m
!
Also
m X
1
t
2!
2
t
2 3!
3
t
3
Note:
m X
Also
e x t
2
e
2
x x
2 2!
x
3 3!
2
t x
4 4!
t
3
t
2 2 2!
3!
m X
k
t
2 2
k th
t
4 2 2 2!
1
t
t
6 3 2 3!
2 2!
moment
t
2
t
2
m
2
m
3
t
3 3!
k
m
!
Equating coefficients of
t k
, we get
k
for
k
1 2
m m
!
2 2
m m
!
hence 1 0, 2 1, 3 0, 4 3
Using of moment generating functions to find the distribution of functions of Random Variables
Example
Suppose that
X
has a normal distribution with mean and standard deviation
.
Find the distribution of
Y = aX + b
Solution:
m X m
e
t
2
bt e m X
bt e e
2 2 2
e
a
2 2 2
a t
2 = the moment generating function of the normal distribution with mean
a
+ b
and variance
a
2 2
.
Thus
Y = aX + b
has a normal distribution with mean
a
+ b
and variance
a
2 2
.
Special Case:
the
z
transformation
Z
X
1
X
aX
b
Z
Z
2
a
a
2 2 1 2 2 1 0 Thus
Z
has a standard normal distribution .
Example
Suppose that
X
and
Y
are independent each having a normal distribution with means
X
and
Y
, standard deviations
X
and
Y
Find the distribution of
S = X + Y
Solution:
m X
e
X t
X
2
m Y
e
Y t
Y
2 Now
m
m X
Y
e
X t
X
2
e
Y t
Y
2
or
m
e
X
X
t
2
X
2
Y
t
2 2 = the moment generating function of the normal distribution with mean variance 2
X
2
Y
X +
Y
and Thus
Y = X + Y
has a normal distribution with mean
X +
Y
and variance 2
X
2
Y
Example
Suppose that
X
and
Y
are independent each having a normal distribution with means
X
and
Y
, standard deviations
X
and
Y
Find the distribution of
L = aX + bY
Solution:
m X
e
X t
X
2
m Y
e
Y t
Y
2 Now
m
m aX
e
X
bY
2
X
2 2
e
Y
m X
2
Y
2 2
or
m
e
a
X
b
X
t
a
2 2
X
b
2 2
Y
t
2 2 = the moment generating function of the normal distribution with mean
a
and variance
a
2 2
X
b
2 2
Y
X + b
Y
Thus
Y = aX + bY
has a normal distribution with mean
a
variance
a
2 2
X
b
2 2
Y
X + B
Y
and
Special Case:
a
= +1 and
b
= -1.
Thus
Y = X - Y
has a normal distribution with mean
X -
Y
and variance 2 2
X
2
Y
2 2
X
Y
2
Example
(Extension to
n
independent RV’s) Suppose that
X
1 ,
X
2 , …,
X n
are independent each having a normal distribution with means (for
i =
1, 2, … ,
n
)
i
, standard deviations
i
Find the distribution of
L = a
1
X
1
+ a
1
X
2
+ …+ a n X n
Solution:
m X i
e
i t
i
2 (for
i =
1, 2, … ,
n
) Now
m
1
n
m
m X
1
1 1
m X n
n m
e
1 1 2 2 2
n e
n
2
n
2 2
or
m
1
e
a
a n
n
t
a
1 2 1 2 2
a n
2 2
n
t
2
n
= the moment generating function of the normal distribution with mean and variance
a
1 2 1 2
a n
2
a
1 2
n
1 ...
a n
n
Thus
Y = a
1
X
1 and variance
+ … + a n X n
distribution with mean
a
1
a
1 2 1 2 1
a n
2 has a normal
+ …+ a n
2
n n
Special case:
a
1
a
2 1 2 1 2 1 2
a n
n
1
n
1 2 2 In this case
X
1 ,
X
2 , …,
X n
is a sample from a normal distribution with mean , and standard deviations , and
L
1
n
X
1
X
2
X n
X
the sample mean
Thus
Y a x
1 1
a x n n
x
1
x n
has a normal distribution with mean
x
a
1 1
a n
n
and variance
x
2
a
1 2 1 2 1 2 2
a n
2
n
2 2 2
n
2 2 2
n
Summary
If
x
1 ,
x
2 , …,
x n
is a sample from a normal distribution with mean , and standard deviations , then
x
the sample mean has a normal distribution with mean
x
and variance 2
x
2
n
standard deviation
x
n
0.4
0.3
0.2
0.1
0 20 Sampling distribution of
x
30 40 50 Population 60
The Central Limit theorem
If
x
1 ,
x
2 , …,
x n
with mean is a sample from a distribution , and standard deviations , then if
n
is large
x
the sample mean has a normal distribution with mean
x
and variance
x
2 2
n
standard deviation
x
n
Proof:
(use moment generating functions) We will use the following fact: Let
m
1 (
t
),
m
2 (
t
), … denote a sequence of moment generating functions corresponding to the sequence of distribution functions:
F
1 (
x
) ,
F
2 (
x
), … Let
m
(
t
) be a moment generating function corresponding to the distribution function
F
(
x
) then if then
i
lim
i
lim
i
Let
x
1 ,
x
2 , … denote a sequence of independent random variables coming from a distribution with moment generating function
m
(
t
) and distribution function
F
(
x
).
Let
S n
=
x
1 +
x
2 + … +
x n
then
m S n
m x
1
x n
=
m x
1
t m x
2
m x n n
= now
x
or
m t x
x
1
x
2
n
m
1
S n
x n m
S n S n n t
m
n
Let
z
x
n
n x
n
then
m t z
e
n
t m x
nt
e
n
t
m
nt n
n
and ln
z
n
t
n
ln
m
t n
Let
u
t n
or
n
t
u
and
n
t
2 2
u
2 Then ln
z
n
t
n
ln
m
t
t
2 2
u
t
2 2
u
2 ln
n
t
2 2 ln
u
2
u
n
z
u
lim ln 0
z
t
2 2
u
lim 0 ln
u
2
u
t
2 2
u
lim 0
using L'Hopital's rule 2
u
2 2
t
2 2
u
lim 0 2 using L'Hopital's rule again
t
2 2
u
lim 0
2 2
2 using L'Hopital's rule again
t
2 2
m
t
2 2 thus lim ln
n
2 2 2
m
i
2
t
2 2
z
t
2 2 and lim
n
z
t
2
e
2
Now
t
2
e
2 Is the moment generating function of the standard normal distribution Thus the limiting distribution of
z
is the standard normal distribution i.e. lim
n
x
1 2
e
u
2 2
du
Q.E.D.