Transcript Chapter 2
Review of Probability Concepts
Prepared by Vera Tabakova, East Carolina University
B.1 Random Variables
B.2 Probability Distributions
B.3 Joint, Marginal and Conditional Probability
Distributions
B.4 Properties of Probability Distributions
B.5 Some Important Probability Distributions
Principles of Econometrics, 3rd Edition
Slide B-2
A random variable is a variable whose value is unknown until it is
observed.
A discrete random variable can take only a limited, or countable,
number of values.
A continuous random variable can take any value on an interval.
Principles of Econometrics, 3rd Edition
Slide B-3
The probability of an event is its “limiting relative frequency,” or the
proportion of time it occurs in the long-run.
The probability density function (pdf) for a discrete random
variable indicates the probability of each possible value occurring.
f ( x) P X x
f ( x1 ) f ( x2 )
Principles of Econometrics, 3rd Edition
f ( xn ) 1
Slide B-4
Principles of Econometrics, 3rd Edition
Slide B-5
Figure B.1 College Employment Probabilities
Principles of Econometrics, 3rd Edition
Slide B-6
The cumulative distribution function (cdf) is an alternative way to
represent probabilities. The cdf of the random variable X, denoted
F(x), gives the probability that X is less than or equal to a specific
value x.
F x P X x
Principles of Econometrics, 3rd Edition
Slide B-7
Principles of Econometrics, 3rd Edition
Slide B-8
For example, a binomial random variable X is the number of
successes in n independent trials of identical experiments with
probability of success p.
n x
P X x f x p (1 p)n x
x
n
n!
where n! n (n 1 n 2
x x! n x !
Principles of Econometrics, 3rd Edition
(B.1)
2 1
Slide B-9
Figure B.2 PDF of a continuous random variable
Principles of Econometrics, 3rd Edition
Slide B-10
P 20 X 40
40
f x dx .355
20
P X x
x
f t dt F x
P 20 X 40 F (40) F (20) .649 .294 .355
Principles of Econometrics, 3rd Edition
Slide B-11
1
2
X
3
4
high school diploma or less
some college
four year college degree
advanced degree
0 if had no money earnings in 2002
Y
1 if had positive money earnings in 2002
Principles of Econometrics, 3rd Edition
Slide B-12
f x, y 1
x
Principles of Econometrics, 3rd Edition
y
Slide B-13
f X ( x) f ( x, y )
for each value X can take
y
fY ( y) f ( x, y)
(B.2)
for each value Y can take
x
4
f Y y f x, y
y 0,1
x 1
fY 1 .19 .06 .04 .02 .31
Principles of Econometrics, 3rd Edition
Slide B-14
Principles of Econometrics, 3rd Edition
Slide B-15
P(Y y, X x) f ( x, y )
f ( y | x) P(Y y | X x)
P X x
f X ( x)
Principles of Econometrics, 3rd Edition
y
f y | X 3
0
.04/.18=.22
1
.14/.18=.78
(B.3)
Slide B-16
Two random variables are statistically independent if the conditional
probability that Y = y given that X = x, is the same as the
unconditional probability that Y = y.
P Y y | X x P Y y
(B.4)
f ( x, y )
f ( y | x)
fY ( y )
f X ( x)
(B.5)
f ( x, y) f X ( x) fY ( y)
(B.6)
Principles of Econometrics, 3rd Edition
Slide B-17
Principles of Econometrics, 3rd Edition
Slide B-18
Principles of Econometrics, 3rd Edition
Slide B-19
Principles of Econometrics, 3rd Edition
Slide B-20
Principles of Econometrics, 3rd Edition
Slide B-21
B.4.1
Mean, median and mode
E[ X ] x1P X x1 x2 P X x2
xn P X xn
(B.7)
For a discrete random variable the expected value is:
E[ X ] x1 f ( x1 ) x2 f ( x2 )
n
xi f ( xi ) xf ( x)
i 1
Principles of Econometrics, 3rd Edition
xn f ( x n )
(B.8)
x
Slide B-22
For a continuous random variable the expected value is:
E X
xf x dx
The mean has a flaw as a measure of the center of a probability
distribution in that it can be pulled by extreme values.
Principles of Econometrics, 3rd Edition
Slide B-23
For a continuous distribution the median of X is the value m such that
P X m P( X m) .5
In symmetric distributions, like the familiar “bell-shaped curve” of
the normal distribution, the mean and median are equal.
The mode is the value of X at which the pdf is highest.
Principles of Econometrics, 3rd Edition
Slide B-24
E[ g ( X )] g ( x) f ( x)
(B.9)
x
E aX aE X
(B.10)
E g X g x f x axf x a xf x aE X
Principles of Econometrics, 3rd Edition
Slide B-25
E aX b aE X b
(B.11)
E g1 X g2 X E g1 X E g2 X
(B.12)
The variance of a discrete or continuous random variable X is the
expected value of
g X X E X
Principles of Econometrics, 3rd Edition
2
Slide B-26
The variance of a random variable is important in characterizing the
scale of measurement, and the spread of the probability distribution.
Algebraically, letting E(X) = μ,
var( X ) E X E[ X 2 ] 2
2
Principles of Econometrics, 3rd Edition
2
(B.13)
Slide B-27
Figure B.3 Distributions with different variances
Principles of Econometrics, 3rd Edition
Slide B-28
var(aX b) a2 var( X )
(B.14)
var(aX b) E aX b E aX b E aX b a b
2
2
E a X a E X a 2 var X
2
Principles of Econometrics, 3rd Edition
2
2
Slide B-29
3
E X
skewness
3
4
E X
kurtosis
4
Principles of Econometrics, 3rd Edition
Slide B-30
E[ g ( X , Y )] g ( x, y) f ( x, y)
x
(B.15)
y
E X Y E( X ) E(Y )
(B.16)
E X Y x y f x, y xf x, y yf x, y
x
y
x
y
x
y
x f x, y y f x, y xf x yf y
x
y
y
x
x
y
E X E Y
Principles of Econometrics, 3rd Edition
Slide B-31
E (aX bY c) aE ( X ) bE (Y ) c
(B.17)
E XY E g X , Y xyf x, y xyf x f y
x
y
x
y
xf x yf y E X E Y if X and Y are independent.
x
y
g ( X , Y ) ( X X )(Y Y )
Principles of Econometrics, 3rd Edition
(B.18)
Slide B-32
Figure B.4 Correlated data
Principles of Econometrics, 3rd Edition
Slide B-33
cov( X ,Y ) XY E X X Y Y E XY X Y
cov X , Y
XY
var( X ) var(Y ) X Y
(B.19)
(B.20)
If X and Y are independent random variables then the covariance and
correlation between them are zero. The converse of this relationship is
not true.
Principles of Econometrics, 3rd Edition
Slide B-34
If a and b are constants then:
var aX bY a2 var( X ) b2 var(Y ) 2ab cov( X ,Y )
(B.21)
var X Y var( X ) var(Y ) 2cov( X , Y )
(B.22)
var X Y var( X ) var(Y ) 2cov( X ,Y )
(B.23)
Principles of Econometrics, 3rd Edition
Slide B-35
If X and Y are independent then:
var aX bY a2 var( X ) b2 var(Y )
(B.24)
var X Y var( X ) var(Y )
(B.25)
var X Y Z var X var Y var Z
Principles of Econometrics, 3rd Edition
Slide B-36
4
E X xf x 1 .1 2 .2 3 .3 4 .4 3 X
x 1
E X X
2
X
2
2
2
2
2
1 3 .1 2 3 .2 3 3 .3 4 3 .4
4 .1 1 .2 0 .3 1 .4
1
Principles of Econometrics, 3rd Edition
Slide B-37
B.5.1
The Normal Distribution
If X is a normally distributed random variable with mean μ and
variance σ2, it can be symbolized as X ~ N , 2 .
( x )2
f ( x)
exp
,
2
22
2
1
Principles of Econometrics, 3rd Edition
x
(B.26)
Slide B-38
Figure B.5a Normal Probability Density Functions with Means μ and Variance 1
Principles of Econometrics, 3rd Edition
Slide B-39
Figure B.5b Normal Probability Density Functions with Mean 0 and Variance σ2
Principles of Econometrics, 3rd Edition
Slide B-40
A standard normal random variable is one that has a normal
probability density function with mean 0 and variance 1.
X
Z
~ N (0,1)
(B.27)
The cdf for the standardized normal variable Z is
( z) P Z z .
Principles of Econometrics, 3rd Edition
Slide B-41
a
X a
a
P[ X a] P
P
Z
(B.28)
a
X a
a
P[ X a] P
P
Z
1
(B.29)
b
a
b
a
P[a X b] P
Z
(B.30)
Principles of Econometrics, 3rd Edition
Slide B-42
A weighted sum of normal random variables has a normal
distribution.
X 1 ~ N 1 , 12
X 2 ~ N 2 , 22
Y a1 X1 a2 X 2 ~ N Y a11 a22 , Y2 a1212 a2222 2a1a212 (B.27)
Principles of Econometrics, 3rd Edition
Slide B-43
V Z12 Z22
Zm2 ~ (2m)
(B32)
E[V ] E (2m ) m
var[V ] var (2m ) 2m
Principles of Econometrics, 3rd Edition
(B.33)
Slide B-44
Figure B.6 The chi-square distribution
Principles of Econometrics, 3rd Edition
Slide B-45
A “t” random variable (no upper case) is formed by dividing a
standard normal random variable Z ~ N 0,1 by the square root of an
independent chi-square random variable, V ~ (2m) , that has been
divided by its degrees of freedom m.
Z
t
V
Principles of Econometrics, 3rd Edition
~ t( m )
(B.34)
m
Slide B-46
Figure B.7 The standard normal and t(3) probability density functions
Principles of Econometrics, 3rd Edition
Slide B-47
An F random variable is formed by the ratio of two independent chisquare random variables that have been divided by their degrees of
freedom.
V1 m1
F
~ F( m1 ,m2 )
V2 m2
Principles of Econometrics, 3rd Edition
(B.35)
Slide B-48
Figure B.8 The probability density function of an F random variable
Principles of Econometrics, 3rd Edition
Slide B-49
binary variable
binomial random variable
cdf
chi-square distribution
conditional pdf
conditional probability
continuous random variable
correlation
covariance
cumulative distribution function
degrees of freedom
discrete random variable
expected value
experiment
F-distribution
Principles of Econometrics, 3rd Edition
joint probability density function
marginal distribution
mean
median
mode
normal distribution
pdf
probability
probability density function
random variable
standard deviation
standard normal distribution
statistical independence
variance
Slide B-50