Transcript Some Basic Probability Concepts

Some Basic Probability Concepts
Some Basic Probability Concepts
Experiments, Outcomes and Random Variables
• An experiment is the process by which an observation is made.
• Sample Space: ‘set of all possible well distinguished outcomes of an
experiment’ and is usually denoted by the letter ‘S’.
• For example, Tossing a coin: S= {H, T}, Tossing a die: S = {1,2,3,4,56}
• Sample Point: ‘each outcome in a sample space’
• Event: ‘Subset of the sample space’
• A random variable is ‘a real valued function defined on the sample
space’.
• A random variable is a variable whose value is unknown until it is
observed. The value of a random variable results from an experiment; it
is not perfectly predictable.
Some Basic Probability Concepts
2
Some Basic Probability Concepts
• A discrete random variable can take only a finite
number of values that can be counted by using the
positive integers.
• A continuous random variable can take any real
value (not just whole numbers) in an interval on the
real number line
• A continuous random variable can take any real
value (not just whole numbers) in an interval on the
real number line.
Some Basic Probability Concepts
3
The Probability Distribution of a
Random Variable
• The term Probability is used to give a quantitative
measure to the uncertainty associated with outcomes
of a random experiment.
• Probability: The Classical Definition
• In a random experiment, if there are ‘n’ equally likely
and mutually exclusive outcomes, of which ‘f’ are
favorable to an event ‘A’, then the probability of
occurrence of event A, denoted by P(A), is given by
the ratio, f/n.
• The frequency approach: ‘the limit of relative
frequency as the number of observations approached
infinity’.
Some Basic Probability Concepts
4
Some Basic Postulates
• Postulate 1: The probability of an event is a nonnegative
real number; that is, 0  P (Ai)  1 for each subset Si of S;
• Postulate 2: P(S) = 1
• Postulate 3: If S1, S2, S3,…Sn are mutually exclusive events
defined on the sample space S, then P(S1U S2U S3… U Sn) =
P(S1) + P(S2) P(S3)+…+P(Sn)
• An Illustration:
• Suppose we have information about the population in
Comilla . We are interested in two characteristics only, Sex
(M or F) and economic status (Poor or Non poor). The two
characteristics are not mutually exclusive.
• S = { (M & P), (F & P), (M & NP), (F &NP)}
Some Basic Probability Concepts
5
If the population is finite, then the distribution is
Economic Status
Sex
Totals
Poor
Non poor
Male

β
α+β
Female
γ
δ
γ+δ
Total
α+γ
+δ
α +β +γ +δ =
N
Some Basic Probability Concepts
6
In terms of probabilities, the distribution would look
like
Economic Status
Poor
Non poor
Male
P(M∩P)
P(M∩NP)
P(M)
Female
P(F∩P)
P(F∩NP)
P(F)
P(poor)
P(Non poor)
1
Sex
Some Basic Probability Concepts
7
• The probabilities pertaining to intersection of sets are
called joint probabilities. For instance, P (Male ∩ Poor)
is the probability that a person selected at random in
Comilla will be both male and poor, i.e., has two joint
characteristics.
• The probabilities that appear in the last row and in the
last column of the table are called marginal
probabilities. P (M) gives the probability of drawing a
male regardless of his economic status.
• It may be noted that marginal probabilities are equal to
the corresponding joint probabilities.
Some Basic Probability Concepts
8
• What is the probability that a person of given sex is
poor, or that a person of given economic status is a
male (female)? Such probabilities are called
conditional probabilities. For instance,
P(Poor/Male) means that we have a male and we
want to find out the probability that he is poor,
which is given by
  
P( M  P )
 
P( P / M )  
P( M )
   
Some Basic Probability Concepts
9
• When the values of a discrete random variable are listed
with their chances of occurring, the resulting table of
outcomes is called a probability function.
• For a discrete random variable X the value of the
probability function f(x) is the probability that the random
variable X takes the value x, f(x) =P(X=x).
• Therefore, 0  f(xi)  1 and, if X takes n values x1, .., xn,
then.
• For the continuous random variable Y the probability
density function f(y) can be represented by an equation,
which can be described graphically by a curve. For
continuous random variables the area under the
probability density function corresponds to probability.
Some Basic Probability Concepts
10
Probability function & Its Advantages
• Consider the experiment of tossing two six-sided
dice. Define the random variable as the sum total of
dots observed. Its values range from 2, 3.. to 12.
The sample space will consist of all possible
permutations of the two sets of numbers from 1 to
6. In sum, there will be 36 permutations.
Some Basic Probability Concepts
11
The resulting probability distribution will be as
follows:
X
2
3
4
5
6
7
8
9
10
11
12
Elements of sample
space
11
12, 21
13, 31, 22
14, 41, 23, 32
15, 51, 24, 42, 33
16, 61, 25, 52, 34, 43
26, 62, 35, 53, 44
36, 63, 45, 54
46, 64, 55
56, 65
66
f ( x)

F(x)
1/36
2/36
3/36
4/36
5/36
6/36
5/36
4/36
3/36
2/36
1/36
1
6 x 7
36
Some Basic Probability Concepts
12
Expected Values Involving a Single
Random Variable
• The Rules of Summation
• If X takes n values x1, ..., xn then their sum is
x  x  x   x
• If a is a constant, then
 a  na
n
i 1
i
1
2
n
n
i 1
• If a is a constant then
n
 ax
i 1
i
n
a  xi
i 1
Some Basic Probability Concepts
13
• If X and Y are two variables, then
 ( x  y )  x   y
• If X and Y are two variables, then
 (ax  by )  a x  b y
n
i 1
n
i
i
i 1
n
i 1
n
i
i 1
n
i
i
i 1
i
n
i
i 1
i
• The arithmetic mean (average) of n values of X is
x x  x   x
n
• Also,
x
i
i 1
n

1
2
n
n
n
 (x  x )  0
i 1
i
Some Basic Probability Concepts
14
• We often use an abbreviated form of the summation
notation. For example, if f(x) is a function of the values of X,
n
 f (x )  f (x )  f (x ) 
i 1
i
1
2
 f ( xn )
=  f ( xi ) ("Sum over all values of the index i")
i
  f ( x) ("Sum over all possible values of X ")
x
• Several summation signs can be used in one expression.
Suppose the variable Y takes n values and X takes m values,
and let f(x, y) =x+y. Then the double summation of this
function is
Some Basic Probability Concepts
15
m
n
m
n
 f ( x , y )   ( x  y )
i 1 j 1
i
j
i 1 j 1
i
j
• To evaluate such expressions work from the
innermost sum outward. First set i=1 and sum over
all values of j, and so on.
• To illustrate, let m = 2 and n = 3. Then
 f  x , y     f  x , y   f  x , y   f  x , y 
2
3
i 1 j 1
2
i
j
i 1
i
1
i
2
i
3
 f  x1 , y1   f  x1 , y2   f  x1 , y3  
f  x2 , y1   f  x2 , y2   f  x2 , y3 
Some Basic Probability Concepts
16
• The order of summation does not matter, so
m
n
n
m
 f ( x , y )   f ( x , y )
i 1 j 1
i
j
j 1 i 1
i
j
Some Basic Probability Concepts
17
The Mean of a Random Variable
• The expected value of a random variable X is the average
value of the random variable in an infinite number of
repetitions of the experiment (repeated samples); it is
denoted E[X].
• If X is a discrete random variable which can take the values
x1, x2,…,xn with probability density values f(x1), f(x2),…, f(xn),
the expected value of X is
E[ X ]  x1 f ( x1 )  x2 f ( x2 ) 
 xn f ( xn )
n
  xi f ( xi )
i 1
  xf ( x)
x
Some Basic Probability Concepts
18
Expectation of a Function of a Random Variable
• If X is a discrete random variable and g(X) is a function of it,
then
E[ g ( X )]   g ( x) f ( x)
x
• However, in general, if X is a discrete random variable and
g(X) = g1(X) + g2(X), where g1(X) and g2(X) are functions of X,
then
E[ g ( X )]   [ g1 ( x)  g 2 ( x)] f ( x)
x
  g1 ( x) f ( x)   g 2 ( x) f ( x)
x
x
 E[ g1 ( x)]  E[ g 2 ( x)]
Some Basic Probability Concepts
19
• The expected value of a sum of functions of random
variables, or the expected value of a sum of random
variables, is always the sum of the expected values.
• If c is a constant,
E[c]  c
• If c is a constant and X is a random variable, then
E[cX ]  cE[ X ]
• If a and c are constants then
E[a  cX ]  a  cE[ X ]
Some Basic Probability Concepts
20
The Variance of a Random Variable
var( X )   2  E[ g ( X )]  E  X  E ( X )   E[ X 2 ]  [ E ( X )]2
2
• Let a and c be constants, and let Z = a + cX. Then Z is
a random variable and its variance is
var(a  cX )  E[(a  cX )  E (a  cX )]2  c 2 var( X )
Some Basic Probability Concepts
21
A Recap
•
•
•
•
•
•
Probability: Basic Concepts
Classical & Frequency approaches
Some Basic Postulates
Some Examples
Probability function & its advantages
Mathematical expectation
Some Basic Probability Concepts
22
Using Joint Probability Functions
• Marginal Probability Functions
• If X and Y are two discrete random variables then
f ( x )   f ( x, y )
for each value X can take
y
f ( y )   f ( x, y )
for each value Y can take
x
• Conditional Probability Functions
f ( x | y )  P[ X  x | Y  y ] 
f ( x, y )
f ( y)
Some Basic Probability Concepts
23
Independent Random Variables
• If X and Y are independent random variables, then
f ( x, y)  f ( x) f ( y)
for each and every pair of values of x and y. The
converse is also true.
• If X1, …, Xn are statistically independent the joint
probability function can be factored and written as
f ( x1 , x2 ,
, xn )  f1 ( x1 )  f 2 ( x2 ) 
 f n ( xn )
Some Basic Probability Concepts
24
• If X and Y are independent random variables, then
the conditional probability function of X given that
Y=y is
f ( x | y) 
f ( x, y ) f ( x ) f ( y )

 f ( x)
f ( y)
f ( y)
for each and every pair of values x and y. The
converse is also true.
Some Basic Probability Concepts
25
The Expected Value of a Function of Several Random
Variables: Covariance and Correlation
• If X and Y are random variables, then their
covariance is
cov( X , Y )  E[( X  E[ X ])(Y  E[Y ])]
• If X and Y are discrete random variables, f(x,y) is
their joint probability function, and g(X,Y) is a
function of them, then
E[ g ( X , Y )]   g ( x, y ) f ( x, y )
x
y
Some Basic Probability Concepts
26
• If X and Y are discrete random variables and f(x,y) is
their joint probability function, then
cov( X , Y )  E[( X  E[ X ])(Y  E[Y ])]
 [ x  E ( X )][ y  E (Y )] f ( x, y )
x
y
• If X and Y are random variables then their
correlation is
=
cov( X , Y )
var( X ) var(Y )
Some Basic Probability Concepts
27
• The Mean of a Weighted Sum of Random
Variables
E[aX  bY ]  aE ( X )  bE (Y )
• If X and Y are random variables, then
E  X  Y   E  X   E Y 
Some Basic Probability Concepts
28
The Variance of a Weighted Sum of
Random Variables
• If X, Y, and Z are random variables and a, b, and c
are constants, then
var  aX  bY  cZ   a 2 var  X   b 2 var Y   c 2 var  Z 
 2ab cov  X , Y   2ac cov  X , Z   2bc cov Y , Z 
• If X, Y, and Z are independent, or uncorrelated,
random variables, then the covariance terms are
zero and:
var  aX  bY  cZ   a2 var  X   b2 var Y   c2 var  Z 
Some Basic Probability Concepts
29
• If X, Y, and Z are independent, or
uncorrelated, random variables, and if a = b
= c = 1, then
var  X  Y  Z   var  X   var Y   var  Z 
Some Basic Probability Concepts
30
Theoretical Derivation of Sampling Distribution of
Estimators & Test Statistics:
• Binomial Distribution:
• Comilla Story: Picking a BPL Person
• Let p be the proportion of BPL population in Comilla
and q be the proportion of APL population.
• Let n denote the sample size.
• Let be the proportion of BPL in the sample.
• Let X denote the number of poor in the sample.
• If the person picked up happens to be poor, the
experiment is a success and its probability is p.
Otherwise, it is a failure with a probability given by q,
that is, (1-p).
Some Basic Probability Concepts
31
• Let us define the sampling distributions of X
and for samples of various sizes. Since pˆ =
(X/n) or X = n , by the different results that
we have learnt so far, we can determine the
distribution of , if we know that of X and vice
versa.
Some Basic Probability Concepts
32
Sampling Distribution for n = 1
Number of Poor Probability: f(x)
x f(x)
x2 f(x)
0
P(F) = q
0
0
1
P(S) = p
p
p
Sum
p+q=1
p
p
Some Basic Probability Concepts
33
• Mean and variance of X:
E( X )
x

i

f ( xi )
p
i
Var ( X )

E ( X 2 )  [ E ( X )]2

x
i

p  p2
2
i
f ( xi )  [ xi f ( xi )]2
i
 p(1  p)  pq
• Mean and variance of pˆ :
ˆ) – E(X/n) = E(X) = p
E( p
^
Var( p )=Var(X/n)=Var(X)=pq
Some Basic Probability Concepts
34
Sampling Distribution for n = 2
Number of Poor
Probability: f(x)
xf(x)
x2 f(x)
0
P(F)P(F) = q2
0
0
2pq
2pq
1
P(F)P(S)+P(S)P(F)
= 2pq
2
P(S)P(S) = p2
2p2
4p2
Sum
(p+q)2 = 1
2p(p+q)
2p(2p+q)
Some Basic Probability Concepts
35
• Mean and variance of X:

x
f ( xi )

2 p( p  q )
Var( X )  E ( X 2 )  [ E ( X )]2

x
E( X )
i
2 p
i
2
i
f ( xi )  [ xi f ( xi )]2
i
 2 p ( q  2 p)  ( 2 p )
2
i
 2 pq  4 p  4 p 2  2 pq
2
• Mean and variance of pˆ:
^
E(p)  E(X/n) E(X/2) p
^
Var (p)  Var(X/2) (1/4)Var(X)  (pq/2)
Some Basic Probability Concepts
36
Sampling Distribution for n = 3
Number of
Probability:
Poor
f(x)
0
P(F) = q3
pqq+ qpq+
1
2
qqp= 3pq2
ppq+ pqp+
qpp=3p2q
3
ppp = p3
Sum
(p+q)3 = 1
xf(x)
x2 f(x)
0
0
3p2q
3q2p
6pq2
12qp2
3p3
9p2
Some Basic Probability Concepts
37
• Mean and variance of X:
E( X ) 
x
i
f (xi )
 3q 2 p  6qp 2  3 p 3  3 p(q 2  2 pq  p 2 )  3 p( p  q) 2  3 p
i
• Mean and variance of
Some Basic Probability Concepts
38
• In general, we have:
Some Basic Probability Concepts
39
• That is, the probability of getting x poor
people in a sample size of ‘n’ is
• Properties:
Some Basic Probability Concepts
40
• E( ) = p, that is, unbiased estimator.
•
,that is, the distribution gets concentrated
as sample size increases. This property together
with (i) implies p is a consistent estimator. The
dispersion of the sampling distribution
decreases in inverse proportion to the square
root of sample size. That is, if sample size
increases k times, then the std. deviation of the
sampling distribution decreases k times.
^
p
^
Some Basic Probability Concepts
41
^
• The sampling distribution of p is most
dispersed when the population parameter p
is equal to ½ and is least dispersed when p is
0 or 1.
Some Basic Probability Concepts
42
• The asymmetry (skewness) of the sampling
distribution of p decreases in inverse
proportion to the square root of sample size
(since ))))))))))))))))).
• It is least skewed when p = ½ and is most
skewed when p is 0 or 1.
^
Some Basic Probability Concepts
43
The Normal Distribution
Properties:
•The distribution is continuous and symmetric
around its mean μ. This implies: (i) mean = median =
mode; and (ii) the mean divides the area under the
normal curve into exact halves.
•The range of the distribution extends from -∞ to +
∞. In other words, the distribution is unbounded.
Some Basic Probability Concepts
44
• The curve attains maximum height at x = μ; the points
of inflection occur at x = μ σ(which means the
standard deviation measures the distance from the
center of the distribution to a point of inflection).
• Normal distribution is fully specified by two
parameters, mean (μ) and variance (σ2). If we know
these two parameters, we know all there is to know
about it.
• If X, Y,…, Z are normally and independently
distributed random variables and a,b,…,c are
constants, then the linear combination aX+bY+…+cZ is
also normally distributed.
Some Basic Probability Concepts
45
How to calculate probabilities for a
normal random variable?
• From tabulated results
• Different normal distributions lead to different probabilities due
to differences in mean and variance. For the same reason, if we
know the area under one specific normal curve, the area under
any other normal curve can be computed by accounting for the
differences in mean and variance.
• One specific distribution for which areas have been tabulated is a
normal distribution with mean μ = 0 and variance σ2 = 1, called
the standard normal distribution (also called unit normal
distribution).
• Given that (i) X is normally distributed with mean μ and variance
σ2; and (ii) the areas under the standard normal curve, how to
determine the probability that x lies in some interval, say, (x1 and
x 2) ?
Some Basic Probability Concepts
46
• Let Z denote a normally distributed variable
with mean zero and variance equal to unity.
That is,
• P(x1 < x < x2) = probability that X will lie between
x1 and x2(x1 < x2); and P(z1< z <z2) = probability
that Z will lie between z1 and z2 (z1 < z2) .
• Since X is normally distributed, a linear function
of X will also be normal.
Some Basic Probability Concepts
47
• Let it be denoted by aX + b, where a and b
are constants.
• Choose a and b such that (aX+b) is a standard
normal variable. That is,
Some Basic Probability Concepts
48
• Solving for a and b , we get
• Thus, we have aX+b =
=Z
• In other words, any variable with mean μ and
variance σ2 can be transformed into a standard
normal variable by expressing it as a deviation
from its mean and dividing by σ.
Some Basic Probability Concepts
49
• Consider P(x1 < x < x2) where x1 < x2.
(X  )
• From  = Z, we get X = Z+. Hence, we
can write x1 = z1 +  and x2 = z2 + 
• Now, P(x1 < x < x2) = P(z1 +  < Z +  <z2 +
 ) = P(z1 < Z < z2)
(X  )
(X   )
• where z1 =  and z2 = 
Some Basic Probability Concepts
50
Thank You