Joint Distributions of 2 or More Random Variables
Download
Report
Transcript Joint Distributions of 2 or More Random Variables
Joint Distribution of two or More Random Variables
• Sometimes more than one measurement (r.v.) is taken on each member of
the sample space. In cases like this there will be a few random variables
defined on the same probability space and we would like to explore their
joint distribution.
• Joint behavior of 2 random variable (continuous or discrete), X and Y
determined by their joint cumulative distribution function
FX ,Y x, y P X x, Y y .
• n – dimensional case
FX1 ,...,X n x1 ,...,xn P X 1 x1 , , X n xn .
week 7
1
Discrete case
• Suppose X, Y are discrete random variables defined on the same probability
space.
• The joint probability mass function of 2 discrete random variables X and Y
is the function pX,Y(x,y) defined for all pairs of real numbers x and y by
pX ,Y x, y P X x and Y y
• For a joint pmf pX,Y(x,y) we must have: pX,Y(x,y) ≥ 0 for all values of x,y and
p x, y 1
X ,Y
x
y
week 7
2
Example for illustration
• Toss a coin 3 times. Define,
X: number of heads on 1st toss, Y: total number of heads.
• The sample space is Ω ={TTT, TTH, THT, HTT, THH, HTH, HHT, HHH}.
• We display the joint distribution of X and Y in the following table
• Can we recover the probability mass function for X and Y from the joint table?
• To find the probability mass function of X we sum the appropriate rows of the
table of the joint probability function.
• Similarly, to find the mass function for Y we sum the appropriate columns.
week 7
3
Marginal Probability Function
• The marginal probability mass function for X is
p X x p X ,Y x, y
y
• The marginal probability mass function for Y is
pY y p X ,Y x, y
x
• Case of several discrete random variables is analogous.
• If X1,…,Xm are discrete random variables on the same sample space with
joint probability function
pX1,...X n x1,...xm P X1 x1,..., X m xm
The marginal probability function for X1 is
pX1 x1
p
X1 ,...X n
x1,...xm
x2 ,...,xm
• The 2-dimentional marginal probability function for X1 and X2 is
pX1X 2 x1, x2 pX1,...X n x1, x2 , x3 ,...,xm
x3 ,...,xm
week 7
4
Example
• Roll a die twice. Let X: number of 1’s and Y: total of the 2 die.
There is no available form of the joint mass function for X, Y. We display
the joint distribution of X and Y with the following table.
• The marginal probability mass function of X and Y are
• Find P(X ≤ 1 and Y ≤ 4)
week 7
5
Independence of random variables
• Recall the definition of random variable X: mapping from Ω to R such that
: X x F for x R .
• By definition of F, this implies that (X > 1.4) is and event and for the
discrete case (X = 2) is an event.
• In general X A : X A is an event for any set A that is formed
by taking unions / complements / intersections of intervals from R.
• Definition
Random variables X and Y are independent if the events X A and Y B
are independent.
week 7
6
Theorem
• Two discrete random variables X and Y with joint pmf pX,Y(x,y) and
marginal mass function pX(x) and pY(y), are independent if and only if
pX ,Y x, y pX x pY y
• Proof:
• Question: Back to the rolling die 2 times example, are X and Y
independent?
week 7
7
Conditional Probability on a joint discrete distribution
• Given the joint pmf of X and Y, we want to find
P X x and Y y
P X x | Y y
PY y
and
PY y | X x
P X x and Y y
P X x
• Back to the example on slide 5,
PY 2 | X 1 0
2
1
PY 3 | X 1 36
10
5
36
PY 12 | X 1 0
• These 11 probabilities give the conditional pmf of Y given X = 1.
week 7
8
Definition
• For X, Y discrete random variables with joint pmf pX,Y(x,y) and marginal mass
function pX(x) and pY(y). If x is a number such that pX(x) > 0, then the
conditional pmf of Y given X = x is
pY | X y | x pY | X y | X x
• Is this a valid pmf?
p X ,Y x, y
p X x
• Similarly, the conditional pmf of X given Y = y is
p X |Y x | y p X |Y x | Y y
p X ,Y x, y
pY y
• Note, from the above conditional pmf we get
pX ,Y x, y pX |Y x | y pY y
Summing both sides over all possible values of Y we get
p X x p X ,Y x, y p X |Y x | y pY y
y
y
This is an extremely useful application of the law of total probability.
• Note: If X, Y are independent random variables then PX|Y(x|y) = PX(x).
week 7
9
Example
• Suppose we roll a fair die; whatever number comes up we toss a coin that
many times. What is the distribution of the number of heads?
Let X = number of heads, Y = number on die.
We know that
1
pY y
, y 1, 2, 3, 4, 5, 6
6
Want to find pX(x).
• The conditional probability function of X given Y = y is given by
y 1
p X |Y x | y
x 2
2
for
• By the Law of Total Probability we have
x = 0, 1, …, y.
y 1 1
p X x p X |Y x | y pY y
6
y
y x x 2
6
y
Possible values of x: 0,1,2,…,6.
week 7
10
Example
• Interested in the 1st and 2nd success in repeated Bernoulli trails with probability of
success p on any trail. Let X = number of failures before 1st success and
Y = number of failures before 2nd success (including those before the 1st success).
• The marginal pmf of X and Y are
for
x = 0, 1, 2, ….
p X x P X x pqx
• and
pY y PY y y 1 p 2 q y for
y = 0, 1, 2, ….
• The joint mass function, pX,Y(x,y) , where x, y are non-negative integers and x ≤ y is
the probability that the 1st success occurred after x failure and the 2nd success after
y – x more failures (after 1st success), e.g.
pX ,Y 2,5 PFFSFFFS p 2 q5
• The joint pmf is then given by
p 2q y
p X ,Y x, y
0
0 x y
otherwise
• Find the marginal probability functions of X, Y. Are X, Y independent?
• What is the conditional probability function of X given Y = 5
week 7
11
Some Notes on Multiple Integration
• Recall: simple integral f x dx cab be regarded as the limit as N ∞ of
a
the Riemann sum of the form N
b
f x x
i
i 1
i
where ∆xi is the length of the ith subinterval of some partition of [a, b].
• By analogy, a double integral
f x, y dA
D
where D is a finite region in the xy-plane, can be regarded as the limit as
N ∞ of the Riemann sum of the form
N
N
f x , y A f x , y x y
i 1
i
i
i
i 1
i
i
i
i
where Ai is the area of the ith rectangle in a decomposition of D into
rectangles.
week 7
12
Evaluation of Double Integrals
• Idea – evaluate as an iterated integral. Restrict D now to a region such that
any vertical line cuts its boundary in 2 points. So we can describe D by
yL(x) ≤ y ≤ yU(x) and xL ≤ x ≤ xU
• Theorem
Let D be the subset of R2 defined by yL(x) ≤ y ≤ yU(x) and xL ≤ x ≤ xU
If f(x,y) is continuous on D then
D
xU
yU
f x, y dA f x, y dy dx
xL
yL
• Comment: When evaluating a double integral, the shape of the region or
the form of f (x,y) may dictate that you interchange the role of x and y and
integrate first w.r.t x and then y.
week 7
13
Examples
1) Evaluate
x ydA where D is the triangle with vertices (0,0), (2,0) and (0,1).
D
2) Evaluate
2
2
xy
x
y
dA where D is the region in the 1st quadrate bounded
D
by y = 0, y = x and x2 + y2 = 1.
3) Integrate f x, y xe x2 y over the set of points in the positive quadrate for
which y > 2x.
week 7
14
The Joint Distribution of two Continuous R.V’s
• Definition
Random variables X and Y are (jointly) continuous if there is a non-negative
function fX,Y(x,y) such that
P X , Y A f X ,Y x, y dxdy
A
for any “reasonable” 2-dimensional set A.
• fX,Y(x,y) is called a joint density function for (X, Y).
• In particular , if A = {(X, Y): X ≤ x, Y ≤ x}, the joint CDF of X,Y is
FX ,Y x, y
x
y
f X ,Y u, v du, dv
• From Fundamental Theorem of Calculus we have
2
2
f X .Y x, y
FX ,Y x, y
FX ,Y x, y
xy
yx
week 7
15
Properties of joint density function
•
f X ,Y x, y 0 for all x, y R
• It’s integral over R2 is
f X ,Y x, y dxdy 1
week 7
16
Example
• Consider the following bivariate density function
12 2
x xy
f X ,Y x, y 7
0
0 x 1, 0 y 1
otherwise
• Check if it’s a valid density function.
• Compute P(X > Y).
week 7
17
Properties of Joint Distribution Function
For random variables X, Y , FX,Y : R2 [0,1] given by FX,Y (x,y) = P(X ≤ x,Y ≤ x)
lim FX ,Y x, y 0
x
y
lim FX ,Y x, y 1
x
y
• FX,Y (x,y) is non-decreasing in each variable i.e.
FX ,Y x1 , y1 FX ,Y x2 , y2
if x1 ≤ x2 and y1 ≤ y2 .
• lim FX ,Y x, y FY y
x
and
lim FX ,Y x, y FX x
y
week 7
18
Marginal Densities and Distribution Functions
• The marginal (cumulative) distribution function of X is
FX x P X x
x
f X ,Y u, y dydu
• The marginal density of X is then
f X x FX' x f X ,Y x, y dy
• Similarly the marginal density of Y is
fY y f X ,Y x, y dx
week 7
19
Example
• Consider the following bivariate density function
6 xy 2
f X ,Y x, y
0
0 x 1, 0 y 1
otherwise
• Check if it is a valid density function.
• Find the joint CDF of (X, Y) and compute P(X ≤ ½ ,Y ≤ ½ ).
• Compute P(X ≤ 2 ,Y ≤ ½ ).
• Find the marginal densities of X and Y.
week 7
20
Generalization to higher dimensions
Suppose X, Y, Z are jointly continuous random variables with density f(x,y,z), then
• Marginal density of X is given by:
f X x
f x, y, z dydz
• Marginal density of X, Y is given by :
f X ,Y x, y f x, y, z dz
week 7
21
Example
Given the following joint CDF of X, Y
FX ,Y x, y x 2 y y 2 x x 2 y 2
0 x 1, 0 y 1
• Find the joint density of X, Y.
• Find the marginal densities of X and Y and identify them.
week 7
22
Example
Consider the joint density
2 e y
f X ,Y x, y
0
0 x y
otherwise
where λ is a positive parameter.
• Check if it is a valid density.
• Find the marginal densities of X and Y and identify them.
week 7
23