Transcript PowerPoint 簡報 - National Chung Cheng University

Chapter 4. Multiple Random Variables
In some random experiments, a number of different
quantities are measured.
S

X   X1, X 2 ,
, Xn 
Ex. 4.1. Select a student’s name from an urn. 
H   : height
W   : weight
A   : age
tch-prob
1
4.1 Vector Random Variables
A vector random variable X is a function that assigns a
vector of real numbers to each outcome  in S, the sample
space of the random experiment.
The vector ( H ( ),W ( ), A( )) is a vector random variable.
Each event involving an n-dimensional random variable
X   X1 , X 2 ,
, X n  has a corresponding region in an
n-dimensional real space.
tch-prob
2
Event Examples
• Consider the two-dimensional random variable X=(X,Y).
Find the region of the plane corresponding to events
A   X  Y  10 ,
B  min( X , Y )  5 ,


C  X 2  Y 2  100 .
tch-prob
3
Product Form
• We are particularly interested in events that have the
product form
A   X1 in A1
X 2
in A2  ...
X n
in An 
where Ak is a one-dimensional event that involves X k only.
( x1 , y2 ) ( x2 , y2 )
y2
y1
x1
{x1  X  x2 } {Y  y2 }
tch-prob
x2
{x1  X  x2 } { y1  Y  y2}
4
Product Form
• A fundamental problem in modeling a system with a vector
random variable involves specifying the probability of
product-form events
P[ A]  P  X 1 in A1
X 2
in A2  ...
X n
P  X1 in A1 , X 2 in A2 ,..., X n in An 
in An 
• Many events of interest are not of product form.
• However, the non-product-form events can be
approximated by the union of product-form events.
Ex. B  X  5 and Y  
X  5 and Y  5
tch-prob
5
4.2 Pairs of Random variables
A. Pairs of discrete random variables
- Let X=(X,Y) assume values fromS   x j , yk  , j  1, 2,..., k  1, 2,....
- The joint pmf of X is
pX ,Y  x j , yk   P  X  x j 
Y  yk 
P  X  x j , Y  yk 
j  1, 2,...,
k  1, 2,...
It gives the probability of the occurrence of the pair  x j , yk 
- The probability of any event A is the sum of the pmf over
the outcomes in A:
P[ X in A] 
  p x , y 
XY
j
k
x j , yk in A

- When A=S,

 p  x , y   1
j 1 k 1
XY
j
k
tch-prob
6
Marginal pmf
• We are also interested in the probabilities of events
involving each of the random variables in isolation.
• These can be found in terms of the Marginal pmf.
px  x j   P  X  x j   P  X  x j , Y  anything 

  p X ,Y  x j , yk 
k 1

p y  yk   P Y  yk    p X ,Y  x j , yk 
j 1
• In general, knowledge of the marginal pmf’s is insufficient
to specify the joint pmf.
tch-prob
7
Ex. 4.6. Loaded dice: A random experiment consists of tossing two loaded
dice and noting the pair of numbers (X,Y) facing up. The joint pmf
pX ,Y ( j, k ) 
j
k
1
2
3
4
5
6
1
2/42
1/42
1/42
1/42
1/42
1/42
2
1/42
2/42
1/42
1/42
1/42
1/42
3
1/42
1/42
2/42
1/42
1/42
1/42
4
1/42
1/42
1/42
2/42
1/42
1/42
5
1/42
1/42
1/42
1/42
2/42
1/42
6
1/42
1/42
1/42
1/42
1/42
2/42
The marginal pmf P[X=j]=P[Y=k]=1/6.
tch-prob
8
Ex. 4.7. Packetization problem: The number of bytes N in a message has a
geometric distribution with parameter 1-p and range SN={0,1,2,….}. Suppose
that messages are broken into packets of maximum length M bytes.Let Q be
the number of full packets and let R be the number of bytes left over. Find the
joint pmf and marginal pmf’s of Q and R.
tch-prob
9
joint cdf of X and Y
y
The joint cdf of X and Y is defined as the
probability of the product-form event
{X  x1} {Y  y1}:
(x1,y1)
x
FX ,Y  x1, y1   P  X  x1, Y  y1 
(i ) FX ,Y ( x1 , y1 )  FX ,Y ( x2 , y2 )
if x1  x2 and y1  y2
(ii ) FX ,Y (, Y1 )  FX ,Y ( X 1 , )  0
(iii ) FX ,Y (, )  1
(iv) FX ( x)  FX ,Y ( x, )  P[ X  x, Y  ]  P[ X  x]
marginal cdf
FY ( y )  P[Y  y ]
tch-prob
10
y
joint cdf of X and Y
(x1,y1)
(v) lim FX ,Y ( x, y )  FX ,Y (a, y )
x
x a
lim FX ,Y ( x, y )  FX ,Y ( x, b)
y b
(vi) P[ x1  X  x2 , y1  Y  y2 ]
 FX ,Y ( x2 , y2 )  FX ,Y ( x2 , y1 )  FX ,Y ( x1 , y2 )  FX ,Y ( x1 , y1 )
tch-prob
11
P[ x1  X  x2 , Y  y1 ]  FX ,Y ( x2 , y1 )  FX ,Y ( x1, y1 )
y
y
x1
x2
x
y2
y1
y1 (x1,y1) A
(x2,y1)
(x1,y1)
x1 x2
x
(x2 ,y2)
B
B
tch-prob
12
joint pdf of two jointly continuous random variables
X and Y are jointly Continuous if the probabilities of events
involving (X,Y) can be expressed as an integral of a pdf, f X ,Y ( x, y) .
P  X in A   f X ,Y ( x ', y ') dx ' dy '
A
 f
1  

  X ,Y ( x ', y ') dx ' dy '
x y f
FX ,Y ( x, y )   
 X ,Y ( x ', y ') dx ' dy '
 2 FX ,Y ( x, y )
f X ,Y ( x, y ) 
x y
b b
P  a1 ,  X  b1 , a2  Y  b2    1  2 f X ,Y ( x ', y ') dx ' dy '
a1 a2
y  dy
P  x  X  x  dx, y  Y  y  dy    x  dx 
f X ,Y ( x ', y ') dx ' dy '
x
y
 f X ,Y ( x, y ) dx dy
tch-prob
13
Marginal pdf: obtained by integrating out the variables that are not of
interest.
f X ( x)   
 f X ,Y ( x, y ') dy '
fY ( y )   
 f X ,Y ( x ', y ) dx '


d x
    
 f X ,Y (x', y')dy' dx'
dx
marginal cdf
tch-prob
14
Ex. 4.10. A randomly selected point (X,Y) in the unit square has uniform
joint pdf given by
1 0  x  1 and 0  y  1
f X ,Y ( x, y )  
elsewhere.
0
Find
v
ii
iv
1
FX ,Y ( x, y).
i
tch-prob
iii
1
15
Ex. 4.11 Find the normalization constant c and the marginal pdf’s for
the following joint pdf:
ce x e y
f X ,Y ( x, y)  
 0
tch-prob
0 y x
elsewhere.
16
Ex. 4.12
Find P[ X  Y  1] in Example 4.11.
0
tch-prob
1
17
Ex. 4.13 The joint pdf of X and Y is
f X ,Y ( x, y) 
1
2 1  
( x2 2  xy  y2 )/2(1  2 )
2
e
   x, y  .
We say that X and Y are jointly Gaussian. Find the marginal pdf’s.
tch-prob
18
4.3 Independence of Two Random Variables
X and Y are independent random variables if any event A1 defined
in terms of X is independent of any event A2 defined in terms of Y;
P[ X in A1, Y in A2 ] = P[ X in A1 ] P[ Y in A2 ]
Suppose that X,Y are discrete random variables, and suppose we are
interested in the probability of the event A  A1 A2 , where A1
involves only X and A2 involves only Y.
“”If X and Y are independent, then A1 and A2 are independent
events.A  X  x and A  Y  y 
1
Let

j

2
k
p X ,Y ( x j , yk )  P  X  x j , Y  yk 
 P  X  x j  P Y  yk 
 p X ( x j ) pY ( yk ) for all x j and yk .
tch-prob
19
“”
If pX ,Y ( x j , yk )  pX ( x j ) pY ( yk ) for all x j and yk ,
then P  A 
 
p X ,Y ( x j , yk )
x j in A1 yk in A2
=
 
p X ( x j ) pY ( yk )
x j in A1 yk in A2
=

pX ( x j )
x j in A1

pY ( yk )
y k in A2
=P  A1  P  A2 
tch-prob
20
In general, X, Y are independent iff
FX ,Y ( x, y )  FX ( x) FY ( y )
or f X ,Y ( x, y)  f X ( x) fY ( y) if X , Y are jointly continuous.
If X and Y are independent r.v. ,then g(X) and h(Y) are also independent.
P  g ( X ) in A, h(Y ) in B  P  X in A', Y in B'
 P  X in A' P Y in B'
 P  g ( X ) in A  P  h(Y ) in B.
# A and A’ are equivalent events; B and B’ are equivalent events.
tch-prob
21
Ex.4.15 In the loaded dice experiment in Ex. 4.6, the tosses are not
independent.
Ex. 4.16 Q and R in Ex. 4.7 are independent.
P[Q  q, R  r ]  (1  p) p qM  r
P[Q  q]  (1  p M )( p M ) q , q  0,1, 2,...
P[ R  r ]  (1  p) p r /(1  p M ), r  0,1, 2,..., M  1.
Ex.4.17 X and Y in Ex. 4.11 are not independent, even though the joint
pdf appears to factor.
2e x e y 0  y  x  
f X ,Y ( x, y)  
elsewhere.
 0
f X ( x)  2e x (1  e x )
fY ( y)  2e2 y
tch-prob
22
4.4 Conditional Probability and Conditional Expectation
Many random variables of practical interest are not independent. We
are interested in the probability P[Y in A] given X=x?
P[Y in A, X  x]
P[Y in A X  x] 
P[ X  x]
conditional probability
A. If X is discrete, can obtain conditional cdf of Y given X=xk
FY ( y xk ) 
P[Y  y, X  xk ]
, for P  X  xk   0.
P[ X  xk ]
The conditional pdf, if the derivative exists, is
fY ( y xk ) 
d
FY ( y xk )
dy
P Y in A | X  xk   
y in A
fY ( y | xk )dy
tch-prob
23
If X and Y are independent
P[Y  y, X  xk ]  P[Y  y ]P[ X  xk ]
FY ( y x)  FY ( y )
fY ( y x)  fY ( y )
- If X and Y are discrete
PY ( y j xk ) 
P[ X  xk , Y  y j ]
P[ X  xk ]

PXY ( xk , y j )
PX ( xk )
If X and Y are independent
PY ( y j xk )  PY ( y j )
tch-prob
24
B. If X is continuous, P[ X = x] = 0
conditional cdf of Y given X = x
FY ( y x)
lim FY ( y | x  X  x  h)  lim
h0
h0
PY  y, x X  x  h
P x X  x h
y xh f ( x ', y ')dx 'dy ' y f ( x, y ')dy 'h
X ,Y
 lim  x
  X ,Y
xh f ( x ')dx '
f X ( x)  h
h0
x
X
y f ( x, y ')dy '
 X ,Y

f X ( x)
conditional pdf. fY ( y x) 
f X ,Y ( x, y)
f X ( x)
tch-prob
25
pX ,Y ( xk , y j )
pY ( y j xk ) 
 pX ,Y ( xk , y j )  pY ( y j xk ) pX ( xk ).
pX ( xk )
Discrete
continuous
discrete
f X ,Y ( x, y)
fY ( y x) 
 f X ,Y ( x, y)  fY ( y x) f X ( x)
f X ( x)
P[Y in A]  
 p X ,Y ( xk , y j )
all xk y j in A
 
 pY ( y j | xk ) pX ( xk )   p X ( xk )  pY ( y j | xk )
all xk y j in A
all xk
  P[Y in A X  xk ] p X ( xk )
all xk
continuous
y j in A
Theorem on total probability
 P[Y in A X  x] f (x)dx
P[Y in A]  
X
tch-prob
26
Ex 4.22. The total number of defects X on a chip is a
Poisson random variable with mean  . Suppose that
each defect has a probability p of falling in a specific
region R and that the location of each defect is
independent of the locations of all other defects. Find
the pmf of the number of defects Y that fall in the region R.
P[ X  k ] 
k
k!
e  ,
k  0,1, 2,...
 k  j
k j
p
(1

p
)

P[Y  j | X  k ]   j 

0

0 jk
jk
( p) j  p
P[Y  j ]   P[Y  j | X  k ]P[ X  k ]  ... 
e
j!
k 0

tch-prob
27
Ex. 4.23 The number of customers that arrive at a service station during
a time t is a Poisson random variable with parameter  t. The time
required to service each customer is an exponential random variable with
parameter  . Find the pmf for the number of customers N that arrive
during the service time T of a specific customer. Assume that the customer
arrivals are independent of the customer service time.
( t )k   t
P[ N  k | T  t ] 
e , k  0,1, 2,...
k!
fT (t )   e
 t
, t 0

P[ N  k ]   P[ N  k | T  t ] fT (t )dt
0
k
    


 , k  0,1, 2,...
        
tch-prob
28
Ex. 4.24 The random variable X is selected at random from
the unit interval; the random variable Y is then selected at
random from the interval (0, X ). Find the cdf of Y .
f X ( x)  1,
0  x 1
1/ x 0  y  x
fY |x ( y | x)  
 0 otherwise
y / x 0  y  x
P[Y  y | X  x]  
x y
 1
1
FY ( y)  P[Y  y]   P[Y  y | X  x] f X ( x)dx
0
y
FY ( y )   1dx '  dx '  y  y ln y
0
y x'
y
1
tch-prob
29
Conditional Expectation
The conditional expectation of Y given X=x is
 yf ( y x)dy
E[Y | x] 
Y
or
  y j pY ( y j x)
yj
if X,Y are discrete.
The conditional expectation E Y | x  can be viewed as defining
a function of x : g ( x)  E Y | x .
g ( x) can be used to define a random variable g ( X )  E Y | X .
What is E  g ( X )   E  E Y | X  ?
 E[Y | x] f ( x)dx
E  EY | X   
X
  EY | xk pX ( xk )
X continuous
X discrete
xk
tch-prob
30
E[ g ( X )]  E[ E[Y X ]]
 E[Y | x] f ( x)dx
 
X
  yf ( y x)dy f ( x)dx
 
 Y
X
 y  f ( x, y)dxdy
 
 XY
 yf ( y)dy
 
Y
 E[Y ]
can be generalized to E[h(Y )]  E[ E[h(Y ) X ]]
tch-prob
31
y
[XY]
[ 0,0 ]
0.1
[ 1,0 ] [ 1,1 ]
[ 2,0 ] [ 2,1 ] [ 2,2 ]
[ 3,0 ] [ 3,1 ] [ 3,2 ] [ 3,3 ]
1
p XY ( x, y ) 
10
0.4
0.3

pY ( y )  
0.2
0.1
E[Y] = 1
for 0  y  x  3
y0
1
2
3
·
· ·
· · ·
· · · ·
0.1
0.2

p X ( x)  
0.3
0.4
x
x0
1
2
3
E[X] = 2.0
tch-prob
32
1

pY ( y x  2)   3
0
1

pY ( y x  1)   2
0
y  0,1,2
y
·
· ·
· · ·
· · · ·
y  0,1
x
1
3
E[Y x  3]  (0  1  2  3) 
4
2
1
1
1
E[Y x  2]  .0  .1  .2  1
3
3
3
1
1
1
E[Y x  1]  .0  .1 
2
2
2
E[Y x  0]  1.0  0
3
1
E[Y ]   0.4  1 0.3   0.2  0  0.1
2
2
 0.6  0.3  0.1  1.0
tch-prob
33
Ex. 4.25 Find the mean of Y in Ex. 4.22 using conditional expectation.


E Y    E Y | X  k P  X  k  
 kpP  X  k 
k 0
k 0
 pE[ X ]  p
Ex. 4.26 Find the mean and variance of the number of customer arrivals N
during the service time T of a specific customer in Ex. 4.23.
E N | T  t  t
E  N 2 | T  t    t  ( t )2

EN  
E[ N | T  t ] fT (t )dt 
E  N  
2

0


0


0
 tfT (t )dt   E[T ]   / 

E[ N | T  t ] fT (t )dt   (  t   2t 2 ) fT (t )dt
2
0
  E[T ]   2 E[T 2 ]   /   2 2 /  2
tch-prob
34
4.5 Multiple Random Variables
Extend the methods for specifying probabilities of pairs of random variables
to the case of n random variables.
The joint cdf of X1, X 2, is defined as
FX , X ,, X ( x1, x2, , xn )  P  X1  x1, X 2  x2,..., X n  xn 
1
2
n
We say that X 1 , X 2 ,..., X n

are jointly continuous random variables if

P   X ,........X n  in A   X in A  f X , X ,, X ( x1 ', x2 ', , xn ')dx1 ' dx2 ' dxn '
1 2
n

 1

where f X , X ,, X n ( x1 ', x2 ', , xn ') is the joint pdf function.
1
2
The joint cdf is given by
FX , X ,, X ( x1, x2, , xn )   x  x f X , X ,, X ( x1 ', x2 ', , xn ')dx1 ' dx2 ' dxn '
1 2
n
1 2
n
1

n

tch-prob
35
The joint pdf, if it exists, is given by
n

f X , X ,, X ( x1, x2, , xn ) 
F
( x , x , , xn )
x1 xn X1, X 2 ,, X n 1 2
1 2
n
The marginal pdf of X1 is

 f
f X ( x1)  
( x1, x2 ', , xn ')dx2 ' dxn '


X
,
X
,

,
X
1
1 2
n
The marginal pdf for X1, X 2, , X n1 is
fX
1, X 2 ,, X n 1

( x1, x2, , xn1)  
f X , X ,, X ( x1, x2 , , xn1, xn ')dxn '
1 2
n
The conditional pdf of X n given the values of X1, , X n1 is
f X ,, X ( x1, , xn )
1
n
if f
( x1, , xn )  0
f X ( xn x1, , xn 1) 
X
,
,
X
f X ,, X ( x1, , xn1)
n
1
n1
n 1
1
Repeated applications of above
f X , X ,, X (x1, x2, , xn )  f X (xn x1, , xn1) f X (xn1 x1, , xn2 ) f X (x2 x1) f X (x1).
1 2
n
n1
n
tch-prob
2
1
36
EX. 4.29 Random variables X1, X 2 , and X 3 have joint Gaussain pdf
( x12 x22 2 x1x2  x32 /2)
f X , X , X ( x1, x2 , x3)  e
.
2


1 2 3
Find the marginal pdf of X1 and X 3.
1 x2
( x2 x2 2 x x )
3
2
12
 e 1 2
f X , X ( x1, x3)  e
dx

2
1 3
2
2 2
[( x  1 x )21 x2]
2 2 1 21

e
e

dx

2
2
2 2
( x  1 x )2
1 x2 1 x2
2 3 e 21  e 2 2 1
e


dx

2
2
2
2 2
1 x2 1 x2
1 x12 1 x32
( x' x )2 2
21 e 2 3  e
1
'  e 2 e 2
e

dx

2
2 
2
2
2
1
 x32
2
X1 and X3 are independent zero-mean, unit-variance Gaussian r.v.s.
tch-prob
37
Independence
X1, X 2 ,, X n are independent if and only if
FX , X ,, X ( x1, x2, , xn )  FX ( x1) FX ( x2 ) FX ( xn )
1
2
or f X
n
1, X 2 ,, X n
1
2
( x1, x2, , xn )  f X ( x1)
p( x1, , xn )  pX ( x1)
1
1
pX ( xn )
n
f X ( xn ) if continuous
n
if discrete
n
tch-prob
38
4.6 Functions of Several Random Variables
Quite often we are interested in one or more functions of random
variables involved with some experiment. For example, sum, maximum or
minimum of X1, X2, …,Xn.
Let random variable Z be defined as
Z  g ( X1, X 2, , X n ).
The cdf of Z
FZ ( z)  P[Z  z]
 P[{x  ( x1, , xn ) such that g(x)  z)}]
 x in eqv.event  f X ,, X ( x1' , , xn' )dx1' dxn'
1
n
pdf f Z ( z)  d FZ ( z)
dz
tch-prob
39
Example 4.31
y
Z=X+Y
FZ ( z)  P[Z  z]
 P[ X  Y  z]
z-x' f ( x', y')dy'dx'
 -
 - X ,Y
 f ( x', z  x')dx'
f Z ( z)  d FZ ( z)  
X ,Y
dz
y=-x+z
x
Superposition integral
If X and Y are independent r.v.,
 f (x ') f (z  x ')dx '
f Z (z)  
X
Y
tch-prob
convolution integral
40
Example 4.32 Sum of Non-Independent r.v.s
Z=X+Y , X,Y zero-mean, unit-variance with
correlation coefficient    1
2
f X ,Y ( x, y) 
1
2 1  2
22 xy y2) 2(1 2)

(
x
e
-  x, y  
 f ( x ', z  x ')dx '
f Z ( z)  
X ,Y

1
2 1-
 e[ x'2 2 x'( zx')( zx')2]/2(1 2)dx '

2 
x'2  2 x'( z  x')  ( z  x ')2  x'2  2 x'z  2 x'2  z 2  2zx'  x'2
 2(1  ) x'2  2(1  ) x'z  z 2  2(1  )[ x'  z ]2  1  z 2  z 2
2
2
 2(1  )[ x'  z ]2  1  z2
2
2
tch-prob
41
 (1 ) z2
( x' z )2 (1 )
4(1 2)

e
2
f Z ( z) 
dx'
 e
2 1  2
 z2
' z )2 21

(
x
4(1


)

2
2 dx'
e
 e
2 1  2
 z2
4(1  )
e


2 1   2
1
( x' z )2 21
2
2 dx'
e
1 
2
 z2
e 22(1  )
2 
 z2
4(1  )
 e

2 1   2
2 2(1  )
 -1
2

e z2 2
2
Sum of these two non-independent Gaussian r.v.s is also a Gaussian r.v.
tch-prob
42
Ex.4.33 A system with standby redundancy.
Let T1 and T2 be the lifetimes of the
two components. They are independent
exponentially distributed with the same
mean.
T1
T2
The system lifetime is T  T1  T2

 x
e
fT ( x)  
1
0


 x
e
fT ( x)  
2
0

x0
x0
x0
x0

 ( z x)
e
fT ( z  x)  
2
0

xz
xz
f ( z)  0z  e x  e ( z  x) dx
T
  2e z 0z dx   2 ze z
Erlang m=2
tch-prob
43
The conditional pdf can be used to find the pdf of a function of
several random variables.
Let Z = g (X,Y).
Given Y = y,
Z = g (X,y) is a function of one r.v. X.
Can first find f Z ( z Y  y ) from f X (x )
 f (z y ') f ( y ')dy '
then find f Z (z)  
Z
Y
tch-prob
44
Example 4.34 Z = X/Y
X,Y indep., exponentially distributed with mean one.
Assume Y = y, Z = X/y is a scaled version of X
f Z ( z y)  f X ( x y) x yz  dx
dz
 y f X ( yz y)
 y ' f ( y ' z y ') f ( y ')dy '   y ' f ( y ' z, y ')dy '
f Z ( z)  

X
Y
X ,Y
 0 y ' f X ( y ' z) fY ( y ')dy '  0 y 'e y'ze y'dy '
y'
 0 y 'e y'(1 z)dy '
dy'

y
'

y
'(1

z
)

y
'(1

z
)
1


e
e
dy '
0 1 z 0
(1 z)
 1  1 e y'(1 z) 
0
1 z 1 z
 1
z0
2
(1 z)
tch-prob
'
e y (1 z)dy'
'
 1 e y (1 z)
(1 z)
45
y
x
y
z
x  yz
if y  o
x  yz
if y  0
x  yz
y
x  yz
(z>0)
(z<0)
x  yz
x  yz
x
x
y  yz
yz f (x,y)dxdy   f (x,y)dxdy
FZ (z)   
  yz X ,Y
X ,Y
yz f (yz,y)d(yz)dy   f (yz,y)d(yz)dy
  
  yz X ,Y
X ,Y
0 yf (yz,y)dy
f Z (z)  0 yf X ,Y (yz,y)dy  
X ,Y
 y f (yz,y)dy
 
X ,Y

0
0


0
0

tch-prob
46
f X ,Y (x,y)  f X ,Y (  x, y)
yz

FZ (z)       yz f X ,Y (x, y)dxdy
yz  yz f (  x, y)dxdy
  
  X ,Y
yz
 2  f X ,Y (x,y)dxdy
if


0
0


0
0

0
tch-prob
47
Ex. Z  min( X , Y )
FZ (z)  FX ( z )  FY ( z )  FX ,Y ( z, z )
If X , Y are independent,
FZ (z)  FX ( z )  FY ( z )  FX ( z) FY ( z)
f Z ( z )  f X ( z )  fY ( z )  f X ( z ) FY ( z )  FX ( z ) fY ( z )
(z, z)
tch-prob
48
Ex. Z  max( X , Y )
(z, z)
FZ (z)  FX ,Y ( z, z )
If X , Y are independent,
FZ (z)  FX ( z) FY ( z)
f Z ( z )  f X ( z ) FY ( z )  FX ( z ) fY ( z )
tch-prob
49
Transformation of Random Vectors
Z1  g1( X1, X 2 , , X n )
Z2  g2 ( X1, X 2 , , X n )
Zn  gn ( X1, X 2 , , X n )
Joint cdf of ( Z , Z ,  , Z n )
1 2
FZ ,
1
,Zn
( z1, , zn )  P[ g1(X)  z1, , gn (X)  zn ]
tch-prob
50
Example 4.35 W = min (X,Y) , Z = max (X,Y)
FW ,Z (w, z)  P[{min( X ,Y )  w} {max( X ,Y )  z}]
If z>w
FW ,Z (w, z)  FX ,Y ( z, z) {FX ,Y ( z, z)  FX ,Y (w, z)  FX ,Y ( z, w)  FX ,Y (w, w)}
y
 FX ,Y (w, z)  FX ,Y ( z, w)  FX ,Y (w, w)
(z,z)
If z<w
(w,w)
FW ,Z (w, z)  FX ,Y ( z, z)
min( X ,Y )  z
max(X ,Y )  z
x
min(X ,Y )  z
tch-prob
max(X ,Y )  z
51
pdf of Linear Transformation
Linear Transformation
V=aX+bY
W=cX+eY
assume
V  a b   X 
W   c e  Y  
  
 
A  ae  bc  0
x 
v 

1
 y   A  w
 
 
y
(x,y+dy)
X 
A 
Y 
w
(v+adx+bdy,w+cdx+edy)
(v+bdy,w+edy)
(x+dx,y+dy)
dP
(v,w)
(x,y)
(v+adx,w+cdx)
(x+dx,y)
v
x
Equivalent event
tch-prob
52
f X ,Y ( x, y)dxdy
fV ,W (v, w)dP
f X ,Y ( x, y)
fV ,W (v, w) 
dP
dxdy
stretch factor
dP = ?
v1  v2
(c,e)
(a,b)
V2
b
a

(a,b)
v
1
v2
sin
 ae  bc
V1
tch-prob
53


a
b

  Projection of (c, e) on (a, b)
(c, e)  
,

 a2  b2 a2  b2 
(-b, a)  (a, b)  0 Perpendicu lar

b
a
,
 a2  b2 a2  b2
(c, e)  


 cdx
adx


h  (bdy, edy )  
,

 c 2d 2 x  a 2d 2 x a 2d 2 x  c 2d 2 x 
(bdy,edy)
h
(adx,cdx)
o

  Projection of (c, e) on (-b, a)


dP  h  a 2 d 2 x  c 2 d 2 x
 ( ae  bc ) dxdy
dP  ae  bc  A  a b
dxdy
c e
fV ,W (v, w) 
f X ,Y ( x(v, w), y(v, w))
A
f X ( A1Z)
For n-dimensional vector Z  AX, f Z ( z) 
A
tch-prob
54
Example 4.36 X,Y jointly Gaussian
f
X ,Y
( x, y) 
1
( x22 xy y2)
2(1 2)
e
2 1  2
 V  1 1 1  X 
X 


A
W 
Y 

 
-1
1
Y
2



 
 
 
A 1

 
1 1 -1 V 
2 1 1  W 
X  (V W )
2
X


Y




Y  (V W )
tch-prob
2
55
vw vw
,
)
X ,Y
2
2
(v, w) 
f
f
V ,W



(
1
2 1 - 
2
1
2 1 - 
2
1
2


[( vw)2 2  (vw)(v w)  v w  ] 2(1  2 )
2
2
2  2
e
e
1
2 1  

2
2 
 v
 w 
 2(1  ) 2(1  ) 
e
2
 v
2(1  )

1
2 1  
e
2
 w
2(1  )
V, W are independent , zero mean, Gaussian r.v.s with variance 1   ,
and 1   , respectively.
see Fig 4-16 Contours of equal
value of the joint pdf of XY
tch-prob
56
Pdf of General Transformation
V  g1 ( X , Y )
W  g2 ( X , Y )
Assume that v( x, y ) and w( x, y ) are invertiable, i.e.,
v  g1 ( x, y )
w  g 2 ( x, y )
x  h1 (v, w)
invertible

y  h2 (v, w)
Fig 4.17a
g1 ( x  dx, y )  g1 ( x, y ) 
g1
dx
x
g
 v  1 dx
x
v
 v  dx
x
g 2 ( x  dx, y )  w 
v
a
x
v
b
y
w
c
x
w
e
y
w
dx
x
tch-prob
57
y
y
(g1(x+dx,y+dy),g2(x+dx,y+dy))
(x,y+dy)
(g1(x,y+dy),g2(x,y+dy)
(x+dx,y+dy)
(g1(x+dx,y),g2(x+dx,y))
(x,y)
(x+dx,y)
(g1(x,y),g2(x,y))
x
x
(v 
(v 
g1
g
dy, w  2 dy )
y
y
(v 
v  g1 ( x, y )
w  g 2 ( x, y )
g1
g
g
g
dx  1 dy , w  2 dx  2 dy )
x
y
x
y
g1
g
dx, w  2 dx )
x
x
(v, w)
tch-prob
58
fV ,W (v, w) 
f X ,Y (h1 (v, w), h2 (v, w))
v v
x y
w w
x y
Can be shown that

Jacobian of the
Inverse Transformation
x
v
y
v

Jacobian of the transformation
x  v

w  x

 w
y

w  x
v
y
w
y






1
x
v
fV ,W (v, w)  f X ,Y (h1 (v, w), h2 (v, w))
y
v
tch-prob
x
w
y
w
59
Example 4.37 X,Y zero mean , unit-variance , indep. Gaussian r.v.s
1
2
2
V  (X Y ) 2
radius
W  ( X , Y )
x  v cos w,
x
v
y
v
angle
in (0,2 )
y  v sin w
x
w cos w

y
sin w
w
 v sin w
v
v cos w
tch-prob
60
1  x2 2 1  y2 2
fV ,W (v, w) 
e

e
v
2
2
v v2 cos2 w 2 v2 sin2 w 2

e
e
2
2
1 v 2

ve
Rayleigh
2

V,W independent
v0
0  w  2
uniform
 fW (w) fV (v)
Linear transformation method can be used even if we are interested in only
one function of random variables.
-by defining an “auxiliary” r.v.
tch-prob
61
Ex. 4.38 X: zero-mean , unit-variance Gaussian
Y: Chi-square r.v. with n degrees of freedom
X and Y are independent
X
V

find pdf of
Y
n
Let W=Y, then
X V W
n
Y W
x
v
y
v
x
w

y
w
w
 w
n
0
v
2 wn
1
n
tch-prob
62
n 1  y
y ) 2 e 2
2
(
e
f X ,Y ( x, y ) 
 2
2
2 ( n )
2
 x2
n 1 w
2n ( w ) 2 e 2
e
fV ,W ( v, w) 
 2
 w
n
2
2 ( n )
2
n1 [ w (1v2 )]
n
2
(w ) 2 e
2

2 n  ( n )
2
v 2 w
tch-prob
63






2  
w
v

n1  1  
2  n  
1
 w  2

fV (v) 
e
dw


0 
n
2 nπ Γ(
) 2
2
w  v2 
'
Let w  1  
2
n
 n1
 v2  2
1 

n1


n


2 e w'dw '
fV (v)  
(w
'
)

0
nπ Γ( n )
2
 n1
 v2  2
 n 1 
Γ
1 




n
2





nπ Γ( n )
2
tch-prob
Student's t - distribution
64
4.7 Expected Value of Function of Random Variables
Z=g(X,Y)
 

 g ( x, y) f X ,Y ( x, y)dxdy X , Y jointly continuous

E[ Z ]  
X , Y discrete
  g ( xi , yn ) pX ,Y ( xi , yn )
i
n

Ex. 4.39 Z=X+Y
E[ Z ]  E[ X  Y ]
 
   ( x ' y ') f X ,Y ( x ', y ')dx ' dy '
 
 
   x ' f X ,Y ( x ', y ')dy ' dx '    y ' f X ,Y ( x ', y ')dx ' dy '


  x ' f X ( x ') dx '   y ' fY ( y ') dy '
 E[ X ]  E[Y ]
X, Y need not be independent
In general, E[ X1  X 2    X n ]  E[ X1]    E[ X n ]
tch-prob
65
Ex. 4.40. X,Y independent r.v.s and let
g ( X , Y )  g1 ( X ) g 2 (Y )
E[ g ( X , Y )]  E[ g1 ( X ) g 2 (Y )]
 
   g1 ( x ') g 2 ( y ') f X ( x ') fY ( y ')dx ' dy '


  g1 ( x ') f X ( x ')dx '   g 2 ( y ') fY ( y ')dy '
 E[ g1 ( X )]  E[ g 2 (Y )]
The jkth joint moment of X and Y is
  j k
 
 x y f X ,Y ( x, y )dxdy

j k
E[ X Y ]  
j k
x
i yn p X ,Y ( xi , yn )
 
i
n
X , Y jointly continuous
X , Y discrete
when j=1 , k=1
E[XY]: the correlation of X and Y
If E[XY]=0 , then X and Y are orthogonal.
tch-prob
66
The jkth central moment of X and Y
j
E [( X  E [ X ]) (Y  E [Y ]) k ]
When j=1 , k=1
E[(X-E[X])(Y-E[Y])]=COV(X,Y) covariance of X and Y
COV(X,Y)=E[XY-XE[Y]-YE[X]+E[X]E[Y]]
=E[XY]-2E[X]E[Y]+E[X]E[Y]
=E[XY]-E[X]E[Y]
Ex. 4.41. X,Y independent
COV(X,Y)=E[(X-E[X])(Y-E[Y])]
=E[X-E[X]]E[Y-E[Y]]
=0
tch-prob
67
The correlation coefficient of X and Y
 XY 
COV ( X , Y )
 XY

E[ XY ]  E[ X ]E[Y ]
 XY
where  X  Var ( X ) ,  Y  Var (Y ) are the standard deviation of X and Y .
1   X ,Y  1.
 X  E[ X ] Y  E[Y ] 2 
pf : 0  E 

 


X
Y
 

 1  2  X ,Y  1  2(1   X ,Y )
X,Y are uncorrelated if  XY  0
If X,Y are independent , then COV(X,Y)=0 ,  XY  0 , X, Y uncorrelated.
X,Y uncorrelated does not necessarily imply X,Y are independent.
tch-prob
68
X,Y uncorrelated does not necessarily imply X,Y are independent.
Ex. 4.42  : uniformly distributed in (0,2 )
X =cos and Y =sin
tch-prob
69
Joint characteristic Function
 X1 , X 2 ,..., X n (1 , 2 ,
, n )  E  e j (1 X1 2 X 2 n X n ) 
Consider the case n=2:
 X ,Y (1 , 2 )  E  e j (1 X 2Y ) 
j ( x2 y )
 
   f X ,Y ( x, y)e 1
dxdy
If X and Y are independent r.v.s
 X ,Y (1 , 2 )  E e j (1 X 2Y )   E e j1 X e j2Y 
 E e j1 X  E e j2Y    X (1 )Y (2 )
tch-prob
70
If Z=aX+bY
 Z ( )  E[e
j (aX bY )
]  E[e
j (aX bY )
]   X ,Y (a, b )
If X , Y are independent,  Z ( w)   X (a )  Y (b )
The ikth joint moment of X , Y is
E[ X iY k ] 
1
j ik
i
k


 X ,Y (1 , 2 )  0, 0
i
k
1
2
1 2
tch-prob
71
4.8 Jointly Gaussian Random Variables
X,Y are said to be jointly Gaussian if

1

exp 
2 1  ρX2 ,Y


f X ,Y  x, y  


2
 x  m  2






x

m
y

m
y

m

1
1
2
2



2
ρ



 
 
X ,Y 
σ
σ
σ
σ
 1 
2
2
 1 
 
  

2 1σ 2 1  ρX2 ,Y
   x  ,   y  
Contours of constant pdf



 xm1 2
xm1

2

 1
X ,Y
1


y m2
2
 2 X ,Y1 2 
1
  arctan  2 2 
2
 1  2 
tch-prob
 

y m2 2 
 
  constant
2


72
Marginal p.d.f.
 x  m1  / 212
fX  x  e
2
21
 x  m2  / 2 22
2
fY  y   e
Conditional pdf
fX  x | y 
f X ,Y  x, y 
fY  y 
2 2
2 

1

1


exp  
x   X ,Y   y  m2   m1 
2  2 
2
 
2
1


X ,Y  1
 


2 12 1   X2 ,Y



is Gaussian with conditional mean m1   X ,Y  1  y  m2 


2
and conditional variance  12 1   X2 ,Y .
If X ,Y  0, f X  x | y   f X  x  X , Y are independent.
* For X , Y jointly Gaussian, X , Y uncorrelated  X , Y independent.
tch-prob
73
We now show that  X ,Y is indeed the correlation coefficient.
Cov  X ,Y     X  m1 Y  m2 
    X  m1 Y  m2  | Y   .
   X  m1  Y  m2  | Y  y 
  y  m2    X  m1 | Y  y 
  y  m2     X | Y  y   m1 


  y  m2   X ,Y  1  y  m2 
2


  X  m1 Y  m2  | Y    X ,Y  1 Y  m2 
2
2
2

Cov( X , Y )     X ,Y  1 Y  m2  

2

2

  X ,Y  1  Y  m2  
2
  X ,Y 1 2
 X ,Y 
tch-prob
C ov  X ,Y 
1 2
Correlation
Coefficient
74
n jointly Gaussian Random Variables
X1, X 2 ,..., X n are jointly Gaussian if


T
1
exp   x  m  K 1  x  m 
2
f X  x   f X1 , X 2 ,..., X n  x1 , x2 ,..., xn  
n/2
1/ 2
2

K
 
 x1 
 m1   E  X 1  
x 
m   E X 
 2  and K is the covariance matrix
2
where x    , m   2   

 
  

 
  
 xn 
 mn   E  X n 
COV ( X 1 , X 2 )
 VAR( X 1 )
COV ( X , X )
VAR ( X 2 )
2
1
K 


COV ( X n , X 1 ) COV ( X n , X 2 )
tch-prob
COV ( X 1 , X n ) 
COV ( X 2 , X n ) 


VAR ( X n ) 
75
The pdf of the jointly Gaussian random variables is completely
specified by the individual means and variances and the pairwise
covariances.
Ex. 4.46 Verify that (4.83) becomes (4.79) when n=2.
Ex. 4.48
X1 , X 2 ,..., X n are jointly Gaussian.
If COV ( X i , X j )  0, i  j,  X1 , X 2 ,..., X nindependent.
tch-prob
76
Linear Transformation of Gaussian Random Variables
 x1 
 y1 
x 
y 
Let X   2  be jointly Gaussian, and define Y   2  by Y =AX,
 
 
 
 
x
 n 
 yn 
fY  y  


f X A1y

A


exp  1 A1y  m
2
 2 
n/2

T

K 1 A1y  m
A K

1/ 2
From elementary properties of matrices,
tch-prob
A
A
1
1

y  m    y  Am 
y  m  A1  y  Am 
T
T
A
1T
77


T
1T
1
exp   y  Am  A K 1 A1  y  Am 
2
fY  y  
n/2
1/ 2
 2  A K
1T
Since A
1
1

K A  AKA
T

1


det  C   det AKAT  det  A det  K 
let C  AKA , n  Am.
T
fY  y  
,
2
1
 (y n)T C 1 (y n)
e 2
 2 
n/2
C
1/ 2
Thus, Y jointly Gaussian with mean n and covariance matrix C.
tch-prob
78
If we can find a A s.t. AKAT   , a diagonal matrix
f Y (y ) 
 1  y n T 1 y n 
e 2
 2 

n/2

2
 1 n

exp    yi  ni  / i 
 2 i 1

1/ 2
 21  22  .....  2n  
1  y  n 2 / 
n exp 
i
2 i i

1/ 2
2

i 1
 i
1/ 2


Y1, Y2 ,......Yn are independent.
If we can select matrix A that diagonalize K with A  1,
then the linear transformation corresponds to a rotation.
tch-prob
79
Ex. 4.49
 2 X ,Y1 2 
1
  arctan  2 2 
2
 1  2 
V   cos 
W     sin 
  
sin    X 
cos    Y 
Cov V,W   Ε V  Ε V   W  Ε W   
 E  X  m1  cos   Y  m2  sin    X  m1  sin   Y  m2  cos  
E V   m1 cos   m2 sin 

E W   m1 sin   m2 cos 
tch-prob
80
Ex.4.50
X1, X 2 ,..., X n are jointly Gaussian.
Z  a1 X1  a2 X 2  ....  an X n
Let Z2  X 2 , Z3  X 3 ,..., Zn  X n .
Define Z   Z , Z2 ,..., Zn  , then
Z  AX
 a1 a2
0 1
where A  


0 0
an 
0 
.


1
is jointly Gaussian with mean n  Am
and covariance matrix C  AKAT
n
E  Z   n1   ai E  X i 
i 1
n n
VAR  Z   C11    ai a j COV  X i , X j 
i 1 j 1
tch-prob
81
Joint Characteristic Function of n jointly Gaussian random variables
X1 , X 2 ,..... X n is
 X1 , X 2 ,.... X n
n
n n
j  i mi  1   
i k COV  X i , X k 
2
i1k 1
1 , 2 ,..., n   e i1
1
jωT m ωT Kω
2
e
tch-prob
82
4.9 Mean Square Estimation
We are interested in estimating the value of an inaccessible random variable Y
in terms of the observation of an accessible random variable X.
The estimate for Y is given by a function of X, g(X).
1. Estimating a r.v. Y by a constant a so that the mean square error (m.s.e) is
minimized :
 
2
min  Y  a     Y 2  2aY   a 2

a 
2
d min  Y  a  

a 
 2Y   2a  0
da
a  Y 
2


m.s.e.    Y  a
  VAR Y 




tch-prob
83
2. Estimating Y by g(X) = a X + b
2
min  Y  aX  b  

a,b 
best b is b   Y  aX    Y   a  X 

best a is by min   Y   Y   a  X    X 
a 

2



Differentiate w.r.t. a
 2Y  Y   a X  X  X  X 
 2  COV  X , Y   aVAR  X    0
Y
COV  X , Y 

a 
  X ,Y
VAR  X 
X
tch-prob
84
Minimum mean square error (mmse) linear estimator for Y

Y  a X  b
 X  EX 
  X ,Y  Y 
  E Y 
X


Zero-mean, unit-variance version of X
In deriving a*, we obtain




 X  X
Y


Y

a


 




error of the best linear estimator

 X    X    0

observation 
Orthogonality condition
tch-prob
85
Mean square error of best linear estimator

E  Y   Y    a  X    X 



 E  Y  Y 


2


  a  X   X    Y  E Y  

aE  Y  Y    a  X   X 

2 

  Y   Y    a  X    X 

 VAR Y  aCOV X , Y

   X   X  

Y


Y





 
 

 VAR Y    X ,Y  Y  X  Y  X ,Y
X

 VAR Y  1   X2 ,Y

tch-prob
86
3. Best mmse estimator of Y is in general a non-linear function of X, g(X)
2

min  Y  g  X   
g .


2
2
 Y  g  X      Y  g  X  | X 



 

2
   Y  g  X  | X  x  f X x dx

 

constant when X  x


The constant that minimizes E  Y  g  X  | X  x 
2

is
g  x   E Y | X  x
Regression curve
Y | X  is the estimator for Y in terms of X that yields the smallest m.s.e.
tch-prob
87
2
Ex. 4.51 Let X be uniformly distributed in (-1,1) and let Y=X .
Find the best linear estimator and best estimator of Y in terms of X.
Ex. 4.52 Find the mmse estimator of Y in terms of X when X and Y are
jointly Gaussian random variables.
tch-prob
88