Transcript Lecture 5-dpdate

Operations on Multiple Random Variables
Previously we discussed operations on one Random Variable:
Expected Value
Moment
Central moment
  xf (x)dx
  X
E X = X =  N
  x i P(x i )
 i=1
Continuous
  x n f (x)dx
X



m n = E  X n  =  N
  x in P(x i )
 i=1
Discrete
Continuous
Discrete
  (x  X) n f (x)dx
X



μ n = E (X  X) n  =  N
  (x in  X) n P(x i )
 i=1
Continuous
Discrete
Characteristic Function
ΦX (ω) = E e
jωX
 =
Moment Generation


jωx
f (x)e dx
 X
1 
 jωx
f X (x) =
Φ
(ω)e
dω
X

2π 
n
d
Φ X (ω)
n
m n = (  j)
dωn ω = 0
Function of Random Variable
Nonmonotone Transformation
Monotone Transformation
1
dT (y)
f Y (y) = f X T (y) 
dy
1
f Y (y) = 
n
f X (x n )
dT(x)
dx x = x n
EXPECTED VALUE OF A FUNCTION OF Multiple RANDOM
VARIABLES
Two Random Variables
   g(x,y)f (x,y)dxdy
X,Y
  
g = E  g(X,Y) = 
  g(x i ,y k )PX,Y (x i ,y k )
 i k
Continuous
Discrete
N Random Variables
g = E g(X1 ,…,X n ) =
EX 5.1-1






g(x1 ,…,x n )f X1 ,…,Xn (x1 ,…,x n )dx1
dx n
Joint Moment about the Origin
mnk = E  X Y  =
n
mn0 = E[X n ]
k
m0k = E[Y ]
n+k
k

 

 
x n yk f X,Y (x,y)dxdy
the nth moment mn of the one random variable X
the kth moment mk of the one random variable Y
is called the order of the moments
Thus m02 , m20 and m11 are all 2ed order moments of X and Y
The first order moments m10 = E[X]  X and m01 = E[Y]  Y
are the expected values of X and Y and are the coordinates of
the center of gravity of the function fXY(x,y)
The second-order moment m11 = E[XY] is called the correlation
of X and Y and given the symbol RXY hence
R XY = m11 = E[XY] = 



 
xyf X,Y (x,y)dxdy
If the correlation can be written as
R XY = E[X]E[Y]
Then the random variables X and Y are said to be uncorrelated.
Statistically independence of X and Y → X and Y are uncorrelated
The converse is not true in general
R XY = m11 = E[XY] =


  xyf X,Y (x,y)dxdy

f X,Y (x,y) = f X (x)f Y (y)


R XY   xf X (x)dx  yf Y (y)dy = E[X]E[Y]





 
xyf X (x)f Y (y)dxdy
Uncorrelated of X and Y does not imply that X and Y are
Statistically independent in general, except for the Gaussian random
variables as will be shown later
If RXY = 0 then the random variables X and Y are called orthogonal.
For N random variables X1, X2, …,XN the (n1 + n2 + … nN) order
joint moments are defined by
mn n
= E X1 1X22 ...X NN 
n
...n N
1 2
=


n

 
...
n

n
n
X1 1...X NN f X ,...,X (x1,...,x N )dx1...dx N
1
N
where n1,n2 ,… nN are all integers 0, 1, 2,
EX 5.1-2
Joint Central Moments
We define the joint central moments for two random variables X
and Y as follows
μnk =
E (X


X)n (Y

Y)k 

=



 
(x  X)n (y  Y)kf X,Y (x,y)dxdy
The second-order ( n + k =2) central moments are
μ 20 = E (X  X)2  = σ2X
The variance of X
μ02 = E (Y  Y)2  = σ2Y
The variance of Y




The second-order joint moment µ11 is very important. It is called the
covariance of X and Y and is given the symbol CXY
The second-order joint moment µ11 is very important. It is called the
covariance of X and Y and is given the symbol CXY
CXY = μ11 =
E (X

 X)(Y 
Y)

=



 
(x  X)(y  Y)f X,Y (x,y)dxdy
By direct expansion of the product (X  X)(Y  Y)
we get
CXY = μ11 = E (XY  XY  XY  XY)


 E[XY]  E[X]Y  XE[Y]  XY = E[XY]  XY  XY  XY
 CXY = R XY  XY= R XY  E[X]E[Y]
Note on the 1-D , the variance was defined as
 X = μ2 = E[(X  X) ] = E[X2 ]  X2
2
Which can be written as
 X = μ2 = E[(X  X)(X  X)] = E[XX]  XX
CXY = R XY  XY= R XY  E[X]E[Y]

0

CXY  

E[X]E[Y]
CXY = 0
if
if X and Y are independents
if X and Y are orthogonal
X=0 or Y=0
The normalized second order-moments defined as
is known as the correlation coefficient of X and Y
It can be shown that 1 ≤ r ≤ 1
CXY
ρ=
σXσY
For N random variables X1, X2, …,XN the (n1 + n2 + … nN) order
joint central moments are defined by
μn n
= E (X1  X1) 1(X2  X2 )
n
...n N
1 2





n2
... (X N  X N )
nN 


=  ...  (x1  X1) 1 ...(x N  X N ) N f X ,...,X (x1,..., x N )dx1 ... dx N
n
n
1
N
Problem 5.1-1
Random variables X and Y have the joint density
1
0 < x < 6 and 0 < y <4

f X ,Y ( x, y )   24
 0
elsewhere
What is the expected value of the function g(X , Y )=(XY ) 2 ?
Solution
y  4 x 6
 
 1 
E[(XY ) ]    (xy) f X ,Y ( x, y)dxdy =   x y   dxdy  64
 24 
 
y 0 x 0
2
2
2
2
Problem 5.1-3
Random variables X and Y have the joint density
f X ,Y ( x, y )  u ( x)u ( y )16e 4( x  y )
Find the mean value (Expected) of the function g(X , Y )
1
1

5
0
<
x

and
0
<
y


2
2

1
1

g(X , Y )  1
< x and/or
<y
2
2

all other x and y
0


f X ,Y ( x, y)  u( x)u( y)16e4( x y )
1
1

 5 0 < x  2 and 0 < y  2

1
1

g(X , Y )  1
< x and/or
<y
2
2

all other x and y
0


Solution
 
E[g(X , Y )] 
  g(x, y) f
X ,Y
( x, y) dxdy
 
y
=
1
1
x
2
2
  5 16e
y 0 x 0
y 
4( x  y )
x
1
2
 dxdy    (1) 16e
y
1 x 0
2
4( x  y )
 dxdy  3.486
f X ,Y ( x, y)  u( x)u( y)16e4( x y )
f X ,Y ( x, y )  0
1
1

5
0
<
x

and
0
<
y


2
2

1
1

g(X , Y )  1
< x and/or
<y
2
2

all other x and y
0


f X ,Y ( x, y )  0
f X ,Y ( x, y )  0
Solution
 
E[g(X , Y )] 
  g(x, y) f
X ,Y
( x, y) dxdy
 
1
1
y  x
2
2
=
 
y 0 x 0
5 16e4( x  y )  dxdy
y 
x
1
2
   (1) 16e
y
4( x  y )
 dxdy
1 x 0
2
y  x 
  
y 0 x  1
2
(1) 16e 4( x  y )  dxdy
 3.486
Problem 5.1-19
Two Random variables X and Y have the joint density
2
2
 ( x  0.5 y )
f X ,Y ( x, y )   43

0
0 < x < 2 and 0 < y <3
elsewhere
(a) Find all the First and second order moments ?
(b) Find the covariance ?
(c) Are X and Y uncorrelated ?
JOINT CHARACTERISTIC FUNTIONS
Let u first review the characteristic function for the single random
variable
Let X be a random variable with probability density function fX(x)
We defined the characteristic function FX(w) as follows
Φ X (ω) = E e
jωX
 =


jωx
f (x)e dx
 X
1 
 jωx
f X (x) =
Φ
(ω)e
dω
X


2π
FX(w) is the Fourier transform of fX(x) with the sign of w is reverse
The moments mn can be found from the characteristic function as
follows:
n
d
Φ X (ω)
n
m n = (  j)
dωn ω = 0
We now define the joint characteristic function of two random
variables X and Y with joint probability density function fXY(x,y)
as follows
Φ X,Y (ω1 ,ω2 ) = E e jω1X + jω2Y 
ΦX,Y (ω1 ,ω2 ) =

 

 
were w1 and w2 are real numbers
f X,Y (x,y) e jω1x + jω2ydxdy
FX,Y(w1,w2) is the two-dimensional Fourier Transform of fXY(x,y)
with reversal of sign of w1 and w2
From the inverse two-dimensional Fourier Transform we have
1
f X,Y (x,y) =
(2π)2

 

 
ΦX,Y (ω1 ,ω2 ) e  jω1x  jω2y dω1dω2
By setting w2= 0 in the expression of FX,Y(w1,w2) above we obtain
the following
ΦX,Y (ω1,0) = 



 
=


f X,Y (x,y) e

f (x,y)
  X,Y

=  f X (x) e

jω1x
e
dx = Φ (ω )
X 1
Similarly
ΦY (ω2 ) = ΦX,Y (0,ω2 )
jω1x + j(0)y
dxdy
jω1x
  

f (x,y)dy 

   X,Y

dxdy = 

e
jω1x
dx
The characteristic function for the
marginal probability density function
fX(x)
The joint moments can be found from the joint characteristic function
m nk = (  j) n+k
 n+k Φ X,Y (ω1 ,ω2 )
ω1n ωk2
ω1 =0, ω2 0
This expression is the two-dimensional extension of the one-dimension
n
d
Φ X (ω)
n
m n = (  j)
dωn ω = 0
The joint characteristic function for N random variables X1 ,…,X n
Φ X1 ,...,X N (ω1 ,...,ω N ) = E e jω1x1 + ... + jωN x N 
The joint moments are obtained from
m n1n 2 ...n N = (  j) R
 R X1,....X N (ω1 ,...,ωN )
ω1n1 ω2n 2 ...ωnNN
were R = n1  n 2 
all ωi =0
nN
The Gaussian Random Variable
A random variable X is called Gaussian if its density function has the
form
f X (x ) 
1
2 x2
e
 ( x  aX ) 2 2 x2
where  X2 (the varinace) and aX (the mean)
The “spread” about the point
x = ax is related to σx
JOINTLY GAUSSINA RANDOM VARIABLES
Two random variables X and Y are said to be jointly gaussian or
Bivariate gaussian density if their joint density function is of the form
f X,Y (x,y) =
1
2πσ X σ Y 1  ρ 2

 (x  X) 2
1
2ρ(x  X)(y  Y) (y  Y)  
 exp 

+


2 
2
2
σXσY
σY  
 2(1  ρ )  σ X
were
X = E X
Y = E Y
σ 2X = E (X  X) 2 
ρ = E (X  X)(Y  Y)  σ X σ Y
σ 2Y = E (Y  Y) 2 
The joint Gaussian density function and its maximum is located at
the point (X,Y)
The maximum value is obtained from
f X,Y (x,y)  f X,Y (X,Y) =
The locus of constant values of
1
2πσ X σ Y 1  ρ 2
f X,Y (x,y) will be an ellipse
This is equivalent to the intersection of
with the plane parallel to the xy-plane
f X,Y (x,y)
If ρ = 0 then the joint Gaussian density
f X,Y (x,y) =
1
2πσ X σ Y 1  ρ 2

 (x  X) 2
1
2ρ(x  X)(y  Y) (y  Y)
 exp 

+
2 
2
2
2(1

ρ
)
σ
σ
σ
σ
X
X Y
Y


 1  (x  X) 2
1
(y  Y)  
 f X,Y (x,y) =
 exp  
+


2
2
2πσ X σ Y
σY  
 2  σX
  (x  X) 2  
  (y  Y) 2  
1
1
=
 exp  
 exp  




2
2
2
2
2πσ X
2πσ Y


  2σ X
  2σ Y
f X (x)
f Y (y)
= f X (x) f Y (y)
1
dT (y)
f Y (y) = f X T (y) 
dy
1
f Y (y) = 
n
Yi = Ti (X1 ,X 2 ,...,X N )
i = 1,2, ..., N
X j = Tj1 (Y1 ,Y2 ,...,YN )
j = 1,2, ..., N
f X (x n )
dT(x)
dx x = x n
f Y1 ,Y2 ,...,YN (y1 ,...,y N )  f X1 ,X2 ,...,XN (x1 = T11 ,...,x N = TN1 ) J
T11
Y1
T11
YN
TN1
Y1
TN1
YN
J=
Example
u  h  x, y   x  y
v  g  x, y   x  y
1
x  h 1  u , v   (u  v)
2
1
v  g 1  u , v   (u  v)
2
 f
A
X ,Y
( x, y)dxdy  fU ,V (u, v)dudv
B
Left side

A
f X ,Y ( x, y ) dxdy   f X ,Y  h 1  u , v  , g 1  u, v   | J ( x, y ) | dudv
B
were
 x
 u
J ( x, y )  
 y
 u
1
x   h  u, v 

v  
u

1
y   g  u , v 
v  
u
h 1  u, v  

v
 is the Jacobian
1
g  u , v  

v

| J ( x, y ) | is magnitude  absolute value  of the determinant of the Jacobian
Comparing it with the Right side
 fU ,V (u , v)  f X ,Y  h 1  u , v  , g 1  u , v   | J ( x, y ) |
fU ,V (u, v)  f X ,Y  h 1  u , v  , g 1  u , v   | J ( x, y ) |
Example
1
x  h 1  u , v   (u  v)
2
1
v  g 1  u , v   (u  v)
2
u  h  x, y   x  y
v  g  x, y   x  y
 x
 u
J ( x, y )  
 y
 u
x   h  u , v 

v  
u

y   g 1  u , v 
v  
u
 J ( x, y ) 
1
1
2
h
1
 u, v  

v

1
g  u , v  

v

 uv
 2 

 
u

 u v 


  2 
 u
 u  v 


 2 
v 
 u v 


 2 
v 
1

 2
1
 2
1 
2 

1

2 
Example Let X and Y are two zero mean Gaussian independent random variables
f X ( x) 
1
2
2
Let R  X  Y
2
Find f R (r , ) ?
e
2
 x 2 2 2
fY ( y ) 
Y 
=tan  
X
1
1
2 2
e
 y 2 2 2
 Rectangular to Polar 
Transformation



Solution Since X and Y are independent
 f X ( x, y )  f X ( x ) f Y ( y ) 
1
2
2
e
 ( x 2  y 2 ) 2 2
Assuming r  0 and 0    2
The Transformation r  x 2  y 2
 y
 
 =tan 1  
x
has the inverse Transformation
x  h (r , )  r cos( )
1
y  g (r , )  r sin( )
1
 f R , (r ,  )  f X ,Y  h 1  r ,   , g 1  r ,    | J ( x, y ) |
 Polar to Rectangular 
Transformation



f R , (r , )  f X ,Y  h 1  r ,   , g 1  r ,    | J ( x, y ) |
 x
 r
J ( x, y )  
 y
 r
x   
(r cos  )
   r

y   
(r sin  )
   r
 | J ( x, y ) | 


(r cos  ) 



(r sin  ) 


cos  r sin( )
r
sin( ) r cos 
 cos  r sin( ) 


sin( ) r cos  
r
f R , (r , )  f X ,Y  h 1  r ,   , g 1  r ,    | J ( x, y ) |
y  g 1 (r , )  r sin( )
x  h1 (r , )  r cos( )
f R , ( r ,  )
r
f R , (r , )  f X ,Y  h 1  r ,   , g 1  r ,    | J ( x, y ) |
y  g 1 (r , )  r sin( )
x  h1 (r , )  r cos( )
f R , ( r ,  ) 
1
2
e
2

 ( r cos )2  ( r sin  )2

2 2
r 

r
2 2
e r
2
2 2
f R , ( r ,  )
f R , (r ,  ) 
r
2
r
e
2
2
2 2

 1 
r2

re

 2 
2 2

Rayleigh density function
Uniform density between 0, 2 
Let X and Y be independent uniform random variables over(0,1).
Find the pdf of Z  XY , f Z ( z ) ?
Solution We seek the density of Z , f Z ( z) ?
If we can find a joint density between Z and another random variable say W , then we can
find the marginal density f Z ( z ) by integrating the joint density f ZW ( z, w) with respect to w

fZ ( z) 

f ZW ( z, w)dw

Q : How can we find the joint density f ZW ( z, w) ?
A : Using the Jacobian method
f ZW ( z, w)  f XY  x  h 1  z , w  , y  g 1  z , w   J ( x, y )
Since X and Y are independent then
1 0  x  1 , 0  y  1
 f XY ( x, y )  
otherwise
0
f XY ( x, y )  f X ( x) fY ( y )

fZ ( z) 

Z  XY
f ZW ( z, w)dw

f ZW ( z, w)  f XY  x  h 1  z , w  , y  g 1  z , w   J ( x, y )
we seek another
transformation W
1 0  x  1 , 0  y  1
f XY ( x, y )  
otherwise
0
Let Z  XY and W  X
The transformation z  xy, w  x has the inverse transformation
 x
 z
J ( x, y )  
 y
 z
x 
0
w  
 1
y  
w
w 
1 
1

J
(
x
,
y
)


w
z 
 2
w 
xw
yz/w
z 1

 f ZW ( z, w)  f XY  x  w, y   
w w

z

1
1
0

w

1
,
0

 1 1 0  z  w  1
z

0  z  w 1


f XY ( w, )  
w

f
(
z
,
w
)

w
ZW
0
otherwise
w 

otherwise
 0
0
otherwise

 fZ ( z) 


1
f ZW ( z, w)dw  
z
1
dw   ln z 0  z  1
w