Transcript Discrete Random Variables
4/27/2020
EE255/CPS226 Stochastic Processes
Dept. of Electrical & Computer engineering Duke University Email: [email protected]
1
What is a stochastic process?
Stochastic Process: is a family of rv s {
X(t)|t
ε
T
} (T is an index set; it may be discrete or continuous) Values assumed by
X
(
t
) are called
states. State space
(I): set of all possible states Example: cosmic radio noise at antenna {a 1 , a 2 , .., a k }. t 1 Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University
Stochastic Process Characterization
Sample space
S:
set of antennas.
Sample the output of all antennas at time t 1 ( rv {X(t 1 )}.
In general, we can define: rv), i.e. we can define At a fixed time
t=t 1
, we can define X t1 (s) = X(t 1 ,s) (rv X(t 1 )). Similarly, we can define, X(t 2 ), .., X(t k ). X(t 1 ) can be characterized by its distribution function, We can also a joint variable, characterized by its CDF as, Discrete and continuous cases: States X(t) (i.e. time t) may be discrete/continuous State space I (i.e. sample space S) may be discrete/continuous Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University
Classification of Stochastic Processes
Four classes of stochastic processes: Discrete-state process chain (e.g., DJIA index at any time) discrete-time process stochastic sequence {X n (e.g., probing a system every 10 ms.) | n є T} Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University
Example: a Queuing System
Random arrivals Inter arrival time distribution fn. F Y Queue (waiting station) m servers Service time distribution fn. F S Inter arrival times
Y 1 , Y 2 , …
(mutually independent) (
F Y
) Service times:
S 1 , S 2 , …
(mutually independent) (
F S
) Notation for a queuing system:
F y
/
F Y
/
m
Possible arrival/service time distributions types are: M: Memory-less (i.e., EXP) D: Deterministic G: General distribution
E k
:
k
-stage Erlang etc.
M/M/1 Memory-less arrival/departure processes with 1-service station Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University
Discrete/Continuous Stochastic Processes
N k : Number of jobs waiting in the system at the time of k th departure Stochastic process {N k |k=1,2,…}: job’s Discrete time, discrete space N k Discrete k Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University
Continuous Time, Discrete Space
X(t)
: Number of jobs in the system at time
t.
{
X(t)|t є T
} forms a
continuous-time, discrete-state
stochastic process, with, X(t) Continuous Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University
Discrete Time, Continuous Space
W k
: wait time for the
k th
job. Then {
W k | k є T
} forms a
Discrete-time, Continuous-state
stochastic process, where, W k Discrete Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University k
Continuous Time, Continuous Space
Y(t)
: total service time for all jobs in the system at time
t. Y(t)
forms a
continuous-time, continuous-state
stochastic process, Where, Y(t) t Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University
Further Classification
(1 st order distribution) (2 nd Similarly, we can define n th order distribution: order distribution) Difficult to compute n th order distribution. Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University
Further Classification (contd.)
Can the n th order distribution computations be simplified?
Yes. Under some simplifying assumptions: Stationary (strict) F(
x
;
t
) = F(
x
;
t
+τ) all moments are time-invariant Independence As consequence of independence, we can define
Renewal Process
Discrete time independent process {X n |n=1,2,…} (X 1 , X 2 , .. are
iid, non-negative
rvs), e.g., repair/replacement after a failure. Markov process removes independence restriction.
Markov Process
Stochastic proc. {X(t) | t є T} is Markov if for any t 0 the conditional distribution < t 1 < … < t n < t, Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University
Markov Process
Mostly, we will deal with discrete state Markov process i.e., Markov chains In some situations, a Markov process may also exhibit invariance
wrt
to the time origin, i.e. time-homogeneity time-homogeneity does not imply stationarity. This also means that while conditional pdf may be stationary, the joint pdf may not be so.
Homogeneous Markov process process is completely summarized by its current state (independent of how it reached this particular state). Let,
Y
: time spent in a given state Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University
Markov Process-Sojourn time
Y is also called the
sojourn time
This result says that for a homogeneous discrete time Markov chain,
sojourn time
in a state follows EXP( ) distribution.
Semi-Markov
process is one in which the sojourn time in state may not be EPX( ) distributed.
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University
Renewal Counting Process
Renewal counting process: # of renewals (repairs, replacements, arrivals) in time
t
: a continuous time process: If time interval between two renewals follows EXP distribution, then Poisson Process Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University
Stationarity Properties
Strict sense Stationarity Stationary in the mean E[X(t)] = E[X] In general, if Then, a process is said to be
wide-sense stationary Strict-sense
stationarity
wide-sense
stationarity Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University
Bernoulli Process
A set of Bernoulli sequences, {
Y i
|
i=1,2,3,..
},
Y i =1 or 0
{
Y i
} forms a Bernoulli Process. Often
Y i
’s are independent.
E[
Y i
] =
p
; E[
Y i 2
] =
p
; Var[
Y i
] =
p
(
1-p
) Define another stochastic process , {
S n
|
n=1,2,3,..
}, where
S n = Y 1 + Y 2 +…+ Y n
(
i.e. S n
:sequence of partial sums)
S n = S n-1 + Y n
(recursive form) P[
S n = k| S n-1 = k
] = P[
Y n = 0
] = (
1-p
) and, P[
S n = k| S n-1 = k-1
] = P[
Y n = 1
] =
p
{
S n
|
n=1,2,3,..
}, forms a Binomial process P[
S n = k
] = Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University
Binomial Process Properties
Viewing successes in a Bernoulli process as arrivals, then, define discrete rv
T 1 : #
trials up to & including 1 st success (arrival)
T 1
: First order inter-arrival time and
v
has a Geometric distribution P[
T 1 =i
]
= p(1-p) i-1 , i=1,2,…;
E[ Geometric Distribution
T 1
]
= 1/p;
Var[ memory-less property.
T 1
]
= (1-p)/p 2
Cond.
pmf
P[
T 1 = i
| no success in the previous
m
trials ]
= p
Since we treat arrival as success in {
S n
}, occupancy time in state memory-less
S n
is Generalization to
r th order inter-arrival time T r :
# trial trials up to including
r th
arrival.
Distribution for
T r : r-
fold
convolution
of
T 1 ’s
distribution.
Non-homogeneous
Bernouli process.
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University
Poisson Process
A continuous time, discrete state process.
2.
3.
1.
N(t): no. of events occurring in time (0, t]. Events may be, # of packets arriving at a router port # of incoming telephone calls at a switch # of jobs arriving at file/computer server 4.
Number of failed components in time interval Events occurs successively and that intervals between these successive events are
iid
rv s , each following EXP( ) 1.
2.
λ: average arrival rate (1/ λ: average time between arrivals) λ: average failure rate (1/ λ: average time between failures) Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University
Poisson Process (contd.)
1.
2.
3.
N(t) forms a Poisson process provided: N(0) = 0 Events within non-overlapping intervals are independent In a very small interval
h
, only one event may occur (prob. p(h)) Letting,
p n (t)
=
P
[
N(t)=n
], Hence, for a Poisson process, interval arrival times follow EXP( ) (memory-less) distribution. Such a Poisson process is non-stationary. Mean = Var = λt ; What about E[N(t)/t], as t infinity?
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University
Merged Multiple Poisson Process Streams
Consider the system, + Proof: Using z-transform. Letting, α = λt, Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University
Decomposing a Poisson Process Stream
Decompose a Poisson process into multiple streams + N arrivals decomposed into {n 1 , n 2 , .., n k }; N= n 1 +n 2 , ..,+n k Cond. pmf Since, The uncond. pmf Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University
Renewal Counting Process
Poisson process EXP( ) distributed inter-arrival times.
What if the EXP( ) assumption is removed renewal proc.
Renewal proc. : {
X i |i=1,2,…
} (
X i
’s are
iid
X i : time gap between the occurrence of i th non-EXP rv s ) and (i+1) st event
S k
=
X 1
+
X 2
+ .. +
X k
time to occurrence of the k th event.
N(t)- Renewal counting process is a
discrete-state, continuous-time
stochastic. N(t) denotes no. of renewals in the interval (0, t]. Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University
Renewal Counting Processes (contd.)
S n t For N(t), what is P(N(t) = n)?
t n More arrivals possible 1.
2.
n th renewal takes place at time
t
(account for the equality) If the n th renewal occurs at time
t n
occur in the interval (
t n < t
].
< t
, then one or more renewals Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University
Renewal Counting Process Expectation
Let,
m(t)
= E[
N(t)
]. Then,
m(t)
= mean no. of arrivals in time (0,t].
m(t)
is called the
renewal
function
.
Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University
Renewal Density Function
Renewal density function: For example, if the renewal interval
X
is EXP(
λ x
), then
d(t) = λ , t >= 0
and
m(t) = λ t , t >= 0.
P[
N(t)
=n]
= e –λ t (λ t) n
/
n! i.e
Poisson process pmf
F n
(
t
) will turn out to be
n
-stage Erlang Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University
Availability Analysis
Availability: is defined is the ability of a system to provide the desired service.
If no repairs/replacements, Availability = Reliability.
If repairs are possible, then above def. is pessimistic.
MTBF T 1 D 1 T 2 D 2 T 3 D 3 T 4 D 4 ……. MTBF = E[D i +T i+1 ] = E[T i +D i ]=E[X i ]=MTTF+MTTR Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University
Availability Analysis (contd.)
renewal x Repair is completed with in this interval t 2.
1.
Two mutually exclusive situations: System does not fail before time
t
A(t) = R(t)
System fails, but the repair is completed before time
t
Therefore, A(t) = sum of these two probabilities Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University
Availability Expression
dA(x) : Incremental availability Repair is completed with in this interval 0 x x+dx Renewed life time >= (t-x) t dA(x) = Prob(that after renewal, life time is > (t-x) & that the renewal occurs in the interval (x,x+dx]) Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University
Availability Expression (contd.)
A(t) can also be expressed in the Laplace domain .
Since, R(t) = 1-W(t) or L R (s) = 1/s – L W (s) = 1/s –L w (s)/s What happens when
t
becomes very large?
However, Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University
Availability, MTTF and MTTR
Steady state availability
A
is: for small values of
s
, Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University
Availability Example
Assuming EXP( ) density fn for g(t) and w(t) Bharat B. Madan, Department of Electrical and Computer Engineering, Duke University