Transcript Document

Probability and Statistics with
Reliability, Queuing and Computer
Science Applications: Chapter 6 on
Stochastic Processes
Kishor S. Trivedi
Visiting Professor
Dept. of Computer Science and Engineering
Indian Institute of Technology, Kanpur
What is a Stochastic Process?




Stochastic Process: is a family of random variables
{X(t) | t ε T} (T is an index set; it may be discrete or
continuous)
Values assumed by X(t) are called states.
State space (I): set of all possible states
Sometimes called a random process or a chance
process
Stochastic Process Characterization

At a fixed time t=t1, we have a random variable X(t1).
Similarly, we have
X(t2), .., X(tk).
X(t1) can be characterized by its distribution function,

We can also consider the joint distribution function,


Discrete and continuous cases:
 States X(t) (i.e. time t) may be discrete/continuous
 State space I may be discrete/continuous
Classification of Stochastic Processes



Four classes of stochastic processes:
discrete-state process  chain
discrete-time process  stochastic sequence {Xn | n є T}
(e.g., probing a system every 10 ms.)
Example: a Queuing System
Queue (waiting station)
Random arrivals
Inter arrival time
distribution fn. FY

m
servers
Interarrival times Y1, Y2, … (common dist. Fn. FY)

Service times: S1, S2, …

Notation for a queuing system: FY /FS/m

Some interarrival/service time distributions types are:
 M: Memoryless (i.e., EXP)
 D: Deterministic
 Ek: k-stage Erlang etc.
 Hk: k-stage Hyper exponential distribution
 G: General distribution
 GI: General independent inter arrival times

Service time
distribution fn. FS
(iid with a common cdf FS)
M/M/1  Memoryless interarrival/service times with a single server
Discrete/Continuous Stochastic Processes

Nk: Number of jobs waiting in the system at the time of kth job’s
departure  Stochastic process {Nk| k=1,2,…}:

Discrete time, discrete state
Nk
Discrete
k
Continuous Time, Discrete Space

X(t): Number of jobs in the system at time t. {X(t) | t є T} forms a
continuous-time, discrete-state stochastic process, with,
X(t)
Continuous
Discrete Time, Continuous Space

Wk: waiting time for the kth job. Then {Wk | k є T} forms a Discrete-time,
Continuous-state stochastic process, where,
Wk
Discrete
k
Continuous Time, Continuous Space

Y(t): total service time for all jobs in the system at time t. Y(t)
forms a continuous-time, continuous-state stochastic process,
Where,
Y(t)
t
Further Classification

(1st order distribution)

(2nd order distribution)


Similarly, we can define nth order distribution:
Formidable task to provide nth order distribution for all
n.
Further Classification (contd.)


Can the nth order distribution be simplified?
Yes. Under some simplifying assumptions:

Independence


As example, we have the Renewal Process

Discrete time independent process {Xn | n=1,2,…} (X1, X2, .. are iid,
non-negative rvs), e.g., repair/replacement after a failure.

Markov process introduces a limited form of dependence

Markov Process

Stochastic proc. {X(t) | t є T} is Markov if for any t0 < t1< … < tn<
t, the conditional distribution satisfies the Markov property:
Markov Process





We will only deal with discrete state Markov
processes i.e., Markov chains
In some situations, a Markov chain may also exhibit
time-homogeneity
Future of process (probabilistically) determined by its
current state, independent of how it reached this
particular state; but in a non homogeneous case,
current time can also determine the future.
For a homogeneous Markov chain current time is also
not needed to determine the future.
Let Y: time spent in a given state in a hom. CTMC
Homogeneous CTMC-Sojourn time




Since Y, the sojourn time, has the memoryless prop.
This result says that for a homogeneous continuous
time Markov chain, sojourn time in a state follows
EXP( ) distribution (not true for non-hom CTMC)
Hom. DTMC sojourn time dist. Is geometric.
Semi-Markov process is one in which the sojourn
time in a state is generally distributed.
Bernoulli Process



A sequence of iid Bernoulli rvs, {Yi | i=1,2,3,..}, Yi =1 or 0
{Yi} forms a Bernoulli Process, an example of a renewal
process.
Define another stochastic process , {Sn | n=1,2,3,..}, where
Sn = Y1 + Y2 +…+ Yn (i.e. Sn :sequence of partial sums)
 Sn = Sn-1+ Yn
(recursive form)
 P[Sn = k | Sn-1= k] = P[Yn = 0] = (1-p) and,
 P[Sn = k | Sn-1= k-1] = P[Yn = 1] = p

{Sn |n=1,2,3,..}, forms a Binomial process, an example
of a homogeneous DTMC
Renewal Counting Process


Renewal counting process: # of renewals
(repairs, replacements, arrivals) by time t: a
continuous time process:
If time interval between two renewals follows
EXP distribution, then  Poisson Process
Note:
For a fixed t, N(t) is a random variable
(in this case a discrete random variable
known as the Poisson random variable)
The family {N(t), t  0} is a stochastic
process, in this case, the homogeneous
Poisson process
{N(t), t  0} is a homogeneous CTMC as
well

Poisson Process



A continuous time, discrete state process.
N(t): no. of events occurring in time (0, t]. Events may be,
1.
# of packets arriving at a router port
2.
# of incoming telephone calls at a switch
3.
# of jobs arriving at file/compute server
4.
Number of component failures
Events occurs successively and that intervals between these
successive events are iid rvs, each following EXP( )
1.
2.
λ: arrival rate (1/ λ: average time between arrivals)
λ: failure rate (1/ λ: average time between failures)
Poisson Process (contd.)

1.


N(t) forms a Poisson process provided:
1.
N(0) = 0
2.
Events within non-overlapping intervals are independent
3.
In a very small interval h, only one event may occur (prob.
p(h))
Letting, pn(t) = P[N(t)=n],
For a Poisson process, interarrival times follow EXP( )
(memoryless) distribution.
E[N(t)] = Var[N(t)] = λt ; What about E[N(t)/t], as t infinity?
Merged Multiple Poisson Process Streams

Consider the system,
+

Proof: Using z-transform. Letting, α = λt,
Decomposing a Poisson Stream

Decompose a Poisson process using a prob. switch

N arrivals decomposed into {N1, N2, .., Nk}; N= N1+N2, ..,+Nk
Cond. pmf

Since,

The uncond. pmf

Generalizing the Poisson Process
Poisson Process
Non-Homogeneous Poisson
Process (NHPP)
Non-Homogeneous Poisson
Process (NHPP)
If the expected number of events per unit time, l, changes with
age (time), we have a non-homogeneous Poisson model. We
assume that:

1.
If 0  t, the pmf of N(t) is given by:

PN t   k   mt  / k!em t 
k
k  0, 1, 2, ...
where m(t)  0 is the expected number of events in the time
period [0, t]


2.
Counts of events in non-overlapping time periods are
mutually independent.
m(t) : the mean value function. l(x) :the time-dependent rate of
occurrence of events or time-dependent failure rate
m(t )   0 l (x) dx
t
NHPP(cont.)
Generalizing Poisson Process
Poisson Process
Non-Homogeneous Poisson
Process (NHPP)
Renewal Counting
Process
Renewal Counting Process

Poisson process  EXP( ) distributed interarrival times.
What if the EXP( ) assumption is removed  renewal proc.
Renewal proc. : {Xi | i=1,2,…} (Xi’s are iid non-EXP rvs)
Xi : time gap between the occurrence of (i-1) st and ith event

Sk = X1 + X2 + .. + Xk  time to occurrence of the kth event.




N(t)- Renewal counting process is a discrete-state, continuoustime stochastic process. N(t) denotes no. of renewals in the
interval (0, t].
Renewal Counting Processes (contd.)

Sn
For N(t), what is P(N(t) = n)?
tn

t
More arrivals possible
Renewal Counting Process Expectation

Let, m(t) = E[N(t)]. Then, m(t) = mean no.
of arrivals in time (0,t]. m(t) is called the
renewal function.
Renewal Density Function


Renewal density function:
For example, if the renewal interval X is EXP(λ),
then



d(t) = λ , t >= 0 and m(t) = λ t , t >= 0.
P[N(t)=n] = e–λ t (λ t)n/n! i.e Poisson pmf
Fn(t) will turn out to be n-stage Erlang
Alternating Renewal Process
I(t)
1
Operating
Restoration
0
Time
Where:
Failure times T1, T2, … are mutually independent with
a common distribution function W
Restoration times D1, D2, … are mutually independent
with a common distribution function G
The sequences {Tn} and {Dn} are independent

Availability Analysis



Availability: is defined is the ability of a system to
provide the desired service.
If no repair/replacement,Availability(t)=Reliability(t)
If repairs are possible, then above is pessimistic.
MTBF
T1

D1
T2
D2
T3
D3
T4
D4 …….
MTBF = E[Di+Ti+1] = E[Ti+Di]=E[Xi]=MTTF+MTTR
Availability Analysis (contd.)
renewal
x

Repair is completed with in this interval
t
Two mutually exclusive situations:
1. System does not fail before time t  A(t)
= R(t)
System fails, but the repair is completed
before time t
Therefore, A(t) = sum of these two
probabilities
2.

Availability Expression

dA(x) : Incremental availability
Repair is completed with in this interval
0
x
x+dx
t
Renewed life time >= (t-x)

dA(x) = Prob(that after renewal, life time is > (t-x) &
that the renewal occurs in the interval (x,x+dx])
Availability Expression (contd.)


A(t) can also be expressed in the Laplace domain.
Since, R(t) = 1-W(t) or LR(s) = 1/s – LW(s) = 1/s –
Lw(s)/s

What happens when t becomes very large?

However,
Availability, MTTF and MTTR


Steady state availability A is:
Taking the expression of sLA(s) and taking the
limit via L’Hospital rule and using the moment
generating property of the LT, we get the
required result for the steady-state
A=MTTF/(MTTF+MTTR)
Availability Example

Assuming EXP( ) density fn for g(t) and w(t)
Generalizing Poisson Process
Bernoulli Process
Poisson Process
Renewal Counting
Process
Compound Poisson
Process
Non-Homogeneous Poisson
Process (NHPP)
Non-Homogeneous
Continuous Time
Markov Chain
Homogeneous
Continuous Time
Markov Chain
Homogeneous
Discrete Time
Markov Chain
Semi-Markov
Process
Markov Regenerative
Process