Transcript Operationele Research II (vakcode 158006)

Flows and Networks
Plan for today (lecture 2):
•
•
•
•
•
•
•
•
•
•
•
Questions?
Birth-death process
Example: pure birth process
Example: pure death process
Simple queue
General birth-death process: equilibrium
Reversibility, stationarity
Truncation
Kolmogorov’s criteria
Summary / Next
Exercises
Last time on Flows and Networks:
Highlights: continuous time Markov chain
• stochastic process X(t)
countable or finite state space S
Markov property
P( X (t  s)  j | X (t )  i, X (tn )  jn ..., X (t1 )  j1 )
 P( X (t  s)  j | X (t )  i)
transition rates
Ph (i, j )
q(i, j )  lim
h 0
h
i j
independent t
irreducible: each state in S reachable from any
other state in S
Assume ergodic and regular
global balance equations (equilibrium eqns)
0   [ (k )q(k , j )   ( j )q( j, k )]
k j
π is stationary distribution
solution that can be normalised is
equilibrium distribution
if equilibrium distribution exists, then it is
unique and is limiting distribution
lim P ( X (t )  k | X (0)  j )   (k )
t 
Flows and Networks
Plan for today (lecture 2):
•
•
•
•
•
•
•
•
•
•
•
Questions?
Birth-death process
Example: pure birth process
Example: pure death process
Simple queue
General birth-death process: equilibrium
Reversibility, stationarity
Truncation
Kolmogorov’s criteria
Summary / Next
Exercises
Birth-death process
• State space S  Z
• Markov chain, transition rates
 ( j)
k  j  1 birth rate


 ( j)
k  j  1 death rate

q( j , k )  
k j
  ( j )   ( j )

0
otherwise
• Bounded state space:
q(J,J+1)=0 then states space bounded above at J
q(I,I-1)=0 then state space bounded below at I
• Kolmogorov forward equations
dPt (i, j )
 Pt (i, j  1) ( j  1)  Pt (i, j )[ ( j )   ( j )]  Pt (i, j  1)  ( j  1)
dt
• Global balance equations
0   ( j  1) ( j  1)   ( j )[ ( j )   ( j )]   ( j  1)  ( j  1)
Flows and Networks
Plan for today (lecture 2):
•
•
•
•
•
•
•
•
•
•
•
Questions?
Birth-death process
Example: pure birth process
Example: pure death process
Simple queue
General birth-death process: equilibrium
Reversibility, stationarity
Truncation
Kolmogorov’s criteria
Summary / Next
Exercises
Example: pure birth process
• Exponential interarrival times, mean 1/
• Arrival process is Poisson process
• Markov chain?
• Transition rates : let t0<t1<…<tn<t
P( X (t  h)  j  1 | X (t )  j , X (t 0)  j 0,..., X (tn)  jn) 
P( X (t  h)  j  1 | X (t )  j )  h  o(h)
P( X (t  h)  j  2 | X (t )  j )  o(h)
P( X (t  h)  j | X (t )  j )  1  h  o(h)

q( j, k )  
 
k  j 1
k j
• Kolmogorov forward equations for P(X(0)=0)=1
dPt (0, j )
 Pt (0, j  1)  Pt (0, j )
dt
dPt (0,0)
 Pt (0, j )
dt
• Solution for P(X(0)=0)=1
(t ) j t
Pt (0, j ) 
e ,
j!
j  0,1,2,...,t  0
Flows and Networks
Plan for today (lecture 2):
•
•
•
•
•
•
•
•
•
•
•
Questions?
Birth-death process
Example: pure birth process
Example: pure death process
Simple queue
General birth-death process: equilibrium
Reversibility, stationarity
Truncation
Kolmogorov’s criteria
Summary / Next
Exercises
Example: pure death process
• Exponential holding times, mean 1/
• P(X(0)=N)=1, S={0,1,…,N}
• Markov chain?
• Transition rates : let t0<t1<…<tn<t
P( X (t  h)  j  1 | X (t )  j , X (t 0)  j 0,..., X (tn)  jn) 
P( X (t  h)  j  1 | X (t )  j )  jh  o(h)
P( X (t  h)  j  21| X (t )  j )  o(h)
P( X (t  h)  j | X (t )  j )  1  jh  o(h)
 j
q( j, k )  
  j
k  j 1
k j
• Kolmogorov forward equations for P(X(0)=N)=1
dPt ( N , j )
 ( j  1) Pt (0, j  1)  jPt (0, j )
dt
dPt ( N , N )
  NPt (0, N )
dt
• Solution for P(X(0)=N)=1
 N  t j
Pt ( N , j )    e
1  et
 j
 

N j
,
j  0,1,2,...,N , t  0
Flows and Networks
Plan for today (lecture 2):
•
•
•
•
•
•
•
•
•
•
•
Questions?
Birth-death process
Example: pure birth process
Example: pure death process
Simple queue
General birth-death process: equilibrium
Reversibility, stationarity
Truncation
Kolmogorov’s criteria
Summary / Next
Exercises
Simple queue
• Poisson arrival proces rate , single server
exponential service times, mean 1/
• Assume initially empty:
P(X(0)=0)=1,
S={0,1,2,…,}
• Markov chain?
• Transition rates :
P( X (t  h)  j  1 | X (t )  j )  h  o(h)
P( X (t  h)  j  1 | X (t )  j )  h  o(h)
P( X (t  h)  j | X (t )  j )  1  [h  h]  o(h)
k  j 1
 
 
k  j  1, j  0

q( j, k )  
 [    ] k  j , j  0
  
k  j, j  0
Simple queue
• Poisson arrival proces rate , single server
exponential service times, mean 1/
k  j 1
 
 
k  j  1, j  0

q( j, k )  
 [    ] k  j , j  0
  
k  j, j  0
• Kolmogorov forward equations, j>0
dPt (i, j )
 Pt (i, j  1)  Pt (i, j )[   ]  Pt (i, j  1) 
dt
dPt (i,0)
  Pt (i,0)  Pt (i,1) 
dt
• Global balance equations, j>0
0   ( j  1)   ( j )[   ]   ( j  1) 
0   (0)   (1) 
Simple queue (ctd)



j

j+1

Equilibrium distribution: <
 ( j )   (0)( /  ) j
 (1   /  )( /  ) j
Stationary measure; summable  eq. distrib.
Proof: Insert into global balance
Detailed balance!
 ( j )q( j, j 1)   ( j 1)q( j 1. j )
Flows and Networks
Plan for today (lecture 2):
•
•
•
•
•
•
•
•
•
•
•
Questions?
Birth-death process
Example: pure birth process
Example: pure death process
Simple queue
General birth-death process: equilibrium
Reversibility, stationarity
Truncation
Kolmogorov’s criteria
Summary / Next
Exercises
Birth-death process
• State space S  N  {0,1,2,...}
• Markov chain, transition rates
 ( j)
k  j 1
birth rate


 ( j)
k  j  1, j  0 death rate

q( j, k )  
k  j, j  0
  ( j )   ( j )
   (0)
k  j, j  0
• Definition: Detailed balance equations
 ( j )q( j, j 1)   ( j 1)q( j 1. j )
• Theorem: A distribution that satisfied detailed
balance is a stationary distribution
• Theorem: Assume that

 (0)  
 jS
then
1
q(r  1, r ) 
 
q(r , r  1) 
j

r 1
j
 ( j )   (0)
r 1
q(r  1, r )
,
q(r , r  1)
jS
is the equilibrium distrubution of the birth-death
prcess X.
Flows and Networks
Plan for today (lecture 2):
•
•
•
•
•
•
•
•
•
•
•
Questions?
Birth-death process
Example: pure birth process
Example: pure death process
Simple queue
General birth-death process: equilibrium
Reversibility, stationarity
Truncation
Kolmogorov’s criteria
Summary / Next
Exercises
Reversibility; stationarity
• Stationary process: A stochastic process is
stationary if for all t1,…,tn,
( X (t1 ), X (t2 ),...,X (tn )) ~ ( X (t1   ), X (t2   ),...,X (tn   ))
• Theorem: If the initial distribution is a
stationary distribution, then the process is
stationary
• Reversible process: A stochastic process is
reversible if for all t1,…,tn,
( X (t1 ), X (t2 ),...,X (tn )) ~ ( X (  t1 ), X (  t2 ),...,X (  tn ))
  ( j)  1
jS
NOTE: labelling of states only gives suggestion of
one dimensional state space; this is not
required
Reversibility; stationarity
• Lemma: A reversible process is stationary.
• Theorem: A stationary Markov chain is
reversible if and only if there exists a
collection of positive numbers π(j), jS,
summing to unity that satisfy the detailed
balance equations
 ( j )q( j, k )   (k )q(k , j ), j, k  S
When there exists such a collection π(j), jS, it
is the equilibrium distribution
• Proof
Flows and Networks
Plan for today (lecture 2):
•
•
•
•
•
•
•
•
•
•
•
Questions?
Birth-death process
Example: pure birth process
Example: pure death process
Simple queue
General birth-death process: equilibrium
Reversibility, stationarity
Truncation
Kolmogorov’s criteria
Summary / Next
Exercises
10
Truncation of reversible processes
Lemma 1.9 / Corollary 1.10:
If the transition rates of a reversible Markov process with
state space S and equilibrium distribution  ( j ), j  S are
altered by changing q(j,k) to cq(j,k) for j  A, k  S \ A
where
c>0
then the resulting Markov process is
reversible in equilibrium and has equilibrium distribution
where B
is the normalizing constant.
 B ( j )

Bc ( j )
If
c=0
jA
jS \ A
then the reversible Markov process
is truncated to A
and the resulting Markov
process is reversible with equilibrium distribution
 ( j)
  (k )
jA
k A
S\A
A
Time reversed process
X(t) reversible Markov process  X(-t) also, but
Lemma 1.11: tijdshomogeneity not inherited for nonstationary process
Theorem 1.12 : If X(t) is a stationary Markov process
with transition rates q(j,k), and equilibrium
distribution π(j), jS, then the reversed process
X(-t) is a stationary Markov process with
transition rates
 (k )q(k , j )
j, k  S
q' ( j , k ) 
 ( j)
and the same equilibrium distribution
Theorem 1.13: Kelly’s lemma
Let X(t) be a stationary Markov processwith
transition rates q(j,k). If we can find a collection of
numbers q’(j,k) such that q’(j)=q(j), jS, and a
collection of positive numbers (j), jS, summing to
unity, such that
 ( j )q( j , k )   (k ) q' ( k , j )
j, k  S
then q’(j,k) are the transition rates of the timereversed process, and (j), jS, is the equilibrium
distribution of both processes.
Flows and Networks
Plan for today (lecture 2):
•
•
•
•
•
•
•
•
•
•
•
Questions?
Birth-death process
Example: pure birth process
Example: pure death process
Simple queue
General birth-death process: equilibrium
Reversibility, stationarity
Truncation
Kolmogorov’s criteria
Summary / Next
Exercises
Kolmogorov’s criteria
• Theorem 1.8:
A stationary Markov chain is reversible iff
q( j1 , j2 )q( j2 , j3 )...q( jn1 , jn )q( jn , j1 )
 q( j1 , jn )q( jn , jn1 )...q( j3 , j2 )q( j2 , j1 )
for each finite sequence of states
j1, j2 ,..., jn  S
Notice that
 ( j )   (0)
q(0, j1 )q( j1 , j2 )q( j2 , j3 )...q( jn1 , jn )q( jn , j )
q( j, jn )q( jn , jn1 )...q( j3 , j2 )q( j2 , j1 )q( j1 ,0)
Flows and Networks
Plan for today (lecture 2):
•
•
•
•
•
•
•
•
•
•
•
Questions?
Birth-death process
Example: pure birth process
Example: pure death process
Simple queue
General birth-death process: equilibrium
Reversibility, stationarity
Truncation
Kolmogorov’s criteria
Summary / Next
Exercises
Summary / next:
•
•
•
•
•
Birth-death process
Simple queue
Reversibility, stationarity
Truncation
Kolmogorov’s criteria
• Next
input / output simple queue
Poisson proces
PASTA
Output simple queue
Tandem netwerk
Exercises
[R+SN] 1.3.2, 1.3.3, 1.3.5, 1.5.1, 1.5.2, 1.5.5,
1.6.2, 1.6.3, 1.6.4