Operationele Research II (vakcode 158006)
Download
Report
Transcript Operationele Research II (vakcode 158006)
Flows and Networks
Plan for today (lecture 2):
•
•
•
•
•
•
•
•
•
•
•
•
Questions?
Continuous time Markov chain
Birth-death process
Example: pure birth process
Example: pure death process
Simple queue
General birth-death process: equilibrium
Reversibility, stationarity
Truncation
Kolmogorov’s criteria
Summary / Next
Exercises
Discrete time Markov chain: summary
• stochastic process X(t)
countable or finite state space S
Markov property
P( X (tn1 ) jn1 | X (tn ) jn ,..., X (t1 ) j1 )
P( X (tn1 ) jn1 | X (tn ) jn )
time homogeneous
P( X (t ) k | X (t ) j )
independent t
irreducible: each state in S reachable from any
other state in S
transition probabilities
p( j, k ) P( X (t 1) k | X (t ) j )
p( j , k ) 1
kS
Assume ergodic (irreducible, aperiodic)
global balance equations (equilibrium eqns)
( j ) ( k ) p( k . j )
kS
solution that can be normalised is
equilibrium distribution
if equilibrium distribution exists, then it is
unique and is limiting distribution
lim P ( X (t ) k | X (0) j ) (k )
t
Random walk
http://www.math.uah.edu/stat/
• Gambling game over infinite time horizon:
on any turn
– Win €1 w.p. p
– Lose €1 w.p. 1-p
– Continue to play
– Xn= amount after n plays
– State space S = {…,-2,-1,0,1,2,…}
– Time homogeneous Markov chain
p(i, i 1) p P( X n1 j | X n i)
p(i, i 1) 1 p
– For each finite time n:
– But equilibrium?
P( X n j | X 0 i)
Continuous time Markov chain
• stochastic process X(t)
countable or finite state space S
Markov property
P( X (t s) j | X (t ) i, X (tn ) jn ..., X (t1 ) j1 )
P( X (t s) j | X (t ) i)
transition probability
Pt (i, j) P( X (t ) j | X (0) i)
irreducible: each state in S reachable from any
other state in S
Chapman-Kolmogorov equation
Pt s (i, j ) Pt (i, k ) Ps (k , j )
k
transition rates or jump rates
Ph (i, j )
q (i, j ) lim
h 0
h
i j
Ph (i, j) q(i, j)h o(h)
Continuous time Markov chain
• Chapman-Kolmogorov equation
Pt s (i, j ) Pt (i, k ) Ps (k , j )
k
transition rates or jump rates
Ph (i, j )
q(i, j ) lim
h 0
h
i j
• Kolmogorov forward equations: (REGULAR)
Pt h (i, j ) Pt (i, k ) Ph (k , j )
k
Pt h (i, j ) Pt (i, j ) Pt (i, k ) Ph (k , j ) Pt (i, j )[Ph ( j , j ) 1]
k j
[ Pt (i, k ) Ph (k , j ) Pt (i, j ) Ph ( j , k )]
k j
Pt ' (i, j ) [ Pt (i, k )q(k , j ) Pt (i, j )q( j , k )]
k j
Global balance equations
0 [ (k )q(k , j ) ( j )q ( j , k )]
k j
Continuous time Markov chain: summary
• stochastic process X(t)
countable or finite state space S
Markov property
P( X (t s) j | X (t ) i, X (tn ) jn ..., X (t1 ) j1 )
P( X (t s) j | X (t ) i)
transition rates
Ph (i, j )
q(i, j ) lim
h 0
h
i j
independent t
irreducible: each state in S reachable from any
other state in S
Assume ergodic and regular
global balance equations (equilibrium eqns)
0 [ (k )q(k , j ) ( j )q( j, k )]
k j
π is stationary distribution
solution that can be normalised is
equilibrium distribution
if equilibrium distribution exists, then it is
unique and is limiting distribution
lim P ( X (t ) k | X (0) j ) (k )
t
Flows and Networks
Plan for today (lecture 2):
•
•
•
•
•
•
•
•
•
•
•
Questions?
Birth-death process
Example: pure birth process
Example: pure death process
Simple queue
General birth-death process: equilibrium
Reversibility, stationarity
Truncation
Kolmogorov’s criteria
Summary / Next
Exercises
Birth-death process
• State space S Z
• Markov chain, transition rates
( j)
k j 1 birth rate
( j)
k j 1 death rate
q( j , k )
k j
( j ) ( j )
0
otherwise
• Bounded state space:
q(J,J+1)=0 then states space bounded above at J
q(I,I-1)=0 then state space bounded below at I
• Kolmogorov forward equations
dPt (i, j )
Pt (i, j 1) ( j 1) Pt (i, j )[ ( j ) ( j )] Pt (i, j 1) ( j 1)
dt
• Global balance equations
0 ( j 1) ( j 1) ( j )[ ( j ) ( j )] ( j 1) ( j 1)
Flows and Networks
Plan for today (lecture 2):
•
•
•
•
•
•
•
•
•
•
•
Questions?
Birth-death process
Example: pure birth process
Example: pure death process
Simple queue
General birth-death process: equilibrium
Reversibility, stationarity
Truncation
Kolmogorov’s criteria
Summary / Next
Exercises
Example: pure birth process
• Exponential interarrival times, mean 1/
• Arrival process is Poisson process
• Markov chain?
• Transition rates : let t0<t1<…<tn<t
P( X (t h) j 1 | X (t ) j , X (t 0) j 0,..., X (tn) jn)
P( X (t h) j 1 | X (t ) j ) h o(h)
P( X (t h) j 2 | X (t ) j ) o(h)
P( X (t h) j | X (t ) j ) 1 h o(h)
q( j, k )
k j 1
k j
• Kolmogorov forward equations for P(X(0)=0)=1
dPt (0, j )
Pt (0, j 1) Pt (0, j )
dt
dPt (0,0)
Pt (0, j )
dt
• Solution for P(X(0)=0)=1
(t ) j t
Pt (0, j )
e ,
j!
j 0,1,2,...,t 0
Flows and Networks
Plan for today (lecture 2):
•
•
•
•
•
•
•
•
•
•
•
Questions?
Birth-death process
Example: pure birth process
Example: pure death process
Simple queue
General birth-death process: equilibrium
Reversibility, stationarity
Truncation
Kolmogorov’s criteria
Summary / Next
Exercises
Example: pure death process
• Exponential holding times, mean 1/
• P(X(0)=N)=1, S={0,1,…,N}
• Markov chain?
• Transition rates : let t0<t1<…<tn<t
P( X (t h) j 1 | X (t ) j , X (t 0) j 0,..., X (tn) jn)
P( X (t h) j 1 | X (t ) j ) jh o(h)
P( X (t h) j 2 | X (t ) j ) o(h)
P( X (t h) j | X (t ) j ) 1 jh o(h)
j
q( j, k )
j
k j 1
k j
• Kolmogorov forward equations for P(X(0)=N)=1
dPt ( N , j )
( j 1) Pt (0, j 1) jPt (0, j )
dt
dPt ( N , N )
NPt (0, N )
dt
• Solution for P(X(0)=N)=1
N t j
Pt ( N , j ) e
1 et
j
N j
,
j 0,1,2,...,N , t 0
Flows and Networks
Plan for today (lecture 2):
•
•
•
•
•
•
•
•
•
•
•
Questions?
Birth-death process
Example: pure birth process
Example: pure death process
Simple queue
General birth-death process: equilibrium
Reversibility, stationarity
Truncation
Kolmogorov’s criteria
Summary / Next
Exercises
Simple queue
• Poisson arrival proces rate , single server
exponential service times, mean 1/
• Assume initially empty:
P(X(0)=0)=1,
S={0,1,2,…,}
• Markov chain?
• Transition rates :
P( X (t h) j 1 | X (t ) j ) h o(h)
P( X (t h) j 1 | X (t ) j ) h o(h)
P( X (t h) j | X (t ) j ) 1 [h h] o(h)
k j 1
k j 1, j 0
q( j, k )
[ ] k j , j 0
k j, j 0
Simple queue
• Poisson arrival proces rate , single server
exponential service times, mean 1/
k j 1
k j 1, j 0
q( j, k )
[ ] k j , j 0
k j, j 0
• Kolmogorov forward equations, j>0
dPt (i, j )
Pt (i, j 1) Pt (i, j )[ ] Pt (i, j 1)
dt
dPt (i,0)
Pt (i,0) Pt (i,1)
dt
• Global balance equations, j>0
0 ( j 1) ( j )[ ] ( j 1)
0 (0) (1)
Simple queue (ctd)
j
j+1
Equilibrium distribution: <
( j ) (0)( / ) j
(1 / )( / ) j
Stationary measure; summable eq. distrib.
Proof: Insert into global balance
Detailed balance!
( j )q( j, j 1) ( j 1)q( j 1. j )
Flows and Networks
Plan for today (lecture 2):
•
•
•
•
•
•
•
•
•
•
•
Questions?
Birth-death process
Example: pure birth process
Example: pure death process
Simple queue
General birth-death process: equilibrium
Reversibility, stationarity
Truncation
Kolmogorov’s criteria
Summary / Next
Exercises
Birth-death process
• State space S N {0,1,2,...}
• Markov chain, transition rates
( j)
k j 1
birth rate
( j)
k j 1, j 0 death rate
q( j, k )
k j, j 0
( j ) ( j )
(0)
k j, j 0
• Definition: Detailed balance equations
( j )q( j, j 1) ( j 1)q( j 1. j )
• Theorem: A distribution that satisfies detailed
balance is a stationary distribution
• Theorem: Assume that
(0)
jS
then
1
q(r 1, r )
q(r , r 1)
j
r 1
j
( j ) (0)
r 1
q(r 1, r )
,
q(r , r 1)
jS
is the equilibrium distrubution of the birth-death
prcess X.
Flows and Networks
Plan for today (lecture 2):
•
•
•
•
•
•
•
•
•
•
•
Questions?
Birth-death process
Example: pure birth process
Example: pure death process
Simple queue
General birth-death process: equilibrium
Reversibility, stationarity
Truncation
Kolmogorov’s criteria
Summary / Next
Exercises
Reversibility; stationarity
• Stationary process: A stochastic process is
stationary if for all t1,…,tn,
( X (t1 ), X (t2 ),...,X (tn )) ~ ( X (t1 ), X (t2 ),...,X (tn ))
• Theorem: If the initial distribution is a
stationary distribution, then the process is
stationary
• Reversible process: A stochastic process is
reversible if for all t1,…,tn,
( X (t1 ), X (t2 ),...,X (tn )) ~ ( X ( t1 ), X ( t2 ),...,X ( tn ))
( j) 1
jS
NOTE: labelling of states only gives suggestion of
one dimensional state space; this is not
required
Reversibility; stationarity
• Lemma: A reversible process is stationary.
• Theorem: A stationary Markov chain is
reversible if and only if there exists a
collection of positive numbers π(j), jS,
summing to unity that satisfy the detailed
balance equations
( j )q( j, k ) (k )q(k , j ), j, k S
When there exists such a collection π(j), jS, it
is the equilibrium distribution
• Proof
Flows and Networks
Plan for today (lecture 2):
•
•
•
•
•
•
•
•
•
•
•
Questions?
Birth-death process
Example: pure birth process
Example: pure death process
Simple queue
General birth-death process: equilibrium
Reversibility, stationarity
Truncation
Kolmogorov’s criteria
Summary / Next
Exercises
10
Truncation of reversible processes
Lemma 1.9 / Corollary 1.10:
If the transition rates of a reversible Markov process with
state space S and equilibrium distribution ( j ), j S are
altered by changing q(j,k) to cq(j,k) for j A, k S \ A
where
c>0
then the resulting Markov process is
reversible in equilibrium and has equilibrium distribution
where B
is the normalizing constant.
B ( j )
Bc ( j )
If
c=0
jA
jS \ A
then the reversible Markov process
is truncated to A
and the resulting Markov
process is reversible with equilibrium distribution
( j)
(k )
jA
k A
S\A
A
Time reversed process
X(t) reversible Markov process X(-t) also, but
Lemma 1.11: tijdshomogeneity not inherited for nonstationary process
Theorem 1.12 : If X(t) is a stationary Markov process
with transition rates q(j,k), and equilibrium
distribution π(j), jS, then the reversed process
X(-t) is a stationary Markov process with
transition rates
(k )q(k , j )
j, k S
q' ( j , k )
( j)
and the same equilibrium distribution
Theorem 1.13: Kelly’s lemma
Let X(t) be a stationary Markov processwith
transition rates q(j,k). If we can find a collection of
numbers q’(j,k) such that q’(j)=q(j), jS, and a
collection of positive numbers (j), jS, summing to
unity, such that
( j )q( j , k ) (k ) q' ( k , j )
j, k S
then q’(j,k) are the transition rates of the timereversed process, and (j), jS, is the equilibrium
distribution of both processes.
Flows and Networks
Plan for today (lecture 2):
•
•
•
•
•
•
•
•
•
•
•
Questions?
Birth-death process
Example: pure birth process
Example: pure death process
Simple queue
General birth-death process: equilibrium
Reversibility, stationarity
Truncation
Kolmogorov’s criteria
Summary / Next
Exercises
Kolmogorov’s criteria
• Theorem 1.8:
A stationary Markov chain is reversible iff
q( j1 , j2 )q( j2 , j3 )...q( jn1 , jn )q( jn , j1 )
q( j1 , jn )q( jn , jn1 )...q( j3 , j2 )q( j2 , j1 )
for each finite sequence of states
j1, j2 ,..., jn S
Notice that
( j ) (0)
q(0, j1 )q( j1 , j2 )q( j2 , j3 )...q( jn1 , jn )q( jn , j )
q( j, jn )q( jn , jn1 )...q( j3 , j2 )q( j2 , j1 )q( j1 ,0)
Flows and Networks
Plan for today (lecture 2):
•
•
•
•
•
•
•
•
•
•
•
Questions?
Birth-death process
Example: pure birth process
Example: pure death process
Simple queue
General birth-death process: equilibrium
Reversibility, stationarity
Truncation
Kolmogorov’s criteria
Summary / Next
Exercises
Summary / next:
•
•
•
•
•
Birth-death process
Simple queue
Reversibility, stationarity
Truncation
Kolmogorov’s criteria
• Next
input / output simple queue
Poisson proces
PASTA
Output simple queue
Tandem netwerk
Exercises
[R+SN] 1.3.2, 1.3.3, 1.3.5, 1.5.1, 1.5.2, 1.5.5,
1.6.2, 1.6.3, 1.6.4