Transcript No Slide Title

Things I Thought I Knew About
Queueing Theory, but was Wrong
About: Part 1, Multiserver Queues
Alan Scheller-Wolf
Tepper School of Business
Carnegie Mellon University
Joint work with
Karl Sigman, Rein Vesilo, and Michele Dufalla
January 15, 2013
Alan Scheller-Wolf
Lunteren, The Netherlands
Page 1
A Great Day
As a graduate student, one of the greatest things that
can happen is the following:
Student: “I can prove that XXX is true!”
Advisor: “That can’t be right, everyone knows YYY
is true.”
Student: “I know, but I can prove it.”
Advisor: “There must be a bug, show me.”
Time passes...
Advisor: “Huh, you are right, XXX is true!”
January 15, 2013
Alan Scheller-Wolf
Lunteren, The Netherlands
Page 2
My Goals
• Take you through “my great day” as a student
• Help you understand why everyone believed YYY
• Help you understand, mathematically, why XXX
is true
• Help you see why, intuitively, XXX “has to be
true”
• Discuss what general insights this gives
January 15, 2013
Alan Scheller-Wolf
Lunteren, The Netherlands
Page 3
Mean Delay in GI/GI/1 Queues
i.i.d. arrivals ~ T
Single Server Queue
i.i.d. service ~ S
rate l
rate m
• Jobs are served in a FIFO (FCFS) fashion
• Delay: Time customers spend
in queue
S should be “heavy tailed” -• Question: What are conditions
for mean
think Pareto
distribution
(stationary) delay to be finite?
• Kiefer and Wolfowitz (K&W, 1956)
 r =l/m < 1
E[D] finite iff E[S2] finite
January 15, 2013
Alan Scheller-Wolf
Lunteren, The Netherlands
Page 4
2
E[S ]
Why is
Necessary
(for Single-Server Queues)?
• The inspection paradox
A randomly arriving job sees an equilibrium service
time Se (or the excess of a service time)
Length biasing causes the tail of Se [P(Se>x)] to behave
as the integrated tail of S – you are more likely to “see”
longer service times
E[Se] is finite iff E[S2] is finite
• The upshot: If an arriving job “sees” a job in
service, this job has mean remaining size on the
order of E[S2]. The arriving job can’t enter
service until this job is done.
January 15, 2013
Alan Scheller-Wolf
Lunteren, The Netherlands
Page 5
2
E[S ]
Why is
Sufficient
(for Single-Server Queues)? (I)
• The Lindley Equation (Lindley, 1952)
Dn+1 = (Dn+Sn-Tn)+, where Sn is the service time of
customer n and Tn is the interarrival time between
customers n and n+1
Why?
Let’s say customer n arrives at time t
Then customer n leaves service at time t+Dn+Sn
Customer n+1 arrives at time t + Tn
Customer n+1 enters service at time min{t+Dn+Sn,t + Tn}
Customer n+1’s delay is min{t+Dn+Sn,t + Tn} – (t + Tn)
This is a reflected random walk
January 15, 2013
Alan Scheller-Wolf
Lunteren, The Netherlands
Page 6
2
E[S ]
Why is
Sufficient
(for Single-Server Queues)? (II)
Reflected Random Walks:
• For Zn+1 = (Zn+Xn)+, E[Z2] is finite if:
E[(X+)2] is finite
E[Xn] < -d < 0 for all n and on every sample path
• For our GI/GI/1 queue:
E[(X+)2] = E[S2] is finite
E[Xn] = E[S] - E[T] < -d < 0 ; for all n and on every
sample path, since r<1
January 15, 2013
Alan Scheller-Wolf
Lunteren, The Netherlands
Page 7
Mean Delay in GI/GI/k Queues
Multi-Server Queue
i.i.d. arrivals ~ T
i.i.d. service ~ S
rate l
rate m
• Question: What are conditions for mean
stationary delay to be finite?
• K&W, 1956
 r =l/ km < 1
E[D] finite if E[S2] finite
January 15, 2013
Alan Scheller-Wolf
Lunteren, The Netherlands
Page 8
Why is E[S2] Sufficient?
• Split the GI/GI/k into k GI/GI/1 queues
This performs no better than the original system
• Route arriving jobs to queues in cyclic (or round
robin) fashion
• Each GI/GI/1 queue has finite mean delay
• Therefore the original GI/GI/k queue has finite
mean delay as well
January 15, 2013
Alan Scheller-Wolf
Lunteren, The Netherlands
Page 9
What about Necessity?
“Everyone knew” that
r =l/ km < 1
E[S2] finite
were necessary as well.
This had even appeared in the literature.
January 15, 2013
Alan Scheller-Wolf
Lunteren, The Netherlands
Page 10
Pictorially:
M
o
m
e
2
n
t
s
o
f
Requirement for finite mean delay
1
S
1
2
3
4
Servers
January 15, 2013
Alan Scheller-Wolf
Lunteren, The Netherlands
∞
Page 11
Back to the GI/GI/1
• Why is finite E[S2] Necessary?
• The inspection paradox
If an arriving job “sees” a job in service, this
job has mean remaining size on the order of
E[S2]. The arriving job can’t enter service until
this job is done.
• Does this logic hold for the GI/GI/k?
NO!
• Can we capture this effect mathematically?
January 15, 2013
Alan Scheller-Wolf
Lunteren, The Netherlands
Page 12
Lindley Equation for GI/GI/1
• Our logic for GI/GI/1
Let’s say customer n arrives at time t
Then customer n leaves service at time t+Dn+Sn
Customer n+1 arrives at time t + Tn
Customer n+1 enters service at time min{t+Dn+Sn,t +
Tn}
Customer n+1’s delay is min{t+Dn+Sn,t + Tn} – (t + Tn)
• Dn+1 = (Dn+Sn-Tn)+
January 15, 2013
Alan Scheller-Wolf
Lunteren, The Netherlands
Page 13
Lindley Equation for GI/GI/k
• For GI/GI/k
Let’s say customer n arrives at time t
Then customer n leaves service at time t+Dn+Sn
Customer n+1 arrives at time t + Tn
Customer n+1 enters service at time min{t+Dn+Sn,t + Tn}
NO!
January 15, 2013
Alan Scheller-Wolf
Lunteren, The Netherlands
Page 14
When Does Customer n+1
Enter Service? (I)
n
n+1
j
• Customer n arrives at time t
• Customer n leaves service at time t+Dn+Sn
January 15, 2013
Alan Scheller-Wolf
Lunteren, The Netherlands
Page 15
When Does Customer n+1
Enter Service? (II)
n
n+1
j
If there were multiple
other servers, Un would
be the minimum
remaining time
• When does customer j leave service?
Let’s say that when customer n enters service, she
looks over at the other server, and observes the
amount of time customer j has left
Let’s call this amount of time Un
Then customer j leaves service at time t+Dn+Un
January 15, 2013
Alan Scheller-Wolf
Lunteren, The Netherlands
Page 16
When Does Customer n+1
Enter Service? (III)
n
n+1
j
• Customer n leaves service at time t+Dn+Sn
• Customer j leaves service at time t+Dn+Un
• So the next server comes free at time
t+Dn+min{Sn,Un} = t+Dn+Pn
January 15, 2013
Alan Scheller-Wolf
Lunteren, The Netherlands
Page 17
Lindley Equation for GI/GI/k
• For GI/GI/k:
Dn+1 = (Dn+Pn-Tn)+
We have replaced Sn with Pn
How big of difference could one letter make?
January 15, 2013
Alan Scheller-Wolf
Lunteren, The Netherlands
Page 18
Back to Reflected Random
Walks
• For our GI/GI/k queue, for finite mean delay it is
sufficient that:
E[(X+)2] = E[P2] is finite
E[Xn] = E[Pn] - E[Tn] < -d < 0 ; for all n and on every
sample path
How does this help?
We don’t know the first thing about P!
January 15, 2013
Alan Scheller-Wolf
Lunteren, The Netherlands
Page 19
When is E[P2] Finite?
E[P2] = E[min{S,U}2]
= E[min{S,U}3/2min{S,U}1/2]
S is independent of U
3/2
1/2
< E[S U ]
= E[S3/2] E[U1/2]
W is the total work
in system
And E[U1/2] < E[W1/2]
And E[W1/2] is finite iff E[S3/2] (K&W, 1956)
So for a GI/GI/2 queue, E[P2] is finite if E[S3/2] is finite!
January 15, 2013
Alan Scheller-Wolf
Lunteren, The Netherlands
Page 20
What about Negative Drift?
E[Xn] = E[Pn] - E[Tn]
= E[min{Sn,Un}] - E[Tn]
< E[Sn] - E[Tn]
= E[S] - E[T]
This seems like a really
weak bound, but is in fact
the best we can do, since this
must hold on all sample paths
and for all n; Un could be
arbitrarily large on a sample
path
I have negative drift for all n on every sample path if
E[S] < E[T] (r < ½)
but not otherwise
January 15, 2013
Alan Scheller-Wolf
Lunteren, The Netherlands
Page 21
Finite Mean Delay in GI/GI/2 (I)
So it appears that:
• Finite E[S3/2] is sufficient for finite mean delay in
GI/GI/2 queues if E[S] < E[T]
• Finite E[S3/2] is not sufficient for finite mean delay
in GI/GI/2 queues if E[T] < E[S] < 2E[T]
Does this make sense?
YES!
January 15, 2013
Alan Scheller-Wolf
Lunteren, The Netherlands
Page 22
Finite Mean Delay in GI/GI/2
(II)
• The inspection paradox
If an arriving job “sees” a job in service, this
job has mean remaining size on the order of
E[S2]
The mean time this job has been in service also
is on the order of E[S2]
The key question is what has been going on
while this long job has been in service
January 15, 2013
Alan Scheller-Wolf
Lunteren, The Netherlands
Page 23
Finite Mean Delay in GI/GI/2
(III)
• If E[T] < E[S] < 2E[T]:
While a single extremely large job is in service
the remaining server cannot keep up with the
workload, and the system “crashes”
• If E[S] < E[T]:
A single extremely large job is not typically
enough to crash the system – the remaining
server can keep up with the workload
It takes two extremely large jobs to crash the
system
January 15, 2013
Alan Scheller-Wolf
Lunteren, The Netherlands
Page 24
Redefining r
• For a k-server queue r is traditionally defined as
r =l/ km
This is WRONG!
• If instead r =l/m
E[S] < E[T] means r<1, and we have a “spare server”
to absorb the workload if a long job arrives
E[T] < E[S] < 2E[T] means 1<r<2, and we have no
spare capacity
r =l/m denotes the number of servers needed
to handle the mean load on the system
January 15, 2013
Alan Scheller-Wolf
Lunteren, The Netherlands
Page 25
What about GI/GI/3?
• If 2<r<3 (no spare servers):
E[D] is finite if E[S2] is finite
• If 1<r<2 (one spare server):
E[D] is finite if E[S3/2] is finite
• If 0<r<1 (two spare servers):
E[D] is finite if E[S4/3] is finite
January 15, 2013
Alan Scheller-Wolf
Lunteren, The Netherlands
Page 26
What about GI/GI/k?
• If k-1<r<k (no spare servers):
E[D] is finite if E[S2] is finite
• If k-2<r<k-1 (one spare server):
E[D] is finite if E[S3/2] is finite
• If j<r<j+1 (k-j-1 spare servers):
E[D] is finite if E[S(k+1-j)/(k-j)] is finite
January 15, 2013
Alan Scheller-Wolf
Lunteren, The Netherlands
Page 27
Pictorially: Fixed Number of
For moment
conditions,
Pictorially:
20-server queue with load
18.5
Spare
Servers
is the same as a 2-server queue
withRequirement
load 0.5
for finite mean delay
M
o
m
e
2
n
t
s
o
f
No spare servers
Diminishing benefits
One spare server
Two spare servers
3/2
4/3
1
S
1
2
3
4
Servers
January 15, 2013
Alan Scheller-Wolf
Lunteren, The Netherlands
∞
Page 28
Pictorially: As Spare Servers
Grow
M
o
m
e
2
n
t
s
o
f
Requirement for finite mean delay
As the number of spare servers
approaches infinity, the moment
conditions approach those of an
infinite-server queue, as they should
3/2
4/3
1
S
1
2
3
4
Servers
January 15, 2013
Alan Scheller-Wolf
Lunteren, The Netherlands
∞
Page 29
Other Results (I)
• Higher Moments:
 If j<r<j+1 (k-j-1 spare servers):
E[Dr] is finite if E[S(k+r-j)/(k-j)] is finite
(r has replaced 1 in the previous formula)
 If you have just two spare servers, E[D2] is
finite if E[S5/3] is finite – you get finite variance
of delay without finite variance of service time
January 15, 2013
Alan Scheller-Wolf
Lunteren, The Netherlands
Page 30
Other Results (II)
• Necessary Conditions:
 The sufficient conditions above are also
provably necessary for all but a small set of
service time distributions
January 15, 2013
Alan Scheller-Wolf
Lunteren, The Netherlands
Page 31
Other Results (III)
• Integral Load:
 Drift has been crucial in our analysis – if r<1
E[S3/2] is sufficient for finite mean delay; if r>1
we require E[S2]
 What if r=1?
 We have been able to show that E[S3/2] is not
sufficient, but have not been able to get close
to showing E[S2] is necessary
 We are still working
January 15, 2013
Alan Scheller-Wolf
Lunteren, The Netherlands
Page 32
Insights
• Delay moments in multi-server queues with heavytailed service times depend on the moments of S
and the number of spare servers
r is most usefully defined as l/m for multiserver queues
• Adding spare serves helps with delay moments,
but in diminishing fashion
• This works because under heavy-tailed service
times queues build up due to large jobs blocking
servers
January 15, 2013
Alan Scheller-Wolf
Lunteren, The Netherlands
Page 33
Open Problems
1. Dependencies:
Everything we showed was for GI/GI/k; what if
interarrival and/or service times are not i.i.d?
Our “Lindley” equation for the GI/GI/k:
Dn+1 = (Dn+Pn-Tn)+
also holds for a special type of GI/G/1 queue.
(It actually defines a GI/G/1 queue.) So there
must be a relationship/duality here.
January 15, 2013
Alan Scheller-Wolf
Lunteren, The Netherlands
Page 34
Open Problems (II)
2. Bounds:
 As of now, the best analytical bounds on multiserver delay under heavy-tailed service time are
in general very poor
These cannot, in general, be easily simulated
 See work by Daley, Scheller-Wolf, Vesilo
January 15, 2013
Alan Scheller-Wolf
Lunteren, The Netherlands
Page 35
Open Problems (III)
• Tail Asymptotics
Moment conditions for delay are really just a
proxy for what we really want to know: How
the tail of the delay distribution behaves
See work by Boxma, Foss, Korshunov, Whitt,
Zwart
January 15, 2013
Alan Scheller-Wolf
Lunteren, The Netherlands
Page 36
References
Scheller-Wolf, A. (2003). Necessary and Sufficient Conditions for Delay
Moments in FIFO Multiserver Queues: Why s Slow Servers are Better than
One Fast Server for Heavy-Tailed Systems. Operations Research 51, No. 5,
748-758.
Scheller-Wolf, A. and R. Vesilo (2006). Structural Interpretation and
Derivation of Necessary and Sufficient Conditions for Delay Moments in
FIFO Multiserver Queues. Queueing Systems, 54, No. 3 221-232.
January 15, 2013
Alan Scheller-Wolf
Lunteren, The Netherlands
Page 37
Dank U!
Any Questions?
January 15, 2013
Alan Scheller-Wolf
Lunteren, The Netherlands
Page 38