Transcript 3rd Edition: Chapter 1

Introduction to Queuing
Theory
Queueing theory definitions
 (Bose) “the basic phenomenon of queueing arises
whenever a shared facility needs to be accessed for
service by a large number of jobs or customers.”
 (Wolff) “The primary tool for studying these
problems [of congestions] is known as queuing
theory.”
 (Kleinrock) “We study the phenomena of standing,
waiting, and serving, and we call this study Queuing
Theory." "Any system in which arrivals place demands
upon a finite capacity resource may be termed a
queueing system.”
 (Mathworld) “The study of the waiting times, lengths,
and other properties of queues.”
2
Applications of Queuing Theory
 Telecommunications
 Traffic control
 Determining sequence of computer operations
 Predicting computer performance
 Health services (e.g., control of hospital bed
assignments)
 Airport traffic, airline ticket sales
 Layout of manufacturing systems.
3
Example application of queuing
theory
 In many retail stores and banks


multiple line/multiple checkout system  a
queuing system where customers wait for the
next available cashier
We can prove using queuing theory that :
throughput improves increases when queues are
used instead of separate lines
4
Example application of queuing
theory
5
Queuing theory for studying
networks
 View network as collections of queues
 FIFO data-structures
 Queuing theory provides probabilistic
analysis of these queues
 Examples:




Average length
Average waiting time
Probability queue is at a certain length
Probability a packet will be lost
6
Model Queuing System
Queuing System
Queue
Queuing System
Server
Server System
 Use Queuing models to
 Describe the behavior of queuing systems
 Evaluate system performance
7
Characteristics of queuing
systems
 Arrival Process

The distribution that determines how the tasks
arrives in the system.
 Service Process

The distribution that determines the task
processing time
 Number of Servers

Total number of servers available to process
the tasks
8
Kendall Notation 1/2/3(/4/5/6)
 Six parameters in shorthand
•
1.
2.
3.
4.
5.
6.
First three typically used, unless specified
Arrival Distribution
Service Distribution
Number of servers
Total Capacity (infinite if not specified)
Population Size (infinite)
Service Discipline (FCFS/FIFO)
9
Distributions
 M: stands for "Markovian", implying
exponential distribution for service times
or inter-arrival times.
 D: Deterministic (e.g. fixed constant)
 Ek: Erlang with parameter k
 Hk: Hyperexponential with param. k
 G: General (anything)
10
Kendall Notation Examples
 M/M/1:
 Poisson arrivals and exponential service, 1 server, infinite
capacity and population, FCFS (FIFO)
 the simplest ‘realistic’ queue
 M/M/m
 Same, but M servers
 G/G/3/20/1500/SPF
 General arrival and service distributions, 3 servers, 17
queue slots (=20-3), 1500 total jobs, Shortest Packet
First
11
Analysis of M/M/1 queue
 Given:
•
•
l: Arrival rate of jobs (packets on input link)
m: Service rate of the server (output link)
 Solve:
 L: average number in queuing system
 Lq average number in the queue
 W: average waiting time in whole system
 Wq average waiting time in the queue
12
M/M/1 queue model
L
Lq
l
m
Wq
W
13
Little’s Law
System
Arrivals
Departures
 Little’s Law:
Mean number tasks in system = mean arrival rate x
mean response time

Observed before, Little was first to prove
 Applies to any system in equilibrium, as long as nothing
in black box is creating or destroying tasks
14
Proving Little’s Law
Arrivals
Packet 3
#
2
1
# in 3
System 2
1
Departures
1 2 3 4 5 6 7 8
Time
1 2 3 4 5 6 7 8
Time
J = Shaded area = 9
Time in 3
System 2
1
Same in all cases!
1 2 3
Packet #
15
Definitions
 J: “Area” from previous slide
 N: Number of jobs (packets)
 T: Total time
 l: Average arrival rate
 N/T
 W: Average time job is in the system
 = J/N
 L: Average number of jobs in the system
 = J/T
16
Proof: Method 1: Definition
# in 3
System 2
(L) 1
=
Time in 3
System 2
(W)
1
1 2 3 4 5 6 7 8
Time (T)
J  TL  NW
1 2 3
Packet # (N)
L  ( TN )W
L  (l )W
17
Proof: Method 2: Substitution
L  (l )W
L  ( TN )W
J
T
 ( TN )( NJ )
J
T
 TJ
Tautology
18
M/M/1 queue model
L
Lq
l
m
Wq
L=λW
Lq=λWq
W
19
Poisson Process
 For a poisson process with average arrival
rate l , the probability of seeing n arrivals
in time interval delta t
e  lt (lt ) n
Pr(n) 
n!
E (n)  lt
(lt ) 2
Pr(0)  e  1  lt 
...  1  lt  o(t )  Pr(0)  1  lt
2!
(lt ) 2
 lt
Pr(1)  lte  lt[1  lt 
...]  lt  o(t )  Pr(1)  lt
2!
Pr( 2)  ...  0
 lt
20
Poisson process & exponential
distribution
 Inter-arrival time t (time between
arrivals) in a Poisson process follows
exponential distribution with parameter
P r(t )  le
1
E (t ) 
l
 lt
l
21
M/M/1 queue model
L
Lq
l
m
1
Wq
L=λW
Lq=λWq
W = Wq + (1/μ)
m
W
22
Solving queuing systems
 4 unknowns: L, Lq W, Wq
 Relationships:
 L=lW
 Lq=lWq
 W = Wq + (1/m)
 If we know any 1, can find the others

L   nPn
n0
23
Analysis of M/M/1 queue
 Goal: A closed form expression of the
probability of the number of jobs in the
queue (Pi) given only l and m
24
Equilibrium conditions
l
l
n-1
m
l
n
m
l
n+1
m
m
Define Pn (t ) to be the probability of having n tasks in the system at time t
P0 (t  t )  P0 (t )[(1  mt )(1  lt )  mtlt ]  P1 (t )[(mt )(1  lt )]
Pn (t  t )  Pn (t )[(1  mt )(1  lt )  mtlt ]  Pn 1 (t )[(mt )(1  lt )]  Pn 1 (t )[(lt )(1  mt )]
P0 (t  t )  P0 (t )
 lP0 (t )  mP1 (t )
t
Pn (t  t )  Pn (t )
 lPn 1 (t )  (l  m ) Pn (t )  mPn 1 (t )
t
P (t  t )  Pn (t )
St ablize when l  m ,
lim Pn (t )  Pn , lim n
0
t 
t 
t
25
Equilibrium conditions
l
l
n-1
m
l
n
m
l
n+1
m
m
lP0  mP1
(l  m ) Pn  lPn1  mPn1
26
Solving for P0 and Pn
 Step 1
2
P1 
 Step 2
n
l
l
l
P0 , P2    P0, Pn    P0
m
m
m
n
l
P

1
,
then
P

n
0 
 m   1,  P0 
n 0
n 0 



1
l
 

n 0  m 

n
27
Solving for P0 and Pn
 Step 3
n




l
l
1

1
n
  , then      

 1
m
m
1  1 
n 0 
n 0

 Step 4


P0 
1
 1  and Pn   1  
n


n
n 0
28
Solving for L



n1

(
1


)

n

L   nPn   n (1   )

n
(1   ) 
n1
n 0
n0

n
     (1 ) dd
 n0 

d
d
(1  )
 

1
(1  )2

(1  )

 
1
1 
l
m l
29
Solving W, Wq and Lq
L
l
m l
  
 W      
Wl
L
Wq
l
m l
1
m
Lq  lWq  l
1
1
m l
l
l
1
m l
l
m( m l )
m

l
m( m  l )
l2
m( m  l )
30
Online M/M/1 animation
 http://www.dcs.ed.ac.uk/home/jeh/Simjav
a/queueing/mm1_q/mm1_q.html
31
Response Time vs. Arrivals
Waiting vs. Utilization
0.25
W(sec)
0.2
0.15
0.1
0.05
0
0
0.2
0.4
0.6
0.8
1
1.2
%
W
1
m l
32
Stable Region
Waiting vs. Utilization
0.025
W(sec)
0.02
0.015
linear region
0.01
0.005
0
0
0.2
0.4
0.6
0.8
1
%
33
Example
 On a network gateway, measurements show that
the packets arrive at a mean rate of 125 packets
per second (pps) and the gateway takes about 2
millisecs to forward them. Assuming an M/M/1
model, what is the probability of buffer overflow
if the gateway had only 13 buffers. How many
buffers are needed to keep packet loss below one
packet per million?
34
Example
 Measurement of a network gateway:
 mean arrival rate (l): 125 Packets/s
 mean response time (m): 2 ms
 Assuming exponential arrivals:
 What is the gateway’s utilization?
 What is the probability of n packets in the gateway?
 mean number of packets in the gateway?
 The number of buffers so P(overflow) is <10-6?
35
Example
 Arrival rate λ =
 Service rate μ =
 Gateway utilization ρ = λ/μ =
 Prob. of
n packets in gateway =
 Mean number of packets in gateway =
36
Example





Arrival rate λ = 125 pps
Service rate μ = 1/0.002 = 500 pps
Gateway utilization ρ = λ/μ = 0.25
Prob. of n packets in gateway =
n
n
(1 )  0.75(0.25)
Mean number of packets in gateway =


0.25

 0.33
1  0.57
37
Example
 Probability of buffer overflow:
 To limit the probability of loss to less than
10-6:
38
Example


Probability of buffer overflow:
= P(more than 13 packets in gateway)
To limit the probability of loss to less
than 10-6:
39
Example


Probability of buffer overflow:
= P(more than 13 packets in gateway)
= ρ13 = 0.2513 = 1.49x10-8
= 15 packets per billion packets
To limit the probability of loss to less
than 10-6:
40
Example


Probability of buffer overflow:
= P(more than 13 packets in gateway)
= ρ13 = 0.2513 = 1.49x10-8
= 15 packets per billion packets
To limit the probability of loss to less
than 10-6:
 n  106
41
Example



To limit the probability of loss to less
than 10-6:
 n  106
or


n  log 106 / log0.25
42
Example



To limit the probability of loss to less
than 10-6:
 n  106
or


n  log 106 / log0.25
= 9.96 ≈10 buffers
43
Example
 One fast server vs. m slow servers?

In terms of delay, what will happen?
44
Example
 Customers arrive at a fast-food restaurant
at a rate of 5/minute and wait to receive
their order for an average of 5 minutes.
Customers eat in the restaurant with
probability 0.5 and carry out their order
without eating with probability 0.5. A meal
requires an average of 20 minutes. What is
the average number of customers in the
restaurant?
45
Example
 Empty taxis pass by a street comer at a
Poisson rate of 2 per minute and pick up a
passenger if one is waiting there.
Passengers arrive at the street corner at a
Poisson rate of 1 per minute and wait for a
taxi only if there are fewer than 4 persons
waiting; otherwise, they leave and never
return. Find the average waiting time of a
passenger who joins the queue.
46
Example
 The average time,T, a car spends in a
certain traffic system is related to the
average number of cars N in the system by
a relation of the form T = a + bN2, where a
> 0, b > 0 are give in scalars.

What is the maximal car arrival rate that the
system can sustain?
47
Example
 A person enters a bank and finds all of the four
clerks busy serving customers. There are no other
customers in the bank, so the person will start
service as soon as one of the customers in service
leaves. Customers have independent, identical,
exponential distribution of service time.


What is the probability that the person will be the last
to leave the bank assuming that no other customers
arrive?
If the average service time is 1 minute, what is the
average time the person will spend?
48