Queueing Theory

Overview

• • • • Introduction Basic Queue Properties – Kendall Notation – Little’s Law Stochastic Processes – Birth-Death Process – Markov Process Queueing Models

Why Queueing Theory

• • Mathematical properties of lines or “queues” Useful to understand delays and congestions in computer networks – – Tool for packet switched networks Limited service/processing capability – Probabilistic arrivals Customers (Arrivals) Server Queue

Example Queueing System

Departures

Analysis

• Analysis of the queue can help identify numerous transient and steady state properties about the system, including – # of customers in system/queue – Response time – Wait time – Utilization – Throughput

Queue Properties

• • • • Arrival process – The rate and distribution that customers arrive to the system Service patterns – The rate and distribution at which the system can process customers Number of servers Queueing discipline – First in-First out, Last in-first out, Round Robin

Queue System Notation

• Kendal Notation

Interarrival time distribution

M – Exponential D – Deterministic G – General

# of parallel service channels

A/B/X/Y/Z

Service time distribution

M – Exponential D – Deterministic G – General Capacity of the System

(Default = infinite)

Queue discipline

FIFO – First in first out LIFO – Last in first out RSS – Random selection of service GD – General discipline

(Default = FIFO)

Queue Performance Parameters

T – Time in system, W = E[T]

T q

– Time in queue

(W q = E[T q ])

S – Service time

(1/μ = E[S]) Time in the system

Arrival rate (λ) Server Departures Queue

L q N q – # customers in queue – avg. # customers in queue N s – # customers in service N – # customers in system L – avg. # customers in system # customers in system

Queue Stability

• • • μ – service rate λ – arrival rate λ/μ - traffic intensity – λ/μ < 1 for stable queue • λ/μ = .1 – light load • λ/μ = .5 – moderate load • λ/μ = .9 – heavy load

Little’s Law

• Average number of customers in a queueing system equals the arrival rate of new customers times the customer service rate

L L q

= = l

W

l

W q

• Intuitively, longer service times equals longer queues – Examples • Traffic jams occur with accidents, bad weather • Jimmy John’s has less seating than Zoe’s

Little’s Law

• Your computer networking professor receives 30 emails per day, on average he has 15 unchecked messages, how long until he responds to your email? – L= λW – W = L/λ = 15/30 = .5 day • A switch receives 100 packets every second, the switch can process each packet in 8ms, what’s the number of packets in system?

• L = λ/W • L= 100 pps x .008sec = 8 packets

Little’s Law Derivation

4 3

N

2 1

Time

t 1 t 2 t 3 t 4 t 5 t 6 t 7 T

• Show: LT = WN – c N c /T – arrival rate

G/G/c General Properties

• Useful Equations (c = number of servers): • L = λW [Little’s Law (also L q = λW q )] • r = λ/μ [work load rate] • ρ = λ/cμ [utilization/traffic intensity] • p 0 = 1 – ρ [probability system is empty]

Utilization vs Queueing Delay

Avg. Queuing Delay [W = 1/(μ-λ)] Utilization [ρ = λ/μ] Can’t fully utilize network without long delays!!!!

Stochastic Processes

Stochastic Process

• • Probability process that takes random values,

X(t)=x(t 1 ),…,x(t n

) for times t

1 ,….,t n

Can be – Discrete -time •

T = {0, 1, 2, ….}

• Example: coin flip – Continuous -time •

T = {0< t < ∞}

• Example: stock market, weather

Probability Review

• Bernoulli trial – Random experiment with only two outcomes – Example, flip of coin (heads and tails) • If I flip 5 coins, what is the probability of 3 heads?

– Binomial distribution Possible combinations of k events

P

(

X

=

k

) = è

n k

ö

p k

(1 ø

p

)

n

-

k

Probability of n-k non-events Probability of k events

Poisson Distribution

• • • Assume arrival times of some event follow an exponential distribution X(t) for t≥0 represents the number of arrivals up to time period t p x (t) = probability x arrivals in time t

p x

(

t

)  ( 

t

)

x e

 

t x

!

Poisson Distribution

• Example: Assume 5 (λ=5) packets arrive per second • Probability of seeing exactly 5 packets – p(5) • Probability of seeing less than 10 packets in a second – 1-[p(0) + p(1) + … + p(10)] P(0) P(1) P(2) P(3) P(4) P(5) P(6) P(7) P(8) P(9) P(10) .007

.034

.081

.135

.175

.175

.141

.101

.061

.034

.013

λ .

..

λ 2 λ 1 λ n λ λ 1 λ n λ 2 .

..

Poisson Process

• Superposition – Multiple Poisson processes aggregate to Poisson process with higher rate  

i n

  1 

i

• Decomposition – Single Poisson process decomposes to multiple lower rate Poisson processes

Exponential Distribution

•

F

(

t

) = l

e

l

t E

(

t

) = 1 l Used to model – Arrival rate – Service rate λ=1 • Memoryless (Markov) property Pr(T > s+t |T > s)= Pr(T > t)

Markov Process

• Discrete or continuous process where Pr(X n Pr(X n = x n = x n |X n-1 |X n-1 = x n-1 , X n-2 = x n-1 ) = x n-2 , … ,X 0 = x 0 ) = – Memoryless • process Present state only the precious state, not those earlier • Classified by: – Index set (discrete, continuous) – State space • Markov Chain – discrete • Markov Process – continuous

Birth-Death Process (BDP)

• • Continuous time Markov chain – – State n represents size of population

p n

is probability system in state n Transition types – λ i – birth rate, moves system from state n to n+1 – μ i – death rate, moves system from n to n-1 λ 0 λ 1 λ 2 λ k … 0 1 2 k μ 1 μ 2 μ 3 μ k-1

BDP Probabilities

• Flow balance –

p n

is steady state probability of system being in state n ( l n + m n )p n = l n-1 p n-1 l 0 p 0 = m 1 p 1 + m n+1 p n+1 – Steady State Probability

p n

=

p

0

n

Õ

i

= 1 l

i

1 m

i

Queue Models

Types of Queues

• • • • M/M/1 – Single queue and single server M/M/c – Single queue, c servers M/M/c/m – Single queue, c servers, m buffer size M/G/1 – General service distribution

Single Server Queue

• M/M/1 – Single queue and single server – Customer arrival – • Exponentially distributed with λ – Service time • Exponential distribution with μ Customers (Arrivals) Queue Server Departures

M/M/1 Properties

• Birth-death process where: l

n

m

n

= = l m • Flow equations: ( l + m )p n = l p n-1 l p 0 = m p 1 + m p n+1 λ λ λ λ … 0 1 2 k μ μ μ μ

M/M/1 Probabilities

• Steady State Probabilities for M/M/1

p n p n p

0 =

p

0

n

Õ

i

= 1 l m = (1 = 1 r r ) r

n

=

p

0 æ ç è l m ö ÷ ø

n

M/M/1 Properties

• L – average number of events in system

L

     • W – average time spent in system – Use Little’s Law (L=λW)

W



L

   1  

Example

• Assume a network: – Receives packets at 100pps (λ=100) – Can process 200 pps (μ=200) • What is average # packets in system? • What is average time a packet is in system

Example

• Assume a network where: – Packet sent at 1000pps – Average packet size is 1000bits • Question: To ensure average delay less than 50ms, what should be link speed? – 10ms?

Statistical Multiplexing vs FDM/TDM

• • Network has m users, each send packets at λ/m What’s the average delay? • Statistical Multiplexing – Users share single network which can send μ pps pps –

W

  1   FDM/TDM • Users all allocated μ/m – Essentially m of network bandwidth independent M/M/1 queues

W

 (  /

m

) 1  (  /

m

)  

m

  Usually better to have one big server/network!!!!

Multiple Server Queue

• M/M/c – Single queue with c servers λ Customers (Arrivals) Queue λ/c λ/c λ/c Server 1 .

..

Server c Departures

M/M/c Properties

• Birth/death rates: 

n

m = ì í îï  

c

m

n

m (1 £ (

n n

< ³

c

)

c

) λ λ λ 0 1 2 μ 2μ 3μ … • Utilization:   

c

 λ cμ c λ cμ c+1

M/M/c Probabilities

• Steady state probabilities:

p n

  

n

 

c n n

!

 

n



c c

!

n



n p

0

p

0

p

0 ( 1 

n



c

) (

n



c

)   

r c c

!

( 1   ) 

n

 1

c

  1

r n n

!

   1 (

r

/

c



p

 1 )

M/M/c Properties

• Average time in system:

W

= 1 m + è

r c c

!(

cu

)(1 r ) 2 ø

p

0 • Average number in system:

L

=

r

+ æ ç è

r c

!(1 -

c

r ) 2 ö ÷ ø

p

0

Examples

• Assume a network: – Receives packets at 100pps (λ=100) – Two processor computes 100 pps • (μ=100, c=2) • What is average # packets in system? • What is average time a packet is in system