Transcript Document 7775473

CPE 619
Queueing Networks
Aleksandar Milenković
The LaCASA Laboratory
Electrical and Computer Engineering Department
The University of Alabama in Huntsville
http://www.ece.uah.edu/~milenka
http://www.ece.uah.edu/~lacasa
Overview
Queueing Network: model in which jobs departing
from one queue arrive at another queue
(or possibly the same queue)



Open and Closed Queueing Networks
Product Form Networks
Queueing Network Models of Computer Systems
2
Open Queueing Networks

Open queueing network:
external arrivals and departures



Number of jobs in the system varies with time
Throughput = arrival rate
Goal: To characterize
the distribution of
number of jobs
in the system
3
Closed Queueing Networks

Closed queueing network:
No external arrivals or departures




Total number of jobs in the system is constant
“OUT” is connected back to “IN”
Throughput = flow of jobs in the OUT-to-IN link
Number of jobs is given, determine the throughput
4
Mixed Queueing Networks


Mixed queueing networks: Open for some
workloads and closed for others 
Two classes of jobs. Class = types of jobs
All jobs of a single class have the same service
demands and transition probabilities. Within each
class, the jobs are indistinguishable
5
Series Networks



k M/M/1 queues in series
Each individual queue can be analyzed
independently of other queues
Arrival rate= l. If mi is the service rate for ith server:
6
Series Networks (cont’d)

Joint probability of queue lengths:
 product form network
7
Product-Form Network

Any queueing network in which:

When fi(ni) is some function of the number of jobs at
the ith facility, G(N) is a normalizing constant and is a
function of the total number of jobs in the system
8
Example 32.1




Consider a closed system with two queues and N jobs
circulating among the queues
Both servers have an exponentially distributed service time.
The mean service times are 2 and 3, respectively. The
probability of having n1 jobs in the first queue and n2=N-n1 jobs
in the second queue can be shown to be:
In this case, the normalizing constant G(N) is 3N+1-2N+1.
The state probabilities are products of functions of the number
of jobs in the queues. Thus, this is a product form network.
9
General Open Network of Queues


Product form networks are easier to analyze
Jackson (1963) showed that any
arbitrary open network of m-server queues
with exponentially distributed service times
has a product form
10
General Open Network of Queues (cont’d)

If all queues are single-server queues, the queue
length distribution is:

Note: Queues are not independent M/M/1 queues
with a Poisson arrival process
In general, the internal flow in such networks is not
Poisson. Particularly, if there is any feedback in the
network, so that jobs can return to previously visited
service centers, the internal flows are not Poisson

11
Closed Product-Form Networks


Gordon and Newell (1967) showed that any
arbitrary closed networks of m-server queues with
exponentially distributed service times
also have a product form solution
Baskett, Chandy, Muntz, and Palacios (1975) showed that
product form solutions exist for an even broader class of
networks
12
BCMP Networks
1. Service Disciplines:




First-come-first-served (FCFS)
Processor sharing (PS)
Infinite servers (IS or delay centers) and
Last-come-first-served-preemptive-resume (LCFS-PR)
2. Job Classes: The jobs belong to a single class while awaiting
or receiving service at a service center, but may change
classes and service centers according to fixed probabilities at
the completion of a service request
13
BCMP Networks (cont’d)
3. Service Time Distributions:



At FCFS service centers, the service time distributions must be
identical and exponential for all classes of jobs
At other service centers, where the service times should have
probability distributions with rational Laplace transforms
Different classes of jobs may have different distributions
4. State Dependent Service:



The service time at a FCFS service center can depend only on
the total queue length of the center
The service time for a class at PS, LCFS-PR, and IS center can
also depend on the queue length for that class, but not on the
queue length of other classes
Moreover, the overall service rate of a subnetwork can depend on
the total number of jobs in the subnetwork
14
BCMP Networks (cont’d)
5. Arrival Processes:




In open networks, the time between successive arrivals of a class
should be exponentially distributed
No bulk arrivals are permitted
The arrival rates may be state dependent
A network may be open with respect to some classes of jobs and
closed with respect to other classes of jobs
15
Non-Markovian Product Form Networks
By Denning and Buzen (1978)
1. Job Flow Balance: For each class, the number of arrivals to a
device must equal the number of departures from the device
2. One Step Behavior: A state change can result only from single
jobs either entering the system, or moving between pairs of
devices in the system, or exiting from the system. This
assumption asserts that simultaneous job-moves will not be
observed.
3. Device Homogeneity: A device's service rate for a particular
class does not depend on the state of the system in any way
except for the total device queue length and the designated
class's queue length. This assumption implies the following:
16
Non-Markovian PFNs (cont’d)
a. Single Resource Possession: A job may not be present
(waiting for service or receiving service) at two or more devices
at the same time
b. No Blocking: A device renders service whenever jobs are
present; its ability to render service is not controlled by any
other device
c. Independent Job Behavior: Interaction among jobs is limited
to queueing for physical devices, for example, there should not
be any synchronization requirements
d. Local Information: A device's service rate depends only on
local queue length and not on the state of the rest of the
system
17
Non-Markovian PFNs (cont’d)
e. Fair Service: If service rates differ by class, the service rate
for a class depends only on the queue length of that class at
the device and not on the queue lengths of other classes. This
means that the servers do not discriminate against jobs in a
class depending on the queue lengths of other classes
4. Routing Homogeneity: The job routing should be state
independent. The routing homogeneity condition implies that
the probability of a job going from one device to another device
does not depend upon the number of jobs at various devices
18
Machine Repairman Model



Originally for machine repair shops
A number of working machines with a
repair facility with one or more servers
(repairmen)
Whenever a machine breaks down, it is
put in the queue for repair and serviced
as soon as a repairman is available
 Scherr (1967) used this model to represent a timesharing system
with n terminals
 Users sitting at the terminals generate requests (jobs) that are
serviced by the system which serves as a repairman
 After a job is done, it waits at the user-terminal for a random
``think-time'' interval before cycling again
19
Central Server Model



Introduced by Buzen (1973)
The CPU is the central “server” that schedules
visits to other devices
After service at the I/O devices the jobs return to the
CPU
20
Types of Service Centers
Three kinds of devices
1. Fixed-capacity service centers: Service time does not
depend upon the number of jobs in the device
For example, the CPU in a system may be modeled as a fixedcapacity service center.
2. Delay centers or infinite server: No queueing. Jobs spend
the same amount of time in the device regardless of the
number of jobs in it. A group of dedicated terminals is usually
modeled as a delay center.
3. Load-dependent service centers: Service rates may depend
upon the load or the number of jobs in the device., e.g., M/M/m
queue (with m > 2 )
A group of parallel links between two nodes in a computer
network is another example
21
Summary
Product Form Networks
DB
BCMP
Jackson




Product form networks: Any network in which the system state
probability is a product of device state probabilities
Jackson: Network of M/M/m queues
BCMP: More general conditions
Denning and Buzen: Even more general conditions
22
Operational Laws
Overview

What is an Operational Law?






Utilization Law
Forced Flow Law
Little’s Law
General Response Time Law
Interactive Response Time Law
Bottleneck Analysis
24
Operational Laws




Relationships that do not require any assumptions about the
distribution of service times or inter-arrival times
Identified originally by Buzen (1976) and later extended by
Denning and Buzen (1978)
Operational  Directly measured
Operationally testable assumptions
 assumptions that can be verified by measurements



For example, whether number of arrivals is equal to the number
of completions?
This assumption, called job flow balance, is operationally testable
Statement “a set of observed service times is or
is not a sequence of independent random variables”
is not operationally testable
25
Operational Quantities



Black
Box
Quantities that can be directly measured
during a finite observation period
T = Observation interval
Ai = number of arrivals
Ci = number of completions
Bi = busy time Bi
26
Utilization Law



This is one of the operational laws
Operational laws are similar to the elementary laws of motion
For example,
Notice that distance d, acceleration a, and time t are
operational quantities. No need to consider them as expected
values of random variables or to assume a distribution
27
Example 33.1





Consider a network gateway at which the packets arrive at a
rate of 125 packets per second and the gateway takes an
average of two milliseconds to forward them
Throughput Xi = Exit rate = Arrival rate = 125 packets/second
Service time Si = 0.002 second
Utilization Ui= Xi Si = 125  0.002 = 0.25 = 25%
This result is valid for any arrival or service process.
Even if inter-arrival times and service times to are not IID
random variables with exponential distribution
28
Forced Flow Law






Relates the system throughput to
individual device throughputs
In an open model,
System throughput = # of jobs leaving the system per unit time
In a closed model,
System throughput = # of jobs traversing OUT to IN link per
unit time
If observation period T is such that Ai = Ci
 Device satisfies the assumption of job flow balance
Each job makes Vi requests for i-th device in the system
If the job flow is balanced and C0 is # of jobs traversing the
outside link => Ci – # of jobs visiting the i-th device:
Ci = C0 Vi or Vi =Ci/C0 Vi is called visit ratio
29
Forced Flow Law (cont’d)

System throughput:
30
Forced Flow Law (cont’d)

Throughput of ith device:

In other words:

This is the forced flow law
31
Bottleneck Device

Combining the forced flow law and the utilization law,
we get:

Here Di=Vi Si is the total service demand on the
device for all visits of a job
The device with the highest Di has the highest
utilization and is the bottleneck device

32
Example 33.2

In a timesharing system, accounting log data produced the
following profile for user programs





Each program requires five seconds of CPU time, makes 80 I/O
requests to the disk A and 100 I/O requests to disk B
Average think-time of the users was 18 seconds
From the device specifications, it was determined that disk A takes
50 milliseconds to satisfy an I/O request and the disk B takes 30
milliseconds per request
With 17 active terminals, disk A throughput was observed to be
15.70 I/O requests per second
We want to find the system throughput and device utilizations
33
Example 33.2 (cont’d)

Since the jobs must visit the CPU before going to the disks or
terminals, the CPU visit ratio is:
34
Example 33.2 (cont’d)

Using the forced flow law, the throughputs are:

Using the utilization law, the device utilizations are:
35
Transition Probabilities
pij = Probability of a job moving to jth queue after service
completion at ith queue
 Visit ratios and transition probabilities are equivalent in the
sense that given one we can always find the other
 In a system with job flow balance:
i = 0  visits to the outside link
 pi0 = Probability of a job exiting from the system after
completion of service at ith device
 Dividing by C0 we get:

36
Transition Probabilities (cont’d)



Since each visit to the outside link is defined as the completion
of the job, we have:
These are called visit ratio equations
In central server models, after completion of service at every
queue, the jobs always move back to the CPU queue:
37
Transition Probabilities (cont’d)

The above probabilities apply to exit and entrances from the
system (i=0), also. Therefore, the visit ratio equations become:

Thus, we can find the visit ratios by dividing the probability p1j
of moving to jth queue from CPU by the exit probability p10
38
Example 33.3

Consider the queueing network:

The visit ratios are VA=80, VB=100, and VCPU=181.
After completion of service at the CPU the probabilities of the
job moving to disk A, disk B, or terminals are 80/181, 100/181,
and 1/181, respectively. Thus, the transition probabilities are
0.4420, 0.5525, and 0.005525

39
Example 33.3 (cont’d)

Given the transition probabilities, we can find the visit ratios by
dividing these probabilities by the exit probability (0.005525):
40
Little's Law

If the job flow is balanced, the arrival rate is equal to the
throughput and we can write:
41
Example 33.4



The average queue length in the computer system of Example
33.2 was observed to be: 8.88, 3.19, and 1.40 jobs at the CPU,
disk A, and disk B, respectively. What were the response times
of these devices?
In Example 33.2, the device throughputs were determined to
be:
The new information given in this example is:
42
Example 33.4 (cont’d)

Using Little's law, the device response times are:
43
General Response Time Law




There is one terminal per user and
the rest of the system is shared by
all users.
Applying Little's law to
the central subsystem:
Q=XR
Here,



Q = Total number of jobs in the system
R = system response time
X = system throughput
44
General Response Time Law (cont’d)

Dividing both sides by X and using forced flow law:

or,

This is called the general response time law
This law holds even if the job flow is not balanced

45
Example 33.5

Let us compute the response time for the timesharing system
of Examples 33.2 and 33.4
For this system:

The system response time is:

The system response time is 68.6 seconds

46
Interactive Response Time Law

If Z = think-time, R = Response time



The total cycle time of requests is R+Z
Each user generates about T/(R+Z) requests in T
If there are N users:
or

R = (N/X) - Z
This is the interactive response time law
47
Example 33.6

For the timesharing system of Example 33.2, we can compute
the response time using the interactive response time law as
follows:
Therefore:

This is the same as that obtained earlier in Example 33.5.

48
Bottleneck Analysis

From forced flow law:

The device with the highest total service demand Di has the
highest utilization and is called the bottleneck device
Note: Delay centers can have utilizations more than one
without any stability problems. Therefore, delay centers cannot
be a bottleneck device
Only queueing centers used in computing Dmax
The bottleneck device is the key limiting factor in achieving
higher throughput



49
Bottleneck Analysis (cont’d)



Improving the bottleneck device will provide
the highest payoff in terms of system throughput
Improving other devices will have little effect on
the system performance
Identifying the bottleneck device should be the first step in any
performance improvement project
50
Bottleneck Analysis (cont’d)

Throughput and response times of the system are bound as
follows:
and


Here,
is the sum of total service demands on all
devices except terminals
These are known as asymptotic bounds
51
Bottleneck Analysis: Proof




The asymptotic bounds are based on the following
observations:
1. The utilization of any device cannot exceed one. This puts a
limit on the maximum obtainable throughput
2. The response time of the system with N users cannot be
less than a system with just one user. This puts a limit on the
minimum response time
3. The interactive response time formula can be used to
convert the bound on throughput to that on response time and
vice versa
52
Proof (cont’d)

For the bottleneck device b we have:

Since Ub cannot be more than one, we have:
53
Proof (cont’d)

With just one job in the system, there is no queueing and the
system response time is simply the sum of the service
demands:

Here, D is defined as the sum of all service demands
With more than one user there may be some queueing and so
the response time will be higher. That is:

54
Proof (cont’d)

Combining these bounds we get the asymptotic bounds.
55
Typical Asymptotic Bounds
56
Typical Asymptotic Bounds (cont’d)

The number of jobs N* at the knee is given by:

If the number of jobs is more than N*, then we can say with
certainty that there is queueing somewhere in the system
The asymptotic bounds can be easily explained to people who
do not have any background in queueing theory or
performance analysis

57
Example 33.7

For the timesharing system considered in Example 33.2:

The asymptotic bounds are:
58
Example 33.7: Asymptotic Bounds
59
Example 33.7 (cont’d)

The knee occurs at:
or

Thus, if there are more than 6 users on the system,
there will certainly be queueing in the system.
60
Example 33.8

How many terminals can be supported on the timesharing
system of Example 33.2 if the response time has to be kept
below 100 seconds?
Using the asymptotic bounds on the response time we get:

The response time will be more than 100, if:

That is, if:
the response time is bound to be more than 100. Thus, the
system cannot support more than 23 users if a response time
of less than 100 is required.


61
Summary

Symbols:
62