Transcript Web and Intranet Performance: a quantitative analysis

Part VI
System-level Performance
Models for the Web
(Book, Chapter 8)
8 Menascé & Almeida. All Rights Reserved.
1
Understanding the Environment
Developing a Cost Model
Workload Characterization
Workload
Model
Cost
Model
Validation and Calibration
Performance
Model
Workload Forecasting
Performance Prediction
Cost Prediction
Valid
Model
Cost/Performance Analysis
Configuration
Plan
Investment
Plan
8 Menascé & Almeida. All Rights Reserved.
Personnel
Plan
2
Part VI: Learning Objectives
•Characterize system-level models
•Present State Transition Diagram (STD)
technique
•Show solution for simple models
•Show general solution to STDs
•Show how to obtain performance
metrics from the solution of STDs
8 Menascé & Almeida. All Rights Reserved.
3
System-level Models
• System is seen as a
black box.
• Only its input-output
characteristics are
considered.
• Inputs: arrivals of
requests
• Output: throughput.
8 Menascé & Almeida. All Rights Reserved.
Throughput, X0(k)
(requests/sec)
requests in
the system, k
4
System-level Example
• A Web server receives 10 requests/sec.
• The maximum number of requests in
the server is 3.
• Requests that arrive and find three
requests being processed are rejected.
8 Menascé & Almeida. All Rights Reserved.
5
System-level Example
• The measured throughput as a function
of the number of requests is:
Number of Throughput
requests
(req/sec)
0
0
1
12
2
15
3
16
8 Menascé & Almeida. All Rights Reserved.
6
System-level Example: a few
questions
• Q1: What is the probability that an
incoming request is rejected?
• Q2: What is the average number of
requests in execution?
• Q3: What is the average throughput of
the Web server?
• Q4: What is the average time spent by
an HTTP request in the Web server?
8 Menascé & Almeida. All Rights Reserved.
7
System-level Example
• Characterize the Web server by its state,
i.e., the number k of requests in the Web
server.
• Assumptions made:
– homogeneous workload (or single class): all
requests are statistically equivalent, only the
number of req. counts
– memoryless (markovian): how the system
arrived at state k does not matter.
– operational equilibrium: no. requests at
beginning of interval = no. request at the end.
8 Menascé & Almeida. All Rights Reserved.
8
System-level Example
arrivals
state
10 req/sec
10 req/sec
1
0
12 req/sec
10 req/sec
2
15 req/sec
3
16 req/sec
State Transition Diagram (STD)
completions of requests
8 Menascé & Almeida. All Rights Reserved.
9
System-level Example
• Assume we are able to find the values
of:
– pk = probability that (or fraction of time
where) there are k requests in the Web
server.
• Question: can we answer all the
questions posed before as a function of
the pk’s?
8 Menascé & Almeida. All Rights Reserved.
10
System-level Example: a few
questions
• Q1: What is the probability that an
incoming request is rejected?
• A: It is the probability that an arriving
HTTP request finds 3 requests already
being processed. The answer is then p3.
8 Menascé & Almeida. All Rights Reserved.
11
System-level Example: a few
questions
• Q2: What is the average number of
requests in execution?
• A: using the definition of average:
Nreq = 0 x p0 + 1 x p1 + 2 x p2 + 3 x p3
8 Menascé & Almeida. All Rights Reserved.
12
System-level Example: a few
questions
• Q3: What is the average throughput of
the Web server?
• A: again, using the definition of average:
X = 0 x p0 + 12 x p1 + 15 x p2 + 16 x p3
throughput value at each state
8 Menascé & Almeida. All Rights Reserved.
13
System-level Example: a few
questions
• Q4: What is the average time spent by an
HTTP request in the Web server?
• A: It will be a function of the average
number of requests, Nreq, and the
average throughput X. Can be computed
by applying Little’s law.
8 Menascé & Almeida. All Rights Reserved.
14
System-level Example:
computing the pk’s
10 req/sec
10 req/sec
1
0
12 req/sec
10 req/sec
2
15 req/sec
3
16 req/sec
• use the flow in = flow out principle: the flow
into a set of states is equal to the flow out
of this set of states in equilibrium.
8 Menascé & Almeida. All Rights Reserved.
15
System-level Example:
computing the pk’s
10 req/sec
10 req/sec
1
0
12 req/sec
10 req/sec
2
15 req/sec
3
16 req/sec
flow in = flow out
12 x p1 = 10 x p0
8 Menascé & Almeida. All Rights Reserved.
16
System-level Example:
computing the pk’s
10 req/sec
10 req/sec
1
0
12 req/sec
10 req/sec
2
15 req/sec
3
16 req/sec
flow in = flow out
15 x p2 = 10 x p1
8 Menascé & Almeida. All Rights Reserved.
17
System-level Example:
computing the pk’s
10 req/sec
10 req/sec
1
0
12 req/sec
10 req/sec
2
15 req/sec
3
16 req/sec
flow in = flow out
16 x p3 = 10 x p2
8 Menascé & Almeida. All Rights Reserved.
18
System-level Example:
computing the pk’s
• Putting it all together:
12 x p1 = 10 x p0
15 x p2 = 10 x p1
16 x p3 = 10 x p2
8 Menascé & Almeida. All Rights Reserved.
p1 = 10/12 p0
p2 = 10/15 p1
= 10x10 p0
15x12
p3 = 10/16 p2
= 10x10x10 p0
16x15x12
19
System-level Example:
computing the pk’s
• Putting it all together:
p1 = 10/12 p0 ; p2 = 10x10 p0; and
15x12
p3 = 10x10x10 p0
16x15x12
• But, the Web server has to be in one
of the four states at any time. So,
p0 + p1+ p2 + p3 = 1.
8 Menascé & Almeida. All Rights Reserved.
20
System-level Example:
computing the pk’s
• Solving for p0 and then for the other pk’s
we get:
k
0
1
2
3
8 Menascé & Almeida. All Rights Reserved.
Pk
0.365
0.305
0.203
0.127
21
System-level Example:
answering the questions
• Q1: What is the probability that an
incoming request is rejected?
• A: It is the probability that an arriving
HTTP request finds 3 requests already
being processed. The answer is then
p3 = 0.127 = 12.7%.
8 Menascé & Almeida. All Rights Reserved.
22
System-level Example:
answering the questions
• Q2: What is the average number of
requests in execution?
• A: using the definition of average:
Nreq = 0 x 0.365 + 1 x 0.305 +
2 x 0.203+ 3 x 0.127
= 1.091 requests
8 Menascé & Almeida. All Rights Reserved.
23
System-level Example:
answering the questions
• Q3: What is the average throughput of
the Web server?
• A: again, using the definition of average:
X = 0 x 0.365 + 12 x 0.305 +
15 x 0.203 + 16 x 0.127
= 8.731 requests/sec.
8 Menascé & Almeida. All Rights Reserved.
24
System-level Example:
answering the questions
• Q4: What is the average time spent by an
HTTP request in the Web server?
• A: It is a function of the average number
of requests, Nreq, and the average
throughput X. We need Little’s Law to
answer this question.
8 Menascé & Almeida. All Rights Reserved.
25
Little’s Law
PUB
avg. number
people in the
pub
=
avg. departure
rate from the X
pub
8 Menascé & Almeida. All Rights Reserved.
avg. time
spent at the
pub
26
Little’s Law
Web
server
avg. number
=
requests in the
server
1.091 req
avg. departure
rate from the X
server
8.731 req/sec
8 Menascé & Almeida. All Rights Reserved.
avg. time
spent at the
server
?
27
System-level Example:
answering the questions
• Q4: What is the average time spent by an
HTTP request in the Web server?
• A: From Little’s Law,
R = Nreq / X = 1.091 / 8.731 = 0.125 sec.
8 Menascé & Almeida. All Rights Reserved.
28
Practice Drill
Using Models for Decision Making
• What happens if the maximum number
of allowed TCP connections changes
from 3 to 10?
• What if the load on the server doubles?
• What is the impact of a threefold
increase in the server’s capacity?
8 Menascé & Almeida. All Rights Reserved.
29
Types of System-level Models
• Population Size:
– infinite
– finite
• Service Rate:
– fixed
– variable
• Maximum Queue Size:
– unlimited
– limited
8 Menascé & Almeida. All Rights Reserved.
30
Types of System-level Models
(population size)
• Infinite Population: the number of clients is
very large. The rate at which requests arrive
to the system does not depend on the
number of requests being processed in the
system.
– e.g., requests arriving from the Internet to a
public Web server.
8 Menascé & Almeida. All Rights Reserved.
31
Types of System-level Models
(infinite population)
1
2
arrival rate l
(requests/sec)
completed
requests X
SERVER
M
...
internal requests Nreq
infinite population
8 Menascé & Almeida. All Rights Reserved.
Open Model
32
Practice Drill
Using Models for Decision Making
• Your Web server is used for
e-commerce. The company will
announce a new product in the Fall and
expects the number of current sales to
increase by 50%. Will the server handle
the load?
– What is the new response time?
– What is the server utilization?
8 Menascé & Almeida. All Rights Reserved.
33
Practice Drill
Using Models for Decision Making
• What is the meaning of “ Will the server
handle the load?”
• What is the new response time?
• What is the server utilization?
8 Menascé & Almeida. All Rights Reserved.
34
Types of System-level Models
(population size)
• Finite Population: the number of clients M
is limited. The rate at which requests
arrive to the system depends on how
many have already arrived.
– e.g., requests arriving to an intranet Web
server from a known number of clients within
the organization (Intranet).
8 Menascé & Almeida. All Rights Reserved.
35
Types of System-level Models
(finite population)
arrival rate l
(requests/sec)
think 1
time Z
2
completed
requests X
SERVER
M
internal requests Nreq
finite population M
8 Menascé & Almeida. All Rights Reserved.
Closed Model
36
Practice Drill
Using Models for Decision Making
• Company X’s intranet collects sales
data from 100 stores and processes
reorder data on hundreds of products
electronically. The company will merge
with company Y that has 150 stores.
Does the existing Web site have
enough capacity to handle the merged
company’s demand?
8 Menascé & Almeida. All Rights Reserved.
37
Types of System-level Models
(service rate)
• Fixed Service Rate: the throughput does
not vary with the number of requests
being processed.
Throughput
(requests/sec)
requests in the system
8 Menascé & Almeida. All Rights Reserved.
38
Types of System-level Models
(service rate)
• Variable Service Rate: the throughput
depends on the number of requests being
processed.
Throughput
(requests/sec)
requests in the system
8 Menascé & Almeida. All Rights Reserved.
39
Types of System-level Models
(maximum queue size)
• Unlimited Queue Size: all arriving requests
are queued for service. No requests are
rejected!
• Limited Queue Size: requests that find
more than W requests waiting for service
are rejected.
No
n=W?
Yes
8 Menascé & Almeida. All Rights Reserved.
40
Types of System-level Models
S e rv ic e
Queue
P o p u l a ti o n
R a te
S iz e
in fin it e
fix e d
u n lim it e d
in fin it e
fix e d
lim it e d
in fin it e
va ria b le
u n lim it e d
in fin it e
va ria b le
fin it e
fin it e
fix e d
fin it e
va ria b le
8 Menascé & Almeida. All Rights Reserved.
41
System-level models:
Methodology
• Determine proper representation for the
state of system being modeled (numb.
of requests in system, k)
• Determine set of feasible states
• Determine possible transitions between
states, considering possible events
(arrival, completion of request)
8 Menascé & Almeida. All Rights Reserved.
42
System-level models:
Methodology (2)
• For each possible state transition,
determine transition rate, looking at
event that caused the transition
• Use flow in = flow out principle to write
down equations relating values of pk;
remember that sum of all pks is one.
8 Menascé & Almeida. All Rights Reserved.
43
System-level models:
Methodology (3)
• Solve for pks and use them to compute
performance metrics:
– utilization U
– average throughput X
– average number of requests Nreq
– average response time R
– fraction of requests that are lost because of
queue overflow Ploss
8 Menascé & Almeida. All Rights Reserved.
44
Generalized System-level Models
l0
l
1
0
m
l2
2
m2
l3
3
m3
....
m4
Generalized System-level Models can be solved
using the flow in = flow out principle
8 Menascé & Almeida. All Rights Reserved.
45
Generalized System-level Models
li
pk  
i  0 m i 1
k 1
  k 1 l i 
p0    

k  0 i  0 m i 1 
8 Menascé & Almeida. All Rights Reserved.
1
46
System-level Models
Example
A Web server receives 30 requests/sec. Its
throughput function is given below. The server
queue is limited to five requests. What is the
server utilization, avg. throughput, avg. no.
requests, avg. response time, and fraction of
lost requests?
No. of requests Throughput (req/sec)
1
18
2
35
3 or more
50
8 Menascé & Almeida. All Rights Reserved.
47
System-level Models
Example (cont’d)
Using the Generalized System-level model
equations we get that
Average Number of Requests
1.850
Server Utilization
82.700%
Server Average Throughput 28.4 req/sec
Fraction of Lost Requests
0.05343
From Little’s Law,
avg. response time = avg. no requests/
avg. throughput =
1.85 / 28.4 = 0.065 sec.
8 Menascé & Almeida. All Rights Reserved.
48
System-level Models
Example (cont’d)
Max. Queue Size
Average Number of Requests
Server Utilization
Avg. Server Throughput (req/sec)
Average Response Time (sec)
Fraction of Lost Requests
3
1.433
79.81%
24.8
0.058
0.173077
8 Menascé & Almeida. All Rights Reserved.
5
10
15
1.850
2.212
2.264
82.7%
83.9%
83.96%
28.4
29.9
30
0.065
0.074
0.075
0.05343 0.003869 0.000299
49
System-level Models
Example (cont’d)
Fraction of Lost Requests
Fraction of Lost Requests vs. Max Queue
Size
0.2
0.15
0.1
0.05
0
3
5
10
15
Max Queue Size
8 Menascé & Almeida. All Rights Reserved.
50
Practical computation tools
Help for SysMod.XLS
Main Screen
1. This workbook implements the formulas that solve system models
of C/S systems as discussed in Chapter 8 of the book.
2. The input parameters in each worksheet are entered in cells with
purple background.
3. All cells, except for the ones where you must enter values, are protected
to avoid unintended modifications.
Tips:
a. All input parameters should use compatible units (e.g., seconds for
service demands and tps for arrival rates). The results are given in
the units compatible with the input parameters.
b. Do not change the location of any of the cells. This may render your
workbook unusable or generate incorrect results.
c. Always work on a copy of the original workbook provided with the book.
You may open the original workbook and save it under
another name using the Save As option from the File menu.
8 Menascé & Almeida. All Rights Reserved.
51
Practical computation tools
Infinite population/constant service rate/infinite queue
Formulas: fig. 8.6 pag. 182
This w orksheet comes w ith the book "Capacity Planning for Web Performance",
Main Screen
by D. A. Menascé and V. A. F. Almeida, Prentice Hall, 1998.
Arrival Rate l
Service Rate m
Utilization U
Average Numb. of Requests N
Average Response Time R
Note that the arrival rate must be less than the service rate!
k
Pk
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
30 tps
50 tps
60%
1.5
0.05
Utilization
0.4000000
0.2400000
0.1440000
0.0864000
0.0518400
0.0311040
0.0186624
0.0111974
0.0067185
0.0040311
0.0024186
0.0014512
0.0008707
0.0005224
0.0003135
0.0001881
8 Menascé & Almeida. All Rights Reserved.
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
0.95
0.99
R / S = 1 / (1 - U)
(see fig. 8.5 pag. 180)
1.00
1.11
1.25
1.43
1.67
2.00
2.50
3.33
5.00
10.00
20.00
100.00
52
Practical computation tools
Infinite population/constant service rate/finite queue
Formulas: fig. 8.9 pag. 185
This w orksheet comes w ith the book "Capacity Planning for Web Performance",
by D. A. Menascé and V. A. F. Almeida, Prentice Hall, 1998.
Arrival Rate l
Service Rate m
Max. Number of Requests (W)
Average Number of Requests N
Server Utilization U
Server Throughput X
Average Response Time R
Fraction of Lost Requests Ploss
k
Pk
0
1
2
3
4
5
6
7
8
9
10
30
50
10
1.460
59.9%
29.9
0.049
0.24%
W
0.4014565
0.2408739
0.1445243
0.0867146
0.0520288
0.0312173
0.0187304
0.0112382
0.0067429
0.0040458
0.0024275
1
2
3
4
5
6
7
8
9
10
8 Menascé & Almeida. All Rights Reserved.
Main Screen
tps
tps
tps
sec
Fraction of Lost Requests
37.5%
18.4%
9.9%
5.6%
3.3%
1.9%
1.1%
0.7%
0.4%
0.2%
(see fig. 8.8 pag. 184)
53
Practical computation tools
Infinite population/variable service rate/infinite queue
Formulas: (8.5.1) to (8.5.5) pag. 188-189
This w orksheet comes w ith the book "Capacity Planning for Web Performance",
by D. A. Menascé and V. A. F. Almeida, Prentice Hall, 1998.
Arrival Ratel
Saturation Point (J)
Average Numb. of Requests N
Server Utilization U
Server Average Throughput X
Average Response Time R
Main Screen
30 tps
3
2.271
84.0%
30.0 tps
0.076 sec
Note that the arrival rate must be less than X(J)!
k
X(k)
Throughput values until saturation
1
2
3
18
35
50
8 Menascé & Almeida. All Rights Reserved.
54
Practical computation tools
Infinite population/variable service rate/finite queue
Formulas: (8.5.6) to (8.5.9) pag. 190
This w orksheet comes w ith the book "Capacity Planning for Web Performance",
by D. A. Menascé and V. A. F. Almeida, Prentice Hall, 1998.
Main Screen
Arrival Rate l
30 tps
Saturation Point (J)
3
Max. Number of Requests (W)
5
Average Numb. of Requests N
1.852
Server Utilization U
82.7%
Server Avg. Throughput X
28.4 tps
Average Response Time R
0.065 sec
Fraction of Lost Requests Ploss 0.05343
k
X(k)
Throughput values before saturation
1
2
3
18
35
50
8 Menascé & Almeida. All Rights Reserved.
55
Practical computation tools
Finite population/constant service rate
Formulas: (8.5.10), (8.5.11) pag. 191-192
This w orksheet comes w ith the book "Capacity Planning for Web Performance",
by D. A. Menascé and V. A. F. Almeida, Prentice Hall, 1998. Main Screen
Average Think Time Z
Service Rate m
Number of Clients M
Average Number of Requests N
Server Utilization U
Server Throughput X
Average Response Time R
2
80
50
0.443
31.0%
24.8
0.018
8 Menascé & Almeida. All Rights Reserved.
sec
tps
tps
sec
56
Practical computation tools
Finite population/variable service rate
Formulas: (8.5.15) to (8.5.18) pag. 193-194
This w orksheet comes w ith the book "Capacity Planning for Web Performance",
Main Screen
by D. A. Menascé and V. A. F. Almeida, Prentice Hall, 1998.
Average Think Time Z
Saturation Point (J)
Number of Clients M
Average Number of Requests N
Server Utilization U
Server Throughput X
Average Response Time R
k
X(k)
Throughput values until saturation
1
2
18
35
1
3
50
0.645
97.0%
11.1
0.058
sec
tps
tps
sec
3
50
8 Menascé & Almeida. All Rights Reserved.
57
Part VI: Summary
•System-level models view a server as a
black box. Only its arrival process and
throughput functions are relevant.
•State Transition Diagrams (STDs) can be
used to find the probability that k requests
are in the server. Use the flow in = flow out
principle.
•Little’s Law can be used to compute the
response time from the average number of
requests
and from the throughput.
58
8 Menascé & Almeida. All Rights Reserved.