#### Transcript Web and Intranet Performance: a quantitative analysis

Part VI System-level Performance Models for the Web (Book, Chapter 8) 8 Menascé & Almeida. All Rights Reserved. 1 Understanding the Environment Developing a Cost Model Workload Characterization Workload Model Cost Model Validation and Calibration Performance Model Workload Forecasting Performance Prediction Cost Prediction Valid Model Cost/Performance Analysis Configuration Plan Investment Plan 8 Menascé & Almeida. All Rights Reserved. Personnel Plan 2 Part VI: Learning Objectives •Characterize system-level models •Present State Transition Diagram (STD) technique •Show solution for simple models •Show general solution to STDs •Show how to obtain performance metrics from the solution of STDs 8 Menascé & Almeida. All Rights Reserved. 3 System-level Models • System is seen as a black box. • Only its input-output characteristics are considered. • Inputs: arrivals of requests • Output: throughput. 8 Menascé & Almeida. All Rights Reserved. Throughput, X0(k) (requests/sec) requests in the system, k 4 System-level Example • A Web server receives 10 requests/sec. • The maximum number of requests in the server is 3. • Requests that arrive and find three requests being processed are rejected. 8 Menascé & Almeida. All Rights Reserved. 5 System-level Example • The measured throughput as a function of the number of requests is: Number of Throughput requests (req/sec) 0 0 1 12 2 15 3 16 8 Menascé & Almeida. All Rights Reserved. 6 System-level Example: a few questions • Q1: What is the probability that an incoming request is rejected? • Q2: What is the average number of requests in execution? • Q3: What is the average throughput of the Web server? • Q4: What is the average time spent by an HTTP request in the Web server? 8 Menascé & Almeida. All Rights Reserved. 7 System-level Example • Characterize the Web server by its state, i.e., the number k of requests in the Web server. • Assumptions made: – homogeneous workload (or single class): all requests are statistically equivalent, only the number of req. counts – memoryless (markovian): how the system arrived at state k does not matter. – operational equilibrium: no. requests at beginning of interval = no. request at the end. 8 Menascé & Almeida. All Rights Reserved. 8 System-level Example arrivals state 10 req/sec 10 req/sec 1 0 12 req/sec 10 req/sec 2 15 req/sec 3 16 req/sec State Transition Diagram (STD) completions of requests 8 Menascé & Almeida. All Rights Reserved. 9 System-level Example • Assume we are able to find the values of: – pk = probability that (or fraction of time where) there are k requests in the Web server. • Question: can we answer all the questions posed before as a function of the pk’s? 8 Menascé & Almeida. All Rights Reserved. 10 System-level Example: a few questions • Q1: What is the probability that an incoming request is rejected? • A: It is the probability that an arriving HTTP request finds 3 requests already being processed. The answer is then p3. 8 Menascé & Almeida. All Rights Reserved. 11 System-level Example: a few questions • Q2: What is the average number of requests in execution? • A: using the definition of average: Nreq = 0 x p0 + 1 x p1 + 2 x p2 + 3 x p3 8 Menascé & Almeida. All Rights Reserved. 12 System-level Example: a few questions • Q3: What is the average throughput of the Web server? • A: again, using the definition of average: X = 0 x p0 + 12 x p1 + 15 x p2 + 16 x p3 throughput value at each state 8 Menascé & Almeida. All Rights Reserved. 13 System-level Example: a few questions • Q4: What is the average time spent by an HTTP request in the Web server? • A: It will be a function of the average number of requests, Nreq, and the average throughput X. Can be computed by applying Little’s law. 8 Menascé & Almeida. All Rights Reserved. 14 System-level Example: computing the pk’s 10 req/sec 10 req/sec 1 0 12 req/sec 10 req/sec 2 15 req/sec 3 16 req/sec • use the flow in = flow out principle: the flow into a set of states is equal to the flow out of this set of states in equilibrium. 8 Menascé & Almeida. All Rights Reserved. 15 System-level Example: computing the pk’s 10 req/sec 10 req/sec 1 0 12 req/sec 10 req/sec 2 15 req/sec 3 16 req/sec flow in = flow out 12 x p1 = 10 x p0 8 Menascé & Almeida. All Rights Reserved. 16 System-level Example: computing the pk’s 10 req/sec 10 req/sec 1 0 12 req/sec 10 req/sec 2 15 req/sec 3 16 req/sec flow in = flow out 15 x p2 = 10 x p1 8 Menascé & Almeida. All Rights Reserved. 17 System-level Example: computing the pk’s 10 req/sec 10 req/sec 1 0 12 req/sec 10 req/sec 2 15 req/sec 3 16 req/sec flow in = flow out 16 x p3 = 10 x p2 8 Menascé & Almeida. All Rights Reserved. 18 System-level Example: computing the pk’s • Putting it all together: 12 x p1 = 10 x p0 15 x p2 = 10 x p1 16 x p3 = 10 x p2 8 Menascé & Almeida. All Rights Reserved. p1 = 10/12 p0 p2 = 10/15 p1 = 10x10 p0 15x12 p3 = 10/16 p2 = 10x10x10 p0 16x15x12 19 System-level Example: computing the pk’s • Putting it all together: p1 = 10/12 p0 ; p2 = 10x10 p0; and 15x12 p3 = 10x10x10 p0 16x15x12 • But, the Web server has to be in one of the four states at any time. So, p0 + p1+ p2 + p3 = 1. 8 Menascé & Almeida. All Rights Reserved. 20 System-level Example: computing the pk’s • Solving for p0 and then for the other pk’s we get: k 0 1 2 3 8 Menascé & Almeida. All Rights Reserved. Pk 0.365 0.305 0.203 0.127 21 System-level Example: answering the questions • Q1: What is the probability that an incoming request is rejected? • A: It is the probability that an arriving HTTP request finds 3 requests already being processed. The answer is then p3 = 0.127 = 12.7%. 8 Menascé & Almeida. All Rights Reserved. 22 System-level Example: answering the questions • Q2: What is the average number of requests in execution? • A: using the definition of average: Nreq = 0 x 0.365 + 1 x 0.305 + 2 x 0.203+ 3 x 0.127 = 1.091 requests 8 Menascé & Almeida. All Rights Reserved. 23 System-level Example: answering the questions • Q3: What is the average throughput of the Web server? • A: again, using the definition of average: X = 0 x 0.365 + 12 x 0.305 + 15 x 0.203 + 16 x 0.127 = 8.731 requests/sec. 8 Menascé & Almeida. All Rights Reserved. 24 System-level Example: answering the questions • Q4: What is the average time spent by an HTTP request in the Web server? • A: It is a function of the average number of requests, Nreq, and the average throughput X. We need Little’s Law to answer this question. 8 Menascé & Almeida. All Rights Reserved. 25 Little’s Law PUB avg. number people in the pub = avg. departure rate from the X pub 8 Menascé & Almeida. All Rights Reserved. avg. time spent at the pub 26 Little’s Law Web server avg. number = requests in the server 1.091 req avg. departure rate from the X server 8.731 req/sec 8 Menascé & Almeida. All Rights Reserved. avg. time spent at the server ? 27 System-level Example: answering the questions • Q4: What is the average time spent by an HTTP request in the Web server? • A: From Little’s Law, R = Nreq / X = 1.091 / 8.731 = 0.125 sec. 8 Menascé & Almeida. All Rights Reserved. 28 Practice Drill Using Models for Decision Making • What happens if the maximum number of allowed TCP connections changes from 3 to 10? • What if the load on the server doubles? • What is the impact of a threefold increase in the server’s capacity? 8 Menascé & Almeida. All Rights Reserved. 29 Types of System-level Models • Population Size: – infinite – finite • Service Rate: – fixed – variable • Maximum Queue Size: – unlimited – limited 8 Menascé & Almeida. All Rights Reserved. 30 Types of System-level Models (population size) • Infinite Population: the number of clients is very large. The rate at which requests arrive to the system does not depend on the number of requests being processed in the system. – e.g., requests arriving from the Internet to a public Web server. 8 Menascé & Almeida. All Rights Reserved. 31 Types of System-level Models (infinite population) 1 2 arrival rate l (requests/sec) completed requests X SERVER M ... internal requests Nreq infinite population 8 Menascé & Almeida. All Rights Reserved. Open Model 32 Practice Drill Using Models for Decision Making • Your Web server is used for e-commerce. The company will announce a new product in the Fall and expects the number of current sales to increase by 50%. Will the server handle the load? – What is the new response time? – What is the server utilization? 8 Menascé & Almeida. All Rights Reserved. 33 Practice Drill Using Models for Decision Making • What is the meaning of “ Will the server handle the load?” • What is the new response time? • What is the server utilization? 8 Menascé & Almeida. All Rights Reserved. 34 Types of System-level Models (population size) • Finite Population: the number of clients M is limited. The rate at which requests arrive to the system depends on how many have already arrived. – e.g., requests arriving to an intranet Web server from a known number of clients within the organization (Intranet). 8 Menascé & Almeida. All Rights Reserved. 35 Types of System-level Models (finite population) arrival rate l (requests/sec) think 1 time Z 2 completed requests X SERVER M internal requests Nreq finite population M 8 Menascé & Almeida. All Rights Reserved. Closed Model 36 Practice Drill Using Models for Decision Making • Company X’s intranet collects sales data from 100 stores and processes reorder data on hundreds of products electronically. The company will merge with company Y that has 150 stores. Does the existing Web site have enough capacity to handle the merged company’s demand? 8 Menascé & Almeida. All Rights Reserved. 37 Types of System-level Models (service rate) • Fixed Service Rate: the throughput does not vary with the number of requests being processed. Throughput (requests/sec) requests in the system 8 Menascé & Almeida. All Rights Reserved. 38 Types of System-level Models (service rate) • Variable Service Rate: the throughput depends on the number of requests being processed. Throughput (requests/sec) requests in the system 8 Menascé & Almeida. All Rights Reserved. 39 Types of System-level Models (maximum queue size) • Unlimited Queue Size: all arriving requests are queued for service. No requests are rejected! • Limited Queue Size: requests that find more than W requests waiting for service are rejected. No n=W? Yes 8 Menascé & Almeida. All Rights Reserved. 40 Types of System-level Models S e rv ic e Queue P o p u l a ti o n R a te S iz e in fin it e fix e d u n lim it e d in fin it e fix e d lim it e d in fin it e va ria b le u n lim it e d in fin it e va ria b le fin it e fin it e fix e d fin it e va ria b le 8 Menascé & Almeida. All Rights Reserved. 41 System-level models: Methodology • Determine proper representation for the state of system being modeled (numb. of requests in system, k) • Determine set of feasible states • Determine possible transitions between states, considering possible events (arrival, completion of request) 8 Menascé & Almeida. All Rights Reserved. 42 System-level models: Methodology (2) • For each possible state transition, determine transition rate, looking at event that caused the transition • Use flow in = flow out principle to write down equations relating values of pk; remember that sum of all pks is one. 8 Menascé & Almeida. All Rights Reserved. 43 System-level models: Methodology (3) • Solve for pks and use them to compute performance metrics: – utilization U – average throughput X – average number of requests Nreq – average response time R – fraction of requests that are lost because of queue overflow Ploss 8 Menascé & Almeida. All Rights Reserved. 44 Generalized System-level Models l0 l 1 0 m l2 2 m2 l3 3 m3 .... m4 Generalized System-level Models can be solved using the flow in = flow out principle 8 Menascé & Almeida. All Rights Reserved. 45 Generalized System-level Models li pk i 0 m i 1 k 1 k 1 l i p0 k 0 i 0 m i 1 8 Menascé & Almeida. All Rights Reserved. 1 46 System-level Models Example A Web server receives 30 requests/sec. Its throughput function is given below. The server queue is limited to five requests. What is the server utilization, avg. throughput, avg. no. requests, avg. response time, and fraction of lost requests? No. of requests Throughput (req/sec) 1 18 2 35 3 or more 50 8 Menascé & Almeida. All Rights Reserved. 47 System-level Models Example (cont’d) Using the Generalized System-level model equations we get that Average Number of Requests 1.850 Server Utilization 82.700% Server Average Throughput 28.4 req/sec Fraction of Lost Requests 0.05343 From Little’s Law, avg. response time = avg. no requests/ avg. throughput = 1.85 / 28.4 = 0.065 sec. 8 Menascé & Almeida. All Rights Reserved. 48 System-level Models Example (cont’d) Max. Queue Size Average Number of Requests Server Utilization Avg. Server Throughput (req/sec) Average Response Time (sec) Fraction of Lost Requests 3 1.433 79.81% 24.8 0.058 0.173077 8 Menascé & Almeida. All Rights Reserved. 5 10 15 1.850 2.212 2.264 82.7% 83.9% 83.96% 28.4 29.9 30 0.065 0.074 0.075 0.05343 0.003869 0.000299 49 System-level Models Example (cont’d) Fraction of Lost Requests Fraction of Lost Requests vs. Max Queue Size 0.2 0.15 0.1 0.05 0 3 5 10 15 Max Queue Size 8 Menascé & Almeida. All Rights Reserved. 50 Practical computation tools Help for SysMod.XLS Main Screen 1. This workbook implements the formulas that solve system models of C/S systems as discussed in Chapter 8 of the book. 2. The input parameters in each worksheet are entered in cells with purple background. 3. All cells, except for the ones where you must enter values, are protected to avoid unintended modifications. Tips: a. All input parameters should use compatible units (e.g., seconds for service demands and tps for arrival rates). The results are given in the units compatible with the input parameters. b. Do not change the location of any of the cells. This may render your workbook unusable or generate incorrect results. c. Always work on a copy of the original workbook provided with the book. You may open the original workbook and save it under another name using the Save As option from the File menu. 8 Menascé & Almeida. All Rights Reserved. 51 Practical computation tools Infinite population/constant service rate/infinite queue Formulas: fig. 8.6 pag. 182 This w orksheet comes w ith the book "Capacity Planning for Web Performance", Main Screen by D. A. Menascé and V. A. F. Almeida, Prentice Hall, 1998. Arrival Rate l Service Rate m Utilization U Average Numb. of Requests N Average Response Time R Note that the arrival rate must be less than the service rate! k Pk 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 30 tps 50 tps 60% 1.5 0.05 Utilization 0.4000000 0.2400000 0.1440000 0.0864000 0.0518400 0.0311040 0.0186624 0.0111974 0.0067185 0.0040311 0.0024186 0.0014512 0.0008707 0.0005224 0.0003135 0.0001881 8 Menascé & Almeida. All Rights Reserved. 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0.95 0.99 R / S = 1 / (1 - U) (see fig. 8.5 pag. 180) 1.00 1.11 1.25 1.43 1.67 2.00 2.50 3.33 5.00 10.00 20.00 100.00 52 Practical computation tools Infinite population/constant service rate/finite queue Formulas: fig. 8.9 pag. 185 This w orksheet comes w ith the book "Capacity Planning for Web Performance", by D. A. Menascé and V. A. F. Almeida, Prentice Hall, 1998. Arrival Rate l Service Rate m Max. Number of Requests (W) Average Number of Requests N Server Utilization U Server Throughput X Average Response Time R Fraction of Lost Requests Ploss k Pk 0 1 2 3 4 5 6 7 8 9 10 30 50 10 1.460 59.9% 29.9 0.049 0.24% W 0.4014565 0.2408739 0.1445243 0.0867146 0.0520288 0.0312173 0.0187304 0.0112382 0.0067429 0.0040458 0.0024275 1 2 3 4 5 6 7 8 9 10 8 Menascé & Almeida. All Rights Reserved. Main Screen tps tps tps sec Fraction of Lost Requests 37.5% 18.4% 9.9% 5.6% 3.3% 1.9% 1.1% 0.7% 0.4% 0.2% (see fig. 8.8 pag. 184) 53 Practical computation tools Infinite population/variable service rate/infinite queue Formulas: (8.5.1) to (8.5.5) pag. 188-189 This w orksheet comes w ith the book "Capacity Planning for Web Performance", by D. A. Menascé and V. A. F. Almeida, Prentice Hall, 1998. Arrival Ratel Saturation Point (J) Average Numb. of Requests N Server Utilization U Server Average Throughput X Average Response Time R Main Screen 30 tps 3 2.271 84.0% 30.0 tps 0.076 sec Note that the arrival rate must be less than X(J)! k X(k) Throughput values until saturation 1 2 3 18 35 50 8 Menascé & Almeida. All Rights Reserved. 54 Practical computation tools Infinite population/variable service rate/finite queue Formulas: (8.5.6) to (8.5.9) pag. 190 This w orksheet comes w ith the book "Capacity Planning for Web Performance", by D. A. Menascé and V. A. F. Almeida, Prentice Hall, 1998. Main Screen Arrival Rate l 30 tps Saturation Point (J) 3 Max. Number of Requests (W) 5 Average Numb. of Requests N 1.852 Server Utilization U 82.7% Server Avg. Throughput X 28.4 tps Average Response Time R 0.065 sec Fraction of Lost Requests Ploss 0.05343 k X(k) Throughput values before saturation 1 2 3 18 35 50 8 Menascé & Almeida. All Rights Reserved. 55 Practical computation tools Finite population/constant service rate Formulas: (8.5.10), (8.5.11) pag. 191-192 This w orksheet comes w ith the book "Capacity Planning for Web Performance", by D. A. Menascé and V. A. F. Almeida, Prentice Hall, 1998. Main Screen Average Think Time Z Service Rate m Number of Clients M Average Number of Requests N Server Utilization U Server Throughput X Average Response Time R 2 80 50 0.443 31.0% 24.8 0.018 8 Menascé & Almeida. All Rights Reserved. sec tps tps sec 56 Practical computation tools Finite population/variable service rate Formulas: (8.5.15) to (8.5.18) pag. 193-194 This w orksheet comes w ith the book "Capacity Planning for Web Performance", Main Screen by D. A. Menascé and V. A. F. Almeida, Prentice Hall, 1998. Average Think Time Z Saturation Point (J) Number of Clients M Average Number of Requests N Server Utilization U Server Throughput X Average Response Time R k X(k) Throughput values until saturation 1 2 18 35 1 3 50 0.645 97.0% 11.1 0.058 sec tps tps sec 3 50 8 Menascé & Almeida. All Rights Reserved. 57 Part VI: Summary •System-level models view a server as a black box. Only its arrival process and throughput functions are relevant. •State Transition Diagrams (STDs) can be used to find the probability that k requests are in the server. Use the flow in = flow out principle. •Little’s Law can be used to compute the response time from the average number of requests and from the throughput. 58 8 Menascé & Almeida. All Rights Reserved.