Transcript Performance Engineering Methodology

Performance Engineering
Methodology
Chapter 4
Performance Engineering
• Performance engineering analyzes the
expected performance characteristics of a
system during the different phases of its
lifecycle.
• Performance engineering
– 1) develops practical strategies that help predict
the level of performance a system can achieve and
– 2) provides recommendations to realize the
optimal performance level.
Typical PE Questions
• Can the insurance claim system meet its
performance requirements of sub-second
response time when a natural disaster occurs
(e.g., a hurricane). Response Time
• Is the infrastructure of a government agency
scalable and can it cope with the computing
demands of the new required online security
mechanisms? Scalability
• Is the reservation system for cruise lines able to
respond to anticipated peak of customer inquiries
after a TV ad campaign? Reliability
PE Larger Questions
• How can one plan, design, develop, deploy,
and operate IT services that meet ever
increasing demands for performance,
availability, reliability, and security?
• Is a given IT system properly designed and
sized for a given load condition?
PE Activities
• Understand the key factors that affect a system’s
performance.
• Measure the system and understand its
workload.
• Develop and validate a workload model that
captures the key characteristics of the actual
workload.
• Develop and validate an analytic model that
accurately predicts the system’s performance.
• Use the models to predict and optimize the
system’s performance.
Modeling Process
Motivating Example: a Call Center
Call Center
• Goals:
– Foster better relationships with customers, creating
customer loyalty and ensuring quality service.
– Improve efficiency and service performance.
– Identify and explore new sales opportunities.
• Main Functions:
– Order status inquiry
– Shipment tracking
– Problem resolution status inquiry
• Requirements: sub-second response time and
24x7 operation.
QoS Questions
• Is the system design able to meet the subsecond response time for
all functions? Response Time
• What will be the impact of doubling the number of system
representatives in the next year? Scalability
• Can acceptable performance levels be maintained after integrating
the system with the mainframe-based inventory application?
Scalability
• Is the system capacity adequate to handle up to 1,000 calls in the
busiest hour and yet preserve the subsecond response time goal?
• How do failures in the database server affect the 24x7 availability
goal?
• What is the impact of starting to offer Web-based self-service to
customers?
At the Requirements Analysis
Phase
• Workload definition:
–
–
–
–
Call center’s view: Arrival rate of phone calls
IT system’s view: Functions received from the representatives.
DB server view: SQL requests from the application server.
LAN view: packet size distribution and interpacket arrival time.
At the System Design Phase
• What should the system throughput be to
meet sub-second response times?
– 200 customer service representatives and 80% are
working during the peak hour.
– Average think time of 30 sec.
Model of the call center system
Call Center Model
• Z: average think time, 30sec
• N: number of active
representatives in the
system, 200X80% = 160
• X0: system throughput
• R: average response time <
1sec
• Using the interactive response time law:
At the System Development Phase
• What should be the capacity of the DB server
so that the performance goals are met?
– Each submitted functions requires 2.2 SQL calls
on average.
– From the Forced Flow Law:
At the Operation Phase
• Assume DB server is a problem. Response times
exceed sub-second goal.
• Measurements during peak hour:
– 57600 queries/hour
– Each query needs 50 msec of CPU, performs 4 I/Os on
disk 1 and 2 I/Os on disk 2. Each I/O takes 8 msec on
average.
– X0 = 57600 / 3600 = 16 queries/sec
– Service demands:
• Dcpu = 0.05 sec;
• Ddisk1 = 4 x 0.008 = 0.032 sec;
• Ddisk2 = 2 x 0.008 = 0.016 sec.
At the Operation Phase (cont’d)
At the Operation Phase (cont’d)
• The residence times at the CPU and disks for
open QN model
• Response time of the DB server:
• RDB =
At the Evolution Phase
• The company is considering to develop Web
applications to allow customers access to the
information they need without assistance from a
customer representative. Web self-services
reduce transaction costs and enhance the
customer experience.
• Security requirements mandate that new
applications be developed for Web access
(authentication, auditing, DB access control
mechanisms).
At the Evolution Phase
• Local queries and web queries:
Results for Evolution Scenario
Ui  Si  Xi
Open Multiclass Queuing Networks - Utilizations
This wokbook comes with the books "Performance by Design," "Capacity Planning for Web Services," and "Scaling for E-Business"
by D. A. Menascé and V. A. F. Almeida, Prentice Hall, 2004, 2002 and 2000.
Classes ®
Queues ¯
1
2Total
cpu1
0.80000
0.15000
0.95000
d12
0.51200
0.20000
0.71200
d23
0.25600
0.10000
0.35600
Results for Evolution Scenario
Open Multiclass Queuing Networks - Queue Lengths
This wokbook comes with the books "Performance by Design," "Capacity Planning for Web Services," and "Scaling for E-Business"
by D. A. Menascé and V. A. F. Almeida, Prentice Hall, 2004, 2002 and 2000.
Classes ®
Queues ¯
1
2Total
cpu1
16.00000
3.00000
19.00000
d12
1.77778
0.69444
2.47222
d23
0.39752
0.15528
0.55280
Performance Engineering Methodogy