Transcript Evaluation_of_Load_Testing_Tools

Evaluation of Load Testing Tools
WebLOAD Professional Vs NeoLoad 3.1
PROJECT PRESENTATION
Prof: Daniel Amyot
Presented By…
ANVESH ALUWALA
GURPREET SINGH DHADDA
Agenda…
 Business context
 Our Goal in the project
 Load Testing
 Methodology
 List of Criteria
 Description to WebLOAD Professional & NeoLoad 3.1
 Final Recommendation
Our Business Context…
 Company: weexcel Inc.
 Developers: 25
 QA Testers: 4
 The company planned to launch an web application and they
want to Load Test the application before deployment
 The company needs a recommendation of the tool that meets
the maximum goals set by the company
Our one and only Goal…
 Recommend the tool that best serves the organization’s
purpose to test their web applications before deployment
Intro to Load Testing…
 Although both Load Testing and Performance Testing seem
similar, their goals are different.
 Load testing operates at a predefined load level, usually the
highest load that the system can accept while still functioning
properly.
 On the other hand, performance testing uses load testing
techniques and tools for measurement and benchmarking
purposes and uses various load levels.
 We can generally call the Load Testing as the subset of
Performance testing.
Load Testing…
 Load Tests determine the applications behavior under
load, up to and including its limits (not just at its limits).
 Load tests specifically refer to the load size (number of
concurrent users) and related values.
Methodology…
 We compare two tools:





WebLOAD Professional & NeoLoad 3.1
We use an E-Commerce website for creating the test
process.
It will be a quantitative analysis.
Each criteria will have a specific range.
Both the tools will be marked for each criteria within the
specific range.
The tool which scores high overall will be recommended.
Criteria…
Minimal criteria…
 Scope for automation.
 Detect performance bottlenecks.
 Display response times under variable loads.
 Must support HTTP/S, SOAP and TCP/IP Internet
protocols.
 MS Windows compatible.
Criteria…(Contd.)
Good to have criteria…
• Cost and License.
• Ease of use.
• Steadiness of the tool.
• Documentation available for the tool.
• Variety of reports provided by the tool.
• Support scripting for scenario generation.
• Customer support.
• Reputation of the tool in market.
Range for each criteria…
CRITERIA
RANGE
Scope for automation
0-10
Detect performance bottlenecks
0-10
Display response times under variable loads
0-10
Must support HTTP/S, SOAP and TCP/IP Internet protocols
0-10
MS Windows compatible
0-10
Cost and License
0-5
Ease of use
0-10
Steadiness of the tool
0-5
Quality of reports provided by the tool
0-10
Support scripting for scenario generation
0-5
Customer support
0-5
Documentation available for the tool.
0-5
Reputation of the tool in market
0-5
TOTAL
100
WebLOAD Professional…
 This tool is from the family of “Radview”
Testing process:
• Creating and editing scripts
• Creating and running Load Tests
• Analyzing Load Test results
• Scripts can be added in JavaScript language
• Also provides script correlation, script validations and script
parameterization
Screenshots…Creating agenda
Screenshots…Running Test
Screenshots…Analyzing results
NeoLoad 3.1
 This tool is from “Neotys”
Testing process:
• Record
-Capture a scenario
• Design
-Graphical interface
-Defining dynamic parameters
-Defining logical options(delay, loops, try catch, etc)
-Checking the virtual users
NeoLoad 3.1
Testing process(contd.)
• Monitors
-Create monitoring machine(select OS, Databases and Web
EJB Architecture)
-Select performance counters
System(CPU User, CPU system, CPU idle, etc.)
Memory(memory used, memory free)
• Runtime
-Select load generator
-Select duration
• Analyze results
Screenshots…creating scenario
Screenshots...runtime
Screenshots…analysis
Marking of tools on each criteria…
 Scope for automation: The test cases should run
automatically after defined time intervals.
WebLOAD: This tool supports automation of test cases
NeoLoad: Even this tool supports automation of running
test cases by specifying the time interval in the duration policy
 Detect performance bottlenecks: should specify an
alert message when the system is in critical stage
WebLOAD: Gives a detail report of the errors
NeoLoad: Gives a quick alert message and details regarding
that error
Marking of tools on each criteria…
 Display response times under variable loads
WebLOAD: The reports generated specify clearly the
response times for different loads
NeoLoad: We can track the response times at every instance
during the run time of the test process
 Must support HTTP/S, SOAP and TCP/IP Internet
protocols
WebLOAD: Wide range of protocols are supported
NeoLoad: Wide range of protocols are supported
Marking of tools on each criteria…
 MS Windows compatible: The tool must mainly support
MS Windows OS but support for other platforms is
considered as an advantage.
WebLOAD: Supports MS Windows OS
NeoLoad: Supports Linux, Solaris, IBM AIX, HP-UX and
VMWare other than MS Windows OS
 Cost and License: The competitive price matters a lot in
evaluation.
WebLOAD: Cost is over $5000 for 100 virtual users
NeoLoad: Cost is cheaper when compared to WebLOAD
Marking of tools on each criteria…
• Ease of use: Usability is important. The tool should be easy
to learn for the testers.
WebLOAD: Little tricky to create the script parameters and
validations
NeoLoad: Self-explanatory and easy to use
• Steadiness of the tool: The tool should not crash or hang
up in between.
WebLOAD: Little bit slow in runtime
NeoLoad: Runs perfect on threshold loads too
Marking of tools on each criteria…
• Quality of reports provided by the tool: At the end,
the reports on the test should be precise and of variety of
representation types
WebLOAD: Need not be a performance analyst in order to
examine the test results.
NeoLoad: Provides enough charts to represent the results
efficiently
• Support scripting for scenario generation
WebLOAD: This supports scripting
NeoLoad:This is only GUI based
Marking of tools on each criteria…
• Customer support
WebLOAD: The Radview team provides excellent service
NeoLoad: The Neotys team are equally helpful
• Documentation for the tool
WebLOAD: Precise documentation available
NeoLoad: Well formatted documentation is available
• Reputation of the tool in market
WebLOAD: Captured 3% of the market share
NeoLoad: New tool, but good competitor for WebLOAD.
Final marking of the tools…
CRITERIA
RANGE
WebLOAD
NeoLoad
Scope for automation
0-10
10
10
Detect performance bottlenecks
0-10
10
10
Display response times under variable loads
0-10
9
10
Must support HTTP/S, SOAP and TCP/IP
Internet protocols
0-10
10
10
MS Windows compatible
0-10
9
10
Cost and License
0-5
3
5
Ease of use
0-10
7
10
Steadiness of the tool
0-5
3
5
Quality of reports provided by the tool
0-10
10
10
Support scripting for scenario generation
0-5
5
0
Customer support
0-5
5
5
Documentation available for the tool.
0-5
4
4
Reputation of the tool in market
0-5
5
4
100
90
93
TOTAL
Final Recommendation…
 After the quantitative analysis of both the tools we
recommend NeoLoad 3.1 and the detailed results and
evaluation to choose this tool will be explained in the final
report.
Thank U for kind attention…