Transcript Presentation title

Too Slow?:
Use VS2010 Profiling & Load
Testing to Manage Performance
Issues
Benjamin Day
Benjamin Day
• Consultant, Coach, Trainer
• Scrum.org Classes
 Professional Scrum Developer (PSD)
 Professional Scrum Foundations (PSF)
•
•
•
•
TechEd, VSLive, DevTeach, O’Reilly OSCON
Visual Studio Magazine, Redmond Developer News
Microsoft MVP for Visual Studio ALM
Team Foundation Server, TDD, Testing Best Practices,
Silverlight, Windows Azure
• http://blog.benday.com
• [email protected]
Thanks.
• To my content reviewers
 Jerri Chiu, Microsoft
 Dennis Bass, Microsoft
The Agenda
• Basics of




Web Performance Tests
Load Tests
Load Test Rigs
Performance Explorer
• Existing application
 Identify performance problems
 Go from bad performance to good
 ASP.NET & WCF
Disclaimer: I’m going to use the term
“load testing” imprecisely…a lot.
Here we go.
The Purpose of Load Testing
• Identify the capabilities of your application
• Possible goals




Explore
Verify
Find the limits
Crush
Best Practice:
The Obligatory “Eat Your Vegetables”
• Load test throughout the development cycle.
• Catch problems early
• Establish performance baseline
• Watch for performance trends
• Cheaper to fix early
• Plan for performance
Cheaper To Fix Early
60
Cost to fix
50
40
30
20
10
0
Early
Less Early Nearly Done Production
Time
Dose Of Reality
• You know you should work that way.
• You probably don’t.
• This talk is for you.
• You have problems.
• How do you figure out what’s going wrong?
Inspiration for this talk
•
•
•
•
•
•
•
Real customer engagement
Spent years developing a web application
Released it
Seriously grumpy customers
Huge hardware
~20 simultaneous users
Fixed it
 4000+ simultaneous users
 Reduced hardware
The Tools
Visual
Studio
Ultimate
Visual
Studio
Premium
• Web Performance Tests
• Load Tests
• Load Test Rigs
• Profiling Tools
• Performance Wizard
TOUR OF THE APPLICATION
Disaster Strikes.
• The application is deployed to production.
• Users are losing their minds.
 Intermittent errors on the site.
 Site is slow.
• Your boss is seriously angry.
 “Fix it!”
The Problem.
• “No Repro”
 “Works on my box.”
 You don’t know what the errors are.
 The site seems to work fine.
• Wild guess: it dies under load.
• You need to create some load.
The Plan.
1.
2.
3.
4.
5.
6.
Script user activity with Web Performance Tests
Simulate a mix of users using Load Tests
Simulate lots of users with a Load Test Rig
Reproduce the errors...hopefully
Find code problems with the Performance Wizard
Fix the code
Web Performance Tests.
The Plan.
1.
2.
3.
4.
5.
6.
Script user activity with Web Performance Tests
Simulate a mix of users using Load Tests
Simulate lots of users with a Load Test Rig
Reproduce the errors...hopefully
Find code problems with the Performance Wizard
Fix the code
What is a Web Performance Test?
• Test type in Visual Studio 2010 Ultimate
• Simulate a user using an ASP.NET application
 HTTP traffic
• Building block of Load Tests
Tip: Web Tests & User Stories
• Web Tests should simulate a User Story
• Example:
 As an administrator, I need to search for a person by username so that I
can deactivate that person’s account.
 As a customer, I want to order a pizza and pay with a credit card.
• Helps to organize your suite
CREATE A BASIC WEB TEST
Data-driven Web Tests
• Run web test once per row in data source
• Simulate multiple users doing *similar* actions
• Data source types
 OLE DB, CSV, XML
• Parameterization
• Better tests, better coverage
WEB TESTS IN THE
UNDERPERFORMING
APPLICATION
Load Tests.
The Plan.
1.
2.
3.
4.
5.
6.
Script user activity with Web Performance Tests
Simulate a mix of users using Load Tests
Simulate lots of users with a Load Test Rig
Reproduce the errors...hopefully
Find code problems with the Performance Wizard
Fix the code
What are Load Tests?
• Mix of Web Performance and Unit Tests
• Simulates a mix of users doing a mix of things
• Simulate
 Network speeds
 Different browsers
 Varying user activity loads
Why would you put a Unit Test in a Load
Test?
• Helpful for testing WCF
• Cumbersome to simulate WCF via Web Test
 (You’d have to re-invent the wheel.)
• Unit tests can re-use the WCF libraries
 (Done and done.)
• Important for Silverlight and SOA
• http://wcfloadtest.codeplex.com/
• http://sqlloadtest.codeplex.com/
HEY KIDS,
LET’S CREATE SOME LOAD
TESTS!
(…and then you all say, “Hooray!”)
Running Load Tests.
The Plan.
1.
2.
3.
4.
5.
6.
Script user activity with Web Performance Tests
Simulate a mix of users using Load Tests
Simulate lots of users with a Load Test Rig
Reproduce the errors...hopefully
Find code problems with the Performance Wizard
Fix the code
Two ways to run load tests.
• Generate load from Visual Studio 2010
 Uses only 1 core/processor
 Limited to 250 simulated users
• Generate load from a Test Rig
 Far better test
 Simulate lots more users
Load Test Agent Licensing
• Microsoft Visual Studio Team System 2010
Test Load Virtual User Pack
• Simulate up to 1000 virtual users
• Used to cost $$$
Visual Studio 2010 Load Test Feature
Pack
• Used to cost ~$3500.00 per 1000 simulated users
• Free starting March 8, 2011
• Visual Studio 2010 Ultimate with MSDN
• Simulate unlimited amount of load
 (Limited only by your hardware.)
• http://www.microsoft.com/visualstudio/
en-us/products/2010-editions/load-test-virtual-user-pack
Load Test Rig & The Application
• Controller
 Orchestrates
 Licensing
 Perf Mon
Counters
• Agent (1..n)
 Simulates
users
The Plan.
1.
2.
3.
4.
5.
6.
Script user activity with Web Performance Tests
Simulate a mix of users using Load Tests
Simulate lots of users with a Load Test Rig
Reproduce the errors...hopefully
Find code problems with the Performance Wizard
Fix the code
RUN LOAD TESTS ON THE RIG
The Core Problem.
‘sp_who2’ says lots of open connections.
Performance Profiling
The Plan.
1.
2.
3.
4.
5.
6.
Script user activity with Web Performance Tests
Simulate a mix of users using Load Tests
Simulate lots of users with a Load Test Rig
Reproduce the errors...hopefully
Find code problems with the Performance Wizard
Fix the code
Profiling Tools
• Load Tests help you find issues in a system
 “Surface” issues
• Profiling helps you find issues in the code
• Unit Tests, Web Tests, Load Tests
 Repeatable
 Good for targeted tuning
• System under “natural” load
 Not repeatable
 Looking for rough patterns
Profiling Methods
Sampling
• Gathers information at intervals (clock cycles)
• No code modifications
• Use this for first pass explorations
Instrumentation
• Modifies your code
• Gathers detailed timing and count information
Concurrency
.NET Memory
Tier Interaction
• Multi-threaded code profiling
• Object creation
• Garbage collection
• Interaction between your application and SQL
Server via ADO.NET
Load Test Performance Sessions Via Wizard
• Must be only Web Performance Tests
• Can’t profile via Load Test Rig
• ASP.NET must be running in x86 mode
LET’S GO PROFILE SOME
STUFF.
The Plan.
1.
2.
3.
4.
5.
6.
Script user activity with Web Performance Tests
Simulate a mix of users using Load Tests
Simulate lots of users with a Load Test Rig
Reproduce the errors...hopefully
Find code problems with the Performance Wizard
Fix the code
Any last questions?
For More Information: Articles
http://tinyurl.com/3xjqgvz
http://tinyurl.com/2ulvvvr
For More Information: Microsoft Links
• Visual Studio Performance Testing Quick Reference Guide
http://vsptqrg.codeplex.com/
• Content Index for Web Tests & Load Tests
http://tinyurl.com/ye97e52
For More Information: Microsoft Bloggers
• VS Team Test Blog Site
http://blogs.msdn.com/b/vstsqualitytools/
• Ed Glas - http://blogs.msdn.com/edglas/
• Bill Barnett - http://blogs.msdn.com/billbar/
• Sean Lumley - http://blogs.msdn.com/slumley/
• Dennis Stone - http://blogs.msdn.com/densto/
• Mike Taute - http://blogs.msdn.com/mtaute/
• Neelesh Kamkolkar http://blogs.msdn.com/b/nkamkolkar/
Thank you.
blog.benday.com | www.benday.com | [email protected]
Additional information that I
practically never have time to cover.
Miscellaneous Tips.
Include Load Tests In Your Nightly Build
•
•
•
•
Create a Test List (*.vsmdi)
Include the Test List in the build
Captures trend data at 24 hours intervals
Compare the load test data with Excel
Static Code Analysis / FxCop
• Help you to find issues in your code
• Show you IDispose errors
• Rules are configurable
• Use your judgment about which are relevant for your team
Avoid Static Methods & CA1822
• Static Code Analysis Rule CA1822 is evil.
• Sacrifices maintainability for perfomance.
• Blog post:
“Static Methods Are A Code Smell”
• http://bit.ly/cnEsuV
Only optimize known performance
problems.
• You’d be surprised by what *ISN’T* a performance problem
• Don’t spend a ton of time coding fancy solutions to
performance problems you *THINK* you might have
• Find and fix *REAL* problems
Tip: WCF Message Size
• Data structure formatting can make a huge difference
• Example: Banks and the States they operate in
• Option 1: Dictionary<string, string>
 Key = Bank Id
 Value = State Abbreviation
• Option 2: Dictionary<string, string[]>
 Key = State Abbreviation
 Value = Array of Bank Id’s
• Option 2 was approximately 90% smaller
Tip: Add Performance Counters
• Create Performance Counters (perfmon) in your app
• Do this early in the development cycle
• More directed profiling of your app during Load Tests
• Good for managing your app when it’s in production
 (Operations people love this.)
Using Performance Sessions on x64
• You can’t profile x64 IIS processes from the IDE
• Option #1: Profile from the command line
 Start ASP.NET profiling using VSPerfAspNetCmd.exe
 Run the web test
• Option #2: Run the ASP.NET process in x86 mode
How To Run ASP.NET in x86 Mode
• Set “Enable 32-bit Applications” to true on
your application’s AppPool