Transcript Session Title That is Really Long and Covers Two Lines

Software Testing with
Visual Studio 2013 &
Team Foundation Server 2013
Benjamin Day
Benjamin Day
•
•
•
•
•
Brookline, MA
Consultant, Coach, & Trainer
Microsoft MVP for Visual Studio ALM
Team Foundation Server, Software
Testing, Scrum, Software Architecture
Scrum.org Classes
– Professional Scrum Developer (PSD)
– Professional Scrum Foundations (PSF)
•
www.benday.com,
[email protected], @benday
Online courses at Pluralsight.com
On with the show.
Overview of
Visual Studio ALM.
Three types of testing in the VS
ALM World.
•
Developer
– “Does the code work?”
•
Manual Testing / Quality Assurance (QA)
– “Does the app work?”
•
Load Testing & Performance Testing
– “Does the app work under load?”
Test types & features.
•
Developer
– Unit Tests
•
Manual / QA
– Coded UI tests
•
Load & Performance
– Web Performance Tests (WPTs)
Recorded or Coded
– Load Tests
There are only so many
minutes in the day.
The real question is ‘what to test?’
Testing ROI.
Testing ROI for Developers.
•
Write unit tests.
– Test First. Test-Driven. Test-Eventually. (Whatever.)
– Write something and try to be honest.
– Watch your code coverage.
•
Why?
– Helps you refactor.
– Helps you modify feature functionality.
– Helps you to know if it’s working a lot faster.
– Tends to push you into better/cleaner architecture.
$1m for IT to support an
application.
That’s looking grim.
New Feature Budget vs. Maintenance Budget
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
Q1
Q2
Q3
Q4
Q1
Q2
Q3
Q4
Q1
Q2
Q3
Q4
Q1
Q2
Q3
Q4
2010 2010 2010 2010 2011 2011 2011 2011 2012 2012 2012 2012 2013 2013 2013 2013
Maintenance
New Features
What’s that feel like?
•
Costs more to run the app than you actually
have.
•
App breaks if you breathe funny.
•
Can’t add new features.
•
Changes take *FOR-EVER*.
•
Competitors are catching up.
•
You’re hosed. It’s hopeless.
Why’s it getting like that?
•
Technical Debt.
•
Excess “inventory”.
•
Too much architecture.
•
Brittle code.
•
Too hard to refactor.
Time to re-write the app.
Use tests to keep it clean.
Or start adding tests to change
the ROI trajectory.
$1m for IT to support an
application.
New Feature Budget vs. Maintenance Budget
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
Q1
Q2
Q3
Q4
Q1
Q2
Q3
Q4
Q1
Q2
Q3
Q4
Q1
Q2
Q3
Q4
2010 2010 2010 2010 2011 2011 2011 2011 2012 2012 2012 2012 2013 2013 2013 2013
Maintenance
New Features
Solid developer testing pays huge
dividends.
Better long-term ROI.
Faster time to market on new features.
More responsive to market with
changes to existing features.
Catch problems early in dev process.
Vastly higher quality builds for
QA testing.
Testing ROI for QA / Manual
Testing.
•
Track your requirements with TFS.
•
Track test plans, progress, and do defect
tracking
with TFS and/or Microsoft Test Manager
(MTM).
•
Use MTM Action Recordings to minimize
tedium.
•
Look for opportunities for test automation
with MTM and Coded UI Tests.
Rule of thumb:
At least one Test Case per Scrum PBI.
Coded UI Tests.
Coded UI helps you test a
running application’s user interface.
Unit Tests vs. Coded UI Tests
Unit Tests
•
Classes and methods
at the API level
•
If it tests a UI, it’s
testing an abstraction.
– (not quite testing the UI)
– UI testing has been hard
•
Test stuff as you build
it
Coded UI Tests
•
UI testing
•
Test a running
application
•
•
•
Simulates a user’s
keyboard and mouse
activities
Test stuff that’s pretty
much done
Integration testing
The layers in your app.
User Interface
(ASP.NET, XAML, WinForms, etc.)
Presentation
(MVC / MVVM)
Domain Model / Service Layer
Repository / Data Access
The Relational Database
(Tables, FKs, Views, Stored Proc's)
The layers in your app.
User Interface
(ASP.NET, XAML, WinForms, etc.)
Coded
UI
tests
Presentation
(MVC / MVVM)
Domain Model / Service Layer
Unit tests
Repository / Data Access
The Relational Database
(Tables, FKs, Views, Stored Proc's)
Unit tests test your APIs.
Coded UI Tests test your running user
interfaces.
I tend to think of QA identifying a case
that needs automation and then
requesting a
Coded UI Test from developers.
Action Recordings vs. Coded UI
Tests
•
Action Recordings
– Exist in Microsoft Test Manager
– It’s there to help QA automate away tedious clicks and
typing.
– Make QA testing go faster.
– The idea: created by non-technical users.
•
Coded UI Tests
– Action Recordings on steroids.
– It really helps to be a programmer.
– You can do “asserts”.
(aka. you can do actual checks)
– You can still associate them to MTM Test Cases as
“Associated Automations.”
Structure of a Coded UI Test
•
The Test Fixture Class
•
– [CodedUI] attribute
– Editable like any other
class
– (You can even make it
data-driven!)
•
UIMap.uitest
– Auto-generated XMLbased “map” of your UI
– Not editable
UIMap.designer.cs
– Supporting information for
the test
– Auto-generated by the
recorder
– Partial class
•
UIMap.cs
– Partial class
– Customizations and
extensions to the stuff in
UIMap.designer.cs
Avoiding hard-coded paths in
Coded UIs.
•
BrowserWindow class
•
ApplicationUnderTest window
Load & Performance Testing.
You have an application.
The Goal:
Performance Test The Application
The purpose of performance
tests?
•
Identify the capabilities of your application
•
Possible goals
– Explore
– Verify
– Find the limits
– Crush
Performance testing in a nutshell.
1.
Throw traffic at the app.
2.
Does it perform as expected?
3.
Recreate any performance problems.
4.
Fix the performance problems.
5.
Repeat.
What can you load test with
Visual Studio 2012 Ultimate?
•
ASP.NET with an HTML UI
– Web Forms or MVC
– (This is the sweet spot.)
•
SharePoint Applications
•
HTTP-based applications
– ASP.NET-based SOA apps
– Service-based apps with WebAPI or REST
•
Coded UI Tests
•
Pretty much anything you can call from a Visual
Studio Unit Test
Demos.
Demos
•
•
•
Unit Tests
Code Coverage
QA Testing
–
–
–
–
–
–
Project Management
Sprint Planning
Test Case Management
Test using MTM
Test using web
Test using web + test
runner
– Test environment manager
•
Web & Load Tests
–
–
–
–
–
WPT Basics
Load Test Basics
Custom PerfMon Counters
Load in the Cloud
Load using Test Rigs
Any last questions?
Thanks.
[email protected] | www.benday.com