Transcript Maintaining Quality - EclipseCon France 2015

Can’t Live With It, Can’t
Live Without It: Test
Strategies for Agile
Teams
Brad Hipps, HP Software
EclipseCon 2013
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Traditional waterfall delivery
Defects found too late in the cycle for effective resolution
Jan
R
i
s
k
Feb
Mar
Apr
Jun
Rqmt
Design
Code / unit test
System test
Time
2
May
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Agile: as it should be
Jan
Feb
Mar
Apr
May
May
Jun
Jun
Rqmt
RQMT
Rqmt
Rqmt
Rqmt
Rqmt
Rqmt
Design
Design
Design
Design
Design
Design
DESGN
Code/UT
Code/UT
Code/UTCODE Code/UT
/ UT
Code/UT
Code/UT
Sys. test
Sys. test
Sys. test
Sys. test
TESTSys. test
 Time-boxed for focus
 Hands-on with stakeholders
 Surfaces issues sooner
3
Sys. test
 Continuous testing
 Designed for change
 True measure of progress
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Agile: as it too often is
Jan
Feb
Mar
Rqmt
RQMT
Rqmt
Rqmt
Design
DESGN
Code/UT
Design
May
May
Jun
Jun
Design
Code/UTCODE Code/UT
/ UT
Code/UT
Sys. test
Sys. test
 Time-boxed for focus
 Hands-on with stakeholders
 Surfaces issues sooner
4
Apr
Apr
TESTSys. test
 Continuous testing
 Designed for change
 True measure of progress
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Scrummerfall. n.
The practice of combining scrum and
waterfall so as to ensure failure at a faster
rate than with waterfall alone.
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
C
G
B
Instability & pre-automation
Exploratory test each new element as it’s introduced
Component (non-GUI)
GUI element
Business process
Stable
Sprint 1
6
Sprint 2
1 Sprint 3
Sprint 4
Sprint 5
C1
C1
C1
C1
C1
G1
G1
G1
G1
G1
C2
C2
C2
C2
G2
G2
G2
C3
C3
B1
B1
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Why explore?
Mine
Fix
The best way to mine sweep
Repeatability alone won’t
clear the field
7
Repeatability and variability
are key
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
“Any tests that can be scripted should be
automated. We don’t need humans doing
something that a machine can do. What we do
want the human testers doing is exploratory
testing. Exploratory testing is a creative
endeavor in which human testers explore the
behavior of the system in an attempt to
–
Dr. Robert C. (“Uncle
Bob”) Martin both documented and
characterize
its behaviors,
undocumented.”
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
The “s+1” rule of test automation
Automation never lags by more than one sprint
C
G
B
Component (non-GUI)
GUI element
Business process
Stable
Sprint 1
9
Sprint 2
1 Sprint 3
Sprint 4
Sprint 5
C1
C1
C1
C1
C1
G1
G1
G1
G1
G1
C2
C2
C2
C2
G2
G2
G2
C3
C3
B1
B1
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Solving for multiple app layers
Multi-layered testing allows automation even without GUI
C
G
B
Component (non-GUI)
GUI element
Business process
Stable
Sprint 1
10
Sprint 2
1 Sprint 3
Sprint 4
Sprint 5
C1
C1
C1
C1
C1
G1
G1
G1
G1
G1
C2
C2
C2
C2
G2
G2
G2
C3
C3
B1
B1
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
C
G
B
Solving for multiple app layers
Service virtualization eliminates dependencies
Component (non-GUI)
GUI element
Business process
Stable
Sprint 1
Sprint 2
1 Sprint 3
Sprint 4
Sprint 5
C1
C1
C1
C1
C1
G1
G1
G1
G1
G1
C2
C2
C2
C2
G2
G2
G2
C3
C3
B1
B1
External components
11
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
The ideal picture: continuous testing
A relentless assault on project risk
Sprint 1
Sprint 2
C1
ET
C1
ET
G1
CI
1 Sprint 3
Sprint 4
Sprint 5
C1
ET
C1
ET
C1
ET
G1
G1
G1
G1
C2
CI
C2
CI
C2
CI
C2
CI
G2
G2
G2
C3
C3
Regression 1
Regression 2
B1
Regression
3
B1
Regression 4
12
Longevity testing (independent of sprint cycles)
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Recent means of shifting quality “to the left”
TDD
Red, green, refactor
BDD
TDD’s more mature sibling. More structured: given, when, then
ATDD
13
Upfront customer criteria. Focused on business intent
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
How to avoid the “scrummerfall” trap
Remember...
1. Exploratory testing is perfectly suited to the evolving, unstable
components in an Agile project
2. Time must be explicitly allocated for test automation in each sprint.
3. Automation should begin as early as possible and should never lag by
more than one sprint.
4. When assessing the results of a sprint, the automation goals must be
considered as vital as the development objectives.
5. Multi-layered testing allows for test automation in advance of GUI
stability, and furthers test coverage.
6. Legacy tools won’t deliver modern velocity. Solutions for acceleration,
virtualization and advanced automation are crucial.
14
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Thank you
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.