Transcript Synchronizing Software Testing with Agile Requirements Practic

Synchronizing Software Testing
with Agile Requirements
Practices
Jean McAuliffe, Dean Leffingwell
May 2005
Copyright 2003-2005, Rally Software Development Corp
Background on Speakers
•
Slide 2
Jean McAuliffe
₋ Product Manager, Rally
Software Development
₋ 2 years Agile Development,
Certified Scrum Master
₋ Former Senior QA Manager
for Rational RequisitePro.
₋ Over 15 years experience in
all aspects of software
development (defining,
developing, testing, training
and supporting) for Software
Development, BioEngineering and Aerospace
companies
Copyright 2003-2005, Rally Software Development Corp
•
Dean Leffingwell
₋ Advisor and coach to a
number of developmentalstage software businesses.
₋ Former Senior VP of Rational
Software Corporation where
he was responsible for the
commercial introduction of
the Rational Unified Process.
₋ Lead author of the text
Managing Software
Requirements: Second
Edition: A Use Case
Approach, Addison Wesley,
2003
Abstract
• Agile requirements practices generally defer commitment to
artifacts such as requirements until the “last responsible
moment.” This challenges the test team to stay continuously
synchronized with the product owners and developers as they
have little lead time, if any, from the time requirements are
available until the time the test artifacts must be in place.
• This presentation describes the organizational and technical
challenges associated with just-in-time agile requirements
practices and techniques test teams can use to address them.
Slide 3
Copyright 2003-2005, Rally Software Development Corp
Agenda
•Context for Agile Testing
•Technical Challenges
•Organizational Challenges
•Keys for Success
Copyright 2003-2005, Rally Software Development Corp
Popular Agile Methods
Slide 5
Dynamic System Development
Method (Dane Faulkner)
XP (Kent Beck)
Adaptive Software
Development (Jim Highsmith)
Lean Software Development
(Mary Poppendieck)
Crystal (Alistair Cockburn)
Feature Driven Development
(Jeff DeLuca)
Scrum (Ken Schwaber)
Agile Rational Unified Process
(RUP)
Copyright 2003-2005, Rally Software Development Corp
Excerpts from the Agile Manifesto
http://agilemanifesto.org/principles.html
• Our highest priority is to satisfy the customer through early and
•
•
•
•
•
continuous delivery of valuable software.
Welcome changing requirements, even late in development.
Agile processes harness change for the customer's competitive
advantage.
Working software is the primary measure of progress.
Deliver working software frequently, from a couple of weeks to
a couple of months, with a preference to the shorter timescale.
Business people and developers must work together daily
throughout the project.
Build projects around motivated individuals. Give them the
environment and support they need, and trust them to get the
job done.
Slide 6
Copyright 2003-2005, Rally Software Development Corp
A Generalized Agile Release Process
Release
Backlog
Iteration 1
Iteration 2
• Feature 1
•Do Feature 1
•Do Feature 2
•Do Feature 3a
•Do Feature 3b
•Do Feature 4a
• Feature 2
• Feature 3
•….
• Feature 9
Slide 7
Copyright 2003-2005, Rally Software Development Corp
Iteration 3
Iteration …
Backlog
•Do Feature 4b •Do Feature 4c • Feature 8
•Do Feature 5 •Do Feature 6 • Feature 9
•Do Feature 7 • Feature 10
• ….
Agile Iteration Cadence
Requirements
Are Refined
Accept
Accept
Slide 8
Iteration N
Copyright 2003-2005, Rally Software Development Corp
Demo & Retro
Iteration N-1
Accept
Auto. Tests
Feature 3
Accept
Accept
Detailed Iteration
Planning & Design
Initial Elaboration
Requirements
With Tests
Dev Feature
Dev Feature
Priority 1
Priority 4
Auto. Tests
Auto. Tests
Feature 1
Feature 4
Dev Feature
Dev Feature
Priority 5
Priority 2
Auto. Tests
Auto. Tests
Feature 2
Feature 5
Dev Feature
Priority 3
Iteration N+1
What’s Different about Testing in Agile?
• Just-In Time Requirements Elaboration
₋
₋
No SRS-level waterfall documents to drive testing plan
Requirements and Test Cases developed in parallel or test
first strategy
• More Frequent Iterations, More Frequent Releases
₋
₋
₋
₋
Testing needs to happen Early and Often
Frequent to continuous regression testing
High need to automate nearly everything
Everyone needs to Test
₋
Iteration Vs. Release testing patterns
• Two Levels of Testing
Slide 9
Copyright 2003-2005, Rally Software Development Corp
Technical Challenges
Requirements are changing fast. How does test keep up?
Test early and often. How exactly do we move testing forward?
Need to move off manual testing and more into automation. How
does this happen?
Different kinds of testing need to happen at different times. How
do these get managed?
Copyright 2003-2005, Rally Software Development Corp
Requirements are Changing
Code
& Deliver
Code
& Deliver
UC/SR
Generate
TCs
Slide 11
Code
& Deliver
Fail TCs
Update
TCs
Copyright 2003-2005, Rally Software Development Corp
Run
TCs
Pass
& Accept
Requirements Changing is a Good Thing?
• Probably the hardest agile principle for the team to
embrace.
₋
₋
₋
Need to elaborate the feature ahead of time
There is minimal time to have the team review before the
start.
Sometimes you have to rewrite
• Bottom-line: everyone collaborates to make the
feature as useful for the customer as possible.
Slide 12
Copyright 2003-2005, Rally Software Development Corp
Requirements to Test Cases
• Use Case Scenario Tests are perfect Acceptance Tests
• Use Case A
₋
₋
Scenario 1
Scenario 2
Test Case 1
Test Case 2
• Declarative Requirements that further refine the Use
Case may be better suited to going directly to
automation
₋
₋
Slide 13
Have one Test Case be the container for all of the
automation results.
All automated tests have to pass before the Test Case
passes.
Copyright 2003-2005, Rally Software Development Corp
Need to Test early and often
• Need to test early in the Iteration – do not want
mini-waterfalls
• Need to test on check-in – Don’t break the build
• Need to test nightly – Don’t wait for a Regression
Iteration
Slide 14
Copyright 2003-2005, Rally Software Development Corp
Mike Cohn’s Testing Pyramid
GUI
Acceptance
Tests
•Small number
•Automate many
•Find the right ones
FitNesse
Unit Tests
Start
Start Stop
Stop Look
?
Slide 15
Copyright 2003-2005, Rally Software Development Corp
•Largest numbers
•Foster Test Driven Design
Break the Manual Testing Paradigm
Manual GUI
Acceptance
Tests
Automated
GUI Tests
Unit
Tests
Start
Start Stop
Stop Look
?
Slide 16
Copyright 2003-2005, Rally Software Development Corp
•Easy to Create
•Very familiar – what we always do
•Typically tedious
•How do we know coverage?
•Need Automation specialists
•Automation good for performance
•Seems like we always rewrite
•Sometimes fragile
•What is Dev testing?
•How do we know what these are?
•How do we know when they fail?
Manual Testing Conundrum
• “You can never have too many manual acceptance
tests”
₋
₋
₋
Manual tests are cute little bunnies, before you know it you
have hundreds or thousands in your regression suite
You inadvertently dig a hole you can never get out of
Whole team had to help run regression suite
• Defect count typically is high
₋
₋
₋
Slide 17
Most defects were found as manual tests were elaborated
Regression tests typically didn’t find many defects
Commonly found defects – things we didn’t think of
Copyright 2003-2005, Rally Software Development Corp
Better, But Not Perfect Testing Architecture
Manual GUI
Acceptance
Tests
Automated
GUI Tests
& FitNesse
Unit Tests
Start Stop Look
Slide 18
Copyright 2003-2005, Rally Software Development Corp
•Still too many here
•Add FitNesse
•Increase Coverage
•Increase Capability
Testing Types and Scheduling
Acceptance –
GUI?
•Minimize Manual
•During the Iteration
•Generate off of Use
Cases to get scenario
tests.
Acceptance - •Combination of Unit •Build Verification and Run
Functional
tests, FitNesse
Nightly
Load &
Performance
•Profiling and
Simulation
Automation
Regression
•Acceptance and
Exploratory
•Manual Group
Slide 19
Functional tests from
previous Iterations
Explore
•Roles and Personas
Copyright 2003-2005, Rally Software Development Corp
•Do it periodically
•Don’t wait till the end of
the Release cycle
•Run Nightly
•Before Releasing
Keys to Overcome the Technical Challenges
• Continuous Builds
• Nightly Regression testing
• Find a way to increase FitNesse testing at the
application layer http://fitnesse.org
• Make Unit Testing a priority
• From found defects – create automated tests that go
into Regression
Slide 20
Copyright 2003-2005, Rally Software Development Corp
Organizational Challenges
•Dev as Testers and Testers as Dev – how does that
happen?
•Resistance to Change – how do we get the team to
welcome and embrace changes and not feel threatened?
•Testers are an integral part of the team- do we need to
re-organize to make this happen?
Copyright 2003-2005, Rally Software Development Corp
I’m a Developer, Not a Tester
• Pretty typical to hear push back from developers that
they
₋
₋
₋
Don’t have time to do all of this testing
Number of features delivered will go down
Don’t really want to do all this testing
• Testers can help
₋
₋
Provide guidance on how to break software, art of creative
destruction
Pair testing with developers works well
• Have developers help out with manual regression
testing.
₋
Slide 22
“Can’t I write a test for this instead of running it manually?”
Copyright 2003-2005, Rally Software Development Corp
I’m a Tester Not a Developer
• Pretty typical to hear from testers
₋
₋
That they don’t feel comfortable or knowledgeable about
coding
That they maybe won’t be needed anymore
• Developers can help
₋
₋
Slide 23
Developers can create the fixtures (code running the test)
needed to make FitNesse testing work
To make it easier to auto test the code at the GUI level
Copyright 2003-2005, Rally Software Development Corp
Resisting Change
• Resistance is common
₋
₋
₋
It is easier to do what is familiar, than risk something new
Time-challenges may keep you doing the old way
Fear of failing keeps you in the status quo
• Get the whole team involved in trying to change
₋
₋
₋
Slide 24
Team needs to figure what works best
Don’t feel like you have to do everything all at once
Keep learning and adapting
Copyright 2003-2005, Rally Software Development Corp
Testers on the Team
• Your organization may have testing as a separate
group – look for ways to integrate them into the
team
₋
Creating feature or component teams comprised of all
disciplines is one way
• Co-location is a great way to hear and share
information
• Daily stand-ups with the whole team keeps the
information current
Slide 25
Copyright 2003-2005, Rally Software Development Corp
Keys to Overcome the Organizational Challenges
• Have Dev help run manual Regression tests
• Pair Dev and Test on Unit
and FitNesse Testing
• Co-location of all the team
• Daily Standups
• Do Retrospectives
Slide 26
Copyright 2003-2005, Rally Software Development Corp
Summary
• Agile Pulls Testing Forward
₋
₋
You need to change your tools and approaches to move it
forward
You might need to change the model/structure of your team
• With Agile, you will create faster Release cycles,
shorter Iterations, more satisfied customers, and
team members that enjoy what they are doing
Slide 27
Copyright 2003-2005, Rally Software Development Corp
Useful References
• Beck, Kent, Test-Driven Development By Example,
Addison Wesley, 2003
• Cohn, Mike, User Stories Applied For Agile Software
Development, Addison Wesley, 2003
• Crispin, Lisa, House, Tip, Testing Extreme
Programming, Addison Wesley, 2003
• Leffingwell, Dean, Widrig, Don, Managing Software
Requirements: Second Edition: A Use Case Approach,
Addison Wesley, 2003
Slide 28
Copyright 2003-2005, Rally Software Development Corp
Thank You
Contact Info
Jean: [email protected]
Dean: [email protected]
Questions?
Copyright 2003-2005, Rally Software Development Corp
Copyright
Slide 29 2003-2005, Rally Software Development Corp