Transcript Implementing an Organisation Wide Testing Approach

Test Management for Very Large Programs
A Survival Kit
aka
Program Test Management - A Survival Kit
Graham Thomas
Independent Software Testing Consultant
STARWEST, Anaheim, October 2008
ABSTRACT
In large organizations with multiple, simultaneous, and related projects, how do you
coordinate testing efforts for better utilization and higher quality? Some organizations
have opened Program Test Management offices to oversee the multiple streams of
testing projects and activities, each with its own test manager.
Should the Program Test Manager be an über-manager in control of everything, or is this
office more of an aggregation and reporting function? Graham Thomas examines the
spectrum of possible duties and powers of this position.
He also shares the critical factors for successful program test management, including
oversight of the testing products and deliverables; matrix management of test
managers; stakeholder, milestone, resource and dependency management; and the
softer but vital skills of influence and negotiation with very senior managers.
Relating experience gained on several large testing programs, Graham shares a practical
model—covering the key test management areas of organization, people, process,
tools, and metrics—that your organization can adapt for its needs.
© Graham Thomas 2008
2
AGENDA





Introduction
Definitions
Role
Attributes & Skill Set
Survival Kit
o
o
o
o
o
Organisation
People
Process
Tools
Metrics
 Critical Success Factors
 Lessons Learnt
 Summary
© Graham Thomas 2008
3
INTRODUCTION
 Why Program Test Management?
o Large Organisations
o Big Programs of work
o Many inter-related projects and programs
 Why Program Test Managers?
o To represent testing at the program level
o To manage multiple streams of testing projects and activities
o At a level which is outside the sphere of influence for individual
projects or testing activities
© Graham Thomas 2008
4
DEFINITION
TEST MANAGEMENT
“The direct management of testing activities”
• planning
• execution
• analysis
• validation
• design
• reporting
• preparation
© Graham Thomas 2008
5
DEFINITION
PROGRAM MANAGEMENT
 What is a Program?
“A broad effort encompassing a number of projects and/or functional
activities with a common purpose.”
Association for Project Managers - Web site Definitions
 What is Program Management?
“The management of multiple streams of development activities
(including testing), each with their own teams carrying out detailed
planning, management and execution activities.”
© Graham Thomas 2008
6
DEFINITION
PROGRAM TEST MANAGEMENT
“The management of multiple streams of testing,
each with their own test management in place
which is responsible for detailed planning,
management and control activities.”
© Graham Thomas 2008
7
PgTM ROLE CONTINUUM
 Über Test Manager
o
o
o
o
o
Micro-management
Disempowering
Decision making bottleneck
Stifle creativity
Perpetuates hero culture
 Aggregating View
o
o
o
o
o
Detached and remote
Spy in the camp
Collation and aggregation
Viewed as overhead
No ‘test’ value add
© Graham Thomas 2008
8
IDEAL PgTM ROLE RESPONSIBILITIES
 PgTMs are not responsible for managing the testing
activities. That is what we have Test Managers for!
 So, what are PgTMs responsible for if they are not managing
the testing?
 Responsibilities vary from program to program:



Testing oversight
Analogous to Program
Management
Looking for gaps
across the program


© Graham Thomas 2008
Getting the development
groups to work together
with the testers
Defining and managing the
program testing approach
9
ATTRIBUTES OF A GOOD PgTM
 Honesty, Integrity, Experience, Credibility
 Pro-active, (Dynamic - if you think rushing round helps
without causing panic)
 Politically aware and astute
 Focussed on the program and organisational goals
 Independence
© Graham Thomas 2008
10
ESSENTIAL SKILL SET
 In addition to:
o Good test management
o Good project management skills
 Stakeholder management
o Influence & negotiation skills
 Program Management
o
o
o
o
Milestone management
Resource & Dependency management
Delivery through others
Oversight and awareness
 Motivate and enthuse
© Graham Thomas 2008
11
SURVIVAL TOOLKIT
 3 Good Things
 Swiss Army Knife - That tool
that no-one knows what it is
for!
 What to avoid doing, is as
important to know, as what
to do
© Graham Thomas 2008
12
Video 1
13
THREE GOOD THINGS
A few years ago I was invited to a corporate Clay
Pigeon shooting day. I had never shot before,
but it sounded like fun so I thought that I would
give it a go.
In the morning we had a practice at the six
disciplines. I had never shot before so I was put
into the absolute beginners group.
I was hopeless. I couldn’t hit anything.
The coach was very good. He had a look at what I
was doing, and for each discipline, just told me 3
simple things, that I could relate to, that would
help me.
In the afternoon I won the individual competition –
with a high score, and two 6’s.
And that got me to thinking about 3 good things. If
only we could do that for Software Testing.
14
CIO
ORGANISATION
PO
Bus.
Dev.
Test
BEST PRACTISE
 Define clear test organisation structure with matrixed
relationships
 Clear and agreed interface with stakeholders and sponsors
 Maintain independent reporting for testing
ANTI-PRACTISE
 Testing only reports into delivery
 Testing perceived as secondary activity
© Graham Thomas 2008
15
Video 2
16
PEOPLE
BEST PRACTISE
 Ideally want people who want to be testers
 Identify your best people and give them the right
opportunities
 Create a collaborative environment to share test knowledge
and experience
ANTI-PRACTISE
 Unclear testing roles
 Lack of training and investment in testers
© Graham Thomas 2008
17
Video 3
18
PROCESS
BEST PRACTISE
 Align development and testing activities
 Consistent processes - standards, methods and approach
 Fit for purpose, flexible and effective
ANTI-PRACTISE
 Testing in silos – UT, ST, SIT, UAT, OAT etc.
 Quality police
© Graham Thomas 2008
19
TOOLS
BEST PRACTISE
 You will get as big a benefit from common process
 Match tools and processes e.g. Test Planning
 Not just automation - but Config, Release & Change Mgmt
ANTI-PRACTISE
 Developers get tools, testers don’t
 You have Quality Centre, what more can you want?
 No Silver bullets
© Graham Thomas 2008
20
METRICS
BEST PRACTISE
 Align your measures and metrics to your stakeholders and
sponsors needs
 Keep your measures and metrics consistent
 Trend measures are very powerful
ANTI-PRACTISE
 If it moves measure it!
 Inappropriate use of metrics can drive blame culture
© Graham Thomas 2008
21
CRITICAL SUCCESS FACTORS
 To ensure that you put in place a testing organisation that
meets the testing needs of the program
 To ensure that you have an efficient and effective testing
process that support and integrate with your development
approach
 To ensure that your stakeholders and sponsors clearly
understand what testing is doing for them
 To be able to articulate the program testing message
© Graham Thomas 2008
22
SUMMARY & QUESTIONS





Defined the Term Program Test Management
Examined the PTM role continuum
Looked at responsibilities, attributes and skills
Worked through a model for Program Test Management
Identified Critical Success Factors
 Any Questions
© Graham Thomas 2008
23
CONTACT DETAILS
Graham Thomas
Independent Software
Testing Consultant
 [email protected]
 +44 7973 387 853
 www.badgerscroft.com
© Graham Thomas 2008
24