Transcript Assurance Model - Test Management Forum

Test Assurance – Ensuring
Stakeholders get What They
Want
Paul Gerrard
Gerrard Consulting
PO Box 347
Maidenhead
Berkshire
SL6 2GU UK
e: [email protected]
w: http://gerrardconsulting.com
t: 01628 639173
Slide 1
Why Test Assurance?
Most projects need to be kept ‘honest’
 Sponsors and business

-
Need someone ‘on the inside’ on their side
Need translations from technical, evasive spin
to lay persons’ language
Testing is rarely done well - project needs
support from someone capable and biasfree
 Risks arising need someone to prod the
project into action (who else will do this?)

Slide 2
Objectives of the role

Primary objective: Support

Secondary objective: Assurance

To provide advice, leadership and direction to the
project team to improve the effectiveness of testing
and in particular test reporting
To provide independent assurance to the management
board on the thoroughness, completeness and quality
of testing
Optional

Slide 3
Excluded from scope is any responsibility for the delivery of
testing - specifically excluded to avoid any conflicts of interest
Support


Information Provision
- Including specific points of expertise or experience
Decision Support
- To support the decision making process, where points of
technique, documentation, reporting or appropriate (not always
'best') practice may be suggested to the team

Advisory
- To advise on process for traceability, coverage, sign-off for
assurance purposes


Risk Awareness
- To identify risks that must be addressed by test phases
Review support
- Critical reviews of coverage, documentation, plans, results while
under development.
Slide 4
Assurance

Test Audit

Test Phase report

Independent audit of test records and process
to include checks for traceability, coverage,
consistency, accuracy etc.
Phase report to document the audit, identify
areas for correction, interpretation
Sign-Off
Slide 5
See later.
Assessment of Testing as a
whole






Can the system can be used in a realistic business
environment, processing realistic data?
Does the system meets performance, availability and
reliability objectives?
Can the system can be supported by operations staff so
that exceptions can be handled?
Can the system can be installed, started, stopped,
backed up and restored from a range of failure
scenarios?
Have all outstanding product risks of concern have been
addressed
Have each of the test phases achieved their target level
of coverage so that the project can be confident in their
assessment?
Slide 6
Process and Coverage
Assurance





Is the process sound; will it achieve the test
phase objectives?
Was the process followed, exit criteria met,
waived appropriately?
Did the appropriate, responsibility people
perform sign-offs?
What process was used to design tests; were
coverage targets set?
Were the targets met in test planning and
execution?
Slide 7
Product Risk Assurance
What risks drove the test planning,
prioritisation?
 Were risks addressed through the testing?
 Are there risks, not tested that are
outstanding?

Slide 8
Interventions - Assurance




Test Assurance Notes: Where anomalies or
uncertainties in the planning, scope or approach to
testing are detected, or where a new risk to be addressed
by testing appears, an Assurance Note is a challenge to
the project to clarify the purpose of that aspect of the
project.
Test Audit: Independent audit of test records and
process to include checks for traceability, coverage,
consistency, accuracy etc.
Test Phase report: Phase report to document the audit,
identify areas for correction, interpretation.
Sign-Off: see slides that define precisely what Assurance
Sign-off means (and doesn't mean).
Slide 10
Sign-Off by stakeholders and
other participants

Indicates that the signatories:
- Have read and understood it what they have signed
- Have approved a deliverable, outcome or process
- Made their decision in a transparent fashion
- Commit to a defined course of action – usually to ‘accept’ and/or
implement
Have reached a consensus view
Have based their decision on their (or delegated) judgements
Agree that their views have been taken into consideration



Signatories must be recognised authorities
Sign-off is usually irreversible – once signed, cannot be
undone
Not granted lightly.
Slide 12
Test Assurance Sign-Off (is
different)


The test approach, plan, specifications, scripts,
logs, incident reports have been reviewed and
test lead has been interviewed/consulted
The following have been ‘assured’:
- Objectives and acceptance/exit criteria
- The test design approach enables objectives to be
-
met
Coverage target(s) ensure that enough information will
be gathered to make the acceptance/exit decision
The exit criteria have been met or…


Slide 13
The remaining anomalies have been waived by the business
And… the right people have been involved
The test records indicate that the phase process has
been followed and are a true reflection.
The politics of assurance



Who will pay for the assurance function?
- Would your department employ a critic?
Can the Assurance Manager stay independent?
- Or will he go native with the development group?
Can the Assurance Manager speak the language of
business?
- Are you a techy at heart – unable to see beyond code, bugs,
incident reports?




If you made a recommendation, would the development
organisation attack you, or the problem?
Assurance Managers sign-off – do they take
responsibility for the release?
Do Test Assurance Management skills exist yet?
If you ‘did’ assurance, would it be a career limiting move?
Slide 14