Transcript No Slide Title

CTL WORKSHOP 2008
Session 3
Concepts
•What else do you do with your proficiency testing
data?
•Mini-audits prior to performing PTPs
•Mini-audits and CSP
Group Task - (from earlier today)
• Describe to one another how proficiency tests
(internal and external) are used within your
respective organisations.
• Prepare a list of these uses.
• Discuss what kinds of things you would
further like to gain from proficiency testing
programs, and list these
• How could use of PT be facilitated by
– PT provider
– Management within your organisation
Mini-audits – what are they?
• Equipment checks to specific areas sourced
both from the relevant standards and from
CTL decision sheets (& other documents)
• Performed prior to performing the PTPs.
• Participants have the opportunity to correct
deficiencies prior to performing the tests
• Reduce the chances of gaining a noncompliance after the PTP
Mini audits during 2007
• Glow wire test/ needle flame test
• Ball pressure test
• These topics have operated several times
before and knowledge exists about the
reasons for outlying results
• The mini-audit format was optional:
– Participants could send these results first and test
after they obtained out evaluation report or,
– They could complete the tests and the audits
together.
• The programs were very efficient (turn
around time for reporting was maximum 30
days)
• Audit results were provided within a few days
of submitting results and were known before
issue of the main report.
Participants
• These programs were originally intended for
manufacturer’s labs
• CTL decided in 2007 to include these
programs on the “official” list
• The participants that had no equipment nonconformities or that corrected their equipment
issues prior to testing had a smaller
proportion of outliers than those performing
the tests and submitting the mini-audit results
together. (50% better results)
Why have an equipment audit?
• Equipment compliance is as much part of compliance
with the standard as performing the test (maybe more
so)
• It is clear that the testing staff in many laboratories do
not have sufficient knowledge of equipment
requirements
• Equipment audits have been “standard” in most PTPs
operated for CTL.
• Performing the equipment audit prior to starting the test
is logical
2008 and 2009
• The format of certain programs will include
equipment mini-audits for CBTLs as a trial.
• Participants will be encouraged to do the
equipment audit and get feedback (and act
on it if necessary) prior to performing the
test.
• Lead times for program starts may be increased
to allow this to happen.
• However, the final reports will be issued MUCH
earlier
• Follow up activities will still be required for
equipment non-compliances, particularly if not
addressed by the time the final report is issued
• CTL can feedback on this next year and we will
jointly decide how to proceed after this.
Accreditation Requirements
• ISO IEC 17025 requires proficiency testing to be
performed.
• The same clause in 17025 describes other techniques
for assuring quality of results.
– Duplicate testing by the same staff
• Possible using PT samples when sufficient samples are
provided or when non-destructive tests are done – but PTPs are
rare compared with the volume of normal tests performed.
– Testing the same sample by multiple staff
• PTP samples and other samples
– Testing standard samples (with known results) in parallel.
• Ex-PTP samples, purchased standard materials (rare in
electrical tests), some retained samples
Why do these requirements exist?
• It is normal quality assurance
• A laboratory should know that:
– Any staff member can reproduce his/her own
result when the same sample is tested on another
occasion
– Any staff member will gain an equivalent result to
all other staff members in the same laboratory
– The laboratory as a whole should obtain
equivalent results to other laboratories. (This is
PTP)
Organisation and Commitment
• Some people in this field argue that time is
money, it takes time to perform testing,
and every minute spent testing quality
assurance samples means lost business.
• People in some other fields argue that they
cannot afford NOT to do quality assurance tests
as they need proof of testing integrity and
consistency of results in order to be confident
when they sign their name on a test certificate.
– However, many people in all fields say they have
difficulty planning, organising , analysing and
reporting their full quality activities in a manner that
management can quickly grasp what is presented
(gobbilty gook)
Many organisations use their PTP
programs as
• Opportunities to meet some of the additional
17025 requirements
• PT samples not subjected to destructive tests
can be tested again and again, by different
staff and at different times..
• The expected results and allowable limits are
known from the final report
• Participants should retain samples used for
non-destructive testing for these purposes
What other uses for retained
samples?
• Staff training
• Equipment function checks
• Calibration checks
What about destructive tests?
• ??
• Retained samples can be tested again, but
these are limited in number and not
offered regularly
• IFM operates check sample programs for
other disciplines of testing
What is a check sample program?
• Series of shorter proficiency tests
designed to allow multiple topics to be
covered for multiple operators within the
same laboratory
• Planned and organised proficiency tests
that can cover additional requirements
from 17025
Drawing on others’ experience
• Laboratory A has 3 staff trained to carry out various tests
in a particular area
• When a PT program is offered in this topic, all 3 staff
have input (observe), but only one staff can actually
carry out the test.
• It may be 3 years until the topic is operated again, but
another staff gets to perform the test that time.
• Realistically, even with staff rotation on PTPs, some staff
may never really be tested, and it could take 10 years
before the same staff is tested again.
Alternatives:
• Lab can order more than one sample
when the PTP is operated
– Staff still only test themselves every 3 years
or so, and no catering for new staff in the
meanwhile
• Choose a different format of program
Typical IFM check sample program
• Operates for 6 or 12 months
• Involves 3-8 tests
• All tests are performed on a matrix every
1-2 months (6-12 samples in all)
• Designed to make quality assurance
testing routine – samples would be treated
as all others are (true reflection of the
capabilities of the lab)
• Duplicate tests and repeat tests are built
into the program
• Multiple testing staff can be catered for in
the same program (can either be
“rostered” to perform tests on various
rounds, or can share samples in a round)
• Testing can be fitted to normal day to day
tests, as are simpler in format and
paperwork
• Results are entered on-line by the due
date. Each individual operator has their
own data entry page
• A simple and rapid turn-around report is
generated for each round, and before the
next round starts
• Labs can implement and gauge
effectiveness of any corrective actions
more rapidly than with conventional
program design because they have a
repeat sample rapidly following
• At the end of the cycle, each laboratory
obtains a summary report for the cycle,
including individual breakdown of each
operator
• This style of program promotes consistency of
performance because:
• QA testing becomes routine
• Quality issues have a greater focus in the lab
(because testing staff are involved more often)
• Laboratories gain capability to trend their
performance (another 17025 requirement!)
• Individual and group staff training needs are
easier to identify
For electrical test laboratories
• Required to pass equipment audit prior to
enrolling
• Is more time efficient for labs than the
regular style of program
• IFM will be trialing glow wire test, needle
flame test and ball pressure test as a
check sample program starting in
September/October 2008
• Laboratories can enroll in this program as an alternative
to 08e29 and 09e34. (Total cost will be approximately
25% less).
• The technical requirements of the program will be
exactly the same.
• It is hoped that the check sample program will seem less
onerous, as the testing requirements will be spread over
6 months
• Organisations wishing to volunteer for this trial should
contact IFM ASAP for more details
Summarising:
• 08e29 and 09e34 will be offered in 2 formats:
• traditional program with equipment mini-audit
• Combined check sample program (equipment audit still
required prior to commencing)
• Laboratories enrolling in both programs can opt for either
format
• Feedback on the experience will be presented in 2009
CTL and workshop to decide how to proceed
• Volunteers are required!
Open discussion on any matter