Transcript PROFICIENCY TESTING A BRIEF OVERVIEW

PROFICIENCY
TESTING
A BRIEF OVERVIEW
CPT Anne Sterling
LTC Paul Mann
OUTLINE
 WHAT IS PT? & WHY DO WE DO IT?
 WHO DOES IT & HOW OFTEN?
 HOW IS IT EVALUATED?
 WHAT DO RESULTS MEAN?
 WHAT HAPPENS WHEN WE FAIL?
HOW TO INVESTIGATE PT
PROFICIENCY TESTING
• Is a form of external quality control
• Is a mechanism to ensure standardized
testing across clinical laboratories and
evaluate your lab’s performance in
comparison to peer groups performance
- Uses commercially available materials &
evaluations
- CAP Surveys
WHY DO WE DO IT?
•
•
•
•
•
CLIA 88
CLIP (CCLM)
CAP
TJC
GOOD LABORATORY PRACTICES
- Develops confidence in the accuracy and
reliability of results
WHO IT APPLIES TO?
• CLIA ‘88
- ALL TESTING LOCATIONS PERFORMING
MODERATE OR HIGH COMPLEXITY TESTING
• CLIP
- ALL TESTING LOCATIONS PERFORMING
MODERATE OR HIGH COMPLEXITY TESTING
+ Waived (Minimal Complexity) labs are required
to be enrolled in a PT program if one is
commercially available.
WHO IS EXEMPT?
• CLIA ‘88
- WAIVED & PPM SITES
• CLIP
- nobody
• LOCAL POLICY
- Can be more stringent than CLIA/CLIP. This is
what TJC/CAP will hold you to.
WHAT IT APPLIES TO ?
• CLIA ‘88
- ALL REGULATED ANALYTES
• CLIP
- ALL ANALYTES
Regulated vs UnregulatedAnalytes
• Regulated Analytes: Listed in 42 CFR Part
493 Subpart I
• Unregulated analytes: those not listed in the
CFR
HOW OFTEN?
• CAP SURVEYS
- THREE TIMES A YEAR (MINIMUM OF 5
SAMPLES FOR ALL REGULATED ANALYTES)
Unregulated analytes usually have fewer challenges,
fewer samples per challenge
• FOR TESTS WITH NO COMMERCIAL
SURVEYS AVAILABLE
- Alternate method (Split Testing) will be
performed at least every 6 months
HOW IS IT EVALUATED?
e-Lab Solutions
The best way to view CAP evaluations is to
use e-Lab Solutions
 Request access from your institution’s
administrator
THREE COMPONENTS TO PT
PERFORMANCE
• the actual result
• the target value
• the evaluation interval - or acceptable
error - for that specimen.
THREE DIFFERENT TYPES OF
TARGET VALUES
• method group (“peer group”) means
• means from another group or an allresults mean
• values derived from an external source
(for example, reference laboratory
consensus or definitive/reference
methods).
FOUR GENERAL TYPES OF
EVALUATION INTERVALS
• fixed intervals (e.g., + 4 mmol/L)
• fixed percentages (e.g., + 10% of the target
value)
• a combination of these two (e.g., + 6 mg/dL
or 10 % of the target value, whichever is
greater)
• intervals based on the group standard
deviation (SD) (e.g., + 2 SD).
WHAT DO THE RESULTS
MEAN?
• WOW!!
• IF YOUR IN, YOUR GOOD
- GUESS AGAIN
• IF YOUR OUT, YOUR INCOMPETENT
- PROBABLY, BUT MAYBE NOT
CAP Evaluation
What Should You Review?
• Evaluate each analyte and specimen for
negative/positive bias, trends or shifts
• Evaluate ungraded challenges
EVALUATE RESULTS
• QUANTITATIVE
- MEAN +/- 2 SD
- EVALUATE BIAS
- EVALUATE CV
- EVALUATE TRENDS
• QUALITATIVE
- INTENDED RESPONSE
- MAJORITY OF RESULTS
PT Grading Policy
• Minimum passing score of 80%,
• 100% for ABO/Rh & Compatibility testing
• Passing grades on PT surveys with 5 challenges
will require 4 correct, 4 challenges will require 3
correct, 3 or fewer challenges will require all to be
correct.
• Challenge Failures
• Analyte Failures
Proficiency Testing Exception
Summary (PTES)
• Unsatisfactory PT performance
 Failure to attain the minimum satisfactory score for an analyte,
test, subspecialty, or specialty for a testing event.
Unsuccessful PT performance (2 of 3 testing events)
 Failure to attain the minimum satisfactory score for an analyte,
test, subspecialty, or specialty for two consecutive or two of three
consecutive testing events.
Critical PT performance (3 of 4 testing events)
 Failure to attain the minimum satisfactory score for an analyte,
test, subspecialty, or specialty for three consecutive or three of four
consecutive testing events. A laboratory must immediately cease
testing for that analyte or the discipline.
WHAT HAPPENS WHEN WE
FAIL?
UNSATISFACTORY
• BE PROACTIVE
- determine why
- prevent 2nd unsatisfactory performance
• INITIATE CORRECTIVE ACTION
• PREVENT LIMITATION OF
SERVICES
nd
2
PT Failure
UNSUCCESSFUL
• STOP TESTING
• INVESTIGATE & DETERMINE CAUSE OF
FAILURE
• EVALUATE PATIENT RESULTS
• INITIATE CORRECTIVE ACTION
• TRAIN STAFF
• EVALUATE CORRECTIVE ACTION (verify on 2
separate occasions)
• DOCUMENTATION
• REQUEST APPROVAL TO RESUME TESTING
Do a Root Cause Analysis
• A rigorous systematic approach to
answering:
- What happened & Why?
How do we do Root Cause
Analysis?
• Said simply, Root Cause Analysis is asking
why the problem occurred, and then
continuing to ask why that happened until
we reach the fundamental process element
that failed.
RCA Goals
• Find out:
– What happened?
– Why did it happen?
– What do you do to prevent it from happening
again?
– How do we know we made a difference?
Team Approach to RCA
•
•
•
•
QA
Supervisor
Bench Techs
Director
Involve Techs?
• Techs know what happens at the microprocess (bench)
level
• Will give them a sense of accomplishment/contribution
• Involve in both the investigation and solution
development
• Techs will be the ones that implement the solution
4Ms of a RCA
• Materials
– Defective Survey Material
– Wrong Survey Material
• Machine / Equipment
–
–
–
–
Instrument Malfunction
Maintenance
Calibration
Quality Control
• Methods
– SOP written , current, &
available
– SOP adequate
– SOP followed
• Man (Management)
–
–
–
–
Training
Competency
Result Entry
Review Process
RCA Methods
• How do you determine the cause?
• Examine the original test print outs to ensure accurate
entry
• Verify QC and look at Levy-Jennings graphs to see if
there was a problem (drift or shift) that was not caught
• Re-test CAP specimen to see if the same result is achieved
• Split test with other instruments (other facility)
• Verify staff competency
CLASSIFY THE PROBLEM
Unacceptable results may be classified as follows:
• Clerical error
• Methodological problem
• Technical problem
• Problem with proficiency testing materials
• Problem with evaluation of results
• No explanation after investigation
Reasons for Failure
INITIATE CORRECTIVE ACTIONS
• PROVE ACCURACY & PRECISION
OF METHOD
- RECALIBRATION
-PROCEDURAL UPDATE
- SURVEY MATERIAL HANDLING & PROCESSING
• RETRAINING
• NEW METHOD/INSTRUMENT
EVALUATION OF PATIENT RESULTS
• Review patient data from the time of the
unacceptable PT result, to determine
whether the problem could have affected
patient care
• If so, appropriate follow-up action should
be documented
EVALUATE CORRECTIVE ACTIONS
• Ensure the validity of patient results by verifying
on two consecutive occasions that the corrective
action taken has resolved the problem
• This maybe done by reanalysis and/or retesting of
frozen or additional PT material, purchase of
supplemental PT material, or blind, split-sample
testing of patient material with another certified
laboratory
DOCUMENTATION
• Document investigation, conclusions, and
corrective actions taken
• Maintain documentation for at least 2 years to
include worksheets, instrument tapes, reporting
forms, evaluation reports, participant summaries,
and documentation of follow-up, as applicable. (5
years for Immunohematology) from the date of
event.
Director Review
The laboratory director will review the
effectiveness of the corrective actions and,
if satisfied, will document his/her
recommendation whether to resume testing.
Approval Process
• Unsatisfactory: Local
• Unsuccessful: RMC
• Critical: CCLM
Critical
• 3rd of 4 PT events
• Failed to address the
problem, identify the
cause, and correct to
avoid future
problems
REQUEST APPROVAL TO
RESUME TESTING
• MEDICAL DIRECTORS APPROVAL
- 1 of 2 or 1 of 3 unsatisfactory performance
• RMC APPROVAL
- 2 of 2 or 2 of 3 unsatisfactory performance
• MEDCOM APPROVAL
- 3 of 3 or 3 of 4 unsatisfactory performance
Actions Laboratories Should Take when a PT
Result is not Graded
• Code 11 – unable to analyze
• Code 20 – No appropriate target/response cannot
be graded
• Code 21 – Specimen Problem
• Code 22 – Result is outside the method/instrument
reportable range
• Code 24 – Incorrect response due to failure to
provide a valid response code
• Code 25 – Inappropriate use of antimicrobial
Actions Laboratories Should Take when a PT
Result is not Graded cont’d
• Code 26 – Educational Challenge
• Code 27 – Lack of participant or referee consensus
• Code 28 – Response qualified with a greater than
or less than sign; unable to quantitate
• Code 30 – Scientific committee decision
• Code 33 – Specimen determined to be
unsatisfactory after contacting the CAP
• Code 40 – Results for this kit were not received
Education Challenges Code 26
Phase II
Is there evidence of evaluation and, if indicated,
corrective action in response to "unacceptable" results
on the proficiency testing reports and results of the
alternative performance assessment system?
NOTE: The evaluation must document the specific reason(s)
for the "unacceptable" result(s) and actions taken to reduce the
likelihood of recurrence. This must be done within one month
after the program receives its evaluation. In addition,
each ungraded challenge, each educational challenge, and
each episode of nonparticipation must be reviewed and
corrective action instituted as appropriate.
Actions Laboratories Should Take when a PT
Result is not Graded cont’d
• Code 41 – Results of this kit were received past
the due date
• Code 42 – No credit assigned due to absence of
response
• Code 44 – This drug is not included in our test
menu. Use of this code counts as a correct
response
CONCLUSION
• EVERY EFFORT SHOULD BE MADE
TO FIND THE CAUSE(S) OF AN
UNACCEPTABLE PT RESULT.
• ACTIONS TO IMPROVE THE
LABORATORY SYSTEM WILL
MINIMIZE THE RISK OF RECURRENCE
AND POTENTIALLY IMPROVE THE
QUALITY OF PATIENT RESULTS.
QUESTIONS?