Transcript Gynecologic Cytology - ohiosocietyofcytology

Quality Indicators in Gynecologic
Cytology: Prospective and
Retrospective Review
Dr. Jennifer Brainard
Section Head, Cytopathology
Pathology and Laboratory Medicine
Cleveland Clinic
Gynecologic Cytology
• CLIA ‘88 mandated quality metrics
for gynecologic cytology laboratories
• Gynecologic cytology practice has
changed dramatically
– Liquid based Pap tests
– Automated screening
– High risk HPV testing
– Proficiency testing
– New screening guidelines
Gynecologic Cytology
• New methods of monitoring performance are
necessary
• Groups from major societies convened to
update quality indicators to reflect current
practice
– CAP Gynecologic Cytopathology Quality
Consensus Conference
– ASC Productivity and Quality Assurance in
the Era of Automated Screening Task Force
CAP Consensus Conference
• CAP was awarded a cooperative
agreement by the CDC
Goal: Identify what quality metrics are
collected, how metrics are analyzed and
what benchmarks are used to determine
variance in performance and what actions
are taken to address performance issues
CAP Consensus Conference:
Basic Structure
• Survey:
– 99 question survey sent to 1245 laboratories
participating in PT
– Complete data received from 541
laboratories (43% response rate)
• Working Groups:
– 5 working groups of CT’s and pathologists
from variety of practice types
CAP Consensus Conference:
Working Groups
1. Interpretive rates, concurrence of
CT/CP diagnoses and turnaround time
2. Prospective and retrospective review
3. Proficiency testing, general quality and
workload
4. Cytologic-histologic correlation
5. HPV rates and testing
CAP Consensus Conference:
Working Groups
• Analyzed responses to a portion of the
survey
• Extensive literature review
• Additional questions for clarification
posted to CAP web site
• Reviewed responses and drafted
consensus statements for presentation
and voting at consensus conference
CAP Consensus Conference:
CAP Web Site
• Open to entire cytology community
• Working groups posted survey results
and data thought to generate the most
interest
• New questions posted and open ended
comments solicited
• Trends and controversial topics
identified
• All responses anonymous
CAP Consensus Conference
• Held on June 4, 2011 in Rosemont, IL
• 98 attendees
• Working groups presented summary
findings and proposed consensus
statements for voting
• Consensus statements re-worked based
on attendee feedback and re-presented
• Consensus defined as >50% agreement
among attendees
CAP Consensus Conference:
Post- Conference Work
• Data analyzed and formal ranking of
degree of consensus performed
• Group leaders prepared manuscripts of
the body of their work
• Manuscripts released together in
February 2013 issue of Archives of
Pathology and Laboratory Medicine
CAP Consensus Conference:
Strengths and Limitations
• Strengths: number and diversity of
participating labs; comprehensive review
of gyn cytology QA practices; extensive
literature review
• Weakness: concerns/sensitivity about
potential impact of consensus
statements may lead to “watering down”
of statements and limited adoption
CAP Consensus Conference
Working Group 2: Prospective and
Retrospective Review
•
•
•
•
•
•
•
Jennifer Brainard MD, FCAP, Chair
Michael Henry MD, FCAP, Senior Author
George Birdsong MD, FCAP
Tarik Elsheikh MD, FCAP
Kalyani Naik MS, SCT(ASCP)
Margaret Neal MD, FCAP
David Andrew Hartley CT(ASCP)CM, CAP
Cytotechnologist Specialist
Prospective and Retrospective
Review: Definitions
• Prospective review
– Review, prior to sign-out, by a second
cytotechnologist of a subset of Pap tests
interpreted as NILM in the first
cytotechnologist review
– ≠ Prescreen
• Retrospective review
– Review of NILM (or NILM +) Pap tests that
have been signed out: an example is NILM
slides from the preceding 5 years in patients
with current HSIL+
Background: Prospective Review
• CLIA 88 – Sec 493.1274 (c)1
– Mandates at least 10% of cases screened
by a cytotechnologist be rescreened by a
qualified supervisory cytotechnologist or
pathologist prior to sign-out
– Must be randomly selected and include
cases from women at increased risk for
developing cervical cancer
– Labs must have a written policy that
defines high risk and its method of random
selection
Background: Retrospective Review
• CLIA 88 – Sec 493.1274 (c)3
– Patients with current HSIL+ must have their
negative cases, if available, from the prior 5
years reviewed
– If significant discrepancies are found, which
affect current patient care, an amended
report must be issued
• Serves as a quality monitor and
educational tool for the laboratory
Background: Results of Reviews
• CLIA 88 – Sec 493.1274 (c)4,5,6 and (d)
– Records of all rescreening results must be
documented
– There must be an annual statistical
evaluation of the number of reviews of any
NILM cases reclassified as LSIL+
– These reviews are compared as individuals
against the laboratories’ overall statistics,
and are one of the elements used to
determine workload limits
Background: CAP Checklist
• CYP.07478 : 10% Rescreen – same as
CLIA and clarifies that slides screened
by MD certified in AP and qualified as a
technical director do not need to be
rescreened
• Otherwise, essentially reiterates CLIA
regulations
Prospective and Retrospective
Review
• Attempt to evaluate the quality of Pap
test screening and interpretation
through directed re-examination of Pap
test slides
• Controversial, emotionally charged and
difficult to perform
• Results can dramatically impact labs
and individuals
Prospective and Retrospective
Review
• Can we increase the value of these
metrics for laboratories with a goal of
having a positive impact on patient
care?
Background: Working Group 2
Goal: Discuss the findings of the
CAP/CDC project pertinent to prospective
and retrospective rescreening and to
provide information regarding best
laboratory practices based on survey
results, consensus conference
discussions, published data and expert
opinion
Prospective Review
• Instituted by a number of labs before
studies regarding efficacy carried out
• Incorporated into the CLIA regulations
• Calculated false negative rates (most
<5%) are unrealistically low and inaccurate
• Error identification expected to be low:
non-enriched population
• Definition of “high risk” and % high risk
cases to include left up to labs
• Biased review
Prospective Review
• False negative rate of the rescreen
process far exceeds the false negative
rate of primary screening (21% in one
study)
• Identification of poor screening
performance may not occur quickly
enough to be of benefit
• >90% of errors remain undetected
• General consensus: Poor measure of
quality
Prospective Review: Survey Results
** >85% of labs rescreen >10% NILM cases
Prospective Review: Alternatives/
Enhancements
• Aimed at increasing thoroughness of
rescreening and limiting bias
– Rapid prescreening/ rescreening
– Enriching the random rescreen population
with known SIL
– 100% rescreening of NILM Pap tests
– Sharing cases among laboratories
– Automated quality control slide selection
• More realistic measure with more error
detection, but quite difficult to implement
Prospective Review: Practical
Enhancement
• Maximize the number of high risk cases by
including all readily identifiable high risk cases
• Multiple studies: patients with a false negative
Pap test who develop HSIL+ usually have a
history of an abnormal Pap test and most are
hrHPV positive
• Pap negative, hrHPV negative patients are at
very low risk for progression to HSIL+
• Most labs do not include enough high risk
cases in prospective review
High Risk Cases in Prospective
Review: Survey Results
• % High Risk in NILM Cases Rescreened
% Cases
% Labs
0
2.6%
1-10
51.7%
11-20
18.0%
>20
27.8%
• Majority of labs (72%) include <20% high risk
cases in their prospective rescreen
Prospective Review: Maximizing
High Risk Cases
• Use multiple measures to identify patients at
high risk
• Remove patients who no longer meet high
risk criteria
• Suggested criteria:
– Prior HSIL cytology
– Prior CIN 2+ biopsy result
– Recent/ concurrent hrHPV positivity
– No screening in past 5 years
– Include unsatisfactory Paps
Prospective Review: Maximizing
High Risk Cases
• Consensus achieved:
– Labs should make an effort to maximize
high risk cases in prospective review and
that multiple measures should be used to
identify high risk patients (89%)
– All readily identifiable high risk cases
should be included (80%)
– All readily identifiable hrHPV+ NILM Paps
should be rescreened (84.5%)
Prospective Review: Use of HPV
Test Results
• 5% to 7% of women 30 years of age and
older with NILM Pap tests are hrHPV +
and at increased risk of developing
cervical cancer
• Studies have shown that false negative
Paps are associated with a significant rate
of hrHPV positivity (80% in one study)
• Major barrier to implementation is limited
LIS functionality that does not allow timely
identification of hrHPV + patients
Prospective Review: Unsatisfactory
Pap Tests
• Patients with unsatisfactory Pap tests
are at higher risk for SIL and carcinoma
• Higher incidence of abnormalities
detected in subsequent Pap following
unsatisfactory Pap
• Unsatisfactory Paps should be included
in high risk population (87% consensus)
Removing Patients from High Risk
Category: Survey Results
• How are patients removed from the high risk
category? (n=468)
– Never removed 43.4%
– Negative Pap test diagnoses 36.5%
– Specified time interval 29.7%
• 5 years 52.1%
• 3 years 20.8%
• Majority of respondents (71%) favor removal
of patients from “high risk” category (n=45)
Removing Patients from High
Risk Category
• Proposals for removing patients:
– Consecutive negative Paps over a
specified time interval
– Consecutive negative Paps and negative
hrHPV test results over a specified time
interval
– 3 years from last criterion for high risk
status
• No consensus on criteria for removal
Prospective Review: Tracking Results
• Both the number of cases rescreened and
the number of lesions identified should be
monitored for the laboratory and for
individuals
• CLIA requires tracking LSIL +
• Some variation among labs in reporting
ASCUS/ASC-H upgrades from NILM
Tracking Upgrades: Survey Results
• Does the laboratory track the total number of
prospective rescreen cases for each individual
cytotechnologist? (Required by CLIA and CAP LAP)
(n=512)
Yes
84.8%
No
15.2%
If No, why not? (Results from online questions)
“Why would I need that information?”
“No easy way to do this”
“Small lab. Single cytotechnologist”
“Smears read off site”
Tracking Upgrades: Survey Results
• Does the laboratory track the number of lesions
identified by prospective rescreen? (Required by
CLIA and CAP LAP) (n=513)
Yes
78.6%
No
21.4%
If No, why not? (Results from online questions)
“Never occurred to us to do this. Few lesions are ever
identified”
“What do you mean by “lesions”? We track the number of
false negatives”
Tracking Upgrades: Survey Results
• Upgrade rates from NILM to ECA that
are actively monitored (n=425)
– ASCUS/ASCH
• Labs: 53.4 – 54.8%
• CT’s: 73.9 – 77.9%
– LSIL, HSIL, AGC, SCC, ADC
• Labs: 57.6 – 69.2%
• CT’s: 81.2 – 95.3%
Prospective Review: Tracking Results
• No consensus on monitoring upgrades to
ASCUS
• Strong majority agreement (98%) that
results of prospective review be shared
with individuals, including
cytotechnologists and cytopathologists at
regular intervals
Retrospective Review
• Less controversial than prospective
review
• Most laboratories find this educationally
valuable
• Focus on patients at very high risk for
false negative Pap
• Goal to improve the screening process
by learning from our mistakes
Retrospective Review
• May reveal systematic problems in the
laboratory or with individuals
• Allows for implementation of corrective
measures including education to
improve performance
• Criticism: small number of errors
detected with this measure; use in
monitoring performance of individuals
Retrospective Review
• Errors detected at significantly higher rate
than in prospective review
• While the total number of upgraded cases
identified is low, the rate of false negative
results is substantial, ranging from 10% to
94% in the literature
• Review of the literature suggests most
false negatives occur within 3 years of
current abnormal (86% within 3 years)
Retrospective Review
• Sherman and Kelly: 53% false negatives
(ASCUS threshhold); 25% rate of unsat
Paps called NILM
• Tabbara and Sidawy: 38% false negatives
(ASCUS/AGUS threshhold)
• Hatem and Wilbur: 94% false negatives
• Jones et al: CAP Q-probe 20.4% false
negatives
Retrospective Review: Common
Findings
• Mix of screening and interpretive errors
• High rates of poorly prepared smears with
compromising factors including obscuring
inflammation and blood
• A significant number of unsatisfactory Pap
tests interpreted as NILM
• Presence of few abnormal cells and
“under-interpretation” of atypical immature
squamous metaplasia
Retrospective Review: Survey Results
• How many years back are previous negative
Paps chosen for retrospective rescreen
based on a current HSIL+ Pap?
1-4 years
0.6%
5 years
96.2%
All available years 2.5%
Retrospective Review: Survey Results
• Which diagnostic categories other than
HSIL+ or AIS+ prompt a retrospective
review of NILM Paps?
None
AGC
LSIL
ASC-US
ASC-H
71.4%
15.0%
8.6%
6.8%
2.3%
Retrospective Review: Enhancement
• Use cervical biopsy results of CIN 2+ to
trigger retrospective review
• This practice would increase the
number of patients with significant
lesions included in retrospective review
• Strong consensus agreement achieved
(87%) among conference attendees
Retrospective Review: Enhancement
• Inclusion of unsatisfactory Paps adds
value to retrospective review
• Many studies stress the relationship of
unsatisfactory Paps to false negatives
• Many of the same factors that render a
Pap test unsatisfactory are also strongly
associated with false negatives
• Consensus among conference attendees
achieved (76%)
Retrospective Review: Tracking Results
• Constant monitoring of trends in false
negative rates is required
• Tracking of upgrades from NILM to LSIL+
required by CLIA
• Monitoring of upgrade rates for
pathologists very low
• Most studies report interpretive errors in
which pathologists play a role
Tracking Upgrades: Survey Results
• Monitoring of upgrade rates (375 labs)
– For NILM to HSIL+
Laboratory
Cytotechnologist
Pathologist
72.0%
82.7%
37.3%
– For ASC-US to HSIL+
Laboratory
Cytotechnologist
Pathologist
27.5%
35.7%
18.1%
Tracking Upgrades: Survey Results
• Which review diagnoses should be
monitored as part of this review?
–
–
–
–
–
–
–
ASC-US – 56%
ASC-H – 91%
LSIL – 91%
HSIL – 98%
AGC – 80%
AIS – 93%
Carcinoma/other malignancy – 96%
Retrospective Review: Tracking Results
• Monitoring this metric for individuals
should include both cytotechnologists
and pathologists and results of these
reviews should be shared regularly
• Upgraded diagnoses from NILM to
ASC-H should be monitored
• Importance of monitoring NILM to
ASCUS less clear
Prospective and Retrospective Review:
Summary
• The number of high risk cases should be
maximized
• Unsatisfactory Pap tests should be included
• All readily identifiable hrHPV positive NILM
cases should be prospectively rescreened
• Cervical biopsy results of CIN 2+ should
trigger retrospective review
• Regular feedback should be provided to
cytotechnologists and pathologists
• Upgrades from NILM to ASC-H should be
monitored