Transcript diagnositc studies - the view from the clinical laboratory

Diagnosing Clostridium difficile.
Tim Planche
St. George’s Healthcare NHS Trust
St. George’s, University of London
Laboratory CDI diagnosis
change or confusion or both?
• There is no doubt that the laboratory diagnosis of CDI is in a
state of flux
• For 10-15 years the laboratory diagnosis of CDI has relied
mainly on the use of enzyme immunoassays (EIAs) to detect
the presence of the major C. difficile toxin(s)
Problems with incorrect results
• False negatives
• False positives
• Miss some cases
• Poor isolation
• Possible non/suboptimal therapy
• Unnecessary
investigation for other
conditions
• Missing real cause of
disease
• Unnecessary therapy
• Unnecessary stopping
of antibiotic therapy
• Cohorting with real
cases of C.
diff/MRSA/noro
Natural history of CDI
Detection of toxigenic C. difficile
1-2% of healthy population
10-15% of non-diarhoeal hospital admissions
15-30 % of long stay non-diarrhoeal patients
Example of C. difficle
Substance detected
Testing method
Cell-cytotoxicity assay
C. difficile toxin
Immunoassays (EIA & membrane)
Culture
C. Difficile spores
Antigen (GDH) detection
Cytotoxigenic culture
Toxogenic C. difficile spore
PCR
Reference/Gold standards
• Define True cases of a disease
• Used for comparison of any newer test
det
ected
_
positives
sensitivity =
True _ positives
det
ected
_
negatives
specificity =
True _ negatives
• I often use the false positivity rate
Also can use diagnostic likelihood ratios
Problems with reference standards
• BUT
• Problems if newer standard is better
• Problems if more than one standard
• - Need for clinical validation
Reference tests for C. difficle
Substance detected
Testing method
Cell-cytotoxicity assay
C. difficile toxin
Immunoassays (EIA & membrane)
Culture
C. Difficile spores
Antigen (GDH) detection
Cytotoxigenic culture
Toxogenic C. difficile spore
PCR
Cell-cytotoxicity assay – detects toxin directly in stool – uses cytopathic effects
on cultured cells
Cytotoxigenic culture– cultures after alcohol shock – phenotypic ID and then
detection of toxin
Example of C. difficile?
Clinical significance of reference assays
• 2 studies from 1986 – not much else
•
Gerding et al 1986
– 149 cases either
• 109 CCTA +ve
• 40 only TC +ve
– 148 non-diarrhoeal controls
• Controls no different from 40 TC +ve only (except fever)
• CCTA +ves had higher WCC, fever, longer hospital stay
– 96 cases had colonoscopy
• 35/68 CCTA +ve had PMC
• 2/27 TC only +ve had PMC
Clinical significance of reference assays
• Lashner et al 1986
• Patients with diarrhoea
– 29 untreated TC+ve/CCTA –ve cases
– 22 cases had outcome data
– 17 fully recovered
– 1 had colectomy for IBD
– 4 died - cause not recorded
– 13 cases had diagnostic studies – 6 IBD 7 normal
test performance in general
ROC Curve
Sens 50%
Spec 100%
Sens 80%
Spec 98%
Sens 100%
Spec 75%
Systematic Review of Studies - results
How do we know how accurate our
numbers are?
• Roll 60 dice – obviously will not always roll 10
sixes exactly
• Equally any study only estimates the
proportion
50%
45%
40%
p ´ (1- p)
N
35%
30%
Value
25%
Upper CI
20%
Lower CI
15%
10%
5%
0%
0
100
200
300
400
500
600
The PPV and NPV vary
State of testing in the UK
over the last decade
St. George’s number of monthly
tests
Number of C diFficile toxin tests per month
600
500
number of tests
400
300
200
100
0
date
Laboratory diagnosis of CDI
FOI survey April, 2010 (n=168/170 trusts, England)
positivity rate
number of
(prevalence)
tests
2008
6.45%
4208
2009
4.47%
4241
Of 108 labs using single EIA, 39 (36%) using a
test with estimated* PPV <50%
*Planche T et al. Lancet Infect Dis 2008.
Goldenberg SD , French GL, J Hosp Infect 2011.
Laboratory diagnosis of CDI
FOI survey April, 2010 (n=168/170 trusts, England)
• Survey carried out 1 year after DH/HPA
warning & guidance
– 70% continue to use EIA as single test
– 21% 2-stage test
– 3.6% cytotoxin test
– 0.6% (n=1) PCR
Goldenberg SD , French GL, J Hosp Infect 2011.
Calculated PPV – example of C.
difficile
100%
90%
Calculated PPV
80%
70%
60%
50%
40%
30%
20%
0%
2%
4%
6%
8%
Prevalence of C. diff
10%
12%
14%
16%
18%
20%
CEP report CEP08054
– Wilcox et al 2009
Conclusion
The poor PPVs of toxin detection kits,
especially in the context of widespread
testing, raises doubts about their
appropriateness when used as single tests for
the laboratory detection of C. difficile toxins.
http://www.pasa.nhs.uk/pasa/Doc.aspx?Path=%5bMN%5d%5bS
P%5d/NHSprocurement/CEP/CEP08054.pdf
Solutions
• 1) better test
• 2) combine tests
Other Future Possibilities
• Look at newer diagnostics – such as the stool
proteome
• Consider using markers of gut inflammation
such as calprotectin or lactoferrin
Disadvantages of NAATs for CD
toxin genes
• NAATs do not detect the presence of faecal toxin.
• inpatients typically have C. difficile toxigenic culture positive
rates of 10-20%.
• Colonization by C. difficile is protective against the
development of CDI particularly when accompanied by an
antitoxin antibody response.
Disadvantages of NAATs for CD
toxin genes (ii)
• Diarrhoea is a common symptom, especially in elderly
inpatients, many of whom will have received antibiotics or
laxatives, or have been exposed to other pathogens (notably
norovirus)
• Variable potential therefore to detect toxigenic C. difficile as
an innocent (possibly protective) bystander.
General Ways to combine tests
One after
the other
Unaltered
Cut-offs
Modified
Cut-offs
Simultaneous
Tests in series
Theoretical Screening Assay with
sensitivity of 0.92 & specificity of 0.97
Non reference standard Confirmatory
Assay of sensitivity 0.90 specificity 0.8
Prevalence 12%
PPV=11/14
(79%)
NPV=85/86
(99%)
PPV=9/1o
(90%)
NPV=89/90
(99%)
Sensitivity=sens1 x sens2= 0.83
specificity=spec1 + (spec2 x (1-spec1))=0.994
But number of repeats varies
Optimising the diagnosis of CDI - study plan
June 2010
Faecal samples taken for routine diagnosis tested by
both reference and study methods
Set-up and
preparation
Selection of preferred
algorithm(s)
Stage 1 training set
Stage 2 testing set
4-5 months
4-5 months
Assess 1-2 two stage algorithms
Optimise cut-offs and protocols compared to
reference methods
(n = 7000)
Final assessment of optimised algorithm vs
reference methods
(n = 5500)
12 months, 4 hospitals
Finished collecting clinical data 31/10/11
Analysis
Preliminary data (n~7000)
reference standard = cytotoxigenic culture
single assays -manufacturers' cut-offs
two-stage assays - manufacturers' cut-offs
GDH
EIA tox 1 NAAT
EIA tox 2 GDH
GDH
GDH
NAAT
NAAT
EIA tox 1 NAAT
EIA tox 2 EIA tox 2 EIA tox 1
93.6%
(91.295.5%)
46.9%
(42.651.2%)
93.0%
(90.695.0%)
56.6%
(52.460.8%)
55.0%
(50.759.2%)
89.8%
(87.092.2%)
55.4%
(51.159.5%)
56.4%
(52.260.6%)
46.7%
(42.450.1%)
94.5%
(93.995.0%)
99.2%
(99.099.4%)
97.1%
(96.797.5%)
98.7%
(98.499.0%)
99.6%
(99.499.7%)
98.1%
(97.798.4%)
99.5%
(99.399.6%)
99.5%
(99.399.6%)
99.6%
(99.499.7%)
reference standard = cytotoxin test
single assays -manufacturers' cut-offs
two-stage assays - manufacturers' cut-offs
GDH
EIA tox 1 NAAT
EIA tox 2 GDH
GDH
GDH
NAAT
NAAT
EIA tox 1 NAAT
EIA tox 2 EIA tox 2 EIA tox 1
92.1%
69.2%
94.9%
82.3%
67.4%
94.6%
80.4
82.0%
68.9%
(91.4(64.394.3(78.1(62.4(91.9(76.2(77.8(64.092.8%) 73.8%)
95.4%) 85.9%) 72.1%) 96.6%)
84.3%) 85.7%)
73.6%)
95.9
99.4%
96.9%
98.8%
99.7%
95.6%
99.6%
99.6%
99.7%
(93.4(99.2(94.7(98.5(99.5(95.5(99.4(99.4(99.697.6%) 99.6%)
98.4%) 99.1%) 99.8%) 96.5%)
99.7%) 99.8%)
99.8%)
Further work
• Analysis of the testing set
• What are the questions
• Need for the newer clinical validation
Thank you
• Leeds
• Kerrie Eastwood
• Mark Wilcox
• UCLH
• Nandini Shetty
• Mike Wren
• Pietro Coen
• Oxford
•
•
•
•
Derrick Crook
Sarah Oakley
Lorraine Clark
John Finney
• St. George’s
• Cassie Pope
• Irene Monahan