Transcript BC Diagnostic Accreditation Program

Diagnostic Accreditation Program
A Systematic Approach to Quality and Safety in Diagnostics
with emphasis on Medical Peer Review
Dr. Carlow, MD CCFP
Objectives

To describe why this is important

To identify what is being done throughout
healthcare

To define key principles and practices of a
systematic approach

To define issues and solutions for two diagnostic
modalities including examples from the field
Why is this important?
•
An elderly man underwent chemotherapy for GI cancer
at BCCA in the early 1990’s. In error, he received 10
times the normal dose of 5 FU and died as a
consequence.
•
A Boston health reporter in her late 30’s received a
large overdose of chemotherapy for breast cancer at
the Dana Farber Cancer Institute and died.
•
Both of these tragic events led to major systematic
changes.
Why is this important?
•
A 34 year old woman diagnosed with neuroendocrine
cancer had five surgeries to exise a cyst, remove lower
jaw and teeth, and undergo facial reconstruction. Her
slides were contaminated by cells from another patient.
She did not have cancer.
•
A patient with a positive pregnancy test underwent
pelvic ultrasound. The uterus was described as empty.
The patient received methotrexate for the treatment of
ectopic pregnancy. On review, another radiologist
subsequently diagnosed normal intrauterine pregnancy.
Why is this important?
Diagnostic Errors in the daily News
•
Pathology errors force thousands to be retested in New Brunswick,
G & M Feb 08
•
Disgraced Ontario pathologist says errors not all his fault
G & M Mar 08
•
Serious quality-control problems plague hospital labs in Canada
G & M Mar 08
•
108 women died after botched cancer tests Newfoundland says
G & M Mar 08
•
Errors found in work of another pathologist (6% error rate)
G & M May 08
Why is this important?
More than Anecdotes
•
Harvard medical practices study (Leape NEJM 91) 3.7% with disabling
injuries caused by medical treatment.
•
Institute of Medicine (U.S.) report released in 1999 – To Err is Human:
Building a Safer Health System
•
•
44,000 to 98,000 preventable deaths
Canadian adverse events study:
•
Adverse event rate of 7.5 per 100 hospital admissions
Baker et al
CMAJ 2004
Why is this important?
More than Anecdotes
•
American physicians got it right 55% of the time
McGlynn et al NEJM 2003
•
Many studies reveal significant variations in practice/low
rates of standardization
•
Swiss Cheese
Why is this important?
What about diagnostic errors?
•
•
•
What types of medical errors occur more frequently –
medication or diagnostic errors?
In two recent studies of malpractice claims – diagnostic
errors far outnumbered medication errors as a cause for
claims
Diagnostic errors are underemphasized and
understudied
Why is this important?
What about autopsy discovered errors?
•
Multi decade study
Shojania et al
JAMA, 2003
•
Median error rate 23.5% for major errors
•
Although error rates have declined over the decades,
rates are sufficiently high enough that ongoing use is
warranted
U.S. national average autopsy rate is 5%
•
What is being done?
•
•
•
•
Agency for Health Care Research & Quality (AHRQ)
Canadian Patient Safety Institute (CPSI)
Institute for Health Care Improvement (IHI)
Greater expectations from standard setting bodies:
•
•
•
•
CCHSA
JCAHO
CAP
ACR
What is being done?
•
•
•
•
•
Safer health care now
Collaboratives
5 million lives campaign
Hospital standardized mortality ratio (HSMR)
Global Trigger Tool
What is being done?
•
•
•
•
•
Quality networks
Provincial councils on Quality and Safety
Governing Board’s focusing more on quality and safety
Standards of professional bodies
Recertification
What is being done?
Organizational Initiatives
•
•
Veterans Administration/Kaiser Permanente
BC Cancer Agency
These are two examples of the systematic application of
evidence and the integration of quality and safety.
What is being done?
•
Chronic Disease Management Initiatives
•
•
•
•
Hypertension
CHF
Diabetes
Evidence based stroke program
•
Campbell River hospital
What is being done?
What has been learned about the major attributes of a
systematic approach?
•
•
•
Fragmented and isolated initiatives are quite pervasive
and ineffective
Importance of research driven evidence based care
The important role of clinical decision support systems
and tools
What is being done?
•
•
The integration of quality and safety
The importance of overall system design and clinical
governance:
•
•
•
e.g. Trauma system
e.g. micro systems
Thorough knowledge of improvement methods and
tools including:
•
•
•
•
Knowledge of processes
Quality improvement cycles
Root cause analysis
Rapid cycle improvement
What is being done?
•
Quality planning and priorities
•
•
A clearly set out agenda for quality and safety
An enabling culture
•
•
•
•
•
•
•
•
•
Leadership commitment
Professional responsibility
Inter-professional collaboration
Non-punitive reporting
Disclosure
Improvement mindset
Thinking and acting as a system
Accountability
Breakthrough thinking/aggressive targets
What is being done?
•
•
•
Surveillance/Monitoring/Measurement of processes,
outcomes and benchmarking
Quality and Safety infrastructure support
Technological support
Standards of professional bodies are now reflecting
these attributes
What is being done?
Causes of Error
•
•
•
•
•
•
•
•
Variation in practice with variable inputs
Complexity – too many steps
Inconsistent knowledge, training and language
(terminology)
Human factors in routine repetitive tasks
Deadlines/stress/excessive workload
Handoffs – transfer of information
Cultural issues – lack of openness and freedom of
expression
Unsystematic/adhoc approaches
•
Swiss cheese effect
How should we proceed in
Diagnostic Services?
•
•
Do we know enough about the various attributes of
diagnostic errors?
Are not traditional methods of medical peer review
adequate and working well?
How should we proceed in
Diagnostic Services?
Areas that need attention
•
•
•
Better definition of what constitutes an error
Greater consistency in definitions, terminology and
standardization of reporting
Better tools to assess significance of errors
How should we proceed in
Diagnostic Services?
Areas that need attention
•
•
•
More research on the extent of errors and their causes
More research on the relationship between errors and
adverse affects
Being clearer about acceptable rates of errors
How should we proceed in
Diagnostic Services?
Traditional methods of peer review
•
•
•
•
Morbidity and mortality conferences
Autopsy
Malpractice claims analysis
Error reporting systems
How should we proceed in
Diagnostic Services?
Traditional methods of peer review
•
•
•
•
•
Chart review
Observation of patient care
Clinical surveillance
Administrative data analysis
Electronic medical record review
How should we proceed in
Diagnostic Services?
Many of these have positive attributes, however:
•
•
•
•
•
•
Low case numbers
Hind sight bias
Under reporting
Absence of standardization
Some have a linkage to total organizational effort
Some not specific enough for program or department
Anatomic Pathology Errors
Anatomic Pathology Errors
•
In general anatomic diagnoses are highly accurate?
•
•
•
•
In the opinion of several, errors are not rampant
Diagnostic variation is not uncommon, but not all harmful
Depends on what are acceptable results
Medical quality affected by all phases of the system
Anatomic Pathology Errors
Life Cycle Data
•
Data indicates the importance of gathering information
over the whole testing cycle
•
Carroro et al in Clinical Chemistry 2007 report
•
61.9% pre analytic errors
•
15% analytic
•
23% post analytic
Anatomic Pathology Errors
Pre Analytic Phase
•
In this phase of the test cycle the problems more
frequently relate to:
•
•
•
Specimen I.D.
Sample quality
Availability of clinical information
Anatomic Pathology Errors
Pre Analytic Phase
•
•
In one large study 6% of cases were defective at accessioning with
defective I.D. as the 2nd largest category
Nakhleh et al CAP Q probes
APLM 1996
A survey of 341 labs revealed no clinical history in 2.4% of cases.
When corrected – change in diagnoses in 6.1% of cases
Nakhleh et al CAP Q-probes
APLM 1998
Anatomic Pathology Errors
Pre Analytic Phase
•
Patient I.D. errors in SP are the most rapidly growing
category of malpractice claims in the U.S. Most involve
switch of specimens and most involve needle biopsy of
prostate and breast
Anatomic Pathology Errors
Pre Analytic Phase
•
•
Errors in thyroid gland FNA with relatively high false
positive and false negative rates – quality of tissue
sampling by non-pathologists
FNA\histologic correlation reveals ¼ of thyroid cancer
patients are misdiagnosed as not having cancer due to:
•
•
Errors in specimen quality
Misinterpretation
Raab et al
ASLP 2006
Anatomic Pathology Errors
Analytic Phase
•
In a 4 hospital review up to 12% of tissues examined by
pathologists resulted in errors, more than 1/3 were associated with
harm (AHRQ funded)
Raab, Cancer 2005
•
Up to 15% of patients with lung mass misdiagnosed due to
pathology errors, different rates among hospitals due to “Big Dog”
effect and using different methods
Anatomic Pathology Errors
Analytical Phase
•
Average discrepancy frequency in pathology reports from74 labs on
secondary review is 6.7% with 5% of these having an affect on patient care
(1% of all cases)
•
Canadian Pathology Error Rates:
•
Retro
•
Prospect
14.1% overall rate
1.2% major
13% overall
1.7% major
Lind 1995
AJSP
Anatomic Pathology Errors
Analytic Phase
•
•
•
Most studies are single institution hence variation
However multi institutional studies reveal a discrepancy
rate of 6.7% with between 1 and 1.7% causing harm
What is an acceptable level of performance?
Anatomic Pathology Errors
Analytic Phase
•
•
•
•
Consider that a 1% error rate equates to 10,000 errors
per million
Industrial six sigma standard is 3.4 defects per million
Industry average – four sigma = 6210 defects per million
Should a six sigma standard apply to pathologists?
Anatomic Pathology Errors
Post Analytic Phase
•
Two aspects of the post analytic phase that are the
most important:


completeness of reporting; 28.4% increase in complete
reporting using computer based synoptic reports
Communication of critical results and customization of critical
values for each institution
Anatomic Pathology Errors
Errors

Medical Quality Improvement is most effective if
collection, processing, interpretation and connection to
care providers are considered as an integrated system
Anatomic Pathology Errors
Solutions/Tools

Have a plan and priorities for quality improvement and safety, consider:






Health Authority priorities
Standards of professional bodies
Guidance in literature
Performance data
Internal assessment – Process map
Identify priority projects



Mission
Culturally aligned teams
Improvement methods (PDCA) and root cause analysis, lean design 6 sigma
leap frog
Anatomic Pathology Errors
Solutions/Tools
Generic Laboratory Test Cycle Phases
Test Request
Report Interpretation
Procedural
Patient and specimen
preparation, identification,
transportation, handling,
accession
Technical & Diagnostic
Test method, lab protocols,
criteria, terminology,
accuracy, report content,
analytic timelines
Communication
Report delivery, format,
clarity,
overall timeliness,
integration of
information, satisfaction
Preanalytic
Analytic
Postanalytic
Anatomic Pathology Errors
Solutions/Tools
Error types and test-cycle phases.
Anatomic Pathology Errors
Solutions/Tools
•
Standardization
•
•
•
Terms, language, processes, tasks – work is to be done in a
certain way
Adopting standardized, structured, synoptic reporting formats
province wide
Consider computerized capture of structured data/synoptic
reports linked to databases allowing best practice
comparisons, information distribution, trend analysis and
discrepancy identification
•
e.g. mTuitive
Anatomic Pathology Errors
Solutions/Tools
•
Peer review
•
•
•
Blinded unbiased double slide review, selecting areas of high
risk for error
Amended reports are decreased with 2nd pathologist review
Nakhleh et al
APLM 1998
Prostate cancer – impact of 2nd pathologist on Gleason score:
•
25.2% change
•
14.8% change in management
Thomas et al
Brachytherapy 2007
Anatomic Pathology Errors
Solutions/Tools
•
•
Double viewing dilemma: Is error reduction frequency sufficiently high
to warrant the effort?
Consider digital pathology system (e.g. ScanScope)
•
•
•
•
•
•
Digitize slides
Desktop computer viewing
Multiple viewer conferencing
Can Link through telepathology to remote locations/single pathologists
Improved turnaround and better use of path times
Can correlate slides with CT and MRI scans
Anatomic Pathology Errors
Solutions/Tools
•
Frozen/permanent section, discordant, monitoring – sustained improvement in
performance
Raab et al
ADLM 2006
•
FNA / Histologic correlation
• Toyota production system redesign – standard terminology and immediate
interpretation
• Fewer diagnostic errors
Raab et al
ASCP 2006
Improving skill / concentrating expertise in FNA
Cytology/Histologic correlation Q tracks program showed improvement in pap. smear
performance in preanalytic sampling
Raab et al
APLM Jan 08
•

Anatomic Pathology Errors
Solutions/Tools
•
Adopt a system for measuring performance of key processes
•
•
Participation in cooperative programs access multiple
institutions/databases:
•
•
•
•
e.g. IQLM (U.S.) – 12 core indicators to evaluate lab quality
Q-Tracks
Q-Probes
System wide approach to reporting critical values
Conference, random, focused, amended report, tumour board
reviews
Anatomic Pathology Errors
Solutions/Tools
•
Improve access to clinical information
•
•
•
A culture that supports change
•
•
•
•
•
Electronic Medical Record
Better defined linkages to a large variety of clinical microsystems
(users)
Teamwork
Willingness to challenge each other
Acknowledging error in a non-punitive way
Sharing performance information
Knowledgeable well trained staff
•
•
Departmental CME
Education in QI and safety methods
Diagnostic Imaging Errors
•
•
Technological and manpower factors influence quality in diagnostic
imaging
Radiology’s Achilles heel:
•
Error and variation in the interpretation of the Roentgen Image, now the
weakest aspect of clinical imaging
Robinson, St. James UH
Leads, UK 1997
Diagnostic Imaging Errors
•
Observations of Henry Garland in 1959:
•
•
•
30% of chest radiographs that are positive for disease will be missed
Awakened the profession to the extent of errors
But have things changed?
•
•
Goddard et al BJR 2001 – little change in past 50 years
Internal error rate by same radiologist can be as high as 25% - 30%
Diagnostic Imaging Errors
•
Shively – Imaging economics 2003:
•
•
•
Many could be avoided if a simple protocol followed
Errors in stroke CT fell from 15% to 1%
Shriger, JAMA 1998
•
49% of radiologists reading CT Scans as part of a large study missed at
least 1 stroke
Diagnostic Imaging Errors
•
Turkington et al PMJ
•
•
•
14 out of 57 cases of confirmed lung cancer missed
Delays in diagnosis and treatment
Non-radiologists in emergency departments – rate of misinterpreted
radiographs is high (many studies) 20-25% for CT scans
Diagnostic Imaging Errors
•
Kruskal, Radiology 2006
•
•
On-line quality assurance reporting system – Beth Israel (Harvard) –
329 cases in 9 months
•
Communication errors 18%
•
Interpretation errors 20%
•
Missed diagnoses 30%
•
Procedural complications 16%
Renfrew – Radiology 1992
•
•
•
182 reported errors
126 perceptual
56 mishaps
Diagnostic Imaging Errors
•
Washington Post 2006 based a study by U.S. pharmacopeia
•
Medication errors that cause harm are 7 times more frequent in radiology
departments than in other hospital settings
Diagnostic Imaging Errors
Reasons for Error
•
•
•
•
Failure to consult old reports
Incomplete clinical history
Failure to suggest next appropriate procedure
Technique limitations
Diagnostic Imaging Errors
Reasons for Error
•
Knowledge problems
•
•
•
•
Errors in interpretation
Errors in perception
Failure to communicate in a timely or clinically
appropriate manner
Interpretation by non-radiologists
Diagnostic Imaging Errors
Reasons for Error
•
Quality performance bar, Lau BIIJ 2007
Access
Workload
Access
Workload
Accuracy
TAT
Accuracy TAT
•
Interlinked
•
Output pie is only so big
•
Increased expectations compromise accuracy
Diagnostic Imaging Errors
Solutions/Tools

Have a plan and priorities for quality improvement and
safety, consider:






Health Authority priorities
Standards of professional bodies
Guidance in literature
Performance data
Internal assessment – Process map
Identify priority projects



Mission
Culturally aligned teams
Improvement methods (PDCA) and root cause analysis, lean
design 6 sigma leap frog
Diagnostic Imaging Errors
Solutions/Tools
QUALITY MAP
Patient
Physician
Patient Exam
Appropriateness
Orders Test
Access
Radiology Department
Schedules
Waiting Times
Standard Protocol
Global Outcome
Reviews finding/
treats patient
Finalization Times
Exam Performed
Performance Outcomes
Patient Satisfaction
Interpretation
Structured Report
Radiologist
Protocol Selection
Finalization
Diagnostic Imaging Errors
Solutions/Tools
Radiology scorecard.
Each quality metric
from the quality map
(Figure 1) and key
safety metrics are
listed in the left-hand
column. Departmental
divisions and
operational groups
are listed in the top
row. Metrics
are provided for each
box in the scorecard,
and the box is color
coded (green, yellow,
and red) depending
upon operational
performance. Practice
problems can be
quickly identified
using this tool.
Diagnostic Imaging Errors
Solutions/Tools
•
Examples:
•
•
•
•
Knowledge of history and clinical findings – EMR
Careful selection of radiological investigation and linkage to
clinical protocols
A process to ensure comparisons with previous studies
Improvement in working conditions and available time
Diagnostic Imaging Errors
Solutions/Tools
•
Examples:
•
•
•
•
A process for review and timely follow-up on all discrepancies
on images ordered by and interpreted by non-radiologists
Development of a quality and safety performance
reporting/monitoring system
Develop targeted areas for prospective clinical surveillance to
identify areas needing improvement
Address potential for medication incidents
Diagnostic Imaging Errors
Solutions/Tools
•
Structured anatomic/region specific reports – technology assisted
•
Voice automated
•
Structured report templates
•
Sensitive to clinical requirements
•
Standardization/consistency
•
Reduces transcription errors
•
Faster TAT
•
Improves report clarity
•
Linkage to database
•
Facilitates peer review
Diagnostic Imaging Errors
Solutions/Tools

Peer review, consider:


5% review mandated by ACR. Will small sample size enable
valid individual or departmental reviews?
SMPBC
•
•
False negatives identified through linkage to cancer registry
Feedback to program leaders and individual radiologists
Diagnostic Imaging Errors
Solutions/Tools
•
Technology enabled peer review built into work routine (e.g.
RADPEER)
•
•
•
•
•
•
Software enable second review of past reports/films to be submitted
on electronic format
Can evaluate past reports – scoring system
Can be done rapidly
Central data bank for peer comparisons, departmental reviews and
individual reviews
Meaningful data
International Radiology Quality Network
Pathology/Diagnostic Imaging
Clinical Integration
•
•
•
Closer interaction between clinicians and those in
diagnostic services is associated with better outcomes
The development of clinical protocols and structured
diagnostic reports can enable requirements of each to be
addressed
Consider the clinical microsystem to strengthen
engagement with clinical care teams
Health System as an
inverted Pyramid
Clinical
Microsystem
Patients & Family Needs
CCU
Stroke
ICU
Renal
ED
Mesosystem
• departments
•Programs
• Clinical evidence base
• System support
• Clinical quality measures
Macrosystem
Senior
Leaders
Board
Blunt
end
Quality by Design
Batalden
Sharp
end
Diagnostic Services and Clinical
Microsystems
Clinical
Microsystems
Diagnostic
Services
• Integration
• Information transfer
• Coordination
• Participation in clinical requirements
Summary/Conclusions
•
Diagnostic Services
•
•
•
•
•
•
•
Consider QI and safety as part of a system
Have a plan with priorities
Address cultural barriers
Develop knowledge in QI methods/tools/root cause analysis
Develop performance measurement
Enhance clinical integration
Push for technology
Summary/Conclusions

Provincial Policy and Health Authorities



•
Enabling technology
Redesign/reengineer the system
Capital equipment planning
DAP
•
•
•
Peer review standards
Surveyor preparation/survey tools
Facilitate sharing/best practice dissemination