Transcript Discovering Novel Adverse Drug Events Using Natural
Discovering Novel Adverse Drug Events Using Natural Language Processing and Mining of Electronic Health Records
Carol Friedman, PhD
Department of Biomedical Informatics Columbia University
July 21 - AIME 2009
Motivation: Severity of Problem
• Clinical trials do not test a broad population • Adverse Drug Events (ADEs) world-wide problem • * Expense from ADEs is $5.6 billion annually • * Estimated that over 2 million patients hospitalized due to ADEs • * ADEs are fourth leading cause of death
*In US alone
July 21 - AIME 2009
Motivation: Limitations of Approaches
• Manual review of case reports (Venulet J 1988) • Spontaneous reporting to designated agency (Evans JM 2001; Eland IA 1999; Wysowski DK 2005) – Serious ADEs reported less than 1-10% of time – Reporting is voluntary for physicians/patients – Recognition of ADEs is highly subjective – Difficult to determine cause of ADE – Biased by length of time on market and other factors – Cannot determine number of patients on drug or percent at risk • Drug prescribing/claims data (Hershman D 2007; Ray WA 2009)
July 21 - AIME 2009
Severity of Under Reporting
Study showed 87% of time physicians ignored patient reports of known ADEs (Golumb et al. Physicians response to patient reports of adverse drug effects. Drug Safety 2007)
July 21 - AIME 2009
Related Work
• Automated methods mainly based on spontaneous reporting databases – Most methods use (Evans SJ 2001; Szarfman A 2002) • Surrogate observed-to-expected ratios • Incidence of drug-event reporting compared to background reporting across all drugs and events • Some research aimed at improving effectiveness of SPR databases – Create ontology of higher order adverse events • MedDRA – Avoid fragmentation of signal
July 21 - AIME 2009
Related Work
• Pharmacoepidemiology databases used to confirm suspicions – General practice research database (GPRD) (Wood & Martinez 2004) – New Zealand Intensive Medicines Monitoring (IMMP) (Coulter 1998) – Medicine Monitoring Unit (MEMO) (Evans et al. 2001) • EHR databases used to find signals (Brown JS et al. 2007; Berlowitz DR et al. 2006; Wang X et al. 2009) – Mainly coded data used – Has potential for active real time surveillance – Should reduce biased reporting
July 21 - AIME 2009
Related Work
• Consortiums involving multiple EHRs – EU-ADR project ( http://www.alert-project.org/ ) – eHealth initiative ( http://www.ehealthinitiative.org/drugSafety/ ) • Related work using EHR to detect known ADEs – not aimed at discovering novel ADEs (Bates DW 2003; Hongman B 2001)
July 21 - AIME 2009
D A T A
Exploiting the Electronic Health Record
Text notes
primary care special ties
Applications
inpatient progress admit history
Labs
bun 83 inr hct … 1.3
22 …
Orders
lasix pepcid … … … … Centralized Data NLP + Integration Executable Data •
Decision support
•
Patient Safety
•
Acquire knowledge
•
Discovery
•
Guidelines
•
Surveillance
•
Patient management
•
Clinical Trial recruitment
•
Improved documentation
•
Quality assurance
July 21 - AIME 2009
The Electronic Health Record (EHR)
• Rich source of patient information • Mostly untapped • Primary use for EHR – Documenting care in multi-provider environment – Manual review by providers • More complete than coded ICD-9 codes – Symptoms – Clinical conditions not beneficial for billing • Fragmented • Heterogeneous • Noisy
July 21 - AIME 2009
Research Opportunities: NLP Issues
• Occurrence of clinical events in natural language – Drugs, diseases, symptoms – Temporal information is critical • Irregularity of reports – Section headings important but abbreviated/missing – Use of indentation, lists, run on sentences – Tables & semi-structured data in reports • Abbreviations – 2/2 meaning secondary to – co meaning cardiac output or complaining of • Mapping terms in text to an ontology/controlled vocabulary – infiltrate in chest x-ray means chest infiltrate – ontology terms more limited than language
Research Opportunities: Statistical Issues
• Find associations between drug, symptoms, and diseases – Not explicit in EHR • Large volumes of data – Statistical significance vs. clinical significance • Statistical associations – not relationships – Drug
treats
condition / Drug
causes
condition • Integrating time sequences is important – For
treats
: condition must precede drug event – For
causes
: drug event must precede condition
July 21 - AIME 2009
Research Opportunities: Statistical Issues
• Confounding (indirect associations) – Metolazone
treats
heart failure (HF) – HF is
manifested by
–
Metolazone
and shortness of breath (SOB)
SOB
indirectly related • Higher order associations – Drug interactions: Drug 1 , drug 2 , condition – Drug-contraindications: Drug, disease, condition • Rare ADEs
July 21 - AIME 2009
Other Research Opportunities: Knowledge Acquisition • Structured Knowledge bases – UMLS relations (
may_be_treated_by
) – Proprietary ones – usually unavailable • Text/Semi-Structured Knowledge (need NLP) – Spontaneous reporting databases: indications, drugs, adverse events – Literature (Medline) – Web sites (WebMD, Micromedix) – Online medical textbooks – Claims Data (Health IT payors)
July 21 - AIME 2009
Text Mining for Knowledge Acquisition
• Statistical methods: co-occurrences – Discovered associations between diseases and diets from literature ( Weeber M 2002 ) – Identified disease candidate genes ( Hristovski D 2005 ) • NLP systems – Trends in medications based on the literature and narrative clinical reports (Chen ES 2007, 2008) – Semantic relations in the literature (Hristovski D 2006)
July 21 - AIME 2009
Overview of Our NLP-EHR based Pharmacovigilance System
Narrative records Coded data EHR Medical knowledge MedLEE NLP Standardize & integrate Selecting & filtering Detect associations Eliminate confounding
July 21 - AIME 2009
ADE Signals
Natural Language Processing of EHR
Narrative records Coded data EHR Medical knowledge MedLEE NLP Standardize & integrate Selecting & filtering Detect associations Eliminate confounding
July 21 - AIME 2009
ADE Signals
Meds: Tegretol xr Zocor All: Several sz meds PMHx: sz d/o - well controlled on tegretol high chol - on zocor CAD - 60% lesion in LADM by cath MR - secondary to mitral prolapse PSHx: rib fx in 2001, shoulder fx secondary to trauma Vitals: 130/80 12 80 A/P: 54 y/o m with mult med problems, all relatively well controlled. Pt sz free, not anemic as of 2/2003. Concerned of MR and its possible long term effects.
July 21 - AIME 2009
Coded Output from NLP
med:tegretol xr sectname>> report medication item code>> UMLS:C0592163_Tegretol XR med:zocor sectname>> report medication item code>> UMLS:C0678181_Zocor .........
problem:mitral valve regurgitation sectname>> report past history item code>> UMLS:C0026266_Mitral Valve Insufficiency …….. problem:rib fracture date>> 2001 sectname>> report past history item
July 21 - AIME 2009
Coding Issues
• Not all conditions have codes –
Non-communicative
• Some conditions are combinations of codes – –
Difficulty sleeping Vascular injury
• Granularity of coding system – Many different codes for a concept
Asthma: asthma exacerbation, asthma disturbing sleep, moderate asthma, suspected asthma, … July 21 - AIME 2009
Standardizing Coded Data
HCT:20
Narrative records Coded data EHR Medical knowledge MedLEE NLP Standardize & integrate Selecting & filtering Detect associations Eliminate confounding
July 21 - AIME 2009
C0744727: low hematocrit
ADE Signals
Standardizing Coded EHR Data: Laboratory Tests and Medications • Lab values denoting normal/abnormal vary – Abnormal range may depend on age, sex, ethnicity, weight – Change in lab values and duration must be considered • Standardizing medications is complex & requires additional knowledge – Tradename to generic (
Avandia
rosaglitazone)
– Handling of combination medications •
1.5% Lidocaine with 1:200,000 Epinephrine
– Handling of dose & Route •
Diazepam 2 MG Oral Tablet July 21 - AIME 2009
Narrative records Coded data EHR Medical knowledge
Selecting and Filtering
MedLEE NLP Standardize & integrate Selecting & filtering Detect associations Eliminate confounding •
July 21 - AIME 2009
Select using UMLS classes (diseases, medications) Filter out:
• •
negations, past info, … wrong time order
ADE Signals
Selecting and Filtering
• Dependence on accuracy of semantic classification – UMLS classification errors Finding: birth history, cardiac output, divorce + Finding: cardiomegaly, fever • Temporal information difficult to obtain – An adverse drug event should only follow drug event – Processing of explicit time information is complex and vague •
Yesterday, last admission, 2/5
– Information typically occur in reports without dates
July 21 - AIME 2009
Narrative records
Detect Associations
MedLEE NLP Coded data EHR Standardize & integrate Medical knowledge Selecting & filtering Detect associations Eliminate confounding •
Obtain event frequencies
•
Co-occurrence frequencies
•
Form 2x2 tables
•
Calculate associations
ADE Signals
July 21 - AIME 2009
Detect Associations
• Correct temporal sequence is critical – Drug event should precede adverse event – Dates are not usually stated along with events – Section of reports helpful surrogate • Statistical associations correspond to different clinical relations – For pharmacovigilance: • Want drug
causes
adverse event • Confounding caused by dependencies in data
July 21 - AIME 2009
Confounding Interdependencies
Disease
Manifested by Treats
Drug
Cause_ADE
Adverse Event
July 21 - AIME 2009
Confounding Interdependencies HD ML SOB
ML: Metolazone; HD: Hypertensive Disease; SOB: Shortness of Breath
July 21 - AIME 2009
Drug Associations Network
Rx1-n Rx treatment ADE treatment ADE process Dx Sx association association
July 21 - AIME 2009
Dx1-n Sx1-n process
Reduce Confounding
Narrative records Coded data EHR Medical knowledge MedLEE NLP Standardize & integrate Selecting & filtering Detect associations Eliminate confounding
July 21 - AIME 2009
ADE Signals
Reduce Confounding
• Collect knowledge from external sources and associations – Drug -
treat
disease – Disease -
manifested by
symptom – Drug -
interacts with
drug • Use Information theory – Mutual Information (MI) – Data processing inequality MI 3 < (MI 1 , MI 3 ) MI 1 Drug Disease MI 3 MI 2 Adverse Event
July 21 - AIME 2009
• • • • • •
Initial Study: Methods
– 6 drugs chosen Ibuprofen, Morphine, Warfarin: longtime on market with known ADEs – Bupropion, Paroxetine, Rosiglitazone: ADEs discovered after 2004 – 1 drug class: ACE inhibitors 25,074 textual discharge summaries in 2004 from NYPH processed using MedLEE NLP Reference standard created using expert knowledge sources Drug-potential ADE pairs determined Recall/precision calculated Qualitative analysis performed to classify drug potential ADE pairs detected
July 21 - AIME 2009
Initial Study: Results
• Quantitative – recall (.75), precision (.30) • Qualitative analysis: potential drug-ADE pairs a. Known drug-ADEs: 30% b. Drug-indication pairs: 30% c. Remote drug-indication pair: 33% d. Unknown clinical associations: 6%
July 21 - AIME 2009
Confounding Interdependencies
Disease2 Disease
Manifested by Treats
Drug
Cause_ADE
Adverse Event
July 21 - AIME 2009
Study 2: Reduction of Confounding • Evaluation set • 14 associations related to 2 drugs from Study 1 • Reference standard • Drug-ADE associations determined and MI, DPI used to automatically classify them Direct Indirect Either Drug-ADE Relation Side effects of the drug (
Rosiglitazone-headache
) Conditions related to the disease/symptoms the drug treats (
Metolazone-shortness of breath
) Conditions in both ‘direct’ and ‘indirect’ categories (
Rosiglitazone-chest Pain
)
July 21 - AIME 2009
Results
• Precision • 0.86 when handling confounding • 0.31 when without handling confounding
July 21 - AIME 2009
Discussion: Limitations & Future Directions
• Mutual information only strategy to handle confounding – More complex MI strategy will be explored – Other statistical/knowledge based methods will be explored • Inpatient data only/sicker patient population – The same methods could be used for outpatient data as well possibly more noisy • Drug dosage, drug-drug and more complex interactions should be explored
July 21 - AIME 2009
Discussion: Limitations & Future Directions
• Small evaluation data set – More comprehensive evaluation • Limitations inherent from NLP, coding, association detection • Limitations due to fragmented/incomplete patient data
July 21 - AIME 2009
Summary
• Need for more pharmacovigilance research – Based on the EHR – Using available databases and text • Studies demonstrated promising results • Many interesting research opportunities – Natural language processing – Statistical methods – Integrating different sources of data – Gathering knowledge from different sources – Automated knowledge acquisition for evidence based medicine
July 21 - AIME 2009
Acknowledgement
• NLP Data Mining group at DBMI at Columbia – George Hripcsak – Marianthi Markatou – Herb Chase – Xiaoyan Wang – David Albers – Jung-wei Fan – Lyudmila Shagina – Noemie Elhadad • Grants – R01 LM007659 from NLM – R01 LM008635 from NLM – R01 LM06910 from NLM – 5T15LM007079 from NLM training grant
July 21 - AIME 2009
QUESTIONS
THANK YOU!
July 21 - AIME 2009