On Evidence, Medical and Legal

Donald W. Miller, Jr., M.D.

Professor, Division of Cardiothoracic Surgery University of Washington School of Medicine Director, Cardiothoracic Surgery, VA Puget Sound HCS

Decisions Made

Legal-justice system:

Guilty --or- Liable --or--

Medicine/science:

Research hypothesis True --or- Not Guilty Not Liable Null hypothesis True Examples: TMR relieves angina TMR has no affect on angina Heparin causes thrombocytopenia Heparin does not cause thrombocytopenia Vitamin E supplements increase mortality Vitamin E supplements are safe

Standards of Proof

Kind Level of Evidence Standard ____________________________________________________________________ Regulatory Precautionary Principle Legal--Civil Legal--Civil ★ ★★ Legal--Criminal ★★★ Scientific ★★★★ More likely than not Clear and convincing Beyond a reasonable doubt Irrefutable

Evidenced-Based Medicine

“…use of current best evidence in making decisions about the care of individual patients” Sackett DL, et al.

Evidence-Based Medicine: How to Practice and Teach EBM

(Second Edition). Edinburgh: Churchill Livingstone; 2000

The EBM Evidence Pyramid

• • • • • • • RCTs on Transmyocardial Laser Revascularization Frazier OH, Tuzun E, Eichstadt H, et al. Transmyocardial laser revascularization as an adjunct to coronary artery bypass grafting: a randomized, multicenter study with 4-year follow-up.

Tex Heart Inst J

2004;31(3):231-239.

Horvath KA, Aranki SF, Cohn LH, et al. Sustained angina relief 5 years after transmyocardial laser revascularization with a CO(2) laser.

Circulation

2001;104(Suppl I):I-81-84.

Frazier OH, March RJ, Horvath KA. Transmyocardial revascularization with a carbon dioxide laser in patients with end-stage coronary artery disease.

N Engl J Med

1999;341:1021-1028.

Horvath KA, Cohn LH, Colley DA, et al. Transmyocardial laser revascularization: results of a multicenter trial with transmyocardial laser revascularization used as sole therapy for end-stage coronary artery disease.

J Thorac Cardiovasc Surg

1997;113:645-653.

Schofield PM, Sharples LD, Caine N, et al. Transmyocardial laser revascularization in patients with refractory angina: a randomised controlled trial.

Lancet

1999;353:519-524.

Allen KB, Dowling RD, Fudge TL, et al. Comparison of transmyocardial revascularization with medical therapy in patients with refractory angina.

N Engl J Med

1999;341:1021-1028.

Burkhoff D, Schmidt S, Schulman SP, et al, for the ATLANTIC Investigators. Angina Treatments-Lasers and Normal Therapies in Comparison. Transmyocardial laser revascularization compared with continued medical therapy for treatment of refractory angina pectoris: a prospective randomised trial.

Lancet

1999;354:885-890.

Transmyocardial Laser Revascularization (TMR)

Single high-energy CO the wall of the heart.

2 laser pulse fired through Epicardial surface of channel seals; 20 percent remain patent; and new capillaries 0.1 to .3 mm in diameter, lined with endothelium, extend out into the myocardium from the channels. Channels spaced 1 cm 2 apart.

Angina Relief with TMR

Ciruclation

2001;104(12 Suppl):181-4

Angina Relief in RCT Treatment Groups

Ciruclation

2001;104(12 Suppl):181-4

Change in Myocardial Perfusion in RCT Treatment Groups

N Eng J Med

1999;341(14):1021-8

Event-Free Survival in RCT Treatment Groups

N Eng J Med

1999;341(14):1028-8

Enhanced Quality of Life in RCT Treatment Groups

Ciruclation

2001;104(12 Suppl):181-4

Biologic Plausibility of TMR

CO 2 Laser • • •

Denervation

–

Mylenated fiber

–

Sympathetic fibers Endocardial channels

–

20 % stay remain patent Neoangiogenesis from channels

–

New capillaries 0.1-0.3 diameter with endothelium

–

PET studies show reperfusion

Full-thickness 1 mm channel created with a 20 joules single pulse.

Patient Selection

Indications for TMR

?

CABG to 1mm coronary arteries

?

Reoperative CABG

?

Poor LV function (EF <30%) with intra aortic balloon pump

?

Angina-equivalents (exertional dyspnea)

Epidemiological Evidence:

Randomized Trials and Cohort Studies • Address the incidence of disease and the effects of therapeutic interventions at the population level Has limited usefulness in making clinical decisions in individual patients • Two things it cannot do: Detect rare events Prove (or disprove) that x causes y in a specific individual • Bradford Hill criteria for positing a causal association • Type I and II errors

Bradford Hill’s “Criteria” for a Causal Association

1.

2.

3.

4.

5.

6.

7.

8.

9.

Consistency Strength of Association Temporal Sequence Dose-Response Specificity Coherence Biological Plausibility Analogy Experimental Evidence Hill AB. The environment and disease: association or causation?

Proc R Soc Med

1965;58:293-300.

Verdict on the Null Hypothesis (

H 0

)

Decision The Real Truth H O True H O False (not guilty) (guilty) Reject ( H O false) Type I error Correct Retain ( H O true) Correct Type II error

Cochrane Meta-Analysis

Albumin Administration in Critically Ill Patients Subjects : 30 RCTs, 1419 patients Conclusion : Risk of death 6% higher in patients given albumin Funnel plot for the 24 trials in which deaths occurred Cochrane Injuries Group Albumin Reviewers (CIGAR) . BMJ 1998;317:235-240

Cochrane Meta-Analysis

Albumin Administration in Critically Ill Patients The Fallout The

Times

(London), July 24, 1998: The review “suggests that up to 30,000 patients in Britain alone have died because they were treated human albumin solution.” The

Observer

(London), July 26, 1998 (Criticizing the UK Department of Health on dragging its feet in responding): “300 die as health chiefs dither.” Response of the principal author, Ian Roberts, to this furor: “We were amazed but totally confident we are accurate…having studied all the evidence I am sure we are right.”

Cochrane Meta-Analysis

Albumin Administration in Critically Ill Patients Its Flaws • None of the (seven) authors care for critically ill patients in the ICU • Deaths < 24 hours excluded, > 30 days included • Omitted relevant trials • Included trials that gave albumin on a daily basis for hypoalbuminemia, not hypovolemia • Combined heterogeneous trials (adults and high-risk neonates) • Conflict of interest: study funded by UK’s NHS, which could cut costs by replacing albumin with crystalloid

Cochrane Meta-Analysis

Albumin Administration in Critically Ill Patients Subsequent Developments • Cochrane authors update their review in 2000 • Conclusion remains the same: albumin not safe • Wilkes and Navickis publish systematic review on this subject in 2001 in

Ann Intern Med

; 55 RCTs, 3504 patients • Conclusion: no effect of albumin on mortality, albumin is safe • Cochrane Collaboration • Quietly removes Albumin Review from its library of meta analyses

Scales for Assessing the Quality of RCTs

Does Low Molecular Weight Heparin prevent DVT?

For your meta-analysis to answer the question: Yes or No Choose a quality scale that supports the answer you want

Evidentiary Flaws in Randomized Controlled Trials

Biases in methodology – Faulty trial protocols – Reporting outcome in terms of relative risk without giving absolute risk of all-cause deaths – Justifying intervention on surrogate outcomes (cholesterol level) when the primary outcome (freedom from MI and survival) is not improved Source of funding Kauffman JM. Bias in recent papers on diets and drugs in peer-reviewed medical journals.

J Am Phys Surg

2004;9:11-14.

EBM Guideline Approach to Clinical Problems (Two-Dimensional) Type I Complexity Welsby PD. Reductionism in medicine: some thoughts on medical education from the clinical front line.

Journal of Evaluation in Clinical Practice

1999;5:125-131.

Type I Complexity Guideline NHLBI JNC 7 Clinical Practice Guideline for Hypertension

Hypertension.

2003;42:1206

EBM Guideline Approach to Clinical Problems (Three-Dimensional) Type II Complexity 3-vessel CAD COPD Hypertension Chronic renal insufficiency CLL “Medicine is a science of uncertainty and an art of probability” William Osler Welsby PD. Reductionism in medicine: some thoughts on medical education from the clinical front line.

Journal of Evaluation in Clinical Practice

1999;5:125-131.

EBM Guideline Approach to Clinical Problems (Four-Dimensional) Type III Complexity The Art of Medicine -- Lies within the matrix of these interacting diseases and guidelines Welsby PD. Reductionism in medicine: some thoughts on medical education from the clinical front line.

Journal of Evaluation in Clinical Practice

1999;5:125-131.

The EBM Evidence Pyramid

Observational Studies: Cohort and Case Control

Statistical techniques used to construct matched sets of treatment and control subjects and reduce bias – Propensity score – Multivariate logistic regression modeling

Mangano DT and others. The Risk Associated with Aprotinin in Cardiac Surgery.

N Engl J Med

2006 (January 26);354:353-365. Conclusion: “The association between aprotinin and serious end-organ damage indicates that continued use is not prudent. In contrast, the less expensive generic medications aminocaproic acid and tranexamic acid are safe alternatives.”

Flaws in the NEJM Aprotinin Study

• Conflict of Interest • An observational study (using propensity scores) • Misapplication of earlier work to trial results • Biased selection of references in support of a predetermined position • Ideological bias of the journal’s editor and his appointed peer reviewers

Indications for Aprotinin in Cardiac Surgery at the Seattle VA Medical Center Reoperations Valve surgery Myocardial infarction < 7 days Plavix < 5 days > 75 years old

The EBM Evidence Pyramid

EBM protagonists dismiss case reports as “ anecdotal ”

Value of Case Reports

The most essential evidence in medicine is the patient’s story Eyewitness testimony (i.e., a case report) can meet the highest legal standard of proof A single case report can meet the scientific standard of irrefutability

“Double Hit” CDR Evidence that Heparin Causes Thrombocytopenia Challenge Dechallenge Rechallenge

NEJM

2001;344:1286-1292

Causal Significance of CDR Evidence Institute of Medicine “The recurrence or non recurrence of the adverse event will often have a major impact on the causality assessment.” FDA “Even a single well-documented case report can be viewed as a signal [of causation], particularly if the report describes a positive rechallenge.” Stephens’ Detection of New Adverse Drug Reactions A positive rechallenge is “probably the strongest proof of a causal relationship.”

Brides in Bath Case

George Joseph Smith R v Smith, 1915, (11 Cr App R, 229)

Double Hit: CDR Evidence that MMR Vaccine Causes Autism

Challenge Dechallenge Rechallenge Two months after MMR vaccination After MMR booster shot at age four

Standards of Proof

Kind Level of Evidence Standard ____________________________________________________________________ Regulatory Precautionary Principle Legal--Civil ★ Legal--Civil ★★ Legal--Criminal ★★★ Scientific ★★★★ More likely than not Clear and convincing Beyond a reasonable doubt Irrefutable

The Precautionary Principle

Based on the: – 1990 Bergen Declaration – 1992 Rio Conference of Sustainable Development – 1998 Wingspread Declaration Increasingly governs state regulatory policy and international environmental law Invoked to reduce CO 2 emissions, ban DDT, and bar planting of genetically engineered crops

Calamities Resulting From the Precautionary Principle Standard of Proof Malaria Starvation

Feinstein AR, Horwitz RI.* Problems in “Evidence” of “Evidence-Based Medicine.”

American Journal of Medicine

1997;103:529-535.

* Departments of Medicine and Epidemiology, Yale University School of Medicine

Miller DW, Miller CG. On Evidence, Medical and Legal.

Journal of American Physicians and Surgeons

2005;10:70-73.

Seattle VA Medical Center

Meta-Analysis of Sequential Trials

The Fallacy of Assumed Transitivity • Trial design: drug: Six numerically graded parameters for each -- Clinical Action -- Absorption -- Bioavailability -- Excretion -- Metabolism -- Side-effects • Results: – Drug A vs. B (Trial 1): A is better than B – Drug B vs. C (Trial 2): B is better than C – Drug C vs. D (Trial 3): C is better than D ⁂ Conclude, like C, d rug A will be better than drug D Is this correct? Not necessarily

Non-Transitive Sequences

Drug (Dice) A, B, C, and D with Six Graded Facets The counterintuitive result Efron B. (1990) In:

Innumeracy

1(J.A. Paulos, ed.) Penguin, London. p. 100

Case Series and Case Reports

Information about a single patient or series of patients without a control group

Case Control Studies

Retrospective studies

Cohort Studies

Prospective or retrospective (historical) incidence studies

Randomized Controlled Trials (RCTs)

Answers questions about the safety and efficacy of different therapies

2004 ACC/AHA Guidelines for CABG Recommendations for TMR

,

Class IIA: “[The] weight of evidence is in favor of [its] usefulness/efficacy.”

Level of Evidence:

A (“data derived from multiple randomized clinical trials.”) Recommendation: “TMR, either alone or in combination with CABG, is reasonable in patients with angina refractory to medical therapy who are not candidates for PCI or surgical revascularization.”

Candidates for TMR

• • Not candidates for PCI or CABG •

PCI or CABG performed in 25 % of 6 million people with chest pain each year (1.5 million)

•

12 % of patients with anginal chest pain not a candidate for PCI or CABG (60,000)

Incomplete revascularization with CABG •

15-28 % -- (60,000 to 110,000 patients)

The Evidence-Based Medicine

Gold Standard

Meta-Analyses (Systematic Reviews) Combining trials that address similar questions for greater statistical precision

?

Human-Caused (?) Global Warming