Information Mastery: A Practical Approach to Evidence-Based Care

Course Directors: Allen Shaughnessy, PharmD, MMedEd David Slawson, MD Tufts Health Care Institute Tufts University School of Medicine November 10-12, 2011 Boston, Massachusetts

Information Mastery: A Practical Approach to Evidence-Based Care

Don’t Panic: Basic Statistics You Can Understand

Don’t Panic

Basic Statistics You Can Understand

3

Users

of statistics don’t have to be statisticians

 I am a user of statistics, not a statistician (I have friends, however, who are statisticians)  You don’t have to know a lot about statistics to effectively use statistics  Don’t focus on whether the statistics are

right

• Learn to figure out

what

the statistics are trying to tell you 4

The Shrine of Statistics: The Sacred P-Value P< .05

5

P Value

 "

P

robability" level  The likelihood that the difference observed between two interventions could have arisen by

chance

 Arbitrarily set at 5% risk (P = 0.05) 6

P value example

7

This ain’t physics

“

Unfortunately, the publication standards of medical journals are quite low compared to other science fields such as physics . . . Presumably necessary to assure that possible helpful therapies are not kept from needy patients for far too long.

No respectable physics journal would publish a result with a p value of a few percents. In fact, the publication standard in physics is typically a p value of 0.0001 . . .”

Victor Stenger, PhD Professor Emeritus of Physics, U. of Hawaii Discovered that the neutrino has mass

8

Number Needed to Treat

 The number of patients that need to be treated for one additional patient to receive benefit  The number of patients that need to be treated to prevent one additional outcome  Takes into account the relative risks as well as the absolute risk of no treatment  NNT = 100 % in treatment group - % in control group 9

NNTs for Prevention

Condition

Heart failure (NHYA I or II) Hypertension in patients with type 2 diabetes Hyperlipidemia – primary prevention Hyperlipidemia – secondary prevention DVT

Treatment

Enalapril vs. placebo HTN treatment Various vs. placebo Warfarin (target INR = 1.5 2.0) vs. placebo for 1 yr

Outcome

1 death at one year 1 diabetes-related death over 10 years Simvastatin vs. no treatment 1 death over 1 years 1 MI or CVA over 5 years 1 VTE over 1 year

NNT

100 15 163 16 22 10

NNTs for Treatment

Condition

H. Pylori Peptic Ulcer Migraine Bacterial conjunctivitis Herpes Zoster

Treatment Outcome

Triple therapy H. Pylori tx vs. H 2 for 6-8 wks tx 1 dose sumatriptan vs. placebo Eradication Ulcer cure at 1 year Headache relief at 2 hours Topical abx vs. placebo Acyclovir vs. placebo For early clinical remission (3-5 days) Prevent PHN at 6 months

NNT

1.1

1.8

2.6

5 Not effective 11

Relative Risk

     The risk of harm with one treatment as compared with another The risk of benefit with one treatment as compared with another If RR = 1, then there is no difference between the two treatments Depends only on the relative difference between the two treatments Does not take into account the risk of no treatment — the "absolute risk" 12

Randomised trial of cholesterol lowering in 4444 patients with coronary heart disease: the Scandinavian Simvastatin Survival Study (4S). Scandinavian Simvastatin Survival Study Group

Summary: 4444 patients with high cholesterol and CHD were given either simvastatin or placebo for a median of 5.4 years.

Results: 256 (12%) in the placebo group died 182 (8%) in the simvastatin group died Relative risk = 0.70

Risk reduction = 30% But, what is the NNT = ?

13

Meta-analysis: Statins to prevent stroke and MI

 Meta-analysis of 29 studies, 10,000+ patients  Statins vs. control (usual care) • Stroke risk: 0.82 (18% decrease) • MI risk: 0.74 (26% decrease)  18% from what? 26% from what?

Briel M. Effects of statins on stroke prevention in patients with and without coronary heart disease: A meta-analysis of randomized controlled trials. Am J Med 2004;117

14

Meta-analysis: Statins to prevent stroke and MI

  Stroke Risk • Low risk: 0.2% • High risk (CHD): 0.9%/year MI risk • Low risk: 0.9% • High risk: 3.7%/year NNT t o prevent 1 stroke/1 yr: • Low risk: 2,778 patients • High risk: 617 patients NNT to prevent 1 MI/1 year: • Low risk: 427 patients • High risk: 104 patients

Briel M. Effects of statins on stroke prevention in patients with and without coronary heart disease: A meta-analysis of randomized controlled trials. Am J Med 2004;117:

15

Reformulation of Clarithromycin

Improved GI Tolerance with Biaxin XL Improved GI Tolerance with Biaxin XL 3%

66% decrease

Relative risk = 0.34

1%

Biaxin Biaxin Incidence of GI side effects Incidence of GI side effects Biaxin XL Biaxin XL 16

Confidence Interval

 "A statistic of a statistic"  Statistics are estimates • Confidence intervals tells us the upper and lower possibilities of our statistical estimates 17

Example: Results from the UKPDS

18

95% C.I

.

Since the 95% CI crosses 1.0, the difference is not significant

Risk could be this low

0.80

0.94

1.0

1.1

Risk could be this high 19

What’s new: the Credible Interval

 

Credible interval (CrI) is the confidence interval for Bayesian statistics Bayesian approach



predicts future, not analysis of past

A Word on Combined Outcomes: Truth or Fishing Expedition?



Beware of switched outcomes

• secondary analysis or outcomes vs

a priori

outcome • Pioglitazone trial PROactive 10 (

a priori

primary primary composite outcome not significant so secondary one reported)

Am Heart J 2008;155:712-7.

A Word on Combined Outcomes: Truth or Fishing Expedition?



If the composite outcome is statistically different

• make sure at least 1 individual clinically relevant outcome is significantly different. • CV research: 43% of composite outcomes the most significant outcome was not clinically relevant.

Lincoff, A. M. et al. JAMA 2007;298:1180-1188

Table 3. Cardiovascular Event Rates for Combined Trials Stratified by Study Type.

Lincoff, A. M. et al. JAMA 2007;298:1180-1188

Copyright restrictions may apply.



A Word on Combined Outcomes: Truth or Fishing Expedition?

Beware of “dominant” DOE outcome

• United Kingdom Prospective Diabetes Study • 21 outcomes • “Any diabetes-related outcome” decreased 12% (0.79-0.99) • Only significantly decreased outcome: photocoagulation • • Other 20 outcomes not affected Results not confirmed in ACCORD, VDAT, ADVANCE

Lancet .

1998 Sep 12;352(9131):837-53

.

Don’t Be Afraid of Statistics

 Statistical significance is a requirement for determining clinical significance, but is not enough to

signify

a clinical difference  The P value tells us the risk that the difference between two treatments was due to chance  Relative risk tells part, but not all of the story; NNT does it better  Confidence intervals help us to understand how close our estimate is to the "truth" 25