MDM Review 2009

12.14.09

Jason Sanders

Outline

• • • • • • • • Measures of frequency Measures of association Study designs INTERMISSION Threats to study validity Defining test and study utility Descriptive statistics Q and A

Measures of disease frequency • Incidence (risk, cumulative incidence, incidence proportion)

I = # new cases of disease during time period # subjects followed for time period Important points

: only new cases counted in numerator; time period must be specified

Benefits

: easy to calculate and interpret

Drawback

: competing risks make I inaccurate over long time periods

Measures of disease frequency • Incidence rate (rate)

R = # new cases of disease during time period total time experienced by followed subjects Important points

: only new cases counted in numerator; person time summed for each individual

Benefits

: accounts for competing risks

Drawback

: not as easy to interpret

Measures of disease frequency • Prevalence (prevalence proportion)

P = # subjects with disease in the population # of people in the population Important points

prevalence : All people with active disease in numerator; can calculate “point” or “period”

Benefits

: illustrates disease burden

Drawback

: cross-sectional

Disease frequency example

• You have a group of 100 people. At the start of the study, 10 have active disease. Over the course of 3 years, 18 new cases develop. You accrue 200 person-years of follow-up.

• • • Prevalence at start: 10/100 = 0.1 = 10% Risk over 3 years: 18/(100-10) = 0.2 = 20% Incidence rate: 18/200 = 0.09 cases per py = 9 cases per 100 py

Measures of disease frequency

Property Incidence (risk)

Smallest value 0 Largest value 1 Dimensionality None Interpretation Probability

Incidence rate (rate)

0 Infinity 1/time Rate; inverse of waiting time

Prevalence

0 1 None Proportion

Attributable risk = Risk (E+) – Risk (E-) “Excess risk due to exposure” Attributable risk % = [Risk (E+) – Risk (E-)] / Risk (E+) “% excess risk due to exposure”

Questions on measures of disease frequency?

Measures of association

RR = risk in E+ risk in E RR = rate in E+ rate in E Exposed Unexposed # Cases N E N U Total # or Total person-time N TE or PT E N TU or PT U OR = odds of E+ in cases odds of E+ in controls Case Controls E+ A C E B D

Absolute vs. Relative measures of disease frequency •

Risk, rate, prevalence, AR are absolute measures

– Used for describing disease burden, policy, etc.

•

Relative risk, relative rate, prevalence proportion, odds ratio, AR% are relative measures

– Used to describe etiology, association of disease with exposure, etc.

RR can mean risk ratio or rate ratio

Illustration of cohort study

Time

Risk E+ Risk E RR = Risk E+ Risk E “Exposed people are at X-fold greater risk to develop disease.”

Illustration of case-control study Odds E+ OR = Odds E+ cases Odds E+ controls Odds E+

Time

“Cases have X-fold greater odds of being exposed.”

• What if we could simultaneously achieve: – Prospective measurement of disease (i.e. exposure came before disease) – Measurement of lots of confounders (for adjustment) – Controls coming from same population as cases – Less recall bias – Less selection bias – Efficient, low cost study

Nested case-control or case-cohort study

But you know: 1) E preceded D 2) Other confounders 3) Controls came from same group as cases Time You easily measure case/control status

Study design: observational studies (that count)

Prospective cohort Retrospective cohort Case-control

Study group Measures Temporal relationship E+ and E- groups Rate ratio, risk ratio, odds ratio Possible to establish Time required Long follow-up Cost Expensive When to use? E is rare and/or D is frequent among E+; investigate result of exposures Issues Selection of E-; loss to follow-up; change in E over time E+ and E- groups Cases and controls Rate ratio, risk ratio, odds ratio Odds ratio Possible to establish Difficult to establish (except nested) Can be efficient Depends E is rare and/or D is frequent among E+; save time vs. prospective cohort Selection of E-; loss to follow-up; change in E over time Less time than others Relatively inexpensive D is rare and E is frequent among D+; investigate causes of disease Selection of control group (selection bias); accurate E assessment (recall bias)

• • • • Study design: experimental study (RCT)

Requirement: equipoise Design:

–

Randomize groups to new treatment or standard

•

Benefit: Balance frequency of KNOWN and UNKNOWN confounders in groups (matching)

•

Drawback: Expensive; inefficient; doesn’t always work; can’t analyze variables that are matched on

–

Follow groups through time and assess endpoints (risk, survival, etc.) Analysis:

–

Intent-to-treat (on-treatment)

•

Benefit: Preserve randomization

•

Drawback: Subjects might not have followed treatment

–

Efficacy

•

Benefit: Analyzes subjects who followed treatment for more accurate assessment of treatment effects

•

Drawback: Breaks randomization; introduces more confounding Issues: loss to follow-up; time; cost; changing standard of care during study

Measures in RCTs Absolute risk reduction…attributable risk backwards: ARR = Risk (Placebo) – Risk (Treatment) “Risk reduction attributable to treatment” NNT = 1 / ARR “Number of patients you need to treat to prevent 1 case” Relative risk reduction…attributable risk % backwards: RRR = [Risk (Placebo) – Risk (Treatment)]/Risk (Placebo) “% Risk reduction attributable to treatment”

What if we’re interested in the time to the event, and not just the event?

Survival analysis, log rank, Cox proportional hazards

HR=0.70, 95% CI 0.52-0.95

Proportional?

Bernier et al., NEJM. 2004.

Meta-analysis: steps 1) Formulate purpose 2) Identify relevant studies 3) Establish inclusion and exclusion criteria 4) Abstract data 5) Describe effect measure ( OR, RR ) 6) Assess heterogeneity ( Forrest plot, Q, I 2 ) 7) Perform sensitivity and secondary analyses 8) Assess publication bias ( Funnel plot ) 9) Disseminate results

Can you group data: Forrest plot, Cochrane’s Q, I 2 • Forrest plot – Illustrates size and precision of effect estimates for multiple studies.

• Cochrane’s Q – A hypothesis test of whether variation in effect estimates across studies is due to chance (H 0 ) or not due to chance (H 1 ).

• I 2 – Percent of variation in effect estimates across studies that is due to heterogeneity rather than chance.

Meta-analysis: heterogeneity and dealing with it

Funnel plot: assessing publication bias • Plot Sample size (y-axis) vs. Effect (x-axis)

Unskewed distribution: bias minimal Skewed distribution: bias present

Questions on measures of association or study design?

Break time?

• Scope and Scalpel 2004: Episode 1 • Scope and Scalpel 2004: Episode 2 • Mr. Pitt Med "Blue Steel" Ad • MTV Cribs: Pitt Med • Pitt Med Office: "New PBL Group Day"

Bias, confounding, modification…a wine digression

Bias – systematic error (due to study) resulting in non-comparability; error that will remain in an infinitely large study; difficult to remove once there Will a person who enjoys apricot like Bonny Doon if it comes from a bad barrel?

Confounding – mixing of effects; results in inaccurate estimate of exposure outcome association; is never “controlled,” rather “adjusted for” Peach, Grape Apricot wine Likability Effect modification – difference of effect depending on the presence or absence of a second factor; interesting phenomenon to investigate; detected with stratification or interaction term in model Apricot wine Room Iced Likability Likability If differ by >10%, modification present

Examining your new test: Sn, Sp, PPV, NPV

Positive Negative Gold standard Positive Negative A B C D Prevalence alters PPV most Sn = A / (A + C) “Of those with disease, how many did you identify?” Sp = D / (B + D) “Of those without disease, how many did you identify?” PPV = A / (A + B) “Of those you said had disease, how many truly did?” NPV = D / (C + D) “Of those you said did not have disease, how many truly did not?”

Examining your new test: Likelihood ratios

Positive Gold standard Positive Negative A B Negative C D LR is a ratio of two proportions: proportion of those with a particular result among the diseased compared to the proportion with that result among the non-diseased LR(+) = A / (A + C) = Sn B / (B + D) 1-Sp LR(-) = C / (A + C) = 1-Sn D / (B + D) Sp “The likelihood of a test outcome (+ or -) if you have the disease is X-fold higher than if don’t have the disease.”

Examining various tests: ROC curves

Sn

Picking the best test depends on: 1) Optimizing Sn and Sp (highest AUC) 2) Real world conditions HIV: We want highest Sp and sacrifice Sn

1-Sp

• • • • Parametric biostats: T-test, ANOVA, χ 2 , Pearson

T-test: if you want to test the difference in means of 2 groups (continuous)

–

Assumptions and how to verify them:

•

Independence (are subjects related?)

•

Random sampling (assumed)

• •

Normal distribution of variable (histograms, formal test) Equal variance of variable in each group (F-test) ANOVA: if you want to test the difference in means between ≥2 groups (continuous)

–

Assumptions and how to verify them:

•

Same as T-test χ 2 : if you want to test the difference in frequencies among ≥2 groups (categorical)

–

Assumptions and how to verify them:

•

Cell sizes in table (>5, formal test



Use Fisher’s exact test if unfulfilled) Pearson r: if you want to test the degree of linear relationship between two continuous variables; does not imply causal association or a mathematical association other than linear

–

Assumptions and how to verify them:

•

Linear relationship (look at it)

• •

Independence, random sampling (as above) At least 1 variable must be normally distributed

Nonparametrics: Rank sum, Kruskal-Wallis, Spearman • • • • •

Mann-Whitney rank sum: if you want to test the difference in means of 2 groups (continuous)

–

Assumptions and how to verify them:

•

Independence (are subjects related?)

•

Random sampling (assumed)

•

Variable follows same distribution in both groups, whatever the distribution may be Kruskal-Wallis: if you want to test the difference in means between ≥2 groups (continuous)

–

Assumptions and how to verify them:

•

Same as rank sum Spearman r: if you want to test the degree of linear relationship between two continuous variables; does not imply causal association or a mathematical association other than linear

–

Assumptions and how to verify them:

•

Linear relationship (look at it)

•

Independence, random sampling (as above) Nonparametrics do have assumptions!

Great alternative if assumptions met, but can lack power and don’t give a good idea of how the data are different (rely on significance)

P-values and confidence intervals •

P-value

–

“Is the data consistent with the null hypothesis? If not, then there is a “statistically significant” difference.”

–

Depends upon sample size and magnitude of effect; doesn’t illustrate real values



A POOR MEASURE

•

Confidence interval

–

“What is the range of possible values for the difference observed?”

–

Provides information on precision of data and possible range of values



A BETTER MEASURE

Extra slides

Odds and probability

Odds = Chance of something = p Chance of not something 1-p

If p=50%, odds are 0.5/(1-0.5) = 0.5 / 0.5 = 1. Hence, 50% chance means that it is equally likely that “something” and “not something” will happen.

If p=33%, odds are 0.33/(1-0.33) = 0.33/0.67 = ½. Hence, 33% chance means that it is ½ as likely that “something” will happen compared to “not something” happening. Alternatively, it is twice as likely that “not something” will happen compared to “something” happening.

Standard Deviation vs. Standard Error • The SD is a measure of the variability in the measurements you took. Variability can come from biologic variability, measurement variability, or both. If you believe the tool you use to measure has zero error, then the variability is solely due to biologic variability. If you want to emphasize the biologic variability (i.e. scatter) in your sample, then the SD is the appropriate statistic.

• The SEM is a measure of how well you approximated the true population mean with your sample. Again, error can come from biologic variability, measurement variability, or both. If you assume there is no biologic variability, then the only error comes from the tool you use to measure. With larger sampling sizes from the population, the measurement error becomes less and less because you are more likely to determine the true population mean with sample sizes that become closer to the true population. If you want to emphasize how precisely you determined the true population mean, then the SEM is the appropriate statistic • The SEM is used to calculate Confidence Intervals.

Extra study design: observational studies Study group Measures Temporal relationship Time required Issues

Case report/series Cross-sectional

Patient(s) Defined pop; measure E and O simultaneously in each person None Cost When to use? Interesting/new case(s) Not population based Prevalence Unable to establish Little Intermediate No temporality; prevalence bias

Ecological

Select groups (country, county); measure E and O in population Correlation coefficient Unable to establish Very little Inexpensive E and O are common Aggregate data available; establish hypotheses Ecological fallacy; difficult to adjust for confounding