Issues of Interpretation in Epidemiologic Studies

Developed through the APTR Initiative to Enhance Prevention and Population Health Education in collaboration with the Brody School of Medicine at East Carolina University with funding from the Centers for Disease Control and Prevention

APTR wishes to acknowledge the following individual that developed this module: 

Jeffrey Bethel, PhD

Department of Public Health Brody School of Medicine at East Carolina University This education module is made possible through the Centers for Disease Control and Prevention (CDC) and the Association for Prevention Teaching and Research (APTR) Cooperative Agreement, No. 5U50CD300860. The module represents the opinions of the author(s) and does not necessarily represent the views of the Centers for Disease Control and Prevention or the Association for Prevention Teaching and Research.

1.

2.

3.

4.

Describe the key features of selection and information bias Identify the ways selection and information bias can be minimized or avoided Implement the methods for assessing and controlling confounding Identify uses of the Surgeon General’s Guidelines for establishing causality

Smith, AH. The Epidemiologic Research Sequence. 1984

Exposure or Characteristic Disease or Outcome Observed Association

Exposure or Characteristic Disease or Outcome Observed Association Is it: biased, confounded, or causal?

Any systematic error in the design, conduct, or analysis of a study that results in a mistaken estimate of an exposure’s effect on the risk of disease

 Can create spurious association when there really is not one (bias away from the null)  Can mask an association when there really is one (bias towards the null)  Primarily introduced by the investigators or study participants  Selection and information bias

 Results from procedures used to study participants that lead to a result different from what would have been obtained from the entire population targeted for study  Systematic error made in selecting one or more of the study groups to be compared

 Control selection bias  Self-selection bias  Differential referral, surveillance, or diagnosis bias  Loss to follow-up

Question: Do Pap smears prevent cervical cancer? Cases diagnosed at a city hospital. Controls randomly sampled from household in same city by canvassing the neighborhood on foot. Here is the observed relationship: Cases Controls Pap Smear No Pap Smear 100 150 100 150 Total 250 250

OR = (100 x 150) / (100 x 50) = 1.0 There is no association between Pap smears and risk of cervical cancer (40% of cases and 40% of controls had Pap smears)

Recall:

Cases from the hospital and controls come from the neighborhood around the hospital

Now for the bias:

Only controls who were at home during recruitment for the study were actually included in the study. Women at home were less likely to work and less likely to have regular checkups and Pap smears. Therefore, being included in the study as a control is not independent of the exposure.

Question: Do Pap smears prevent cervical cancer? Cases diagnosed at a city hospital. Controls randomly sampled from household in same city by canvassing the neighborhood on foot. Here is the true relationship: Cases Controls Pap Smear No Pap Smear 100 150 150 100 Total 250 250

OR = (100)(100) / (150)(150) = .44 56% reduced risk of cervical cancer among women who had Pap smears as compared to women who did not (40% of cases had Pap smears versus 60% of controls)

 Refusal or nonresponse by participants that is related to both exposure and disease  e.g. if exposed cases are more/less likely to participate than participants in other categories  Best way to avoid is to obtain high participation rates

 Example related to exposure  CC study: venous thromboembolism (VT) and oral contraceptive (OC) use  Cases: 20-44 yo, hospitalized for VT  Controls: 20-44 yo, hospitalized for acute illness or elective surgery at same hospitals  Result: OR = 10.2

 Authors acknowledged high OR might be due to “bias in the criteria for hospital admission”  Previous studies linked VT to OC  Health care provider more likely to hospitalize women with VT symptoms who were taking OC than symptomatic women who were not taking OC

 Compared HIV incidence rates among IVDU in NYC from 1992-97 through 10 incidence studies to previous years  HIV incidence rates (IR) range from 0 to 2.96 per 100 person-years (py)  Well below IR in NYC from late 70s and early 80s (13 per 100 py) to mid 80s and early 90s (4.4 per 100 py)  Resulted in funding cuts to drug treatment and prevention programs

 Was decline real?

 Follow-up rates in 10 cohorts ranged from 36% to 95%  Only 2 reported >80%  Sample size ranged from 96 to 1,671

Solution

: minimize loss to follow-up

 No – need to avoid it when you design and conduct the study  For example  Use the same criteria for selecting cases and controls  Obtaining all relevant participant records  Obtaining high participation rates  Taking in account diagnostic and referral patterns of disease

 Arises from a systematic difference in the way that exposure or outcome is measured between groups  Can bias towards or away from the null  Occurs in prospective and retrospective studies  Includes recall bias and interviewer bias

 Case-control study of birth defects  Controls: healthy infants  Cases: malformed infants  Exposure data collected at postpartum interviews with infants’ mothers  Controls or cases may have underreported exposure, depending on nature of exposure

 Select diseased control group  Design structured questionnaire  Use self-administered questionnaire  Use biological measurements  Mask participants to study hypotheses

 Systematic difference in soliciting, recording, interpreting information  Case-control study: exposure information is sought when outcome is known  Cohort study: outcome information is sought when exposure is known 

Solutions:

Mask interviewers, use standardized questionnaires or standardized methods of outcome (or exposure) ascertainment

Exposure or Characteristic Disease or Outcome Observed Association Is it: biased, confounded, or causal?

X A B

 A mixing of effects – association between exposure and disease is distorted by the effect of a third variable that is associated with the disease  Alternate explanation for observed association between an exposure and disease

X A B

In order for a factor (X) to be a confounder, all of the following must be TRUE:  Factor X is associated with Disease B (risk factor or preventive factor)   Factor X is associated with Factor A (exposure) Factor X is not a result of Factor A (not on causal pathway)

Smoking Coffee Consumption

   Smoking is associated with pancreatic cancer Smoking is associated with coffee drinking Smoking is not a result of coffee drinking

Pancreatic Cancer

 Pulls the observed association away from the true association  Positive confounding  Exaggerates the true association  True relative risk (RR) = 1.0 and confounded RR = 2.0

 Negative confounding  Hides the true association  True RR = 2.0 and confounded RR = 1.0

Obese Not Obese TOTAL Dementia 400 100 500 No Dementia 600 900 1,500 TOTAL 1,000 1,000 2,000 400/1,000 100/1,000

Age 80-99 Years 45-79 Years TOTAL Yes 400 100 Dementia No 600 900 1,000 400/1,000 100/1,000 1,000 TOTAL 1,000 1,000 2,000

Obese Age 80-99 Years 45-79 Years Yes 900 100 No 100 900 TOTAL 1,000 900/1000 Relative Risk = = 9.0 100 x 100 1,000 TOTAL 1,000 1,000 2,000

Age Obesity Dementia

 Age is associated with dementia (RR=4.0)  Age is associated with obesity (OR=81.0)  Age is not a result of obesity (not from data)



Design phase

 Group or individual matching on the suspected confounding factor  e.g. matching on age in case-control study 

Analysis phase

 Stratification  Standardization  Adjustment (multivariate analysis)



Not an error in the study



Valid finding of relationships between factors and disease



Failure to take into account confounding IS an error and can bias the results!

Exposure or Characteristic Disease or Outcome

Observed Association Is it: biased, confounded, or causal?

 Determine whether a statistical association exists between characteristics or exposures and disease   Study of group characteristics (ecologic studies) Study of individual characteristics (case-control and cohort studies)  Derive inferences regarding possible causal relationship using pre-determined criteria or guidelines

 Association is not equal to causation  Consider the following statement: If the rooster crows at the break of dawn, then the rooster caused the sun to rise  Causation implies there is a true mechanism from exposure to disease

1.

The organism is always found with the disease (regular) 2.

The organism is not found with any other disease (exclusive) 3.

The organism, isolated from one who has the disease, and cultured through several generations, produces the disease (in experimental animals)

Koch added that “Even when an infectious disease cannot be transmitted to animals, the ‘regular’ and ‘exclusive’ presence of the organism [postulates 1 and 2] proves a causal relationship Unknown at the time of Koch-Henle (1840-1880)  Carrier state  Asymptomatic infection  Multifactorial causation  Biologic spectrum of disease



Let’s say you have determined:

 There is a real association  You believe it to be causal (ruled out confounding) 

NOW have you proven CAUSALITY?

6.

7.

8.

9.

1.

2.

3.

4.

5.

Temporal relationship Strength of the association Dose-response relationship Replication of the findings Biologic plausibility Consideration of alternate explanations Cessation of exposure Consistency with other knowledge Specificity of the association

 Exposure to factor must have occurred before disease developed  Easiest to establish in a prospective cohort study  Length of interval between exposure and disease very important  e.g. asbestos and lung cancer  Lung cancer followed exposure by 3 or 20 years?

 The stronger the association, the more likely the exposure is causing the disease  Example: RR of lung cancer in smokers vs. non smokers = 9; RR of lung cancer in heavy vs. non smokers = 20

Which odds ratio (OR) would you be more likely to infer causation from?

OR#1: OR = 1.4

95% CI = (1.2 - 1.7) OR#2:OR = 9.8

OR#3:OR = 6.6

95% CI = (1.8 - 12.3) 95% CI = (5.9 - 8.1)

 Persons who have increasingly higher exposure levels have increasingly higher risks of disease  Example: Lung cancer death rates rise with the number of cigarettes smoked

 The association is observed repeatedly in different persons, places, times, and circumstances  Replicating the association in different samples, with different study designs, and different investigators gives evidence of causation  Example: Smoking has been associated with lung cancer in dozens of retrospective and prospective studies

 Biological or social model exists to explain the association  Does not conflict with current knowledge of natural history and biology of disease  Example: Cigarettes contain many carcinogenic substances  Many epidemiologic studies have identified causal relationships before biological mechanisms were identified

 Did the investigators consider bias and confounding?

 Investigators must consider other possible explanations  Example: Did the investigators consider the associations between smoking, coffee consumption and pancreatic cancer?

 Risk of disease should decline when exposure to factor is reduced or eliminated  In certain cases, the damage may be irreversible  Example: Emphysema is not reversed with the cessation of smoking, but its progression is reduced

 If a relationship is causal, the findings should be consistent with other data  If lung cancer incidence increased as cigarette use was on the decline, need to explain how this was consistent with a causal relationship

 A single exposure should cause a single disease  Example: Smoking is associated with lung cancer as well as many other diseases  Lung cancer results from smoking as well as other exposures  When present, provides additional support for causal inference  When absent, does not preclude a causal relationship

 Remembering distinctions between association and causation in epidemiologic research  Critically reading epidemiologic studies  Designing epidemiologic studies  Interpreting the results of your own study

 Majority of scientists believe HIV causes AIDS  Small group believes AIDS is a behavioral rather than an infectious disease  AIDS is caused by use of recreational drugs and antiretroviral drugs in the U.S. and Europe, and by malnutrition in Africa

   Regular and exclusive Gallo et al. routinely found HIV in people with AIDS symptoms and failed to find HIV among people who either lacked AIDS symptoms or AIDS associated risk factors   Experimental model 3 lab workers accidentally infected in early 1990s with purely molecularly cloned strain of HIV One developed pneumonia (AIDS-defining disease) before started antiretroviral therapy

         Epidemiologic studies have established: Temporal relationship Strong association Dose response Replication of findings Biologic plausibility Cessation of exposure (decrease in deaths after antiretroviral therapy) Specificity Consistency with other knowledge

Associations are observed Causation is inferred

 It is important to remember that these criteria provide evidence for causal relationships  All of the evidence must be considered and the criteria weighed against each other to infer the causal relationship

 Bias is a systematic error that results in an incorrect estimate of association  Selection and information bias  Confounding is an alternate explanation which can be controlled  Design and analysis phases  Causation must be inferred

 Center for Public Health Continuing Education University at Albany School of Public Health  Department of Community & Family Medicine Duke University School of Medicine

Mike Barry, CAE Lorrie Basnight, MD Nancy Bennett, MD, MS Ruth Gaare Bernheim, JD, MPH Amber Berrian, MPH James Cawley, MPH, PA-C Jack Dillenberg, DDS, MPH Kristine Gebbie, RN, DrPH Asim Jani, MD, MPH, FACP Denise Koo, MD, MPH Suzanne Lazorick, MD, MPH Rika Maeshiro, MD, MPH Dan Mareck, MD Steve McCurdy, MD, MPH Susan M. Meyer, PhD Sallie Rixey, MD, MEd Nawraz Shawir, MBBS

 Sharon Hull, MD, MPH President  Allison L. Lewis Executive Director  O. Kent Nordvig, MEd Project Representative