Transcript Experimental Studies Developed through the APTR Initiative to Enhance Prevention and Population Health Education in collaboration with the Brody School of Medicine.

Experimental Studies
Developed through the APTR Initiative to Enhance Prevention and Population
Health Education in collaboration with the Brody School of Medicine at East
Carolina University with funding from the Centers for Disease Control and
Prevention
APTR wishes to acknowledge the following individual that
developed this module:

Jeffrey Bethel, PhD
Department of Public Health
Brody School of Medicine at East Carolina University
This education module is made possible through the Centers for Disease Control and Prevention (CDC) and the
Association for Prevention Teaching and Research (APTR) Cooperative Agreement, No. 5U50CD300860. The module
represents the opinions of the author(s) and does not necessarily represent the views of the Centers for Disease
Control and Prevention or the Association for Prevention Teaching and Research.
1.
Recognize use of experimental studies as an
epidemiologic study design
2.
Distinguish between types of experimental studies
3.
Describe key features of conducting experimental
studies
4.
Recognize special considerations of experimental
studies

Experimental studies (experimental)
 Researcher determines who is exposed
(treatments received)

Cohort studies (observational)

Case-control studies (observational)

Cross-sectional studies (observational)

Goal of public health and clinical medicine is to
modify natural history of disease and improve
morbidity and mortality

How do we select the best preventive and
therapeutic measures?

Carry out studies to determine value of various
measures
Smith, AH. The Epidemiologic Research Sequence. 1984

Most closely resemble controlled laboratory
experiments

Gold standard of epidemiological research

High status and validity and can pick up small and
modest effects

James Lind identified symptoms of scurvy among
sailors at sea after as little as a month

Conducted early experimental study on treatment of
scurvy in mid-1700’s among British sailors

Small sample size (6 groups of 2 ill sailors)

Group eating oranges and lemons were fit for duty in
6 days

Evaluate new drugs and other treatments for
diseases

Evaluate new medical and health care technology

Evaluate new screening programs or techniques

Evaluate new ways of organizing or delivering health
services (e.g. home v. hospital care following
myocardial infarction)

Preventive
 Does prophylactic agent given to healthy or high-risk
individual to prevent disease?

Therapeutic
 Does treatment given to diseased individual reduce risk of
recurrence, improve survival, quality of life?

Individual
 Do women with stage I breast cancer given a lumpectomy
alone survive as long without recurrence of disease as
women given a lumpectomy plus radiation?

Community
 Does fluoride in the water supply decrease the frequency
of dental caries in a community compared to a similar
community without such water treatment?
STUDY
POPULATION
RANDOM ASSIGNMENT
CURRENT
TREATMENT
IMPROVE
DO NOT
IMPROVE
NEW
TREATMENT
IMPROVE
DO NOT
IMPROVE

Hypothesis formed

Participants recruited based on specific criteria and their
informed consent is sought

Eligible and willing subjects randomly allocated to receive
one of the two or more interventions being compared

Study groups are monitored for outcome under study
(recurrence of disease, first occurrence of disease, getting
better, side effects)

Rates of the outcome in the various groups are compared

Women with stage I breast cancer given a
lumpectomy alone will survive as long without
recurrence of disease as women given a lumpectomy
plus radiation

Water supply with fluoride will decrease the
frequency of dental caries in a community compared
to a similar community without water treated with
fluoride




Who will be in the study?
Must be defined specifically before study
begins
Remove subjectivity
Reproducibility

Women’s Health Study
 ≥ 45 years
 No history of coronary heart disease, cerebrovascular
disease, cancer, or other major chronic illness
 No history of side effects to any of study medications
 Were not taking any of following meds more than once per
week: aspirin, NSAIDs, supplements of vitamin A, E, or
beta-carotene
 Were not taking anticoagulants or corticosteroids
NEJM 352;13:1293-1303

How many participants do we need to enroll
in the study?

Programs and tables exist to calculate sample
size based on various parameters
TRUTH IN THE POPULATION
CONCLUSION
FROM SAMPLE
Fail to reject Ho
(no difference)
Ho
(no difference)
H1
(there is a
difference)
Correct decision
Type II error
(Probability = b)
False negative
Reject Ho
(there is a
difference)
Type I error
(Probability = a)
Correct decision
(Probability = 1- b)
False positive
Type I and II errors can be reduced by increasing sample size



The difference in effect to be detected
Estimate of effect in one group
Level of significance (a)
 Probability of concluding treatments differ when
they do not differ

Level of power desired (1 - β)
 Probability of concluding treatments differ when
they do differ

1-sided or 2-sided test

Compare the outcome among “exposed” to what the
outcome would have been if unexposed

This comparison is counterfactual

Instead, compare the outcome among “exposed”
group to the outcome in a “substitute” population

Validity of inference depends on finding a valid
substitute population

Need to randomly assign participants to one of the
intervention groups (test or control)

Randomization
 Next assignment is unpredictable
 Coin toss to determine group allocation
 Random number table, opaque envelopes
 Computer

Main purpose
 Reduces selection bias in the allocation of treatment
 Each participant has an equal chance of being in test or
control group

Secondary purpose
 If large enough sample size, produce treatment and control
groups with similar baseline characteristics
 Control for known and unknown factors
Baseline Characteristics in a study of heart disease patients
Test Group
(n = 9,599)
Control Group
(n = 9,586)
Male (%)
72
72
White (%)
95
95
Current smoker (%)
29
30
Hypertension (%)
52
51
Stable angina (%)
22
22
High cholesterol (%)
41
41
Characteristic
Patients with a history of:
Baseline Characteristics in a study Maternal-Infant HIV Transmission
Test Group
(n = 239)
Control Group
(n = 238)
Median age at entry (yrs)
24
25
White (%)
48
38
Gestational age at entry
29
30
Median (weeks)
26
27
14-26 weeks (%)
52
50
> 26 weeks (%)
48
50
41
41
Characteristic
Median CD4 county at entry

Treatment
 Keep track of which treatment group the participant was
assigned
 Keep track of which therapy received

Baseline data
 Collect baseline demographic and other risk factor data
 Compare treatment groups

Measuring outcome
 Must be conducted in same fashion for all treatment
groups
 Preventive studies
▪ Precursors of disease or first occurrence of disease
 Therapeutic studies
▪ Symptom improvement
▪ Length of survival
▪ Disease recurrence

Myocardial infarction
 Symptoms met WHO criteria
 Abnormal levels of cardiac enzymes or diagnostic
electrocardiograms

Stroke
 New neurologic deficit of sudden onset that persisted for
at least 24 hours

Death from cardiovascular disease
 Examination of autopsy reports, death certificates, medical
records, and information obtained from the next of kin or
other family members

Masking (Blinding)
 Prevents conscious and subconscious bias in research
 Use placebo to mask
 Single blind: participants do not know which treatment
they are receiving
 Double blind: participants and observers (data collectors)
do not know participant treatment status

Parallel
 Participants in each group simultaneously receive one
study treatment
 Treatment and comparison groups consist of different
participants

Crossover
 Planned reversal of intervention and control groups
 Each participant can serve as his/her own control
STUDY
POPULATION
RANDOMLY ASSIGNED
NEW TREATMENT
CURRENT TREATMENT
Group 1
Group 2
Observe and
Measure Effects
Group 1
Group 2
Group 2
Group 1
Observe and
Measure Effects
Group 2
Group 1

Simple
 Each group receives a treatment consisting of one
component (e.g. one drug)

Factorial
 Use same study population to compare 2 or more
treatments
 2 x 2 factorial design
 Similar to 3 arms (drug A, drug B, and placebo) with fewer
participants
Drug A
Yes
No
Yes
Both
A and B
(cell a)
B only
(cell b)
No
A only
(cell c)
Neither
(cell d)
Drug B
Efficacy
of A
a+c v. b+d
Efficacy
of B
a+b
v.
c+d
Aspirin
Betacarotene
Yes
No
Efficacy
of Aspirin
Yes
No
Aspirin and Beta-carotene
Beta-carotene
only
(cell a)
(cell b)
Aspirin only
(cell c)
Neither
(cell d)
a+c v. b+d
Efficacy of
BetaCarotene
a+b
v.
c+d

Overt
 Notify investigators he/she is dropping out of study
 Drop outs

Covert
 Stop taking assigned treatment without telling investigators
 Need to build compliance checks in to the study (e.g. test
urine, count pills, etc.)

Efficacy
 Reduction in risk
 Calculate risk of death, developing disease,
complications in each group

=
Vaccine example
(Rate in placebo group) – (Rate in vaccine group)
Rate in placebo group

Relative risk

Kaplan-Meier plot

Hazard ratio

Number of patients who would need to be treated
(NNT) to prevent 1 adverse event

Number needed to harm (NNH) indicates number
patients treated to cause harm in 1 patient who
would not otherwise have been harmed

Internal validity
 Extent to which the study groups are comparable
 Comparability
 Reflected by selection/randomization

External validity
 Extent to which the results of a study can be applied to
people not in it
 Generalizability
 Representativeness
REFERENCE
POPULATION
External
Validity
STUDY
POPULATION
RANDOMLY ASSIGNED
CURRENT
TREATMENT
NEW
TREATMENT
Internal Validity

Items affecting internal validity
 Loss to follow-up
 Lack of randomization

Items affecting external validity
 Loss to follow-up
 Low response rate
 Narrow inclusion criteria

Randomization
 There must be genuine uncertainty about which treatment
is better

Informed consent
 Some trials enroll participants immediately after diagnosis

When to stop the study?
 Harmful or beneficial effects of one treatment arm
 Outside board monitors study

Expensive and time-consuming

Ethical concerns may arise

A large number of participants may be required

Participant exclusion may limit generalizability

Compliance may be an issue

Influence of sponsorship

Randomization tends to balance risk factors across
study groups

Blinding of participants can reduce bias in
assessment of outcomes

Prospective design

Eliminate bias by comparing two otherwise identical
groups

Detailed information collected at baseline and
throughout study period

Experimental studies top epidemiologic study design
hierarchy in terms of validity

Investigators assign treatment to participants
(experimental)

Randomization reduces selection bias in treatment
allocation

Data collection must be conducted systematically

Noncompliance and drop-outs must be minimized to
increase validity of results

Center for Public Health Continuing Education
University at Albany School of Public Health

Department of Community & Family Medicine
Duke University School of Medicine
Mike Barry, CAE
Lorrie Basnight, MD
Nancy Bennett, MD, MS
Ruth Gaare Bernheim, JD, MPH
Amber Berrian, MPH
James Cawley, MPH, PA-C
Jack Dillenberg, DDS, MPH
Kristine Gebbie, RN, DrPH
Asim Jani, MD, MPH, FACP
Denise Koo, MD, MPH
Suzanne Lazorick, MD, MPH
Rika Maeshiro, MD, MPH
Dan Mareck, MD
Steve McCurdy, MD, MPH
Susan M. Meyer, PhD
Sallie Rixey, MD, MEd
Nawraz Shawir, MBBS

Sharon Hull, MD, MPH
President

Allison L. Lewis
Executive Director

O. Kent Nordvig, MEd
Project Representative