Transcript No Slide Title

Types of Studies and
Study Design
C1, L3-4, S1
Research classifications
• Observational vs. Experimental
Observational – researcher collects info
on attributes or measurements of interest,
but does not influence results.
Experimental – researcher deliberately
influences events and investigates the
effects of the intervention, e.g. clinical
trials and laboratory experiments.
We often use these when we are interested in studying the
effect of a treatment on individuals or experimental units.
C1, L3-4, S2
Experiments & Observational Studies
We conduct an experiment when it is
(ethically, physically etc) possible for the
experimenter to determine which
experimental units receive which treatment.
C1, L3-4, S3
Experiments & Observational Studies
Experiment Terminology
Experimental Unit
patient
patient
mouse
Treatment
Response
drug
pre-surgery antibiotic
radiation
cholesterol
infection
mortality
C1, L3-4, S4
Experiments & Observational Studies
In an observational study, we compare the
units that happen to have received each of
the treatments.
C1, L3-4, S5
Experiments & Observational Studies
Unit
patient
RN
hospital
Observational Study
Treatment
Response
smoking
lung cancer
unit
job stress
ICU
ICU mortality
staffing level
e.g. You cannot set up a control
(non-smoking) group and treatment
(smoking) group.
C1, L3-4, S6
Experiments & Observational Studies
Note:
Only a well-designed and well-executed
experiment can reliably establish
causation.
An observational study is useful for
identifying possible causes of effects, but
it cannot reliably establish causation.
C1, L3-4, S7
1. Completely Randomized Design
The treatments are allocated entirely by
chance to the experimental units.
C1, L3-4, S8
1. Completely Randomized Design
Example:
Which of two varieties of tomatoes (A & B)
yield a greater quantity of market quality
fruit?
Factors that may affect yield:
•
•
•
•
different soil fertility levels
exposure to wind/sun
soil pH levels
soil water content etc.
C1, L3-4, S9
1. Completely Randomized Design
Divide the field into plots and randomly
allocate the tomato varieties (treatments)
to each plot (unit).
8 plots – 4 get variety A
UPHILL
1(A)
2 (A)
3(B)
(A)
4(A)
(B)
5(B)
(A)
6 (B)
(B)
7(A)
(B)
8(B)
Randomly assign A & B varieties in each strip of
similarifelevation.
What
the field sloped upward from left to right?
C1, L3-4, S10
1. Completely Randomized Design
Note:
Randomization is an attempt to make the
treatment groups as similar as possible —
we can only expect to achieve this when
there is a large number of experimental
units to choose from.
C1, L3-4, S11
2. Blocking
Group (block) experimental units by some
known factor and then randomize within
each block in an attempt to balance out
the unknown factors.
Use:
• blocking for known factors
(e.g. slope of field in previous example)
and
• randomization for unknown
factors to try to “balance things
out”.
C1, L3-4, S12
2. Blocking
Example 2: Multi-Center Clinical Trial
Suppose a Mayo clinical trial comparing
two chemotherapy regimens in treatment
of patients with colon cancer will be
conducted using cancer patients in
Scottsdale, AZ and Rochester, MN.
C1, L3-4, S13
2. Blocking
Scottsdale
Rochester
1 (A)
1 (B) 2 (A)
2 (B)
3 (A)
4 (B)
3 (A) 5 (A)
6 (A)
4 (B)
8 (B)
7 (B)
How should we allocate treatments to the
12 patients?
Randomly assign treatments to 4 the patients from
Scottsdale and then to the 8 Rochester patients.
C1, L3-4, S14
2. Blocking
Example 3: Comparing Three Pain
Relievers for Headache Sufferers
• How could blocking be used to increase
precision of a designed experiment to control to
compare the pain relievers?
• What are some other design issues?
C1, L3-4, S15
Example 4: Comparing 17 Different Leg
Wraps on Used on Race Horses
• 17 “boots” tested, each boot is tested
n = 5 times. Why?
• Because of the time constraints all boots
were not tested on the same day.
• 8 tested 1st day, 5 tested 2nd day, 4 tested
3rd day.
• Leg was placed in freezer and thawed
before the 2nd and 3rd days of testing.
C1, L3-4, S16
Horse Leg Wraps (cont’d)
• What problems do you foresee with this
experimental design? Discuss
• What actually happened?
Forces readings obtained
from cadaver leg when no
boot or wrap was used.
What are the implications of
these results? Discuss
C1, L3-4, S17
Horse Leg Wraps (cont’d)
FINAL BOOT COMPARISONS
C1, L3-4, S18
Horse Legs Wraps (cont’d)
• What should have been done?
Discuss
C1, L3-4, S19
3. People as Experimental Units
Example: Cholesterol Drug Study –
Suppose we wish to determine whether
a drug will help lower the cholesterol
level of patients who take it.
How should we design our study?
Discuss for two minutes in groups.
C1, L3-4, S20
Polio Vaccine Example
C1, L3-4, S21
Polio Vaccine Example
Dr. Jonas Salk,
vaccine pioneer
1914-95
Iron Lung
C1, L3-4, S22
The Salk Vaccine Field Trial
• 1954 Public Health Service organized an
experiment to test the effectiveness of
Salk’s vaccine.
• Need for experiment:
– Polio, an epidemic disease with cases
varying considerably from year to year. A
drop in polio after vaccination could mean
either:
• Vaccine effective
• No epidemic that year
C1, L3-4, S23
The Salk Vaccine Field Trial
Subjects: 2 million, Grades 1, 2, and 3
• 500,000 were vaccinated
– (Treatment Group)
• 1 million deliberately not vaccinated
– (Control Group)
• 500,000 not vaccinated - parental
permission denied
C1, L3-4, S24
The Salk Vaccine Field Trial
NFIP Design
• Treatment Group: Grade 2
• Control Group: Grades 1 and 3 + No Permission
Flaws ? Discuss for 30 seconds.
• Polio contagious, spreading through contact.
i.e. incidence could be greater in Grade 2 (bias
against vaccine), or vice-versa.
• Control group included children without parental
permission (usually children from lower income
families) whereas Treatment group could not
(bias against the vaccine).
C1, L3-4, S25
The Salk Vaccine Field Trial
Double-Blinded Randomized
Controlled Experimental Design
• Control group only chosen from those with
parental permission for vaccination
• Random assignment to treatment or control
group
• Use of placebo (control group given injection of
salted water)
• Diagnosticians not told which group the subject
came from (polio can be difficult to diagnose)
• i.e., a double-blind randomized controlled
experiment
C1, L3-4, S26
The Salk Vaccine Field Trial
The double-blind randomized
controlled experiment (and NFIP) results
Size of
group
Rate per
100,000
(NFIP rate)
Treatment
200,000
28
(25) Grade 2
Control
200,000
71
(54) Grade 1/3
No consent
350,000
46
(44) Grade 2
C1, L3-4, S27
3. People as Experimental Units
• control group:
– Receive no treatment or an existing
treatment
• blinding:
– Subjects don’t know which treatment
they receive
• double blind:
– Subjects and administers /
diagnosticians are blinded
• placebo:
– Inert dummy treatment
C1, L3-4, S28
3. People as Experimental Units
• placebo effect:
– A common response in humans when they
believe they have been treated.
– Approximately 35% of people respond
positively to dummy treatments - the
placebo effect
C1, L3-4, S29
Observational Studies
• There are two major types of
observational studies:
prospective and retrospective studies
C1, L3-4, S30
Observational Studies
1. Prospective Studies
– (looking forward)
– Choose samples now, measure variables
and follow up in the future.
– E.g., choose a group of smokers and
non-smokers now and observe their health in
the future.
C1, L3-4, S31
Observational Studies
2. Retrospective Studies
– (looking back)
– Looks back at the past.
– E.g., a case-control study
• Separate samples for cases and controls
(non-cases).
• Look back into the past and compare
histories.
• E.g. choose two groups: lung cancer
patients and non-lung cancer patients.
Compare their smoking histories.
C1, L3-4, S32
Observational Studies
Important Note:
1. Observational studies should use some form
of random sampling to obtain
representative samples.
2. Observational studies cannot reliably
establish causation.
C1, L3-4, S33
Controlling for various factors
• A prospective study was carried out over
11 years on a group of smokers and nonsmokers showed that there were 7 lung
cancer deaths per 100,000 in the nonsmoker sample, but 166 lung cancer
deaths per 100,000 in the smoker sample.
• This still does not show smoking causes
lung cancer because it could be that
smokers smoke because of stress and
that this stress causes lung cancer.
C1, L3-4, S34
Controlling for various factors
• To control for this factor we might divide
our samples into different stress
categories. We then compare smokers
and non-smokers who are in the same
stress category.
• This is called controlling for a
confounding factor.
C1, L3-4, S35
Example 1
• “Home births give babies a good chance”
NZ Herald, 1990
– An Australian report was stated to have said
that babies are twice as likely to die during or
soon after a hospital delivery than those from
a home birth.
– The report was based upon simple random
samples of home births and hospital births.
Q: Does this mean hospitals are dangerous
places to have babies in Australia? Why
or why not? Discuss for 1 minute in groups.
C1, L3-4, S36
Example 2
• “Lead Exposure Linked to Bad Teeth in
Children” ~ USA Today
The study involved 24,901 children ages 2
and older. It showed that the greater the
child’s exposure to lead, the more
decayed or missing teeth.
Q: Does this show lead exposure causes
tooth decay in children? Why or why not?
Discuss for 1 minute.
C1, L3-4, S37
Example 2 ~ cont’d
• “Lead Exposure Linked to Bad Teeth in
Children” ~ USA Today
Researcher:
“We controlled for income level, the
proportion of diet due to carbohydrates,
calcium in the diet and the number of
days since the last dental visit.”
C1, L3-4, S38
Limitations on Scope of Inference
C1, L3-4, S39
Discussion Question 1 – Determine
Whether Age at 1st Pregnancy is a
Risk Factor for Cervical Cancer
How might we proceed?
C1, L3-4, S40
Discussion Question 2 – Determine what
job related factors Mayo nurses are most
dissatisfied with.
How might we proceed?
C1, L3-4, S41
Discussion Question 3 – Determine if a
new pre-operative antibiotic reduces the
risk of infection for patients undergoing
knee replacement.
How might we proceed?
C1, L3-4, S42
Surveys and Polls
(and the errors inherent in them)
C1, L3-4, S43
Sampling
Sampling/Chance/
Random Errors
Nonsampling
Errors
Selection bias
Interviewer effects
Non-response bias
Behavioural considerations
Self selection
Transfer findings
Question effects
Survey-format effects
C1, L3-4, S44
Sources of Nonsampling Errors
Selection bias
Population sampled is not exactly the
population of interest.
e.g. KARE 11 poll, telephone interviews
sample
population
C1, L3-4, S45
Sources of Nonsampling Errors
Non-response bias
People who have been targeted to be surveyed
do not respond.
Non-respondents tend to behave
differently to respondents with
respect to the question being
asked.
C1, L3-4, S46
1936 U.S. Election
• Country struggling to recover from the
Great Depression
• 9 million unemployed
• 1929-1933 real income dropped by 1/3
C1, L3-4, S47
1936 U.S. Election
• Candidates:
– Franklin D Roosevelt (Democrat)
Deficit financing - “Balance the budget of
the people before balancing the budget of
the Nation”
– Albert Landon (Republican)
“The spenders must go!”
C1, L3-4, S48
1936 U.S. Election
• Roosevelt’s percentage
–Digest prediction of the election result
43%
–Gallup’s prediction of the Digest prediction 44%
–Gallup’s prediction of the election result
56%
–Actual election result
62%
• Digest sent out 10 million questionnaires to people
on club membership lists, telephone directories etc.
– received 2.4 million responses
• Gallup Poll used another sample of 50,000
• Gallup used a random sample of 3,000 from the
Digest lists to predict Digest outcome
C1, L3-4, S49
Sources of Nonsampling Errors
Self-selection bias
People decide themselves whether to be
surveyed or not.
Much behavioural research can only
use volunteers.
C1, L3-4, S50
Sources of Nonsampling Errors
C1, L3-4, S51
Sources of Nonsampling Errors
This poll is not scientific and reflects the opinions of
only those Internet users who have chosen to
participate. The results cannot be assumed to
represent the opinions of Internet users in general,
nor the public as a whole. The QuickVote sponsor is
not responsible for poll content, functionality or the
opinions expressed therein.
C1, L3-4, S52
Sources of Nonsampling Errors
Question effects
Subtle variations in wording can have an effect
on responses.
Eg “Should euthanasia be legal?”
vs “Should voluntary euthanasia be legal?”
C1, L3-4, S53
New York Times/CBS News Poll (8/18/80)
“Do you think there should be an
amendment to the constitution prohibiting
abortions?”
Yes 29%
No 62%
Later the same people were asked:
“Do you think there should be an
amendment to the constitution protecting
the life of the unborn child?”
Yes 50%
No 39%
C1, L3-4, S54
Sources of Nonsampling Errors
Interviewer effects
Different interviewers asking the same question
can obtain different results.
Eg sex, race, religion of the interviewer
C1, L3-4, S55
Interviewer Effects in Racial Questions
In 1968, one year after a major racial
disturbance in Detroit, a sample of black
residents were asked:
“Do you personally feel that you trust
most white people, some white people
or none at all?”
• White interviewer:
35% answered “most”
• Black interviewer:
7% answered “most”
C1, L3-4, S56
Sources of Nonsampling Errors
Behavioural considerations
People tend to answer questions in a way they
consider to be socially desirable.
e.g. pregnant women being asked about
their drinking habits
C1, L3-4, S57
Behavioural Considerations in Election
• Official vote counts show that 86.5 million
people voted in the 1980 U.S. presidential
elections.
• A census bureau survey of 64,000
households some weeks later estimated
93.1 million people voted.
C1, L3-4, S58
Sources of Nonsampling Errors
Transferring findings
Taking the data from one population and
transferring the results to another.
e.g. Twin Cities opinions may not be a
good indication of opinions in Winona.
Twin Cities
sample
Winona
C1, L3-4, S59
Sources of Nonsampling Errors
Survey-format effects
Eg question order, survey layout,
interviewed by phone or inperson or mail.
C1, L3-4, S60
Sampling
Sampling/Chance/
Random Errors
Nonsampling
Errors
Selection bias
Interviewer effects
Non-response bias
Behavioural considerations
Self selection
Transfer findings
Question effects
Survey-format effects
C1, L3-4, S61
Survey
Errors
Sampling/Chance/
Random Errors
Nonsampling
Errors
C1, L3-4, S62
Sampling / Chance / Random Errors
• errors caused by the act of taking a
sample
• have the potential to be bigger in smaller
samples than in larger ones
• possible to determine how large they can
be
• unavoidable (price of sampling)
C1, L3-4, S63
Nonsampling Errors
• can be much larger than sampling errors
• are always present
• can be virtually impossible to correct for
after the completion of survey
• virtually impossible to determine how
badly they will affect the result
• must try to minimize in design of survey
(use a pilot survey etc.)
C1, L3-4, S64
Surveys / Polls
A pilot survey is a small survey that is
carried out before the main survey and is
often used to identify any problems with
the survey design (such as potential
sources of non-sampling errors).
C1, L3-4, S65
Surveys / Polls
A report on a sample survey/poll should
include:
– target population (population of interest)
– sample selection method
– the sample size and the margin of error
– the date of the survey
– the exact question(s)
C1, L3-4, S66