Transcript Part - time MSc course Epidemiology & Statistics Module

The following lecture has been approved for
University Undergraduate Students
This lecture may contain information, ideas, concepts and discursive anecdotes
that may be thought provoking and challenging
It is not intended for the content or delivery to cause offence
Any issues raised in the lecture may require the viewer to engage in further
thought, insight, reflection or critical evaluation
[email protected]
Research Design
Experiments
Observations
& Surveys
Prof. Craig Jackson
Head of Psychology Division
School of Social Sciences
BCU
Objectives
Experimental studies
within-subjects studies
between-subjects studies
Observational studies
case-controls
cohorts
RCTs
Bias
Placebo
Control Groups
Introduction
Types of clinical research
Experimental vs. Observational
Longitudinal vs. Cross-sectional
Prospective vs. Retrospective
Experimental
Longitudinal
Observational
Longitudinal
Cross-sectional
Prospective
Prospective
Retrospective
Randomised Controlled Trial
Cohort studies
Case control studies
Survey
Qualititative VS Quantitative Research
False opposition
Observational methods equally valid
Complementary roles
Quantitative
Qualitative equally as hard to do
Qualitative
Experimental Studies
Investigator makes intervention
A “manipulation”
Then studies the effects of that intervention
Features:
Comparison e.g.
Always longitudinal
Always prospective
Experimental
Clinical
Trials
RCTs
before vs. after
control vs. treatment
Rationale of Experimental Studies
Evaluate effectiveness of intervention / therapy
Use similar samples – comparable groups
Samples reflect population
Differences in outcomes due to interventions (not differences between groups)
Independent Variable (IV) alters Dependent Variable (DV)
Best evidence of cause and effect
Sometimes inconclusive
Types of Experimental Studies
Between Subjects Studies
Each group receives different treatment
Groups compared
Within Subjects Studies
Each individual is measured before & after intervention
Advantage that each participant is own control
Between subject variability removed
Traditional Experimental Designs
Between subjects studies
Treatment group
Outcome measured
Control group
Outcome measured
patients
Within Subjects studies
patients
Outcome measured #1
Treatment
Outcome measured #2
Within Subjects Studies
Cross-over-studies
Each patient receives treatment in sequence
“Washout” period between treatments
Order of treatments randomised
Gp A
Treatment 1
Treatment 2
Gp A
Gp B
Treatment 2
Treatment 1
Gp B
Matched-pairs study
Parallel study
Patient in arm 1 matched with patient in arm 2
Matched based on prognostic factors
Data is linked
Paired individuals
Avoiding Bias
Validity of study depends on avoiding bias
Bias = “Systematic distortion of results due to unforeseen factors”
gp1 = pill
gp2 = no pill
How will the “no pill”group progress?
Any effects of them knowing they have no treatment?
Handling differences may influence + complicate trial results
Known as confounding factors
control group
To minimize bias…
randomisation
blinding
Placebo effect – it really does work!
Most effective medication known
In approx. 30% of pop.
Subjected to more clinical trials than any other medicament
Nearly always does better than anticipated
The range of susceptible conditions seems limitless
Does not always occur
Present in subjective and objective outcomes
Negative outcomes can occur (Nocebo effect)
Placebo
•Big pills better than smaller pills
•Red pills better than blue
•4 pills better than 2
•30% of pop.
Patient’s “knowledge” of their treatment causes bias
e.g. ) Benedetti & the Turin study
Control Groups
Allow comparison in Between Group studies
Evaluations without comparison?
Patient knowledge of their treatment causes bias
e.g. ) Benedetti & the Turin study
Types of Control Groups
•“no treatment” group
likely to be confounded by having condition
•“placebo” group
ethically dodgy?
•“low dose” group
avoids ethical issues
•“standard treatment” group
avoids ethical issues
•“gold standard” group
avoids ethical issues
•“historical controls”
unreliable due to many confounders
Control Groups: Random Allocation
Population
(60 million)
Doesn’t guarantee groups will be homogonous
Ensures allocation independent of patient features
Sample
(1000)
Avoids (sub)conscious allocation bias
e.g. severely sick people into treatment groups
Gp A
(500)
Gp B
(500)
Drug X
Drug Y
53 yrs
27 years
Non-homogenous groups may still occur
due to chance – random errors
80% male 50% male
20% fem
50% fem
Stratified randomisation
for each prognostic factor e.g. weight, age, sex
Guarantees allocation to be bias-free
Comparison Groups: Random Sampling
Ensures generalizability of findings to larger pop.
e.g. in-patient sample limitations
Treatment effects better detected if there is little between-group variability
Exclusion Criteria & Inclusion Criteria keep groups comparable
Paradox:
greater uniformity of sample = less generalizable to general population
Blinding: Importance of doing it
Investigator or Subject know treatment = Bias
Observations and Judgements become less reliable
Patient responses change:
Positive outcomes in active arm
Negative outcomes in passive arm
e.g. known cancer diagnoses and deterioration
Use max. degree of blindness possible
e.g. make subject and investigator both blind if possible
e.g. A.A.Mason & Congenital Ichthyosis and Hypnosis
1951
Blinding: Methods of doing it
Double-blind
patient & investigator blind
Treatment type
Patient interaction
Data manager
Un-blinding a problematic study
Breaking code – anticipated in planning
Criteria for breaking code – established and agreed
Emergency access to randomisation code
Treatment stopped and patient withdrawn
Formal monitoring process – review and make recommendations
Blinding: Methods
Double-blind
patient & investigator blind
Single-blind
patient blind
Triple-blind
patient & investigator & data monitor blind
Double-dummy 2 treatments
patients get 2 pills (1 active, 1 dummy)
Open trials
patient & investigator aware of treatment
Randomisation in a double-blind trial
Envelope technique common
Un-blinding – ethical necessity
Subject Variables that potentially confound research
STABLE FACTORS
Age
Education
Sex
Socioeconomics
Language
Handedness
Computer experience
Caffeine (habitual use)
Alcohol (habitual use)
Nicotine (habitual use)
Medication (habitual use)
Paints, glues, pesticides (habitual use)
Diabetes
Epilepsy
Other CNS / PNS disease
Head injury (out >1 hr)
Alcohol / drug addiction
Physical activity
SITUATIONAL FACTORS
Alcohol (recent use)
Caffeine (recent use)
Nicotine (recent use)
Medication (recent use)
Paints, glues, pesticides (recent)
Near visual acuity
Restricted movement (injury)
Cold / flu
Stress
Arousal / Fatigue
Sleep
Screen luminance
Time of day
Time of year
Randomized Controlled Trials in GP & Primary Care
90% consultations take place in GP surgery
RCT is really 50 years old
Potential problems
2 Key areas:
Recruitment Bias
Randomisation Bias
Over-focus on failings of RCTs
RCTs in General Practice & Primary Care
RCTs justified in situations of genuine clinical uncertainty
Samples large enough to establish any worthwhile benefit
(effectiveness or cost, or both)
Need for larger numbers of patients
More than are available to single practices
Requires “club together” approach
GPs: no contractual obligation
(i)
unwilling to take part if no immediate benefit for patients
(ii) while possibly disrupting the delivery of health care
RCTs in General Practice & Primary Care
GPs conflict of interest between:
Role and Wish to benefit future patients
Academic merit
Long term nature of practitioner and patient relationship
may engender loyalties
unfairly coerce patients to give consent
Patients' fears about:
confidentiality
risks of the intervention
apparent disadvantage of being allocated to a control group
may further inhibit recruitment
Fail to recruit consecutive patients may introduce potential for selection bias
RCTs in General Practice & Primary Care
Provides rigorous, sound basis for evaluating treatments
May disrupt primary care
Too much disruption = no reflection of real practice
Methodological problems reduce scientific reliability of the results
(Recruitment & Randomisation)
General practice not a laboratory
Patients are not experimental animals
Case-control studies, retrospective and prospective cohort studies, and
descriptive studies are all acceptable methods.
Observation is OK
Should accept alternative methods when RCT too difficult or flawed
RCT Deficiencies
Trials too small
Trials too short
Poor quality
Poorly presented
Address wrong question
Methodological inadequacies
Inadequate measures of quality of life (changing)
Cost-data poorly presented
Ethical neglect
Patients given limited understanding
Poor trial management
Politics
Marketeering
Why still the dominant model?
Observational Studies
Investigator observes existing situation
Describes
Analyses
Interprets
No influence on events
Longitudinal observation studies
case-control studies: retrospective
cohort-studies: prospective
Cross-sectional observation studies
surveys examining subjects at one point in time
based on random sample of interest population
Observational Studies:
Look for associations
• Cause -> Effect
• Exposure – Illness
• Epidemiological
• Incidence
• Cause
• Prevention
No control
Cannot use classical experimentation
No randomisation
Bias
Case-Control Study
Identify group with condition / illness (cases)
Identify group without condition / illness (controls)
Both groups compared for exposure to (hypothesized) risk factors
Greater exposure to risk factor in cases than controls = “causal relation”
Lead time bias
Recruitment of cases at similar points in time
Newly diagnosed cases
Selection of Controls
Cases have Lung Cancer + Smoking Exposure
Controls could be other hospital patients (other disease) or “normals”
Matched Cases & Controls for age & gender
Option of 2 Controls per Case
Smoking years of Lung Cancer cases and controls
(matched for age and sex)
Cases
n=456
Smoking years 13.75
(± 1.5)
Controls
n=456
6.12
(± 2.1)
F
7.5
P
0.04
Case-Control Study: Other Biases
Recall Bias
Cases > associations with exposures
Unreliable Memories
Retrospective nature
Over-reliance on recall
Unreliable Records
Poor hospital records
Repetitive, incomplete, inaccurate, irretrievable, interpretation
Interview Bias
Different interviewers
Case-Control Study: Measurement Methods
Subj.
Obj.
Present Past
Personal interview



Checklist / questions



JEM



Job-specific questions



Diary




Records


Observation


External environ


Internal concentrations


Biochem. Markers


Cohort Study
ID and examination of a group (cohort)
Followed over time (20 years common!)
Looking for disease development / other end-point
Aetiology of disease (based on data collected)
Data more reliable than case-control studies
• Requires large N
• Requires long follow up
• Inefficient
• Expensive (espec. rare outcomes)
Cohort Study: Methods
Volunteers in 2 groups e.g. exposed vs non-exposed
All complete health survey every 12 months
End point at 5 years: groups compared for Health Status
Comparison of general health between users and non-users of mobile
phones
ill
healthy
mobile phone user
292
108
400
non-phone user
89
313
402
381
421
802
Cohort Study: Other Biases
Lost to follow up
Bias if reason related to exposure
Validity affected
Group sizes change
Membership changes e.g ex-smokers
Differential mortality
Change in circumstance
e.g. job change
Exposures need calculation or re-calculation
Surveillance bias
Investigator aware of group membership
Investigating exposed members more
Observational studies
Cohort (prospective)
cohort
prospectively measure risk factors
end point measured
aetiology
prevalence
development
odds ratios
Case-Control (retrospective)
start point measured
aetiology
odds ratios
prevalence
development
retrospectively measure risk factors
cases
Cross Sectional Study
Subjects contacted & surveyed just once
Questionnaire (post, email, phone)
Random sample of defined pop.
Limited causality
Not temporal relationships
Little insight into aetiology
Source of descriptive data
Prevalence rates
Volunteer bias
Non responses
Self-selection
Unrepresentative sample
Further Reading
Altman, D.G. “Designing Research”. In: Altman, D.G., (ed.) Practical Statistics
For Medical Research. London, Chapman and Hall, 1991; 74-106.
Bland, M. “The design of experiments”. In: Bland, M., (ed.) An introduction to
medical statistics. Oxford, Oxford Medical Publications, 1995; 5-25.
Daly, L.E., Bourke, G.J. “Epidemiological and clinical research methods”.
In: Daly L.E., Bourke, G.J., (eds.) Interpretation and uses of medical statistics.
Oxford, Blackwell Science Ltd, 2000; 143-201.
Jackson, C.A. “Study Design” & “Sample Size and Power”. In: Gao Smith, F.
and Smith, J. (eds.) Key Topics in Clinical Research. Oxford, BIOS scientific
Publications, 2002.
Jackson, C.A. “Planning Health & Safety Research Projects in the
Workplace”. Croner Health and Safety at Work Special Report 2002; 62: 1-16.
Further Reading
Kumar, R. Research Methodology: a step by step guide for beginners.
Sage, London 1999.
Abbott, P. and Sapsford. Research methods for nurses and the caring
professions. Open University Press, Buckingham 1988.
Bowling, A. Measuring Health. Open University Press, Milton Keynes 1994
Polit, D. & Hungler, B. Nursing research: Principles and methods (7th ed.).
Philadelphia: Lippincott, Williams & Wilkins 2003.