Transcript Document

Research Design
Experiments
Observations
& Surveys
Prof. Craig Jackson
Head of Psychology Division
School of Social Sciences
BCU
health.bcu.ac.uk/craigjackson
[email protected]
Objectives
Experimental studies
within-subjects studies
between-subjects studies
Observational studies
case-controls
cohorts
RCTs
Bias
Placebo
Control Groups
Introduction
Types of research
Experimental vs. Observational
Longitudinal vs. Cross-sectional
Prospective vs. Retrospective
Experimental
Longitudinal
Prospective
Randomised Controlled Trial
Qualititative VS Quantitative Research
False opposition
Observational methods equally valid
Complementary roles
Quantitative
Qualitative equally as hard to do
Qualitative
Experimental Studies
Investigator makes intervention
A “manipulation”
Then studies the effects of that intervention
Features:
Comparison e.g.
before vs. after
Always longitudinal
Always prospective
Experimental
Clinical
Trials
RCTs
control vs. treatment
Rationale of Experimental Studies
Evaluate effectiveness of intervention / therapy
Use similar samples – comparable groups
Samples reflect population
Differences in outcomes due to interventions (not differences
between groups)
Independent Variable (IV) alters Dependent Variable (DV)
Best evidence of cause and effect
Sometimes inconclusive
Types of Experimental Studies
Between Subjects Studies
Each group receives different treatment
Groups compared
Within Subjects Studies
Each individual is measured before & after intervention
Advantage that each participant is own control
Between subject variability removed
Within Subjects Studies
Cross-over-studies
Each patient receives treatment in sequence
“Washout” period between treatments
Order of treatments randomised
Gp A
Treatment 1
Gp B Treatment 2
Treatment 2
Treatment 1
Matched-pairs study
Parallel study
Patient in arm 1 matched with patient in arm 2
Matched based on prognostic factors
Data is linked
Paired individuals
Gp A
Gp B
Avoiding Bias
Validity of study depends on avoiding bias
Bias = “Systematic distortion of results due to unforeseen
factors”
gp1 = pill
gp2 = no pill
How will the “no pill”group progress?
Any effects of them knowing they have no treatment?
Handling differences may influence + complicate trial results
Known as confounding factors
To minimize bias…
control group
randomisation
blinding
Placebo effect – it really does work!
Most effective medication known
In approx. 30% of pop.
Subjected to more clinical trials than any other medicament
Nearly always does better than anticipated
The range of susceptible conditions seems limitless
Does not always occur
Present in subjective and objective outcomes
Negative outcomes can occur (Nocebo effect)
Placebo
•Big pills better than smaller pills
•Red pills better than blue
•4 pills better than 2
•30% of pop.
Control Groups
Allow comparison in Between Group studies
Evaluations without comparison?
Patient knowledge of their treatment causes bias
e.g. Benedetti & the Turin study
Types of Control Groups
•“no treatment” group
likely to be confounded by condition
•“placebo” group
ethically dodgy?
•“low dose” group
avoids ethical issues
•“standard treatment” group
avoids ethical issues
•“gold standard” group
avoids ethical issues
•“historical controls”
unreliable due to many confounders
Control Groups: Random Allocation
Population
(60 million)
Sample
(1000)
Gp A
(500)
Drug X
53 yrs
80% male
20% fem
Doesn’t guarantee groups will be
homogonous
Ensures allocation independent of patient
features
Gp B
(500)
Avoids (sub)conscious allocation bias
e.g. severely sick people into treatment
groups
Drug Y
Guarantees allocation to be bias-free
27 years
Non-homogenous groups may still occur
50% male
50% fem
due to chance – random errors
Comparison Groups: Random Sampling
Ensures generalizability of findings to larger pop.
e.g. in-custody sample limitations
Treatment effects better detected if there is little between-group
variability
Exclusion Criteria & Inclusion Criteria keep groups comparable
Paradox:
greater uniformity of sample = less generalizable to gen. pop
Blinding: Importance of doing it
Investigator or Subject know treatment = Bias
Observations and Judgements become less reliable
Patient responses change:
Positive outcomes in active arm
Negative outcomes in passive arm
e.g. known cancer diagnoses and deterioration
Use max. degree of blindness possible
e.g. make subject and investigator both blind if possible
e.g. A.A.Mason & Congenital Ichthyosis and Hypnosis 1951
Blinding: Methods of doing it
Double-blind patient & investigator blind
Treatment type
Patient interaction
Data manager
Un-blinding a problematic study
Breaking code – anticipated in planning
Criteria for breaking code – established and agreed
Emergency access to randomisation code
Treatment stopped and patient withdrawn
Formal monitoring process – review and make
recommendations
Blinding: Methods
Double-blind patient & investigator blind
Single-blind patient blind
Triple-blind
patient & investigator & data monitor blind
Double-dummy
2 treatments
patients get 2 pills (1 active, 1 dummy)
Open trials
patient & investigator aware of treatment
Randomisation in a double-blind trial
Envelope technique common
Un-blinding – ethical necessity
Subject Variables that confound research
STABLE FACTORS
SITUATIONAL FACTORS
Age
Education
Sex
Socioeconomics
Language
Handedness
Near visual acuity
Computer experience
Caffeine (habitual use)
Alcohol (habitual use)
Nicotine (habitual use)
Medication (habitual use)
Paints, glues, pesticides
Diabetes
Epilepsy
Other CNS / PNS disease
Head injury (out >1 hr)
Alcohol / drug addiction
Physical activity
Alcohol (recent use
Caffeine (recent use)
Nicotine (recent use)
Medication (recent use)
Paints, glues, pesticides
Restricted movement (injury)
Cold / flu
Stress
Arousal / Fatigue
Sleep
Screen luminance
Time of day
Time of year
Randomized Controlled Trials in Practice
90% consultations take place in GP surgery
RCT is really 50 years old
Potential problems
2 Key areas: Recruitment Bias
Randomisation Bias
Over-focus on failings of RCTs
RCTs in Practice
RCTs justified in situations of genuine clinical uncertainty
Samples large enough to establish any worthwhile benefit
(effectiveness or cost, or both)
Need for larger numbers of participants
More than are available to single practices
Requires “club together” approach
Practitioners: no contractual obligation
(i)
unwilling to take part if no immediate benefit for clients
(ii)
while possibly disrupting the delivery of service/care
RCTs in Practice
Conflict of interest between:
Role and Wish to benefit future offenders
Academic merit
Long term nature of practitioner and client relationship
may engender loyalties
unfairly coerce clients to give consent
Patients' fears about:
confidentiality
risks of the intervention
apparent disadvantage of being allocated to a control
group
may further inhibit recruitment
Fail to recruit consecutive clients may introduce potential for
RCTs in Practice
Provides rigorous, sound basis for evaluating treatments
May disrupt care
Too much disruption = no reflection of real practice
Methodological problems reduce scientific reliability of the results
(Recruitment & Randomisation)
Practice not a laboratory
People are not experimental animals
Case-control studies, retrospective, prospective cohort studies,
and descriptive studies are all acceptable methods.
Observation is OK
Should accept alternative methods when RCT difficult or flawed
RCT Deficiencies
Trials too small
Trials too short
Poor quality
Poorly presented
Address wrong question
Methodological inadequacies
Inadequate measures of quality of life (changing)
Cost-data poorly presented
Ethical neglect
Participants given limited understanding
Poor trial management
Politics
Marketeering
Why still the dominant model?
Observational Studies
Investigator observes existing situation
Describes
Analyses
Interprets
No influence on events
Longitudinal observation studies
case-control studies: retrospective
cohort-studies: prospective
Cross-sectional observation studies
surveys examining subjects at one point in time
based on random sample of interest population
Observational Studies:
Look for associations
• Cause -> Effect
• Exposure – Illness
• Epidemiological
• Incidence
• Cause
• Prevention
No control
Cannot use classical experimentation
No randomisation
Bias
Case-Control Study
Identify group with condition / offence (cases)
Identify group without condition / offence (controls)
Both groups compared for exposure to (hypothesized) risk
factors
Greater exposure to risk factor in cases than controls = “causal
relation”
Lead time bias
Recruitment of cases at similar points in time
Newly labelled cases
Selection of Controls
Cases have Theft offences
Controls could be other young people patients or “normals”
Matched Cases & Controls for age & gender
Option of 2 Controls per Case
Smoking years of cases and controls
(matched for age and sex)
Cases
n=456
Smoking yrs 13.75
(± 1.5)
Controls
n=456
6.12
(± 2.1)
F
P
7.5
0.04
Case-Control Study: Other Biases
Recall Bias
Cases > associations with exposures / risk factors
Unreliable Memories
Retrospective nature
Over-reliance on recall
Unreliable Records
Poor hospital records
Repetitive, incomplete, inaccurate, irretrievable, interpretation
Interview Bias
Different interviewers
Cohort Study
ID and examination of a group (cohort)
Followed over time (20 years common!)
Looking for condition development / other end-point
Aetiology of condition (based on data collected)
Data more reliable than case-control studies
• Requires large N
• Requires long follow up
• Inefficient
• Expensive (espec. rare outcomes)
Cohort Study: Methods
Volunteers in 2 groups e.g. exposed vs non-exposed
All complete attitude survey every 12 months
End point at 5 years: groups compared for Health Status
Comparison of general health between users and non-users
of mobile phones
ill
healthy
mobile phone user
292
108
400
non-phone user
89
313
402
381
421
802
Cohort Study: Other Biases
Lost to follow up
Bias if reason related to exposure
Validity affected
Group sizes change
Membership changes e.g ex-smokers
Differential mortality
Change in circumstance
e.g. job change
Exposures need calculation or re-calculation
Surveillance bias
Investigator aware of group membership
Investigating exposed members more
Further Reading
Altman, D.G. “Designing Research”. In: Altman, D.G., (ed.)
Practical Statistics For Medical Research. London, Chapman
and Hall, 1991; 74-106.
Bland, M. “The design of experiments”. In: Bland, M., (ed.) An
introduction to medical statistics. Oxford, Oxford Medical
Publications, 1995; 5-25.
Daly, L.E., Bourke, G.J. “Epidemiological and clinical research
methods”.
In: Daly L.E., Bourke, G.J., (eds.) Interpretation and uses of
medical statistics. Oxford, Blackwell Science Ltd, 2000; 143-201.
Jackson, C.A. “Study Design” & “Sample Size and Power”. In:
Gao Smith, F. and Smith, J. (eds.) Key Topics in Clinical
Research. Oxford, BIOS scientific Publications, 2002.
Further Reading
Jackson, C.A. “Planning Health & Safety Research Projects in
the Workplace”. Croner Health and Safety at Work Special
Report 2002; 62: 1-16.
Kumar, R. Research Methodology: a step by step guide for
beginners. Sage, London 1999.
Abbott, P. and Sapsford. Research methods for nurses and the
caring professions. Open University Press, Buckingham 1988.
Bowling, A. Measuring Health. Open University Press, Milton
Keynes 1994
Polit, D. & Hungler, B. Nursing research: Principles and methods
(7th ed.). Philadelphia: Lippincott, Williams & Wilkins 2003.