Transcript Slide 1

Critical Reading
Critical Appraisal
Definition:
assessment of methodological quality
If you are deciding whether a paper is worth reading –
do so on the design of the methods
The specific questions to be asked in the methods
section are:
• Why was the study done?
• What type of study was done?
• Was the study design appropriate?
Why was the study done?
i.e. what was the key research question/ what
hypotheses were the author testing?
Hypothesis is usually presented in the
negative – the
“null hypothesis”
Studies try to disprove this lack of difference
or null hypothesis.
Small Groups
• 15 minutes
• Appoint feedback person
• List the different types of study you have
heard of
• Describe them – give an example
• Advantages & disadvantages
What type of study?
Primary – these report research first hand.
• Experimental i.e. humans, animals; artificial
and controlled surroundings.
• Clinical trials – intervention offered.
• Survey – something is measured in a group.
What type of study?
Secondary – summarise and draw conclusions from
primary studies.
• Overview
– Non systematic (summary)
– Systematic (rigorous and pre-defined
methodology)
– Meta-analyses (integration of numerical data from
more than one study)
• Guidelines (leads to advice on behaviour)
• Decision analyses (to help make choices for doctor or
patient)
• Economic analyses (i.e. is this a good use of
resources?)
Specific Types of Study
Randomised Controlled Trial (RCT)
• Population is randomly allocated to two groups
• One group is given a specific treatment or
intervention
• On average the groups are identical because
they are randomised and therefore any
difference in the measured outcome is due to
the intervention
• Specified follow up period and specified
outcomes
• e.g. drug better than placebo; surgical
procedure compared with sham
Randomised Controlled Trial (RCT)
Advantages
• Allows rigorous evaluation of a single variable in a
previously defined population e.g. a new drug.
• Prospective i.e. collect the information after you
decide to do the study.
• Tries to disprove the null hypothesis
• Tries to eradicate bias because the two groups are
identical.
• Allows for meta-analysis later.
Randomised Controlled Trial (RCT)
Disadvantages
• Expensive and time consuming which can lead to
problems including:
• Too few subjects
• Too short a time
• Who controls the study?
• End point not clinical
• Possibility of hidden bias:
• Imperfect randomisation
• Failure to randomise all eligible patients – who is
included/excluded.
• Assessors not blinded.
Cohort study
• Two (or more) groups of people are selected on a
basis of a difference in exposure to a particular agent
i.e. vaccine, environmental toxin, medicine.
• Group followed up (usually for years) to see how
many in each group develop a particular
disease/outcome.
• e.g. Peto– 40,000 UK doctors.
• e.g. COCP causes breast cancer?
Case Control Study
• Patients with a particular disease are identified and
“matched” with controls.
• Data is collected retrospectively either from medical
records or from memory, looking for a causal agent.
• Looks for associations but not necessarily the same
as cause.
• e.g. SIDS and sleeping position.
• Does whooping cough vaccine cause brain damage?
• Do overhead cables cause leukaemia?
Cross Sectional Survey
• A representative sample of subjects or patients are
studied (interviewed, questionaired, examined) to
answer a specific clinical question at a specific time.
• e.g. normal height of three year olds
• what do most GP’s think about the use of Viagra?
Case Reports
• Medical history of a single patient in a story
form.
• Lots of information given which may not be
seen in a trial or a survey.
• Often written and published fast compared to
studies
• e.g. Thalidomide
Importance of ethics
Hierarchy of Evidence
(Systematic Review and Meta-analysis)
Randomised Controlled Trial
Cohort Studies
Case Control Studies
Cross Sectional Surveys
Case Reports
Was design appropriate?
In general:
• Therapy – i.e. effect of intervention – RCT
• Diagnosis – ? test valid (can we trust it) or reliable (?
same result if repeated) – cross sectional survey
with both gold standard and new test
• Screening – large population, pre-symptomatic –
cross sectional survey
• Prognosis – i.e. what happens to someone if a
disease is picked up at an early stage – longitude
cohort study
• Causation – e.g. ? possible harmful agent leads to
cause – cohort or case control study
- ? case report.
Assessing Methodological Quality
Questions to Ask
• general framework
• specifics dependant on type of paper
Logical Progression
Introduction
- Title
- Abstract
- Introduction
Methods
Results
(Statistics!)
Discussion
Seven essential questions:
Introduction
1. Why was the study done?
• Is the study original or does it add to the
literature in any way? e.g. bigger, better, larger,
more rigorous
• Is it interesting?
• Is there a clear research question?
Seven essential questions:
Methods
2. Who is it about?
• How recruited?
• Who included?
• Who excluded?
• Studied in “real life circumstances”?
• Applicable?
Seven essential questions:
3. Was it well designed?
•
•
•
•
•
•
i.e. does the study make sense?
What specific intervention or manoeuvre was
being considered and what was it being
compared to?
Is what happened what the author said
happened?
What outcome was measured and how?
i.e. length of life, quality of life, reduction in
pain
need to be objective.
Seven essential questions:
4. Was systematic bias avoided?
•
i.e. was it adequately controlled for?
[ Bias = anything that erroneously influences the
conclusions about groups and distorts
comparisons
e.g. RCT – method of randomisation, assessment
? truly blind.
Cohorts – population differences
Case control – true diagnosis, recall (and
influences) ]
Seven essential questions:
5. Was it large enough and long enough to
make results credible?
•
Size is important!
Seven essential questions:
Results
6. What was found?
• Should be logical – simple
complex
Seven essential questions:
Discussion
7. What are the implications?
• For:
- you
- practice
- patients
- further work
• and do you agree?
Four possible outcomes from any study
1.
2.
3.
4.
Difference is clinically and statistically
significant i.e. important and real.
Of clinical significance but not statistically
so. ?sample size too small.
Statistically significance but not clinically
i.e. not clinically meaningful.
Neither clinically nor statistically
significant.
Recommended Reading
Ian Crombie : The Pocket Guide to Critical
Appraisal
Trish Greenhalgh : How to read a paper; the
basis of evidence based medicine