Transcript Slide 1

Critical Reading
Critical Appraisal
Definition:
quality
assessment of methodological
If you are deciding whether a paper is worth
reading – do so on the design of the methods
Need to know two things:
• What type of study was done?
• Was the study design appropriate?
What type of study?
Primary – these report research first hand.
• Experimental i.e. humans, animals;
artificial and controlled surroundings.
• Clinical trials – intervention offered.
• Survey – something is measured in a
group.
What type of study?
Secondary – summarise and draw conclusions
from primary studies.
• Overview
– Non systematic (summary)
– Systematic (rigorous and pre-defined methodology)
– Meta-analyses (integration of numerical data from
more than one study)
• Guidelines (leads to advice on behaviour)
• Decision analyses (to help make choices for
doctor or patient)
• Economic analyses (i.e. is this a good use of
resources?)
Specific Types of Study
Randomised Controlled Trial (RCT)
• Population is randomly allocated to two groups
• One group is given a specific treatment or
intervention
• On average the groups are identical because
they are randomised and therefore any
difference in the measured outcome is due to
the intervention
• Specified follow up period and specified
outcomes
• e.g. drug better than placebo; surgical
procedure compared with sham
Randomised Controlled Trial
(RCT)
Advantages
• Allows rigorous evaluation of a single variable in
a previously defined population e.g. a new drug.
• Prospective i.e. collect the information after you
decide to do the study.
• Tries to disprove the null hypothesis
• Tries to eradicate bias because the two groups
are identical.
• Allows for meta-analysis later.
Randomised Controlled Trial
(RCT)
Disadvantages
• Expensive and time consuming which can lead to
problems including:
• Too few subjects
• Too short a time
• Who controls the study?
• End point not clinical
• Possibility of hidden bias:
• Imperfect randomisation
• Failure to randomise all eligible patients – who is
included/excluded.
• Assessors not blinded.
Cohort study
• Two (or more) groups of people are selected on a
basis of a difference in exposure to a particular
agent i.e. vaccine, environmental toxin, medicine.
• Group followed up (usually for years) to see how
many in each group develop a particular
disease/outcome.
• e.g. Peto– 40,000 UK doctors.
• e.g. COCP causes breast cancer?
Case Control Study
• Patients with a particular disease are identified
and “matched” with controls.
• Data is collected retrospectively either from
medical records or from memory, looking for a
causal agent.
• Looks for associations but not necessarily the
same as cause.
• e.g.SIDS and sleeping position.
• Does whooping cough vaccine cause brain
damage?
• Do overhead cables cause leukaemia?
Cross Sectional Survey
• A representative sample of subjects or patients
are studied (interviewed, questionaired,
examined) to answer a specific clinical question
at a specific time.
• e.g. normal height of three year olds
• what do most GP’s think about the use of
Viagra?
Case Reports
• Medical history of a single patient in a story
form.
• Lots of information given which may not be
seen in a trial or a survey.
• Often written and published fast compared
to studies
• e.g. Thalidomide
Importance of ethics
Hierarchy of Evidence
(Systematic Review and Meta-analysis)
Randomised Controlled Trial
Cohort Studies
Case Control Studies
Cross Sectional Surveys
Case Reports
Was design appropriate?
In general:
• Therapy – i.e. effect of intervention – RCT
• Diagnosis – ? test valid (can we trust it) or
reliable (? same result if repeated) – cross
sectional survey with both gold standard and
new test
• Screening – large population, pre-symptomatic –
cross sectional survey
• Prognosis – i.e. what happens to someone if a
disease is picked up at an early stage –
longitude cohort study
• Causation – e.g. ? possible harmful agent leads
to cause – cohort or case control study
- ? case report.
Assessing Methodological
Quality
Questions to Ask
• general framework
• specifics dependant on type of paper
Logical Progression
Introduction
Methods
Results
Discussion
- Title
- Abstract
- Introduction
(Statistics!)
Seven essential questions:
Introduction
1. Why was the study done?
• Is the study original or does it add to the
literature in any way? e.g. bigger, better,
larger, more rigorous
• Is it interesting?
• Is there a clear research question?
Why was the study done?
i.e. what was the key research question/
what hypotheses were the author
testing?
Hypothesis is usually presented in the
negative – the
“null hypothesis”
Studies try to disprove this lack of
difference or null hypothesis.
Seven essential questions:
Methods
2. What type of study? Who is it about?
• How recruited?
• Who included?
• Who excluded?
• Studied in “real life circumstances”?
• Applicable?
Seven essential questions:
3. Was it well designed?
•
•
•
•
•
•
i.e. does the study make sense?
What specific intervention or manoeuvre
was being considered and what was it
being compared to?
Is what happened what the author said
happened?
What outcome was measured and how?
i.e. length of life, quality of life, reduction
in pain
need to be objective.
Seven essential questions:
4. Was systematic bias avoided?
•
i.e. was it adequately controlled for?
[ Bias = anything that erroneously influences
the conclusions about groups and distorts
comparisons
e.g. RCT – method of randomisation,
assessment ? truly blind.
Cohorts – population differences
Case control – true diagnosis, recall (and
influences) ]
Seven essential questions:
5. Was it large enough and long enough to
make results credible?
•
Size is important!
•
Power!
Seven essential questions:
Results
6. What was found?
• Should be logical – simple
complex
Seven essential questions:
Discussion
7. What are the implications?
• For:
- you
- practice
- patients
- further work
• and do you agree?
Four possible outcomes from
any study
1. Difference is clinically and statistically
significant i.e. important and real.
2. Of clinical significance but not
statistically so. ?sample size too small.
3. Statistically significance but not
clinically i.e. not clinically meaningful.
4. Neither clinically nor statistically
significant.
Recommended Reading
Ian Crombie : The Pocket Guide to Critical
Appraisal
Trish Greenhalgh : How to read a paper;
the basis of evidence based medicine