Transcript Guidelines for Critically Reading the Medical Literature John L. Clayton, MPH

Guidelines for Critically
Reading the Medical
Literature
John L. Clayton, MPH
Purpose of the Study
• What are the objectives in performing
the study?
• What is (are) the research questions
being asked?
• Why was this study conducted?
Study Design
• What type of study was performed?
–
–
–
–
Experimental vs. observational
Prospective vs. retrospective
Cross-sectional vs. longitudinal
Ecological (e.g., vital statistics, mortality trends)
• If you know the study design, you know
which types of analyses are appropriate,
and can be looking for the appropriate test
statistics
Study Design
• How was the population or sample
selected?
• Are there possible sources of bias in
selecting the sample?
– Are there factors that might make the
sample unrepresentative?
Study Design
• Was there a control group?
– How was it selected?
• Was randomization or matching used?
– If so, on what characteristics were subjects
matched?
Study Design
• What were the inclusion criteria used to
choose the study sample?
– Was the definition of a “case” appropriate?
• After identification of the sample, were
there exclusions?
– If so, who was excluded, when were they
excluded, and why were they excluded?
Measurement and Observation
• Are there clear definitions of terms
used, including diagnostic categories,
drugs or compounds used, and
interventions performed?
• Were the outcome criteria defined prior
to examination of the data?
Measurement and Observation
• What was measured during the study?
• How were the measurements made?
• Were the classification criteria and
measurement processes capable of
meeting the study objectives?
Measurement and Observation
• Were the categories and processes
applied consistently for all subjects?
• Are the observations and
measurements reliable (reproducible)?
• What if anything, was done to assess
their reliability?
Presentation of Data
• How are the data presented?
– Tables, charts, graphs, etc.
• Which measures of association were
used?
– Relative risks, odds ratios, correlation
coefficients, etc.
Presentation of Data
• Are results presented clearly and in
enough detail so that the reader is able
to judge them for him/herself?
• Are the data which are presented
relevant to the purpose of the study?
• Are there data which are not presented
which should have been?
Presentation of Data
• Are the data consistent with each other?
– Are the values in one table consistent with
those in another?
– Do the numbers add up appropriately?
– Is everybody accounted for?
Analysis of the Data
• Are the data worthy of analysis?
• What methods of statistical analysis, if
any should have been performed?
• Were the assumptions for various
statistical tests met?
Analysis of the Data
• Was the analysis strategy appropriate
for the study design?
• Were potential confounding variables
taken into account?
• Was the chosen analysis strategy able
to answer the original hypotheses?
Conclusions
• What conclusions did the author(s) reach?
Are they justified?
• Are they generalizable? If so, to what
population?
• What conclusions do you reach based on
the available evidence?
Conclusions
• Would you like more information about the
study in order to reach a conclusion? If so,
what?
• Are the conclusions reached by the authors
relevant to the purpose of the study?
• Do the authors make any recommendations
based on the results of the study? If so, are
they justified? What recommendations would
you make?
General Discussion
• Was the study ethical?
• If (are) the research question(s) being
addressed worthy of being studied?
• What additional research should be
done in order to answer the questions
addressed by this study?
General Discussion
• What type of study design should be
used? What type of analysis strategy
should be used?
• Why was this paper selected for
discussion?
References
• Colton, T., “Critical Reading of the Medical
Literature,” in Statistics in Medicine, Chapter 13,
Boston: Little, Brown and Company, 1974.
• Murphy, E.A., “An exercise in Qualitative
Criticism,” In The Logic of Medicine, Chapter 17,
Baltimore: The Johns Hopkins University Press,
1976.
• R.K. Riegelman, Studying a Study and Testing a
Test: How to Read the Medical Literature, Boston:
Little, Brown and Company, 1981.