Transcript Getting Started: What’s Out in the Literature and Is It

Getting Started: What’s Out in
the Literature and Is It Any Good?
SUSAN B. FOWLER, PHD, RN, CNRN, FAHA
Literature
Research Critique
 Is not synonymous with “criticize”
 Examines the strengths, weaknesses, meaning, and
significance of the study (substantive and theoretical
dimensions)
 Be objective and realistic in identifying the study’s
strengths and weaknesses
Important Points
 All studies have weaknesses or flaws
 Research is critiqued to broaden understanding,
improve practice, and provide background for
conducting a study
 The critique process involves comprehension,
comparison, analysis, and evaluation
Cochrane Systematic Reviews
 Cochrane Library
www3.interscience.wiley.com/cgibin/mrwhome/106568753/HOME
The Cochrane Collaboration. Oxford: Update Software;
1996-. Updated quarterly.
 Cochrane Database of Systematic Reviews
(Cochrane Reviews)




'Gold Standard' for high-quality systematic reviews
Full-text included in Cochrane Library
Cochrane Reviews includes complete reviews and protocols
(reviews that are still in progress)
Cochrane Reviews abstracts are in PubMed
More Reviews
 Database of Abstracts of Reviews of Effects
(Other Reviews - DARE)






prepared by the National Health Service Centre for Reviews
and Dissemination, University of York, England
Complements the Cochrane Reviews by offering a selection of
quality assessed reviews in those subjects where there is
currently no Cochrane review
Brief critical appraisals of previously published reviews of the
effects of health care
Structured abstracts, not full-text
DARE not indexed in PubMed, but original research articles
may be
Also available at no charge on the web from University of York
| www.crd.york.ac.uk/crdweb/
Other Systematic Reviews and Meta-analyses
 PubMed MEDLINE - Systematic Reviews |
www.ncbi.nlm.nih.gov/entrez/query/static/clinical.shtml





access: part of PubMed's Clinical Queries (linked on PubMed Home
and Advanced Search pages)
Identifies systematic reviews and other similar types of studies
found in biomedical journals included in PubMed database
No evaluation of comparative quality of different reviews on a topic
Abstracts supplied by authors/journals. No separate evaluation of
quality of research
Identify systematic reviews in the biomedical area - gathers together
much larger collection than other evidence-based practice resources
Evidence Guidelines/Summaries
 BMJ Clinical Evidence | www.clinicalevidence.com/
 Compendium of evidence on the effects of clinical interventions
 Summarizes the current state of knowledge, including knowns and unknowns,
based on thorough search
 Categorizes interventions as beneficial, likely beneficial, no known benefit,
harmful ...
 DynaMed | www.ebscohost.com/dynamed
 Best available evidence summaries for nearly 2000 topics
 Clinical reference tool developed for use at the 'point-of-care'
 Outline format to quickly identify key conclusions
 Updated daily
 Other Resources
 USPSTF Guidelines | www.ahrq.gov/clinic/prevenix.htm
 AHRQ Evidence Reports | www.ahrq.gov/clinic/epcix.htm
 FPIN Clinical Inquiries included in Journal of Family Practice |
www.jfponline.com and American Family Physician |
www.aafp.org/afp/
RCTs, Case Cohorts, Control Studies
 PubMed | (pubmed.gov )
 note: Filtered by publication type, but not quality
 Clinical Queries - Uses preconfigured search strategies to retrieve
research-based citations on clinical topics in the areas of therapy,
diagnosis, etiology, or diagnosis
 Use Limits to search by specific publication type (e.g. meta-analysis,
randomized controlled trial, review)
 Cochrane Library | www3.interscience.wiley.com/cgi-
bin/mrwhome/106568753/HOME
The Cochrane Collaboration. Oxford: Update Software; 1996-.
Updated quarterly.

Cochrane Central Register of Controlled Trials (Clinical Trials)


Bibliography of controlled trials as part of an international effort to hand search
the world's journals
Includes reports published in conference proceedings and in many other sources
not in PubMed or other bibliographic databases
Content of Research Reports
 The Abstract
 The Introduction (review of the literature, theoretical




framework, significance of and need for the study)
The Methods Section
The Results Section
The Discussion Section
The References
The Beginning
 The title
 Could you tell what the article was about by reading the title?
 The abstract
 Includes a brief description of the problem
 How the study was done (methodology)
 The results
 Conclusions
Next Steps
 The Problem
 Is it clear what the problem the authors are trying to solve?
 Review of the Literature
 Do the articles relate to the problem?
 Is it organized, broad to specific?
 Does it tell a story?
 Comprehensive and current? What years are covered in the ROL?
 Research Question/Aim/Purpose/Hypothesis
 Do authors pose a research question or hypothesis?
 Is it explicitly stated what the aim or purpose of the study is?
 Design
 Is the design stated?
 Does the design flow from the research question or aim?
 Did the authors explain why they choose this research design?
Design
Quantitative
Qualitative
 Analysis of numerical
 Analysis of data such as
data





Experimental
Quasi-experimental
Comparative
Longitudinal
Correlational
words (e.g., from
interviews), pictures
(e.g., video), or objects
(e.g., an artifact)




Grounded theory - social
processes
Phenomonology – lived
experience
Ethnography – culture
Historical
The Journey Continues
 Sample
 Is the population described?
 Is the sample method, or how the sample was chosen
described?
 Is the sample size right for the analysis? Was a power analysis
done to determine sample size?
 Were standards for protection of human subjects discussed?
 Tool
 Was the tool created by the authors or already established?
 Is a copy of the tool included in the article?
 Is reliability discussed?
 Is validity discussed?
Tool, Instruments, Questionnaires, and More
Validity
Reliability
 Extent to which the
 Results are repeatable
tool measures what it
was intended to



Content
Criterion related
construct
and consistent



Internal consistency
Test-retest
Inter-rater
Power
 The probability that a statistical test will detect a




significant difference that exists - the risk of a Type I
error can be calculated using power analysis.
Level of significance
Sample size
Power - acceptable level is .80
Effect size - the degree to which the null hypothesis
is false
Effect Size
 An estimate of how large the treatment effect is, that
is how well the intervention worked in the
experimental group compared to the control group
(intervention studies)
 The larger the effect size, the stronger are the
experimental intervention’s effects.
 Effect size for intervention studies:



.2 = small effect
.5 = medium effect
.8 = large effect
Getting Closer to the End
 Methodology
 Are methods of data collection sufficiently described?
 Is the time frame when the study occurred described?
 Data Analysis
 Is information presented sufficient to answer the research
question(s)?
 Were statistical tests used to analyze the data?
 Were values obtained from the analysis?
 Was statistical significance reported?
 Are the results explained?
 Are tables and figures easy to understand and informative?
Confidence Intervals
 Probability that a value will
fall within a range of
variables
 The larger the CI the less
precise the measurement of
that variable
 A very wide interval may
indicates that more data
should be collected before
anything very definite can
be said about the
parameter
 95% CI most common
The End
 Discussion
 Is a discussion section presented?
 If yes, are the results compared with the literature review?
 Conclusions
 Are conclusions clearly stated?
 Are conclusions directly related to the results?
 Do the findings add to the present nursing knowledge?
 Are study limitations identified?
 Did authors make recommendations for further research?

Carlson, J. (1999). J Emerg Nurs, 25, 330-332.
References
 Relevance (extent to which the reference bears on the
research question)
 Primary sources (descriptions of studies written by
the researchers)
 Secondary sources (descriptions of studies written by
someone other than the original researcher)
 Opinion and anecdotal
Level of Significance
 The researcher does not know when an error in
statistical decision making has occurred. The
researcher can control the risk of making a Type I
Error by setting the level of significance.
 Level of Significance (alpha level) is the probability
of making a Type I Error or the probability of
rejecting a true null hypothesis.
Level of Significance
 Usual level set at .05 (willing to accept the fact that if
the study were carried out 100 times, the decision to
reject the null hypothesis would be wrong 5 times)
 If researcher wants to have smaller risk of falsely
rejecting null hypothesis, set at .01.
 Decreasing risk of making Type I Error increases risk
of Type II Error.
 Findings are significant or not significant.
Clinical Significance
 Related to practical importance of the
findings
 No common agreement in nursing about
how to judge clinical significance


Effect size?
Difference sufficiently important to warrant
changing patient care?
Clinical Significance
 Who should judge clinical significance?
 The patients and their families?
 The clinician or researcher?
 Society at large?
 Clinical significance is ultimately a value judgment
Research Journals
 Examples of peer reviewed / refereed journals: Nursing
Research, Advances in Nursing Science, Applied
Nursing Research, Clinical Nursing Research, Western
Journal of Nursing Research
 Example of Peer Reviewed online resource: Medscape
 Peer Review – Look for statement in first few pages of
journal that it is peer reviewed. Content is “reviewed” by
experts in field before publication
Let’s Get to Work – Critique in Action!!