Critique and Utilization of Research

Presentation and Discussion of Study Findings

 Findings  Discussion of findings  Conclusions  Implications  Recommendations

Findings

 Facts or empirical data objectively reported (results of the data analysis)  No opinions or reactions  Descriptive statistics  Inferential statistics for hypothesis testing

Presentation of findings

 Narrative and in tables  Tables should be explained in text (not in great detail or table)  Equal attention to findings which do not support hypothesis – test used – degrees of freedom – probability value

Discussion of findings

 More subjective  Researcher gives her interpretation  Interpretation must be within context of ROL (no new literature presented here)  Discuss findings which agree and which do not agree with Hypothesis and previous findings  limitations and problems discussed

Statistical vs. Clinical Significance

 Statistical significance means that the Null Hypothesis is rejected  Clinical Significance means the findings may be useful with patients Remember with larger samples a smaller difference between groups may be statistically significant but not clinically significant

Conclusions

 The researcher’s attempt to show what knowledge has been gained by the study and its generalizability – conservatism is the best approach – somewhat abstract and more general (go beyond the findings)

Implications

 This is the creative section of the research report  Based on the conclusions of the study, what changes could be suggested  Contains the “shoulds” of the study  Suggestions for further research – logical extensions of the study – replication

Explanation of Research Findings to clients

 Beer Therapy Newspaper report of JAMA study concluded that “Drinking three beers a day is about as good as jogging when it comes to producing an effect that may decrease the risk of coronary heart disease” The researchers found that joggers registered the same HDL levels as did sedentary men who drank 3 beers/day for three weeks. Moderate drinking causes the liver to produce the same enzymes as exercise.

Explain this to your patient who asks you if he can now substitute three beers a day for the walking that he had been encouraged to do.

Critique of published research

What is a critique?

A critical estimate of a piece of research which has been carefully and systematically studied by a reader who has used specific criteria to appraise the general features of a research report.

Constructive Criticism

A critique:

 Provides a descriptive report of the study  judges scientific merit  judges ultimate worth and applicability  always precedes utilization

Purposes of critique

 Helps researcher refine, and improve the research  Helps future researchers on the topic  Helps consumers use findings while understanding limitations and constraints

Features of a good critique

 Objective  Comprehensive  Correct  Respectful  Humane  Constructive

Do’s for sensitive critiques

      Try to convey a sincere interest and understanding of the article Be sure to emphasize the points of excellence Choose clear concise statements to communicate your observations (avoid ambiguity) When pointing out a study’s weaknesses include explanations that justify your comments Be aware of your own negative attitudes toward the subject matter or the task Include practical suggestions for improvement of the next study on the same topic

Don’ts for a good critique

      Avoid excessive nitpicking and faultfinding on trivial details Never ridicule or demean the project or researcher Don’t use flattery merely to boost researcher’s self esteem Don’t base your summary and recommendations about the study on some loose and biased attitude toward science, the discipline or the topic Don’t use patronizing or condemning language Don’t forget your purpose in critiquing

Criteria for Good Research

       Clarity and relevance of the problem Researchability of the problem Adequacy and relevance of the literature review Match between the purpose, design and method Suitability of the sampling procedure and the sample Correctness of the Analytical procedures Clarity of the findings

               

Errors to look for in research reports

Problem too large or complex Author not qualified Scholarly format not used, too discursive Objectives/purpose not clearly identified Adjusting the purpose to meet the results Different terms to define the same variable Too much pathophysiology in review of literature Too many quotations in review of literature Paraphrasing author after author without synthesis Tables which have no explanation in the text No tables, charts or graphs Incomplete description of methodology No discussion of informed consent Sample too small or inappropriate for study Instruments not described Overgeneralization

Clarity and Relevance of Study’s Purpose

 Will the study solve a problem relevant to nursing?

 Will the facts collected be useful to nursing?

 Will the study contribute to nursing knowledge?

Researchability of Problem

 Can the research question be answered through measuring empirical evidence or data?

 Is the problem stated as a relationship between at least two variables?

Researchability of the problem

 Is the problem presented early in report?

 Is the problem presented in context of ROL?

 Are the hypotheses explicitly stated?

 Are the concepts operationally defined?

 Are the limitations and assumptions stated?

 Are they justifiable?

 Do the problem statement and title match?

 Are the hypotheses testable and consistent with existing knowledge?



Adequacy and relevance of ROL

Are references logical to subject and method?  Synthesized?

 Organized?

 Classics included?

 Non-supportive work included?

 Justifies operational definitions?

 Supports choice of data collection tools?

Agreement of Purpose, Design and Method  Study design named and described?

 Design answers questions and maximizes control?

 Evidence from literature that design is appropriate?

 Data gathering instrument included?

 Validity and reliability reported?

 Checks against data collection errors?

 Consistency of research conditions for all subjects?

 Blind or double blind possible?, desirable?

Suitability of sampling and sample          Probability or non-probability? Why?

Biased sample avoided?

Representative?

Large enough for test? To reduce error?

Too large?

Demographics of sample reported? Match target pop?

Sampling criteria stated and appropriate?

Informed consent and rights of subjects?

Attrition? Why?

Correctness of Analysis

 Statistical tests named, p level?

 Explanation and analysis of qual. Data?

 Tests appropriate to level of measurement?

 Distinction between clinical and statistical significance?

 Right statistical tests?

           

Clarity of findings

Interpretations based on data?

Reasons for ways data presented?

Error in computations?

Discrepancies between tables and text?

Tables and graphs titled?

Actual findings and interpretations distinguished from each other?

Interpretations and implications justified?(N?) Secondary findings overemphasized?

Clearly and logically organized?

Impartial and unbiased?

Generalizations appropriate?

Negative outcomes and limitations?