Transcript Document

Scientific Reviewing for Physicians:
a CLEAR Approach
John Iskander MD MPH
CAPT, U.S. Public Health Service
Senior Medical Consultant
Office of the Associate Director for Science
Centers for Disease Control and Prevention
“What do you read, my Lord?”
“Words, words, words…”
-Hamlet, William Shakespeare
Outline
•Importance of scientific reviewing
•Types of scientific reviews
•The CLEAR framework
•Categorizing comments
•Skill-building, resources, and training
•Questions
Importance of Scientific Review
•Essential to the scientific enterprise
•Building scientific credentials
•Author, reviewer, editorial board member
•Collaborating with colleagues
•Supporting science and the PHS mission
•Little or no formal instruction as part of medical training
•An underappreciated aspect of scientific leadership
•Not much in the “peer reviewed” literature on how to do
a peer review!
USPHS Mission
• The mission of the U.S. Public Health Service
Commissioned Corps is to protect, promote, and
advance the health and safety of our Nation. As
America's uniformed service of public health
professionals, the Commissioned Corps achieves its
mission through:
• Rapid and effective response to public health
needs
• Leadership and excellence in public health
practices
• Advancement of public health science
Basis for this Talk
•“Practice-based evidence” building on experience of
serving as a peer reviewer, clearance reviewer, and
policy reviewer
•OADS fellowship, scientific reviewing curriculum
•Short-term rotations support the professional development of
a scientist interested in pursing a career track in the science
leadership, management and policy field
•Discussions with and feedback from colleagues
Types of Scientific Review
•Subject matter expert (SME)
•Example: a peer review requested by a journal editor (or
internally by a colleague)
•Is content scientifically valid, of importance, novel?
•Policy review
•Example: clearance or agency approval
•Is content consistent with policies and programs?
•Editorial review
•Example: Pre-publication review of an article or book
chapter by editor
•Can clarity be improved without major changes to
content?
Typical 3-Level Comment Framework
•Level 1 or mandatory
•Example: serious errors of fact or data interpretation,
misstatement of agency recommendation or policy
•Level 2 or “revise or respond”
•These should be revised to improve understanding, but
the authors can explain why revision is not possible (e.g.
data to answer question is not available)
•Level 3 or editorial
•Often presented as inline edits
•If any such changes are mandatory, make sure this is
clearly labeled
Before You Start
•Clearly understand:
•Your role, the type of document you are reviewing, and
the type of review requested
•“Who are you, and what is this?”
•Respect the effort that went into creating the document
•Meet deadlines
The CLEAR Framework
•Clarity
•Logic
•Ethics
•“Agency”
•Relevance
What is Clarity?
“Omit needless words.”
-Strunk and White, The Elements of Style
• Shorter sentences, shorter (or no) parenthetical
statements
• Consistent “voice”
• Do ideas presented conflict with each other or appear
contradictory?
What is Logic?
• Flow of ideas
• Being able to follow an article’s “argument”
• Interpretation of findings that is in agreement with
data presented
• Reasonable inferences drawn by authors about
explanations for findings
• Should hold both within paragraphs and sections,
between them, and for the document as a whole
What is Ethics?
•Not only whether there is documentation of IRB
clearance and/or clinical trial registration, but also
whether readers will perceive an ethical issue with the
work that was done
•Vulnerable populations?
•Informed consent?
•Privacy/confidentiality concerns?
What is “Agency”?
•Dual meaning:
•Who is making the recommendations? Individuals,
agency?
•In other words, who is the “agent”?
•Can the agency stand by the science and policy
contained in the work?
•Of special importance for policy reviews
What is Relevance?
•“Information for action”
•What are the authors asking for-awareness of emerging
or rare conditions, use of new diagnostic criteria or
laboratory tests, reporting to public health authorities,
some or all of these?
•Practical value to clinicians
•Rather than saying, “clinicians should look out for this
incredibly rare disease” say “for patients with risk factors,
this disease should be considered and the following
laboratory tests may be obtained”
What Makes a Good Review?
•Feedback that is constructive, specific, and can be
acted on by the author to improve some aspect of
CLEAR
•Not:
“This paragraph needs to be tightened up a little bit”
•Instead:
“The second and third sentences can be eliminated as
they duplicate information presented elsewhere”
What Makes a Good Review?
•A brief (2-3 sentence) summary of the review may be
appropriate
•Might be only for your use, or may be shared with
authors and/or editors
•Crystallizes your thinking, and points out “big picture”
issues
•Do not rewrite extensively unless you are co-authoring
or editing
•Recommending alternative language for a sentence or
part of a sentence often appropriate (choose your words
carefully-you may see them in print)
What Makes a Good Review?
•Be very clear about what must be changed, as
opposed to what are recommendations for improvement
•The “three level” comment structure is one tool to
facilitate this distinction
•Inline edits vs. marginal comments
•Communication
•With author, editor, and/or colleague
Tips and Tricks
•“Three paragraph” rule to see if document is ready for
full review
•Nominalizations
•Strong and weak sentence parts
•Flipping back and forth-usually not a good sign
•Watch for “buried leads”-important ideas or statements
that are hard to find within the document
Tips and Tricks
•For internal reviews, consulting with the author during
review may be appropriate and helpful
•Reviewing and revising your own work-”sleep on it”
•Two stage review-read, then either re-read or review
your comments
•Reviewing prior comments (yours or those of others)
can be helpful
•Don’t repeat comments unnecessarily
Scientific Review Case Study
Review Case Study
•Document: 2012 Surgeon General's Report—
Preventing Tobacco Use Among Youth and Young
Adults (1395 pages, 7 chapters)
•Type of review: policy
•Timeframe for review: 2 weeks
•Methods:
•Multiple reviewers due to length of document
•Discussion with authors during review process to assure
that highest priority comments were addressed
•Consolidation and adjudication of all comments
•If multiple reviewers involved, must “speak with one voice”
Resources for Review
•Medline/PubMed (www.ncbi.nlm.nih.gov/pubmed)
•Example use: does a reference really say what the
authors assert they say? Often can be assessed by
reviewing the abstract
•Clinical Trials registry (www.clinicaltrials.gov)
•Most journals require clinical trials to be registered here
•Agency or other medical libraries (and librarians)
Building Experience as a Reviewer
•How do you get to Carnegie Hall, or become a good
reviewer?
•Practice, practice, practice
•Volunteer to review abstracts for COA or other
scientific meetings
•Serve as a peer reviewer for journals, including Public
Health Reports
•Review colleague’s manuscripts
•Write abstracts, reports, columns, journal articles,
articles for COA Frontline etc.
What’s In It for Me?
•Some journals (e.g. Annals of Internal Medicine) offer
continuing education (CE) credit to reviewers
•Some journals “credit” editors as part of the final
publication
•In some agencies, scientific review may be a part of
leadership responsibilities
•Associate Director for Science, Senior Scientist
•Building scientific and public health fund of knowledge,
in or outside of your specialty
The Lighter Side of Being a Reviewer
•“Employment is a major cause of morbidity and
mortality.”
•“A limitation of this survey is that it is prone to hat
misclassification bias.”
•“We blindfolded reviewers…”
Acknowledgments
•Mary Ari
•James Stephens
•Rick Peavy
•LT Neelam Ghiya
Office of the Associate Director for Science