Transcript Slide 1

Reviewing – The Basics
Toby Charkin – Executive Publisher
SfN - October 2012
What is Peer Review?
Peer Review has 2 key functions:
• Improves the quality of the
research submitted for
publication by giving
reviewers the opportunity to
suggest improvements
Pre-Submission
Peer Review
Authors
• Acts as a filter by ensuring
only good research is
published. Helps to
determine validity,
significance and originality
Reviewers
Editor
Publication
• Importance and clarity of
research hypothesis
• Originality of work
• Delineation of strengths and
weaknesses of methodology,
experimental / statistical
approach, interpretation of
results
• Writing style and figure /
table presentation
• Ethics concerns (animal /
human)
“ Novelty”
Issues to Review
“ Technical” Quality
Purpose of Peer Review
Check the
manuscript
for
4
•
•
•
•
Mistakes in procedures or logic
Conclusions not supported by the results
Errors or omissions in the references
Compliance with ethics standards
– Has the protocol been approved by an appropriate Ethics
Committee?
• Originality and significance of the work
Quality of the Work
Are the methods appropriate and presented in sufficient detail to allow the
results
to be repeated?
Are the data adequate to support the conclusions?
Methods
Results
1. Do all “methods” have a
“results”?
2. Have all “results” been
described in the “Methods”?
5
Conclusions
1. Are all “conclusions” based
on “results”?
Presentation of the Paper
6
Writing
• Clear, concise, good English?
• But no need for reviewers to act as language editor
Title
• Specific, and reflecting the content of the
manuscript?
Abstract
• Brief, and describing the purpose of the work, what
was done, what was found, and the significance?
Figures
• Justified? Clear? Sharp, with fonts proportionate to
the size of the figure? Clear and complete legends?
Tables
• Can they be simplified or condensed? Should any
be omitted?
Trade names,
abbreviations,
symbols
• Properly used where indicated? Abused?
Comments to the Editors
• Comment on novelty and significance
• Recommend whether the manuscript is suitable for publication or not,
usually
– Accept / Minor revision / Major Revision / Reject
Reviewer makes a
recommendation
Editor makes
the decision
• Confidential comments will not be disclosed to author(s)!
7
Comments to the Authors
• Provide specific comments on the design,
presentation of data, results, discussion, and
conclusions
– Do not include recommendations for acceptance / rejection
– Require or suggest other experiments or analyses
– Distinguish between “needs to change” and “nice to
change
– Use neutral language and be constructive
• Ensure that that the comments to the author(s) are
consistent with your recommendation to the editors
Privileged Document
This manuscript is a confidential document. The data
• Are and remain the exclusive property of the authors
• Should not be disclosed to others (who may use the information in their
research)
If you have printed the manuscript
• It must be kept confidential until the review process has been completed
• After final decision by the editor it must be destroyed
If you have shared responsibility for the review of this manuscript with a
colleague, you should provide that person’s name and affiliation to the editors
9
Reviewers
Should only accept to review manuscripts
• In their areas of expertise
• When they can complete the review on time
Should always avoid any conflicts of interest
• If in doubt, consult with the editor
Are not allowed to “use” the data
Must provide an honest, critical assessment
Must analyze the strengths and weaknesses of the research,
and provide specific suggestions for improvement
10
Oversight Function: Ethics
The reviewer also has the (unpleasant) responsibility of reporting
suspicion of
• Duplicate publication
• Plagiarism
– Also self-plagiarism
• Data fabrication or falsification
• Ethics concerns
• Etc.,
These problems are normally followed up by the Editors and the
Publisher
11
Review Process (i)
Regular articles are initially reviewed by at least two reviewers
When invited, the reviewer receives the Abstract of the manuscript
The editor generally requests that the article be reviewed within two weeks
• Limited extensions sometimes acceptable
Articles are revised until the two reviewers agree on either acceptance or
rejection, or until the editor decides that the reviewer comments have been
addressed satisfactorily
The reviewers’ reports help the Editors to reach a decision on a submitted
paper
• The reviewer is the recommends; the editor decides!
12
Review Process (ii)
If a report has not been received after 4 weeks, the Editorial office contacts
the reviewer
If there is a notable disagreement between the reports of the reviewers, a
third reviewer may be consulted
The anonymity of the reviewers is strictly maintained
• Unless a reviewer asks to have his/her identity made known to the authors
13
Review Process (iii)
• Reviewers must not communicate directly
with authors
• All manuscripts and supplementary material must
be treated confidentially by editors and reviewers
– The manuscript cannot be distributed outside
this small group
As author
As editor
As reviewer
As reader
• The aim is to have a “first decision” to the authors
within 4-6 weeks after submission of the
manuscript
• Meeting these schedule objectives requires a
significant effort on the part of the Editorial staff,
Editor and Reviewers
• If reviewers treat authors as they
themselves would like to be treated as
authors, then these objectives can be met
14 January 2012
As a researcher,
you wear many
hats!
Take Home Message


The best reviewers tend to view themselves as
teachers and mentors rather than critics !
January 2012
New initiatives
How can we support and improve
the peer-review process to the
benefit of Editors, authors and
reviewers?
16
Tools for Reviewers
For Editors
• Plagiarism detection tool at time of submission
– Checks the manuscripts against the “entire” published
literature
• “Find a Reviewer” tool, based on “Scopus” database
• Free access to ScienceDirect
– All content published by Elsevier
For
Reviewers
• Free access to Scopus
– The world's largest abstract and citation database of peerreviewed literature and quality web sources with smart tools
to track analyze and visualize research
• Reference-linking and resolution in PDF of the manuscript
17
Peer Review Grand Challenge
March to May 2012 - web-based Challenge invited
submissions on any aspect that could significantly add to the
current peer-review system.
To explore how publishers and Editors can help early career
researchers become reviewers, or how reviewers can be
recognized by either their institutes or publishers.
Winner: Elsevier Reviewer Badges and Rewards scheme by
Simon Gosling (a tangible recognition system for reviewers –
e.g. Most Comprehensive Review”, another for the “Most
Helpful Review”, and one for the “Most Contributions in a
Year)
For more information on this initiative, please contact Clare
Lehane, Executive Publisher, STM Publishing,
[email protected] or visit www.peerreviewfuture.com
18
Neuroscience Peer Review
Consortium
Authors whose papers are not
accepted by one journal in the
Consortium and wish to submit
their manuscripts to a second
participating journal can request
that the previous set of reviews be
forwarded.
Reduce the number of times a
manuscript must be reviewed - it
reduces the workload on
reviewers and speeds up
publication time.
http://nprc.incf.org/
Many Neuroscience journals are
part of this initiative, including
Elsevier titles such as Brain
Research, Neuropsychologia,
Neuroscience, NeuroImage etc
Reviewer Guidance Programme
The Reviewer Guidance Pilot Program aims to provide participants
with the experience of independently reviewing at least two
manuscripts inside a specially-created EES (Elsevier Editorial
System).
Guide participants in how to write review reports in such a way that
they answer the needs of both the Editor and the author
“The Reviewer Guidance Program is not only an experience that
helps early career researchers become better reviewers, but also
to be more critical in analyzing their own papers before submitting.
In addition, this is a great opportunity for junior scientists to
network with their more senior peers.”
For more information on this initiative, please contact Irene KanterSchlifke, Publisher for Pharmacology and Pharmaceutical
Sciences, STM Publishing ([email protected])
20
Published reviewer reports
In this pilot we have published reviewer reports alongside the
final article in Agricultural and Forest Meteorology
Open reviewer reports increase peer-review transparency and
assist good articles to gain authority and provides public
acknowledgement of reviewers’ contributions
Results so far: The pilot launch attracted positive international
media attention. It was also suggested that open reviewer
reports could play a useful role in training early career
researchers as reviewers. So far, reviewer reports have been
published alongside around 13 manuscripts.
For more information on this pilot, please contact Gilles Jonker,
Executive Publisher, Physical Sciences, [email protected]
21
Open Peer Commentary
In this pilot, we have asked experienced researchers to submit
a one page comment on an article for the journal Physics of
Life Reviews. These comments are published in the same
issue as the article. On average, five comments are published
with the article and the author can write a rebuttal article.
Results so far: Since the pilot was launched in January 2010,
the journal has seen an increase in papers (2011 - 85 and
2010 - 74; previously the journal received around 12 papers
per year). There has also been a sharp increase in usage –
roughly 3,000 downloads per month compared to 2,000 per
month in 2009.
For more information please contact Charon Duermeijer,
Publishing Director Physics, [email protected]
22
PeerChoice
Traditionally, Editors chose to approach reviewers they consider
are suitably qualified, or who would find the subject matter
interesting. But what if the reviewer could select the manuscript
themselves?
For a year now, we have been experimenting with this
additional peer-review system on the journal Chemical Physics
Letters. A selected pool of reviewers receives an overview of
the new submissions. If they like a paper because it matches
their expertise and interest, they can decide to review it.
Results so far: The time taken to review the manuscript has
been slightly reduced, while the time taken to accept an
invitation has been halved.
For more information on this pilot, please contact Egbert
van Wezenbeek, Director Publication Process Development,
Publishing Services, [email protected]
23 January 2012
Thank you
For author and reviewer training webcasts:
www.elsevier.com/trainingwebcasts
For reviewer information and guidelines:
www.elsevier.com/reviewers
Effective Reviewing
Dr Michael Rugg – Editor-in-Chief of Neuropsychologia
•
Brainard, D.H. How to write an effective manuscript review. Optics and
Photonics News, 2000, June, 42-43.
•
Roediger, H.L. Twelve tips for reviewers. APS Observer, 2007, 20, part 4
•
Sternberg, R.J. On civility in reviewing. APS Observer, 2002, 15, part 3.
•
Sternberg R.J (ed.) Reviewing Scientific Works in Psychology. American
Psychological Association, 2006
Effective Reviewing
•
Review others as you would want them to review you
•
Turn the review in on time!!!
•
Remember that no paper is perfect – avoid a ‘find the fatal flaw’ mind-set
•
Don’t go overboard summarizing the paper – couple of sentences focusing
on the main point
•
Big picture first – summarize your conclusion and, briefly, its rationale.
Focus on whether the paper tackles an important topic, and whether the
findings and their interpretation advance the field (or could potentially do
so). Expand on the reasons for your conclusion in a series of paragraphs
that each deal with one concern or issue
•
Don’t be picky or get bogged down in minor details. A reviewer is not a
copy-editor
•
Resist the temptation to re-write the paper – restrict recommendations for
revision to those necessary to address your significant concerns.
•
Be constructive and respectful even when being critical - avoid patronizing
comments that imply you’re smart and the authors are not.