Transcript Post-Publication Peer Review

Remember this?
• Haruko Obokata, a stem
cell researcher in Japan,
touted as a rising star in the
field, was discovered to
have fabricated her data.
• How was this discovered?
Post-Publication Peer Review
http://chronicle.com/article/In-Japan-Research-Scandal/147417/
Post-Publication Peer Review
• PubPeer http://pubpeer.com/
• External review of published articles. Notifies authors. Reviewers may be anonymous.
• PubMed Commons http://www.ncbi.nlm.nih.gov/pubmedcommons/
• Comments on PubMed articles with names, usually informal.
• F1000 Research http://f1000research.com/
• No pre-pub review, goes through open post-pub review & author revisions before
finalizing.
• ResearchGate Open Review http://www.researchgate.net/publicliterature.OpenReviewInfo.html
• Users can comment on any article, comments linked to reviewer & article record in RG
Rewarding and Rating Peer Review
• Chetty, Saez, and Sandor (2014) experimented with reviewers at the
Journal of Public Economics
• 4 groups of peer reviewers
6-week deadline
4-week deadline
4-week deadline +$100 for meeting deadline
Social pressure (review times would be publicly posted)
• Fastest group: 4 weeks + $100
• BUT, after the cash incentive period ended, members of this group were no slower than the
4-week deadline group
• Social pressure group also performed faster than 6-week deadline group
• Conclusion: Monetary compensation and social pressure can help incentivize
peer review to occur more quickly
• BUT: at the expense of quality??
• They measured quality, found no significant difference between cash incentive group and
non-incentivized groups
Rewarding and Rating Peer Review
• Stockholm University Library and Ubiquity Press are experimenting with
a badge system for peer review.
• Reviewers make their reviews public and put their names on the
reviews.
• Authors rate the reviewers for helpfulness, clarity, etc.
• Reviewers get badges to display publicly on websites as well as an open
and citable review for their CVs.
• Gamifying a system like this can seem distasteful to veteran peer
reviewers, but they see that early career researchers are most
interested in the system.
Neidenmark, Thomas; Edqvist, Karl; and Wincent, Martin.(2014, March).
"True Benefits of Peer Review". Poster presented at the Library Publishing
Forum, Kansas City, MO.
http://dx.doi.org/10.7710/2162-3309/lpf.1013
Rubriq: Independent Peer Review
• Not associated with a specific journal
• Assessed according to a uniform rubriq
• Can be initiated by the author prior to submission to a journal
• Helps inform revisions prior to formal submission
• Will suggest appropriate journals for submission
• Targeted submissions can lead to quicker acceptance/publication
• Can be used by a journal to outsource peer review
• Peer reviewers are compensated financially
Rubriq Scorecard
• Quality of Research
• Hypothesis, objective, rationale
• Methods and data
• Interpretation
• Quality of Presentation
• Title, abstract, and introduction
• Results (text)
• Results (figures, graphs, and tables)
• Discussion
• Conclusions
• References
• Writing
• Novelty & Interest
• Novelty
• Interest
Public Library of Science: PLosOne
• Peer review assesses only the quality of the research methodology
• Technical soundness of the work
• Rigor of the analysis
• Adherence to data availability policy
• Clear use of English language
• Reviewed by:
• Board of Academic Editors (AEs) member can assign to self
• AEs can reach out to other members
• AEs can solicit additional referees
Postpublication Comments
• Registered users can comment on any article
postpublication
• Commenters cannot be anonymous
• Users can report inappropriate comments for removal
• PLoS One staff have final say in removal of comments
PeerJ
• Selects articles based on scientific and methodological soundness and not impact
• Emphasizes
• Research integrity
• High ethical standards
• Constructive peer review
• Exemplary production quality
• Leading-edge online functionality
• Authors pay for a lifetime publishing plan rather than per article
• Encourages open peer review but standard is single-blind review: reviewers
identify themselves and authors can opt to publish reviews alongside their article
• As part of publishing plan, must do one review per year (comment or formal
review)
PeerJ Preprints
• Authors post preprints—draft, incomplete, or final versions—for
comment
• All commenters are named
• PeerJ reserves the right to edit/remove a comment
• Authors can go on to submit these preprints to PeerJ or elsewhere for
publication