Transcript Constrained Provider Choice

Refereeing and Discussant
Guidelines
Susan Godlonton
AGRODEP AIEN III Workshop
Dakar, Senegal
4th June, 2014
Vu Pham
Referee Report
• Components of a referee report:
• Cover letter
• Summary
• Key concerns
• Decision
• Report
• Overall view
• Main concerns
• Smaller concerns
• No “one-size-fits-all approach”
Vu Pham
Referee Report: Good paper?
• What is a good paper?
• Clear
• Coherent (understandable)
• Correct
• Additional contribution
• Value added of answer to question
• Credibility of identification strategy
• Validity and robustness of approach
Vu Pham
Referee Report: Overall view
• Don’t repeat the abstract, the editor can read
the abstract
• How would you summarize the paper
• What do you think is the key contribution
• Do you think the authors address it correctly
Vu Pham
Referee Report: Main Concerns
• Key issues that the author needs to address
• Ensure that it is clear which of your concerns
are fundamental
• Don’t ask the author to write a different paper
• Be constructive! Try to provide solutions
where possible accounting for data limitations
Vu Pham
Referee Report: Main concerns
Related literature
Inappropriate to the actual material in the paper
Inaccurately described
Incomplete (very common)
Logical argument
Incorrect application of economic concepts
Inaccurate mathematical derivations
Loose ties between the economic model and
the empirical analysis
Vu Pham
Referee Report: Main concerns
Econometric tools are being used inappropriately
Incorrectly used
Assumptions do not hold
Assumptions not defended with evidence
Estimated key coefficients of interest
Not properly identified
Not robust
Additional robustness checks needed
Interpretation of empirical results is inappropriate
Overstates the contribution
Overstates the claim (e.g. causation vs. correlation)
Conclusions are incorrectly made or expressed
Vu Pham
Referee Report: Main concerns
Econometric tools are being used inappropriately
Incorrectly used
Assumptions do not hold
Assumptions not defended with evidence
Estimated key coefficients of interest
Not properly identified
Not robust
Additional robustness checks needed
Interpretation of empirical results is inappropriate
Overstates the contribution
Overstates the claim (e.g. causation vs. correlation)
Conclusions are incorrectly made or expressed
Vu Pham
Referee Report: Minor concerns
Do not bring these up when discussing a paper
Specific places in text where discussion hard to follow
or confusing
Not using standard notation
Spelling and grammatical errors
Missing data sources and poorly constructed Tables or
Figures
References to the literature that are missing or
incorrect
Vu Pham
Additional Resources
• Refereeing
• http://blogs.worldbank.org/impactevaluations/how-much-to-refereeand-how-to-do-it
• http://marcfbellemare.com/wordpress/2012/01/contributing-to-publicgoods-my-20-rules-for-refereeing/
• http://marginalrevolution.com/marginalrevolution/2006/10/how_to_b
e_a_goo.html
• Discussant
• http://chrisblattman.com/2010/02/22/the-discussants-art/
• Other resources will be uploaded to the AIEN workshop
Vu Pham
For today…
Each been assigned a paper/set of slides from
one of your peers
Focus
Clearly define the research question
Define and discuss the identification strategy
Propose alternative credible identification strategy
Vu Pham
For today…
Focus on the identification strategy
What is the identifying variation?
What assumptions are made by using the chosen
identification strategy (implicitly/explicitly)?
Do they make sense?
To what extent are these true?
How else can the author test the assumptions?
What robustness checks or placebo tests could the
author run?
What would be a more preferable strategy?
Vu Pham
Thinking about Identification
Homework Assignment
Vu Pham
Thinking about Identification Exercise
Group into:
Health
Education
Agriculture
Discuss proposed ideas, provide feedback to
one another
Vu Pham