Transcript No Slide Title

NIH Review Procedures
Betsy Myers
Hospital for Special Surgery
General Process for Proposal Review
• Applicant has idea
• Forms are submitted
• Proposal is reviewed for
scientific merit
• Summary statement is
prepared
• Funding Institute or
Center gives information
to applicant
www.csr.nih.gov
Receipt and Review Process at NIH
• >10,000 applications arrive at given
deadline!
• Receipt/review process organized in 3
cycles per year
Assignment Process at NIH
• Assignment based on Abstract, Specific
Aims, more if needed
• Each application assigned to funding
Institute(s)/Center(s)
• Application also assigned to Integrated
Review Group (IRG) within Center for
Scientific Review (CSR) or to
Institute/Center (IC) review group
Assignment Process at NIH
• CSR:
–R01s, R03s, R21s, Small business,
Fellowships
–Reviews for >1 Institute
• IC Review:
–Program projects, Training grants,
Career development awards,
Responses to Requests for Applications
–Specific to Institute
Assignment Process at NIH
• Application then assigned to Study
Section
• NIH officials will consider requests for
these assignments
–Cover letter
Process at NIH
Scientific Review Administrator (SRA)
of Study Section decides on reviewers
from within members of Study Section
or from ad hoc members
Appointment of Reviewers to Study
Section
• SRA recruits members of Study
Section
• Qualifications
–Expert with training and experience in
relevant scientific field
• Level of formal education
• Quantity and quality of relevant research
–PI on research project comparable to
those being reviewed
Appointment of Reviewers to Study
Section
SRA also needs to address
–Diversity in gender, race, ethnicity and
geographic distribution
–Fairness and evenhandedness in
review
–Willingness to do the work required
–Ability to write and present clearly
Appointment of Reviewers to Study
Section
Types of appointments to study section
–Regular: Typically several years
–Temporary: One time on standing study
section, may lead to regular
appointment
–Special emphasis panel: One time only
Roster of Study Section Available Online
Meeting Roster - ZRG1
MOSS-A 91 (4/6/2005 - 4/6/2005)
CENTER FOR SCIENTIFIC REVIEW
SPECIAL EMPHASIS PANEL
ZRG1 MOSSA
4/6/2005-4/6/2005
MEETING ROSTER
CHAIRPERSON
-------------------------------------MYERS, ELIZABETH R. , PHD
ASSOCIATE PROFESSOR
WEILL MEDICAL COLLEGE OF CORNELL UNIVERSITY
ASSOCIATE SCIENTIST, HOSPITAL FOR SPECIAL SURGERY
NEW YORK, NY 10021
MEMBERS
---------------ADAMS, JOHN S. , MD
PROFESSOR
DEPARTMENT OF MEDICINE
CEDARS-SINAI MEDICAL CENTER
LOS ANGELES, CA 90048
AHLGREN, SARA C. , PHD…
http://www.csr.nih.gov/Committees/rosterindex.asp
Assignment of Applications to
Reviewers
• SRA matches grant applications to
specific reviewers
• Tries to ensure
–Appropriate expertise
–Diverse scientific viewpoints
• Tries to avoid
–Overload of particular reviewer
–Potential conflict of interest
Assignment of Applications to
Reviewers
2 reviewers and 1 discussant (typically)
are assigned to each proposal
–Primary reviewer
–Secondary reviewer
–Reader (does not need to prepare
written review prior to meeting of study
section)
Could be more – Tertiary, more Readers
NIH Review Criteria
NIH review criteria for unsolicited
research project grant applications
(R01, R03, R21)
• Significance
–Important problem
–Advancement of scientific knowledge or
clinical practice
–Influence on methods that drive the field
NIH Review Criteria
• Approach
–Adequate development and integration
of design, methods, analyses
–Acknowledgment of potential problems,
alternatives
• Innovation
–Challenge to existing paradigms
–Novel concepts, approaches, methods
NIH Criteria
• Investigator
–Appropriate training, experience
–Complementary and integrated team
• Environment
–Conducive to probability of success
–Unique features of scientific
environment
–Institutional support
NIH Criteria
• Other criteria
–Gender/minority/children inclusion
–Budget
–Protection of humans, animals, and
environment
• Overall rating
–Numerical score that reflects overall
impact
REVIEW CRITERIA: “K” Awards
•
•
•
•
•
Candidate
Career Development Plan
Research Plan
Mentor/Co-Mentor(s)
Environment & Institutional
Commitment
NIH Numerical Rating
Priority score: Single, global score for
proposal
WORST
5.0
Lowest
Priority
BEST
4.0
3.0
Average
application
2.0
1.0
Highest
Priority
Strong in
all categories
Guide to Calibrating Score
Score
Descriptors
1.0 – 1.5
1.6 – 2.0
2.6 – 3.0
Outstanding, Close to flawless
Highly significant, Few
weaknesses
Weaknesses need to be
addressed
Weaknesses balance strengths
>3.0
Weaknesses outweigh strengths
2.1 – 2.5
Submitting Critique Before Meeting
Electronic submission of reviews
• Several days before meeting, reviewers
upload score and written critique
• Once uploaded, can then read other
reviewers’ scores and reviews
• Once uploaded, reviewer cannot make
changes to scores or critiques until after
meeting
Study Section Meeting
Streamlined Applications
• Definition:
–Not in upper half
–Priority score higher than 3
5.0
4.0
3.0
2.0
• Does not apply to career awards,
fellowships (R13, R18, F06, F32/33)
1.0
Study Section Meeting
Streamlining
Streamlining Procedure
• Reviewers asked ahead of time to
recommend applications not in upper
half (“unscored” or “streamlined”)
• SRA compiles list
• List discussed at beginning of meeting
• Any member may ask for proposal to be
discussed
Study Section Meeting
Streamlining
Benefits and rationale
• Gives time for in-depth discussion of
better applications
• Saves costs if meeting is shortened
• Reduces work of scientific review
administrators
Less than 25% of applications will be funded
Study Section Meeting
Streamlining
• If application is streamlined, applicant
receives unaltered written critiques
• Fate of unscored applications?
Study Section Meeting
Review Procedures
Review procedure for proposals to be
scored
–Chair introduces application
–Each reviewer gives preliminary
numerical score or range
–Primary reviewer covers description
and comments
–Other assigned reviewers add
comments
Study Section Meeting
Review Procedures
Review procedure, continued
–Discussion ensues
–Consensus is not necessary
–Chair calls for priority rating
–Every members scores
Resume
Summary Statement is prepared
–SRA asks reviewers to modify critiques
to reflect discussion
–SRA writes resume and summary of
discussion in front
–Summary Statement (“Pink Sheets”)
sent to applicant
Final Score
• Average of all scores multiplied by 100
• Example:
–Average of raw scores from review
panel = 1.88
–Final score = 188
Percentiles
• Percentiles indicate your rank relative to
other applications reviewed by group
• 0.1 (best) to 99.5 (worst)
•  percentage of proposals receiving
better score during last year
• Example: Score: 188, Percentile: 11
Applications Used in Percentile
Conversion
• R01 reviewed at standing study section
–Percentile of score relative to all scores
from current round plus last two rounds
(1 year)
• R01 reviewed at special emphasis
panel
–May be percentiled against distribution
of all CSR scores
Funding
Decision
Flowchart
http://www.niaid.nih.gov/ncn/grants/charts/flowchart_funding.htm
Thank you