Proposals, Grants, Merit Review and You: Focus on trends at NSF with implications for NIH and other agencies

Stephanie Pfirman Environmental Science Member of NSF’s Merit Review Process Advisory Committee

Average Annual Award Size ca. $160k

Average Award Duration ca. 3 yrs

Increase in Submissions and Decrease in Funding Rates

Due to: > applicant pool > # proposals/ applicant – special solicitations

DIFFERENCES AMONG DIRECTORATES

PI CHARACTERISTICS

PRESSURE ON COMMUNITY

Peer Review

2010 average is 5.4 independent reviews/proposal 1998 was 8.6 reviews/proposal Minimum required is 3

OPTIONS AND RECOMMENDATIONS

Reduce # of Special Solicitations?

MRPAC Recommendations for Pilot Experiments

Potential Pilot Score PI response to reviews prior to decision Return non-competitive proposals Wiki-based reviews Increased use of virtual panels

B C A A B

Increased use of ad hoc reviews More use of preliminary proposals Double-blind review Prizes

B C C

The White House Office of the Press Secretary For Immediate Release September 26, 2011

The White House and National Science Foundation Announce New Workplace Flexibility Policies to Support America’s Scientists and Their Families

12:45PM Conference Call with Tina Tchen, John P. Holdren, and Subra Suresh "•Promote family friendliness for panel reviewers – STEM researchers who review the grant proposals of their peers will have greater opportunities to conduct virtual reviews rather than travel to a central location, increasing flexibility and reducing dependent-care needs."

Suggest Potential Reviewers and Offer to be a Reviewer/Panelist

• • If you don’t have time to do a good review – Decline immediately & recommend someone else Once you accept a review assignment … – Take time to do a good job – address all the criteria – Get it in on time – Don’t always give the same rating!

Resubmittals – Communicate with PO Try EAGER or Workshop?

ca. 1.5%? … target is 4%?

EAGER up to $300k over 2 years, RAPID up to $200k over 2 years

Early-concept Grants for Exploratory Research (EAGERs) and Grants for Rapid Response Research (RAPIDs)

Proposed Changes to the Merit Review Criteria

Merit Review Principles and Criteria

The identification and description of the merit review criteria are firmly grounded in the following principles: 1.All NSF projects should be of the highest intellectual merit with the potential to advance the frontiers of knowledge.

2.Collectively, NSF projects should help to advance a broad set of important national goals, including: • Increased economic competitiveness of the United States.

• Development of a globally competitive STEM workforce.

• Increased participation of women, persons with disabilities, and underrepresented minorities in STEM.

• Increased partnerships between academia and industry.

• Improved pre-K–12 STEM education and teacher development.

• Improved undergraduate STEM education.

• Increased public scientific literacy and public engagement with science and technology.

• Increased national security.

• Enhanced infrastructure for research and education, including facilities, instrumentation, networks and partnerships. 3.Broader impacts may be achieved through the research itself, through activities that are directly related to specific research projects, or through activities that are supported by the project but ancillary to the research. All are valuable approaches for advancing important national goals.

4.Ongoing application of these criteria should be subject to appropriate assessment developed using reasonable metrics over a period of time .

Intellectual merit of the proposed activity The goal of this review criterion is to assess the degree to which the proposed activities will advance the frontiers of knowledge. Elements to consider in the review are: 1.What role does the proposed activity play in advancing knowledge and understanding within its own field or across different fields? 2.To what extent does the proposed activity suggest and explore creative, original, or potentially transformative concepts? 3.How well conceived and organized is the proposed activity? 4.How well qualified is the individual or team to conduct the proposed research? 5.Is there sufficient access to resources?

Broader impacts of the proposed activity The purpose of this review criterion is to ensure the consideration of how the proposed project advances a national goal(s). Elements to consider in the review are: 1.Which national goal (or goals) is (or are) addressed in this proposal? Has the PI presented a compelling description of how the project or the PI will advance that goal(s)?

2.Is there a well-reasoned plan for the proposed activities, including, if appropriate, department-level or institutional engagement?

3.Is the rationale for choosing the approach well-justified? Have any innovations been incorporated?

4.How well qualified is the individual, team, or institution to carry out the proposed broader impacts activities? 5.Are there adequate resources available to the PI or institution to carry out the proposed activities?

“Transformative” …

• Flag high risk, game changing ideas • •

From Coburn Report “NSF Under the Microscope” These projects represent good examples transformative science that will change our understanding of important scientific concepts. These research efforts are important scientific ideas that transcend the whims of individual researchers or federal government bureaucrats. And these investments were appropriate expenditures of federal funds. Real, transformative research should be the standard for all NSF supported projects. Recognizing that all scientific endeavors do not result in the intended outcome, NSF investments can advance knowledge and in many cases improve the human condition rather than simply satisfying the random curiosities of some researchers

.

NIH too …

• The Common Fund's NIH Director’s Transformative Research Award initiative, formerly known as the Transformative Research Project (TR01), is created specifically to support exceptionally innovative and/or unconventional research projects that have the potential to create or overturn fundamental paradigms. These projects tend to be inherently risky and may not fare well in conventional NIH review. As compared to the other NIH Director’s Awards - the Pioneer, New Innovator Award, and Early Independence Awards - the primary emphasis of the Transformative Research Awards initiative is to support research on bold, paradigm shifting, but untested ideas, rather than to support exceptionally creative individuals who wish to pursue new, potentially high impact research directions.

Conclusions

• • • • • Big differences between directorates – And differences in recent trends Rejection is the norm – especially for early career applicants Communicate with PO on how to follow up on rejected proposals – EAGER? RAPID?

– Workshop?

Offer to peer review, serve on panels Identify “transformative” aspects of your proposed activities

EXTRA SLIDES

Funding Rates

Old

PI Characteristics