1. Proposals, Grants, Merit Review and You: Focus on trends at NSF with implications for NIH and other agencies
Stephanie Pfirman
Environmental Science
Member of NSF’s Merit Review Process Advisory Committee

4. Average Annual Award Size ca. $160k

5. Average Award Duration ca. 3 yrs

6. Increase in Submissions and Decrease in Funding Rates
Due to:
> applicant pool
> # proposals/ applicant
special solicitations

8. DIFFERENCES AMONG DIRECTORATES

14. PI CHARACTERISTICS

17. PRESSURE ON COMMUNITY

20. Peer Review

23. 2010 average is 5.4 independent reviews/proposal
1998 was 8.6 reviews/proposal
Minimum required is 3

25. Options and recommendations

26. Reduce # of Special Solicitations?

29. MRPAC Recommendations for Pilot Experiments
Potential Pilot
Score
PI response to reviews prior to decision B
Return non-competitive proposals C
Wiki-based reviews A
Increased use of virtual panels
Increased use of ad hoc reviews
More use of preliminary proposals
Double-blind review
Prizes

30. The White House Office of the Press Secretary For Immediate Release September 26, 2011 The White House and National Science Foundation Announce New Workplace Flexibility Policies to Support America’s Scientists and Their Families 12:45PM Conference Call with Tina Tchen, John P. Holdren, and Subra Suresh
"•Promote family friendliness for panel reviewers – STEM researchers who review the grant proposals of their peers will have greater opportunities to conduct virtual reviews rather than travel to a central location, increasing flexibility and reducing dependent-care needs."

31. Suggest Potential Reviewers and Offer to be a Reviewer/Panelist
If you don’t have time to do a good review
Decline immediately & recommend someone else
Once you accept a review assignment …
Take time to do a good job – address all the criteria
Get it in on time
Don’t always give the same rating!

32. Resubmittals – Communicate with PO Try EAGER or Workshop?

33. EAGER up to $300k over 2 years, RAPID up to $200k over 2 years
ca. 1.5%? … target is 4%?
EAGER up to $300k over 2 years, RAPID up to $200k over 2 years

34. Early-concept Grants for Exploratory Research (EAGERs) and Grants for Rapid Response Research (RAPIDs)

35. Proposed Changes to the Merit Review Criteria
Merit Review Principles and Criteria The identification and description of the merit review criteria are firmly grounded in the following principles:
All NSF projects should be of the highest intellectual merit with the potential to advance the frontiers of knowledge.
Collectively, NSF projects should help to advance a broad set of important national goals, including:
Increased economic competitiveness of the United States.
Development of a globally competitive STEM workforce.
Increased participation of women, persons with disabilities, and underrepresented minorities in STEM.
Increased partnerships between academia and industry.
Improved pre-K–12 STEM education and teacher development.
Improved undergraduate STEM education.
Increased public scientific literacy and public engagement with science and technology.
Increased national security.
Enhanced infrastructure for research and education, including facilities, instrumentation, networks and partnerships.
Broader impacts may be achieved through the research itself, through activities that are directly related to specific research projects, or through activities that are supported by the project but ancillary to the research. All are valuable approaches for advancing important national goals.
Ongoing application of these criteria should be subject to appropriate assessment developed using reasonable metrics over a period of time.

36. Intellectual merit of the proposed activity
The goal of this review criterion is to assess the degree to which the proposed activities will advance the frontiers of knowledge. Elements to consider in the review are:
What role does the proposed activity play in advancing knowledge and understanding within its own field or across different fields?
To what extent does the proposed activity suggest and explore creative, original, or potentially transformative concepts?
How well conceived and organized is the proposed activity?
How well qualified is the individual or team to conduct the proposed research?
Is there sufficient access to resources?
Broader impacts of the proposed activity
The purpose of this review criterion is to ensure the consideration of how the proposed project advances a national goal(s). Elements to consider in the review are:
Which national goal (or goals) is (or are) addressed in this proposal? Has the PI presented a compelling description of how the project or the PI will advance that goal(s)?
Is there a well-reasoned plan for the proposed activities, including, if appropriate, department-level or institutional engagement?
Is the rationale for choosing the approach well-justified? Have any innovations been incorporated?
How well qualified is the individual, team, or institution to carry out the proposed broader impacts activities?
Are there adequate resources available to the PI or institution to carry out the proposed activities?

37. “Transformative” … Flag high risk, game changing ideas
From Coburn Report “NSF Under the Microscope”
These projects represent good examples transformative science that will change our understanding of important scientific concepts. These research efforts are important scientific ideas that transcend the whims of individual researchers or federal government bureaucrats. And these investments were appropriate expenditures of federal funds.
Real, transformative research should be the standard for all NSF supported projects. Recognizing that all scientific endeavors do not result in the intended outcome, NSF investments can advance knowledge and in many cases improve the human condition rather than simply satisfying the random curiosities of some researchers.

38. NIH too …
The Common Fund's NIH Director’s Transformative Research Award initiative, formerly known as the Transformative Research Project (TR01), is created specifically to support exceptionally innovative and/or unconventional research projects that have the potential to create or overturn fundamental paradigms. These projects tend to be inherently risky and may not fare well in conventional NIH review. As compared to the other NIH Director’s Awards - the Pioneer, New Innovator Award, and Early Independence Awards - the primary emphasis of the Transformative Research Awards initiative is to support research on bold, paradigm-shifting, but untested ideas, rather than to support exceptionally creative individuals who wish to pursue new, potentially high impact research directions.

39. Conclusions Big differences between directorates
And differences in recent trends
Rejection is the norm – especially for early career applicants
Communicate with PO on how to follow up on rejected proposals
EAGER? RAPID?
Workshop?
Offer to peer review, serve on panels
Identify “transformative” aspects of your proposed activities

40. Extra slides

43. Funding Rates

46. Old

48. PI Characteristics