Major Research Instrumentation- COV Discussion of the Issues and Recommendations with SMART October 18, 2005

Review Process: Uniformity, Quality, and Transparency Significant variations of the review mechanism and associated documentation Recommendation: Utilization of panels across the foundation, augmented by ad hoc reviews when needed Need to pay more attention in documenting decisions and communicating the rationale Recommendation: Provide a more detailed review analysis to PIs. o Document better decisions regarding large proposals and the large proposal panel process. Significant variation in recognition of the value of using diverse reviewers Recommendation: Improve representation and document it.

Using Merit Review Criteria No explicit instructions on how the two criteria are applied to Ph.D.- granting and non-Ph.D. granting institutions Recommendation: develop guidelines for Ph.D. granting institutions and separate guidelines for non-Ph.D. granting institutions Significant disparity in usage of broader impacts criterion across NSF Recommendation: Ensure utilization of broader impact criterion, and require reporting about the related accomplishments in the final report.

Communication to the Research and Education Communities Need to explain clearly that non-Ph.D. granting institutions are competed separately o Help non-Ph.D. granting institutions develop better proposals. Recommendation: provide a web-library of excellent proposals. Need to clearly define “development” proposals Need to convey expected management plans Guidance on how different types of institutions may best fulfill broader impact criterion.

Evaluating Program Impact Need more information (beyond present type of final report) Recommendation: Require post award reporting, including verifying that the instruments have been used as intended. Alternatives include considering 5 year project duration and detailed final report, and/or various incentives to have better final reports. Require more detailed management plans to be used in proposal evaluations-(Final report)

Limits Imposed by the Solicitation Number of proposals per organization Recommendation: Allow each organization to submit one proposal/directorate or office (7) Cap on requested amount Recommendation: Increase cap to $4-6M. Foster collaborative proposals between Ph.D. and non-Ph.D. granting institutions Recommendation: establish a separate limit for this type proposal and/or establish separate funding opportunities. Pay directly operations and maintenance costs.

Program Data Comparison Number of Proposals Reviewed: 838 Dollars Requested: $421,372,027 Average Dollar Requested: $502,831 Median Dollar Requested: $375,822 Number of Awards: 327 MRI Dollars Awarded in Year: $109,067,161 NSF Dollars Awarded in Year: $112,854,311 Average Award (NSF Dollars): $345,120 Median Award (NSF Dollars): $279,983 Success Rate: 39% 2005 Number of Proposals Reviewed: 786 Dollars Requested: $473,579,186 Average Dollar Requested: $602,518 Median Dollar Requested: $445,980 Number of Awards: 255 MRI Dollars Awarded in Year: $89,281,684 NSF Dollars Awarded in Year: $93,690,810 Average Award (NSF Dollars): $367,415 Median Award (NSF Dollars): $294,200 Success Rate: 32%