Performance Assessment Assessment of Organizational Excellence NSF Advisory Committee for Business and Operations May 5-6, 2005

Performance Assessment Develop and use performance assessment tools and measures to provide an environment of continuous improvement in NSF’s intellectual investments as well as its management effectiveness. (one of four areas—ALL now need to be successful)

Expert Assessments Integrated Throughout NSF NSF Mission STRATEGIC OPERATIONAL INDIVIDUAL OUTCOMES OUTPUT ACTIVITY Strategic or Long-Term Planning Scientific Advisory Committee Reviews NSF Performance Planning Advisory Committee for GPRA Performance Assessment (AC/GPA) Business & Operations Advisory Committee Committees of Visitors (COVs) Merit Review Project Reports Program Assessment Ratings Tool (PART) Individual Performance Assessments Directly Linked to Mission and Goals Components In red, added 2001 or later

Performance Assessment: FY 2005 Focus Areas PART Activities –New Efficiency Metric –Highest Ratings in All Programs AC/GPA Activities –Refined processes –Categorizing “Innovative High-Risk”

PART Activities I Achievements Reached consensus with OMB and implemented change. Achievements Reached consensus with OMB and implemented change. Results Factors New Efficiency Measure: Percent of award decisions made available to applicants within six months of proposal receipt or deadline date, while maintaining a credible and efficient competitive merit review system, as evaluated by external experts. Factors New Efficiency Measure: Percent of award decisions made available to applicants within six months of proposal receipt or deadline date, while maintaining a credible and efficient competitive merit review system, as evaluated by external experts.

PART Activities II Achievements 100% of NSF PARTs (8 out of 8 in total) received the highest rating of “Effective” as compared with only 15% of 607 PARTs government- wide. Achievements 100% of NSF PARTs (8 out of 8 in total) received the highest rating of “Effective” as compared with only 15% of 607 PARTs government- wide. Results Factors Completed PARTs in three NSF Investment Categories and one Priority Area: - Institutions (People) - Collaborations (People) - Polar Tools, Logistics, and Facilities (Tools) - Biocomplexity in the Environment (Priority Area) Factors Completed PARTs in three NSF Investment Categories and one Priority Area: - Institutions (People) - Collaborations (People) - Polar Tools, Logistics, and Facilities (Tools) - Biocomplexity in the Environment (Priority Area)

AC/GPA I Achievements - More substantive discussions at AC/GPA meeting -Positive comments from AC/GPA Achievements - More substantive discussions at AC/GPA meeting -Positive comments from AC/GPA Results Factors Work on individual indicators performed through subgroup chairs prior to annual meeting. Factors Work on individual indicators performed through subgroup chairs prior to annual meeting.

AC/GPA II Achievements - Broad perspectives obtained on why NSF-funded projects from various disciplines fall into these categories -Quality and quantity (~200 for “high risk” and ~600 for “multidisciplinary) of program officer responses Achievements - Broad perspectives obtained on why NSF-funded projects from various disciplines fall into these categories -Quality and quantity (~200 for “high risk” and ~600 for “multidisciplinary) of program officer responses Results Factors Identifying “high-risk” and “multidisciplinary” activities Factors Identifying “high-risk” and “multidisciplinary” activities

A STARTING POINT FOR THE AC/GPA DISCUSSION ON INNOVATIVE-HIGH RISK RESEARCH – FROM PROGRAM OFFICER COMMENTS IN NUGGETS “Innovative High-Risk” Forefront, Novel or Transformative but Untried or Untested High Reward but Significant Technical Challenges and/or High Probability of Failure Innovative and Contrary to Current Theory or Conventional Paradigms “Other High Risk” (few in number) Risk from Not Succeeding Risk to Principal Investigators/Others

Accomplishments Indicators of Success Why a Priority Coming Attractions/ Comments PART Activities Priority Initiatives Establishes agency- wide evaluation framework. 4 PARTs completed: - Institutions, - Collaborations, - Polar Tools, and - Biocomplexity All received the highest rating: “effective” 2 PARTs underway: - Fundamental Science and Engineering - FFRDCs AC/GPA Recommendation NAPA Recommendation Broad perspectives obtained on why NSF-funded projects from various disciplines fall into these categories -Quality and quantity (~200 for “high risk” and ~600 for “multidisciplinary) of program officer responses Definitions and categories for “innovative high risk” Performance Assessment: FY 2005 Refining AC/GPA Process Identify Appropriate Efficiency Measure Replacement in PART Use Measure that Combines Quality and Timeliness Reached Consensus with OMB Effective this year Identifying “high-risk” and “multidisciplinary” activities Workload issues for AC/GPA at annual meeting New process well- received -More substantive discussions at AC/GPA meeting -Comments from AC/GPA Creating of separate OE subgroup

Comments 2. “Innovative High Risk Research” Future Initiatives -Complete Fundamental Science and Engineering PART and Federally Funded Research and Development Center PART -Initiate five remaining PARTs for next year 1. PART -Work with AC/GPA to define categories for further use (e.g. by COVs, POs in writing accomplishments (“nuggets”), input to the NSB Task Force on Transformative Research) Initiatives on the Horizon

Conclusion NSF has demonstrated significant achievement in Performance Assessment.