Arlington, VA March 31, 2004 Presentation for the Advisory Committee for Business & Operations Effective Practices Research Overview For Merit Review This document is confidential and is intended solely for the use and information of the client to whom it is addressed. NSF BUSINESS ANALYSIS

1 Today, we will provide…  Brief update on the status of the NSF Business Analysis  Summary of the key findings from the Effective Practices research for Merit Review

2 NSF BUSINESS ANALYSIS NSF Business Analysis has recently competed the following activities and is currently moving into the design phase of the project Completed as of 3/31/04  Completed researching effective practices for the Merit Review and Award Management & Oversight processes  Completed workload study that assessed the amount and distribution of workload across NSF  Completed first full draft of the Target Enterprise Architecture for Applications, Data, Network and Security  Completed researching effective practices for the Merit Review and Award Management & Oversight processes  Completed workload study that assessed the amount and distribution of workload across NSF  Completed first full draft of the Target Enterprise Architecture for Applications, Data, Network and Security “Works in Progress”  Developing process scenarios and preliminary business cases for the Merit Review and Award Management & Oversight processes  Conducting an eJacket study to determine how the electronic-jacket technology tool impacts human capital issues  Developing an IT Technology Governance Framework and IT Implementation Plan that considers the 2-7 year transition plan necessary to move to the Target Enterprise Architecture  Developing process scenarios and preliminary business cases for the Merit Review and Award Management & Oversight processes  Conducting an eJacket study to determine how the electronic-jacket technology tool impacts human capital issues  Developing an IT Technology Governance Framework and IT Implementation Plan that considers the 2-7 year transition plan necessary to move to the Target Enterprise Architecture Quick Snapshot of Project Status

3 NSF BUSINESS ANALYSIS For the “effective” practices research, the team researched the practices of grant-making organizations in order to identify potential alternative practices Research Objectives:  To understand how other federal and non-federal organizations review proposals and manage awards, and the relative emphasis that is placed on either activity  To provide NSF with insights into how other organizations address challenges similar to those faced by the Foundation in the proposal review and award management processes  To identify practices or principles that could be leveraged or modified by NSF in the redesign of its core business processes

4 NSF BUSINESS ANALYSIS The team interviewed representatives of 14 federal and non-federal organizations Organizations Interviewed

5 NSF BUSINESS ANALYSIS Important to note is that few effective practices were identified in the areas of performance measurement, “working across the organization,” and technology usage  Few organizations apply performance metrics to their review or award management practices –Customer satisfaction surveys are performed ‘ad hoc’ –Processing times tend to be the same or longer than NSF –Few agencies have monitoring plans in place to address the degree of award oversight that is applied  Few organizations had a significant need for ‘working across the organization’  Many agencies were either “developing” or “progressing” in their eBusiness capabilities with availability of resources cited as the greatest barrier to development efforts

6 NSF BUSINESS ANALYSIS Effective practices emerged in each of the following areas  Process for how an organization receives, processes, and streamlines activities associated with proposal intake, review, and decision-making Reviewer Community Management Proposal Workflow Management Review Methodologies Description  Various practices an organization applies to review proposals and make funding decisions with limited resources  Management practices that an organization applies to attract, recruit, and reward its reviewers, as well as how the organization maintains a relationship with its community Merit Review Areas

7 NSF BUSINESS ANALYSIS Reviewer Community Management Proposal Workflow Management Review Methodologies Many organizations employ different strategies to mitigate workload demands

8 NSF BUSINESS ANALYSIS Reviewer Community Management Proposal Workflow Management Review Methodologies Most organizations structure the review process to optimize the time of reviewers

9 NSF BUSINESS ANALYSIS Reviewer Community Management Proposal Workflow Management Review Methodologies Most organizations treat reviewers as “customers” throughout the process

10 NSF BUSINESS ANALYSIS Based on the findings in the effective practices research, NSF should consider the following in the scenario design  Broadening the use of Letters of Intent (LOIs) for determining program fit and begin planning and coordinating panels earlier in the process  More pro-actively coordinating program deadlines taking into account the needs of the research community  Developing piloting opportunities to broaden the use of temporary help during peak proposal times or outsourcing certain process functions  Incorporating process changes that reduce the degree of proposal budget rework that occurs throughout the Merit Review process  Compensating off-site reviews, broadening the concept of panel “terms of service”, and providing training for how best to perform a review  Developing strategies that “reach out” to a greater pool of reviewers (e.g., integrating databases for applicants, awardees, reviewers; enabling better remote access; accepting on- line reviewer applications, etc.)

11 NSF BUSINESS ANALYSIS Questions & Answers  For additional questions, please contact: –Abe Zwany, –Tim Koch,