1 © 2000 Technology Commercialization Group LLC. All rights reserved TECHNOLOGY COMMERCIALIZATION GROUP, LLC P AUL G. W AUGAMAN L OUIS G. T ORNATZKY Higher Education Practice W ILLLIAM S. K IRBY A PPLYING B ENCHMARKING TO S PONSORED P ROGRAMS A DMINISTRATION SRA - NCURA Southern Section/Southeastern Region Meeting April 16-17, 2000 TCG

2 © 2000 Technology Commercialization Group LLC. All rights reserved Points to Cover: m Benchmarking in research administration »Overview m The Higher Education Benchmarking Consortium »Overview m The KPMG - NACUBO - SRA Sponsored Programs Benchmarking Effort »Organization »Initial results »Next steps

3 © 2000 Technology Commercialization Group LLC. All rights reserved Benchmarking Systematic comparison of elements of the performance of an organization against that of other organizations, with the aim of mutual improvement. McNair and Leibfried: “Benchmarking.” 1992

4 © 2000 Technology Commercialization Group LLC. All rights reserved The Utility of Benchmarking m Provides a basis for comparison and self-analysis. m Accelerates the sharing of useful and novel approaches. m Energizes everybody to do better.

5 © 2000 Technology Commercialization Group LLC. All rights reserved Purposes and Uses of Measurement in Research Administration m Measure against “industry standards” m Measure against goals » Performance drives improvement interventions. m Identify problems » Diagnose and forecast existing problems. m Build the “business case” for change or new resources.

6 © 2000 Technology Commercialization Group LLC. All rights reserved The Problem: In Research administration.... m There are no generally accepted criteria for success. m We rely on anecdotal data and informal case study rather than systematic data collection and analysis. m Benchmarking and measurement are not part of the improvement culture m Competitive comparisons may be viewed as threats rather than improvement drivers.

7 © 2000 Technology Commercialization Group LLC. All rights reserved The Problem (cont): m Breadth and diversity of the domain make data collection and measurement difficult. m Successful outcomes are difficult to demonstrate objectively. m There is little data available. m Who is going to do it?

8 © 2000 Technology Commercialization Group LLC. All rights reserved The SRA Benchmarking Initiative m Benchmarking and Best Practices Task Force established in m Task Force recommends working with NACUBO and KPMG to facilitate performance data collection. m Collaboration with NACUBO and KPMG initiated in September m Data collection instrument refined with SRA Task Force input in Spring, 1999.

9 © 2000 Technology Commercialization Group LLC. All rights reserved The KPMG Higher Education Benchmarking Consortium The purposes of the consortium are: »To serve as a comprehensive source of institutional comparative performance, benchmarking, and “best practice” information for the higher education community; and »To facilitate the use of comparative performance data to meet strategic, operational improvement, and change management needs.

10 © 2000 Technology Commercialization Group LLC. All rights reserved The KPMG Higher Education Benchmarking Consortium Operating Principles »Participant-driven »“E-nabled” »Focused »Leading to "best practices” »Leveraged, where possible

11 © 2000 Technology Commercialization Group LLC. All rights reserved The KPMG Higher Education Benchmarking Consortium Major Segments »Financial Health and Operations »Student Affairs »Physical Infrastructure »Information Technology »Human Resources »Research: 7 Sponsored Programs 7 Technology Transfer

12 © 2000 Technology Commercialization Group LLC. All rights reserved KPMG - NACUBO - SRA Sponsored Programs Survey Purposes m To provide a common set of sponsored programs performance indicators that will allow meaningful comparisons with other organizations and with peer groups of institutions. m To facilitate the development of a benchmarking data base.

13 © 2000 Technology Commercialization Group LLC. All rights reserved Sponsored Programs Survey SRA-NACUBO-KPMG Roles m NACUBO and KPMG are financing and organizing data collection and analysis, and are promoting the program with their constituencies and clients. m SRA is participating in survey design, is promoting the program with their members, and is participating in dissemination of findings.

14 © 2000 Technology Commercialization Group LLC. All rights reserved Sponsored Programs Survey Features of the Program m Low cost to participating institutions. m Individualized reports for participants. m Expert analysis. m All institution-specific data will be “blinded.” m Only general reports will be available to non-participants.

15 © 2000 Technology Commercialization Group LLC. All rights reserved Sponsored Programs Survey Critical Success Factors for Research Administration m Sustaining or Enhancing Sponsored Research Activity and Funding (competitiveness) m Containing the Costs and Increasing the Efficiency of Sponsored Research Administration m Improving Service to Faculty (in order to achieve Objective 1) m Maintaining and Improving Institutional and Sponsor accountability

16 © 2000 Technology Commercialization Group LLC. All rights reserved Sponsored Programs Survey Indicators: Sustaining or Enhancing Sponsored Research Activity and Funding m Number of proposals submitted per faculty FTE. m Percentage of faculty working as principal investigators. m Sponsored project dollars received per faculty FTE. m Sponsored projects funding growth rate. m Number of new awards as a percent of new proposals submitted.

17 © 2000 Technology Commercialization Group LLC. All rights reserved Sponsored Programs Survey Indicators: Cost and Efficiency m Number of proposals per sponsored projects admin. FTE. m Sponsored projects admin. cost per proposal. m Number of awards per Sponsored projects admin FTE. m Sponsored projects admin costs as percent of dollars received. m Number of awards per research accounting FTE. m Research accounting costs per award m Dollars received per research accounting FTE. m Research accounting cost as percent of dollars received.

18 © 2000 Technology Commercialization Group LLC. All rights reserved Sponsored Programs Survey Indicators: Service to Faculty m Number of funded PI’s per sponsored projects FTE. m Number of active PI’s per sponsored projects FTE. m Number of funded PI’s per research accounting FTE.

19 © 2000 Technology Commercialization Group LLC. All rights reserved Sponsored Programs Survey Indicators: Institutional and Sponsor Accountability m Percent of Sponsored Projects Accounts Receivable over 120 days.

20 © 2000 Technology Commercialization Group LLC. All rights reserved Sponsored Programs Survey Data Needed to Develop Indicators Fy Data: m Number of faculty FTE. m Number of central administrative FTE (sponsored programs, research accounting). m Proposal data (number, dollars, sponsor: federal, industry, other). m Award data (number, dollars, sponsor: federal, industry, other). m Sponsored program expenditure data m Administrative costs allocated to research (central and departmental costs - from indirect cost proposal).

21 © 2000 Technology Commercialization Group LLC. All rights reserved Sponsored Programs Survey Benefits for Participants m Access to useful comparable data for program evaluation and benchmarking. m Low cost. m Ease of preparation. m Uses readily available data assembled for internal reporting or other purposes.

22 © 2000 Technology Commercialization Group LLC. All rights reserved Sponsored Programs Survey Year One m Questionnaire Design (Dec 98-Jun 99) »Survey design was peer-based »Intentions were to use available data »Initial survey requested FY 1998 data m Data Collection (Jul-Oct 99) »A high response rate was a major objective »Timing did not accommodate academic calendar »Correct POC identification was a challenge »Follow up efforts: phone calls, letters, s

23 © 2000 Technology Commercialization Group LLC. All rights reserved Sponsored Programs Survey Year One m Data Collection Outcomes: »62 institutions participated (40% of all US college/university research expenditures) »Definition issues arose $Examples: Who are “research faculty;” new vs. renewal awards $Caused confounding metrics m Data Processing »34 institutions chose peers »3 sub-sets of institutions

24 © 2000 Technology Commercialization Group LLC. All rights reserved Sponsored Programs Survey Year One m Reporting: »Institution-specific reports: $Customized, Institution-specific. Provided ranking information on 16 metrics, and 4 sub-groups for each metric. »Data feedback workshops $Gave participants opportunities to discuss data, and next steps »Revised Reports will be prepared after participants have an opportunity to revise submissions »Public Report in preparation

25 © 2000 Technology Commercialization Group LLC. All rights reserved Sponsored Programs Survey Year One Percentage of faculty working as principal investigators.

26 © 2000 Technology Commercialization Group LLC. All rights reserved Sponsored Programs Survey Year One Sponsored Project dollars per Faculty FTE

27 © 2000 Technology Commercialization Group LLC. All rights reserved Sponsored Programs Survey Year One Sponsored Projects Funding Growth - Fy

28 © 2000 Technology Commercialization Group LLC. All rights reserved Sponsored Programs Survey Year One Number of proposals submitted per central sponsored projects administrative FTE

29 © 2000 Technology Commercialization Group LLC. All rights reserved Sponsored Programs Survey Year One Central sponsored projects administrative cost as a percent of sponsored project dollars

30 © 2000 Technology Commercialization Group LLC. All rights reserved Sponsored Programs Survey Year One Central post-award financial administrative cost per active project

31 © 2000 Technology Commercialization Group LLC. All rights reserved Sponsored Programs Survey Year One Central post-award financial administrative cost as a percent of sponsored project dollars

32 © 2000 Technology Commercialization Group LLC. All rights reserved Sponsored Programs Survey Year One Number of active Principal investigators per central sponsored projects admin. FTE

33 © 2000 Technology Commercialization Group LLC. All rights reserved Sponsored Programs Survey Next Steps m Complete Fy 1998 round » Reopen data base for revisions and additional enrollments (April-May, 2000) » Complete a brief public report (May, 2000) » Complete revised individual reports (June, 2000) » Brief Best Practices study (June - August, 2000)

34 © 2000 Technology Commercialization Group LLC. All rights reserved Sponsored Programs Survey Next Steps m Initiate Fy 2000 round » Target date to begin data collection: September- November, 2000 » Activate Web Site for data collection on-line (September, 2000) and report preparation by participants (January, 2000) » Reports: January, 2001 Web site (for informational purposes):

35 © 2000 Technology Commercialization Group LLC. All rights reserved Contact us... TECHNOLOGY COMMERCIALIZATION GROUP, LLC 2237 O XFORD H ILLS D RIVE R ALEIGH, N ORTH C AROLINA USA T EL : F AX : E -MAIL : P T-C-GROUP.COM P AUL G. W AUGAMAN, P RINCIPAL TCG