Internal Metrics and Benchmarking New Era Measurement of TTO Performance and Impact Speakers Charles Louis - Melba Kurman -

Speakers Charles Louis, Vice Chancellor for Research at the University of California, Riverside Melba Kurman, principal with Triple Helix Innovation 2 Webinar: Internal Metrics and Benchmarking

Overview of today’s program Metrics: how can university technology transfer offices best quantify their performance, impact, and productivity? – The approach taken by the University of California – Adding an additional tool: the technology transfer health index concept – Discussion 3 Webinar: Internal Metrics and Benchmarking

The challenge of benchmarking innovation intermediaries TTO metrics should… Represent effort, activity and outcomes that support a single, tightly-defined mission Capture outcomes the TTO can actually affect Capture the entire big picture of TTO activity, time spent, and results Provide clear insight into TTO activity & performance 4 Webinar: Internal Metrics and Benchmarking

1.Define success criteria for the University's technology transfer program, taking into consideration the documented objectives of the program and the different stages of development of the different campus technology transfer offices. 2.Identify and describe metrics for measuring these criteria. 3.Advise on the scope, format, and frequency of reports of the university’s technology transfer program to both the public and to internal stakeholders. University of California’s technology transfer metrics Webinar: Internal Metrics and Benchmarking

Within UC, at least four stakeholders were identified by the directors of the campus technology programs: 1. The academic community of faculty and students. 2. State and local government. 3. Industry partners. 4. The public. University of California metrics committee Webinar: Internal Metrics and Benchmarking

7 Perceived value of the metric. Is the data for the metric currently captured in a database. If the data is not currently captured in a database, what is the expected level of difficulty in capturing the data. If the level of difficulty is deemed to be high, then the perceived value also must be high if it is to be collected. Criteria for inclusion of a metric to be used and reported Webinar: Internal Metrics and Benchmarking

1. Create public benefit. 2. Provide service to the UC academic community. 3. Create, support and maintain research partnerships with industry. 4. Support economic development. 5. Secure fair compensation for UC technologies. Five identified goals for the UC campus Tech Transfer programs Webinar: Internal Metrics and Benchmarking

The public benefits when the innovations and discoveries of the faculty and staff of the University of California are made available to the public and commercial sectors. Transferring university research outside of the institution enables further research, the creation of new companies, and the development of products and services. By facilitating this transfer, Technology Transfer Offices are instrumental in promoting innovation and public benefit. Goal #1. Create public benefit Webinar: Internal Metrics and Benchmarking

Data collected to measure creating public benefit 1.Number of tech transfer agreements. 2.Number of Start-Up Companies. 3. Number of products on market. Webinar: Internal Metrics and Benchmarking 10

Technology Transfer Offices serve the academic community in all types of intellectual property related activities such as the transfer of proprietary materials into and out of the university. Provides policy guidance, mentoring and education about protecting and commercializing intellectual property. Supports faculty, postdocs and graduate students who have the desire, but not necessarily the experience to champion the development of promising early stage technology and to start new ventures. Goal #2. Provide service to the UC academic community Webinar: Internal Metrics and Benchmarking

Data collected to measure providing service to the academic community 1.Number of disclosures. 2.Number of individuals disclosing for first time. 3.Number of individuals disclosing more than one time. 4.Number of interactions. 5.Number of individuals receiving income from tech transfer agreements. 6.Number of Material Transfer Agreements. Webinar: Internal Metrics and Benchmarking 12

Data collected to measure providing service to the academic community (cont.) 7.Number of copyright licenses. 8.Number of visitor agreements with industry for which the tech transfer office provided input. 9.Number of patents filed. 10.Number of issue patents. 11.Number of researcher-involved start-up companies. 12.Results of customer satisfaction surveys. Webinar: Internal Metrics and Benchmarking 13

Research is a primary mission of the University of California. Partnerships with industry allow for the expertise of university researchers to be focused on solving problems/questions of interest to industry and also allow for innovations resulting from these interactions to be made available to industry. Technology Transfer Offices enhance research partnerships by offering university intellectual property to support and expand interactions with industry. Goal #3. Create, support and maintain research partnerships with industry Webinar: Internal Metrics and Benchmarking

Data collected to measure research partnerships with industry 1.Number of tech transfer agreements resulting from industry research. 2.Number and dollar value of research agreements resulting from tech transfer agreements. 3. Number of agreements for which the tech transfer office provided input. 4.Number of visitor agreements with industry for which the tech transfer office provided input. Webinar: Internal Metrics and Benchmarking 15

Technology-driven businesses create new jobs and drive regional and national economic development. Effective technology transfer from land grant universities is an essential first step in creating new companies employing highly skilled graduates and contributing to the economy through increases in the tax base and the purchase of goods and services. Goal #4: Support economic development Webinar: Internal Metrics and Benchmarking

Data collected to measure support of economic development 1.Number of Start-up companies. 2.Number of jobs created by start-up companies. 3.Funding of start-up companies. 4.Sales revenue of start-up companies. 5.Total gross sales reported by licensees. 6.Number of products on market. 7.Number of individual companies with a tech transfer agreement. Webinar: Internal Metrics and Benchmarking 17

18 Technology Transfer Offices are the stewards of public intellectual property assets. Accordingly, they seek to secure fair compensation for their available technologies keeping in mind the need to provide service to all of the stakeholders involved in the tech transfer process; namely, industry, university researchers and the public. Revenue generated from the technology transfer program is invested in the university’s education and research programs. Goal #5. Secure fair compensation for UC technologies Webinar: Internal Metrics and Benchmarking

Data collected to measure fair compensation for UC technologies 1.Royalty and fee income. 2.Income from the sale of equity. 3.Number of tech transfer agreements generating income. 4.Reimbursement of patent costs. Webinar: Internal Metrics and Benchmarking 19

20 This 2009 report has proven a valuable resource for enhancing the reporting of technology transfer activity of the campuses. Some of the metrics may best serve as a management tool rather than published metrics – especially of value for newer offices. In spite of this document, many senior administrators and faculty still perceive that the primary goal their TTO is #5, generating revenue, not recognizing the many other, non-monetary services that they provide to the researchers on their campuses. Certainly a common view of faculty who do not interact with their TTOs. A number of national organizations have a similar opinion where they compare licensing income $/sponsored funding award $ as the metric for evaluating TTO performance. This report provides a broader discussion of this agenda where TTOs are seen as a service to support the research mission of the University of California. UC metrics project conclusions Webinar: Internal Metrics and Benchmarking

Speaker Melba Kurman, principal at Triple Helix Innovation Webinar: Internal Metrics and Benchmarking 21

Webinar: Internal Metrics and Benchmarking22 A technique & tool that can Capture the entire big picture of TTO activity, time spent, and results Provide clear insight into TTO activity & performance The technology transfer health index

What is the tech transfer health index? Webinar: Internal Metrics and Benchmarking 23 The tech transfer health index quantifies the impact and productivity of the entire long tail curve of technologies in a university’s IP portfolio.

Three universities Each university has 1000 patents and total revenue of $10M. University A: 2 patents earn $5M each University B: 100 patents earn $100K each University C: 1000 patents earn $10K each …Which university has the healthiest IP portfolio? Webinar: Internal Metrics and Benchmarking 24

Three universities Each university has 1000 patents and total revenue of $10M. University A: 2 patents earn $5M each – Too risky University B: 100 patents earn $100K each – Well-balanced and diversified University C: 1000 patents earn $10K each – Nothing is bearing fruit Webinar: Internal Metrics and Benchmarking 25

Simple tallies don’t tell the whole story photo by Petr Jan JuračkaPetr Jan Juračka 26 Webinar: Internal Metrics and Benchmarking

Using the tech transfer health index Quantifying the revenue earned per patent 27 Webinar: Internal Metrics and Benchmarking

Three universities Each university has 1000 patents and total revenue of $10M. University A: 2 patents earn $5M each – Health index = 2 University B: 100 patents earn $100K each – Health index = 100 University C: 1000 patents earn $10K each – Health index = 10 Webinar: Internal Metrics and Benchmarking 28

Comparing small universities to large ones Simple tallies discriminate against universities that have small or newer tech transfer offices. Simple tallies favor universities that have large budgets and large research expenditures The tech transfer health index corrects for size, age, budget and other variables Webinar: Internal Metrics and Benchmarking 29

Large, medium and small universities Three universities, each with 1000 patents and total revenue of $10M. University X: health index = 200. Total research exp. = $500M University Y: health index = 100. Total research exp. = $200M University Z: health index = 20. Total research exp. = $100M Which university is doing better, given their resources? Webinar: Internal Metrics and Benchmarking 30

Answer Three universities, each with 1000 patents and total revenue of $10M. University X: health index = 200. Total research exp. = $500M – 200/500 = 0.4 University Y: health index = 100. Total research exp. = $200M – 100/200 = 0.5 University Z: health index = 20. Total research exp. = $100M – 20/100 = 0.2 Webinar: Internal Metrics and Benchmarking 31

Other ways to apply the health index 32 Webinar: Internal Metrics and Benchmarking Licensing rate FTEs per startup Other categories for the health index Per TTO staff member Per department or college Per regional cluster

Analyze your own data Download the tech transfer health index calculator at uring-performance-with-the-tech- transfer-health-index/677 Webinar: Internal Metrics and Benchmarking 33

Show stakeholders the entire iceberg photo by Ludovic Hirlimann on FlickrLudovic HirlimannFlickr 34 Webinar: Internal Metrics and Benchmarking

The tech transfer health index overcomes the limitations of current metrics Webinar: Internal Metrics and Benchmarking 35 Photograph by Dennis Mojado.

Pulling it all together Quantifying the activity, performance and impact of an innovation intermediary is tough work Education of stakeholders is key It’s possible and some TTOs such as U Cal system are leading the way Additional tools such as the tech transfer health index introduce additional insight and enable comparison across different universities 36 Webinar: Internal Metrics and Benchmarking

Discussion & questions Additional comments or questions? Contact Charles Louis - or Melba Kurman - Webinar: Internal Metrics and Benchmarking 37