Business and Systems Aligned. Business Empowered. TM Three Approaches to Measuring the Economic Impacts of RTD Toronto, ON. October 29, 2005 Presentation.

Slides:



Advertisements
Similar presentations
Anonymous Services, Hard to Reach Participants, and Issues around Outcome Evaluation Centre for Community Based Research United Way of Peel Region Roundtable.
Advertisements

Intelligence Step 5 - Capacity Analysis Capacity Analysis Without capacity, the most innovative and brilliant interventions will not be implemented, wont.
Assessment of Institutional Architecture for Policy Change.
Hawawini & VialletChapter 7© 2007 Thomson South-Western Chapter 7 ALTERNATIVES TO THE NET PRESENT VALUE RULE.
LICENSING “One Way of Putting Your I.P. to Work for Your Organization” Inventing and Patenting Seminar May 16, 2001.
Wider economic impacts from R&D investments Arild HervikResearch seminar
The Technopolis Group Paul Simmonds Director. Introduction Private limited company Founded in 1989 A spinoff from SPRU (University of Sussex) In 2012,
CAA Benchmarking Proposal & Responses to the CAA Paper Presented by Sylvana Thiele & Lori Palotas.
GEOGRAPHY 442 Day 6: The Planning Process. 2 Housekeeping Items I will pass around the usual announcement items, along with the attendance sheet. Start.
Moving from Analysis to Design
New-Product Development and Product Life-Cycle Strategies 9 Principles of Marketing.
Rest of Course Proposals & Research Design Measurement Sampling Survey methods Basic Statistics for Survey Analysis Experiments Other Approaches- Observation,
The Evolution of Evaluation Practices in the Canadian Federal Government Presented at CES / AEA Conference Toronto, Ontario, Canada October.
Horizon 2020 SME Instrument A recipe for success.
The MIS and Performance Measurement Systems of the Industry Energy R&D Program Dennis Rank Dennis Rank and Associates Presentation to the RTD Interest.
Innovation Measurement
Questions from a patient or carer perspective
PROFESSIONALIZATION AND SCIENTIFICATION OF SALES ENGINEERING GSSI 2014 PROF. DR.-ING. J. GÖRNE HTW AALEN, GERMANY Prof. Dr.-Ing J. Görne 1.
. Assessing the Impact of Intellectual Property Rights (IPR) in Agriculture and Responses of Developing Asia Learning from Existing Evaluation Practices.
Boundaries of a Good Price
Slide 1. Thursday, 03 September 2015 Engaging The Board Making CR a boardroom priority Rachel Pickering.
A Comparative Analysis of the Social Dimension surrounding the project partners Hannah Leichsenring Ghent | 12. November 2013.
Michael Abbott The Impacts of Integration and Trade on Labor Markets: Methodological Challenges and Consensus Findings in the NAFTA Context.
Cost-Benefit Analysis for R&D Program Evaluation Jiyoung Park
1 COMMENTS ON THE PAPER “China’s Measure in Real Term for Education” Ramesh Kolli Additional Director General Ministry of Statistics & Programme Implementation.
Measuring the Economic Impacts of Technology Transfer from Federal Labs Strategy and Development Branch Frederick (Rick) Kijek June 12, 2007.
Work Package 4 Learning Process for Reflective Policy Making 3rd TECHNICAL AND SCIENTIFIC COMMITTEE AND STEERING COMMITTEE MEETING 18 Sept – Thessaloniki.
19 th Bled eConference, 06 June Hannes Selhofer European Commission An initiative of the Hannes Selhofer empirica GmbH 19 th Bled eConference –
The regional dimension of innovation and Business Intelligence: the Emilia-Romagna experience Leda Bologni ASTER - Bologna - Italy The regional dimension.
Crossing Methodological Borders to Develop and Implement an Approach for Determining the Value of Energy Efficiency R&D Programs Presented at the American.
1 Analysing the contributions of fellowships to industrial development November 2010 Johannes Dobinger, UNIDO Evaluation Group.
Implications of alternative open access publishing models John Houghton Centre for Strategic Economic Studies Victoria University, Australia
Main results of the “Comparative Report”: an overview of the educational systems in five EU countries and theirs training offer in agricultural machinery.
Meeting of the High Level Group on Administrative Burdens Dr. Johannes Ludewig Chairman of the Nationaler Normenkontrollrat Brussels, 23 July 2013.
How to Define a “Best Practice” Michael J. Spendolini, Ph.D. President, MJS Associates KM World 2000 September 13, 2000.
Cluster Management Scorecard FITT (Fostering Interregional Exchange in ICT Technology Transfer)
McGraw-Hill/Irwin Copyright © 2006 by The McGraw-Hill Companies, Inc. All rights reserved. Chapter 8 Moving from Analysis to Design.
Adapt not Adopt: Sharing Best Practice An overview of the Beacon Scheme evaluation and some key themes and evidence Dr Zoe Radnor Warwick Business School.
The role of gold mining in supporting socio-economic development Andrew Britton | 29 September 2015.
Third Sector Evaluation: Challenges and Opportunities Presentation to the Public Legal Education in Canada National Conference on “Making an Impact” 26.
Statistics Canada Statistics Canada Statistique Canada Statistique Canada Disseminating gender statistics: The Canadian experience Heather Dryburgh, Ph.D.
European Commission Joint Evaluation Unit common to EuropeAid, Relex and Development Methodology for Evaluation of Budget support operations at Country.
& Rabat 20-22/12/
Developing Transnational Project Indicators. The purpose of indicators > Indicators are useful tools that allow programme and project managers to monitor:
Gathering Information Enables You To: 1. Reduce risk 2. Determine consumer attitudes 3. Monitor the environment 4. Coordinate strategy 5. Measure performance.
CDAE 266 Decision Making for Community Entrepreneurship Fall 2007.
Work Package 6 L2C Kick-off meeting Fontainebleau, March 7th 2006.
Performance Assessment and Monitoring of Energy Efficiency Programs Mallika Nanduri Natural Resources Canada May 12 th, 2005.
1 The Use of Surveys in Determining Outcomes of Research, Technology and Development Programs Presented at:Joint CES / AEA Conference Presented.
How to measure the impact of R&D on SD ? Laurence Esterle, MD, PhD Cermes and Ifris France Cyprus, 16 – 17 October L. ESTERLE Linking science and.
Innovativeness as potential benefit of workplace learning in enterprises Innovativeness as potential benefit of workplace learning in enterprises An overview.
Matakuliah : M0624/Information Technology Valuation
Communications Action – Public Awareness Research November 2003.
Audience Profiles RoleKey CharacteristicsValues & NeedsRecommendations C-Level Execs Challenge and opportunity is to capitalize on executives’ critical.
Research and Innovation Research and Innovation Fast Track to Innovation 1.
Methodology: IV to control for endogeneity of the measures of innovation. Results (only for regions with extreme values) Table 2. Effects from the 2SLS.
WHAT INNOVATION FOR WHAT REGIONS? Can benchmarking be a driver? Ronald POHORYLES The Interdisciplinary Centre for Comparative Research in the Social Sciences,
Arindam Bose, Moses Moreri, Thari Pheko1 ICT INDICATORS Botswana Case Study Joint ITU/ECA regional workshop on Information and Communication Technologies.
Rest of Course Proposals & Research Design Measurement Sampling
Economic Value of Geospatial Information: The Albania Case Study March 16, 2016 Annual World Bank Land & Poverty Conference AANCHAL ANAND, LAND ADMINISTRATION.
GREECE: Meeting of the National Councils for S&T policy of the EU Member Countries Prague,25-26 May 2006 National Research Council General Secretariat.
Transnational Working & Reporting Procedures. LP Seminar January 2013 Transnational Cooperation & Reporting Procedures I) Transnational cooperation: Definition.
INTRODUCTION TO BIBLIOMETRICS 1. History Terminology Uses 2.
Skolkovo PRESENTATION
Measuring the User Experience
Principles of Marketing
writing the technical rules for global trade
Module 3b Cost-Benefit-Analysis (CBA)
Competitiveness of the regional market, importance of statistics and innovations THE ROLE OF RESEARCH CENTERS IN PROMOTING OF RESEARCH Sarajevo, 8th.
Project title Company name.
Presentation transcript:

Business and Systems Aligned. Business Empowered. TM Three Approaches to Measuring the Economic Impacts of RTD Toronto, ON. October 29, 2005 Presentation to the CES/AES Conference A. Dennis Rank and Frederick Kijeck

1 © 2005 BearingPoint LP Overview Three studies of three programs. Programs had very different goals, structure, operation, and impacts. Three different challenges, and solutions. Thus three entirely different methods. All studies measure benefits from “national perspective”

2 © 2005 BearingPoint LP Three Programs

3 © 2005 BearingPoint LP Three Studies

4 © 2005 BearingPoint LP Precarn Challenge and Solutions

5 © 2005 BearingPoint LP CSRS Challenges And Solutions

6 © 2005 BearingPoint LP HPPM Challenges And Solutions

7 © 2005 BearingPoint LP Precarn Methodology Bottom-up Partial B-C: Net benefits = (Gross benefits of “big winners”) – (Implementation costs). Total program costs = (Grants and contributions) + (Overhead costs) + (Costs to all other parties). All benefits and costs discounted. Net Present Value (NPV) NPV = (Net benefits of “big winners”) – (Total program costs to all parties). Benefit: Cost ratio (B/C) B/C = (Net benefits of “big winners”) ÷ (Total program costs to all parties).

8 © 2005 BearingPoint LP Precarn Findings Commercial net benefits, cost savings from 6 Big Winner 5.5% discount rate: Lower bound: — Net Benefits = $312 million — NPV = $83.7 million — B/C ratio = 1.37 Upper bound: — Net Benefits = $1.3 billion — NPV = $1.1 billion — B/C ratio = 5.77

9 © 2005 BearingPoint LP Precarn Lessons Learned Crucial to know projects and impacts, likely “big winners”. One technology, one firm, one market always easiest to deal with – makes partial B-C an obvious choice. But ONLY if can get confidential $ data. Clients still need “education” about B-C.

10 © 2005 BearingPoint LP CSRS Methodology Top-down Gross Value Added (GVA): — CSRS influence estimated (Low to High) for 8 sectors & 30+ sub-sectors — Influence quantified by proxy % index. — $ Influence per sector = (Sector GVA) x (Proxy % index). — Total influence of CSRS on Canada = Sum of GVA influences per sector. Bolstered by other study activities: — User surveys & interviews, focus groups, policy analysis, review of international and Canadian literature, qualitative case studies, etc. — Benchmarking to NSRS elements in other countries. — NSRS comparison indices in benchmark countries on $/capita, $/sq. km., $/GDP.

11 © 2005 BearingPoint LP CSRS Findings CSRS “influences” $60B - $90B of Canadian GVA, or about 6% – 9% of total. Backbone of many industrial, government, and research applications, though many users know little about NSRS or CSRS. This figure (as far as possible) modeled on geodetic impacts alone.

12 © 2005 BearingPoint LP CSRS Lessons Learned Top-down sometimes only way to go. But top-down methods difficult to understand, open to interpretation – need other lines of evidence (the more quantitative the better). ID of incremental value-added is harder than in bottom-up methods. Entrenched suspicions difficult to overcome.

13 © 2005 BearingPoint LP HPPM Methodology Qualitative case studies of interesting commercial and policy impacts. Follow-up case studies focused on very specific, concrete long-term impacts first identified by ex-HPPM expert. Impacts confirmed through modified peer review re: research quality, relevance, and impacts.

14 © 2005 BearingPoint LP HPPM Findings HPPM believed to be a commercial failure: No direct commercial applications of technologies (e.g., application of patents). Partially due to nature of technical targets – HPPM would not support applied R&D, and difficult to commercialize R&D not in partners’ product line. Scientific importance actually quite high: HPPM showed better to improve existing membranes than develop new ones. Understood physics and chemistry re. how and why membranes worked. These were breakthrough concepts for all membranes – not just polymers, & not just separation membranes.

15 © 2005 BearingPoint LP HPPM Findings (cont’d) High long-term commercial impacts: New products (e.g., nanofilters) related to HPPM topics, or non-polymer materials (e.g., ceramics), or other types of polymers, or in entirely new fields. Linked – often quite indirectly – to HPPM ideas. Some firms world leaders & “first to market” partially because of HPPM. Impossible to obtain sales figures for these products and processes. Impossible to judge exact role of HPPM, but good reasons to believe it was at least moderately important in these later developments. Also impacts on competencies, researchers’ careers, & innovation networks. Possible impacts on patent laws, R&D policies, & design of R&D programs.

16 © 2005 BearingPoint LP HPPM Lessons Learned Tracking long-term impacts “easier” when considerable time elapsed. Need to be very specific and explicit when trying to ID and track these impacts – they are not well understood by participants. Need external, independent confirmation. Client and analyst need to be flexible re: methods.

17 © 2005 BearingPoint LP Overall Lessons Learned 1. No “magic bullet” – cookie cutter approach won’t work. 2. Difficult to overcome entrenched attitudes and suspicions – many lines of evidence, lots of documentation help, but don’t entirely solve problem. 3. Very useful for program to track “big winners” & users, even for top-down studies. 4. Multi-step process for long-term knock-on & spin-off benefits works, but need to ID very explicit impacts – and don’t ignore non-technology transfer benefits. 5. Follow your nose, be flexible – don’t ignore oddballs.