The Technopolis Group Paul Simmonds Director. Introduction Private limited company Founded in 1989 A spinoff from SPRU (University of Sussex) In 2012,

Slides:



Advertisements
Similar presentations
Setting a Research Agenda: Human Resources and Social Development.
Advertisements

Improving Statistical Systems: Advice from the UIS Technical Guide National training workshop Amman, Jordan October 2010.
Learning from Existing Evaluation Practices on the Impacts and Effects of Intellectual Property on Development Geneva 6th/7th October 2011 Evaluation Section.
Towards 2010 – Common Themes and Approaches across Higher Education and Vocational Education and Training in Europe - New and emerging models in vocational.
Cohesion policies: place based approach. Cohesion policy: a controversial issue Cohesion policy is an highly controversial issue Who pays for cohesion.
Intelligence Step 5 - Capacity Analysis Capacity Analysis Without capacity, the most innovative and brilliant interventions will not be implemented, wont.
Regions as the driving forces of European competitiveness: From theory to practice Interregional Seminar and Partnership Fair Enhancing university-business.
Mywish K. Maredia Michigan State University
Building open regional innovation strategies: New opportunities provided by Smart Specialisation Strategies Claire Nauwelaers Independent STI policy expert.
International Federation of Accountants International Education Standards for Professional Accountants Mark Allison, Executive Director Institute of Chartered.
1. Health Policy Research Group Department of Pharmacology & Therapeutics, College of Medicine, University of Nigeria Enugu Campus 2. Department of Clinical.
Capturing the impact of research Briony Rayfield.
March 5, 2002 Lessons Learned from GAO’s Evaluation of the Outcomes of R&D Programs Presentation to ORNL’s Conference on Estimating the Benefits of Government-Sponsored.
A Methodology for Evaluating Ex Ante the Regional Benefits of R&D Projects Stephen Roper, Nola Hewitt-Dundas and James H Love Aston Business School, Birmingham.
IACT901 - Module 1 Planning Theory - Scope & Integration ABRS Hong Kong 2004 Penney McFarlane University of Wollongong.
Performance-based funding of public research in tertiary education institutions: Country experiences Presentation to the Norwegian Fagerberg Committee:
Tõnis Mets Queensland University of Technology, Brisbane, Australia Aleksei Kelli, Ave Mets University of Tartu, Tartu, Estonia Tiit Tiimann Kaunas, May.
TAFTIE Policy Forum „Measuring innovation” New trends and challenges in innovation measurement Fred Gault UNU-MERIT.
Standards and Guidelines for Quality Assurance in the European
Embedding information literacy into the curriculum - How do we know how well we are doing? Katharine Reedy (Open University) Cathie Jackson (Cardiff University)
Click to add title Household energy efficiency programme evaluation: does it tell us what we need to know? Dr Joanne Wade CXC
Challenges of a Harmonized Global Safety Regime Jacques Repussard Director General IRSN IAEA 2007 Scientific Forum.
Key Findings from the Economic Impact Assessment of the CRC Programme 13 December 2005.
Nov. 18, 2003 TIA Consulting, Inc. Measuring Returns to Research in the Public Sector Rosalie Ruegg TIA Consulting, Inc. Research Money.
Creating Prosperity: the role of higher education in driving the UK’s creative economy UKADIA Conference London, 8 February 2011.
Semester 2: Lecture 9 Analyzing Qualitative Data: Evaluation Research Prepared by: Dr. Lloyd Waller ©
Discussion From Republic of Science to Audit Society, Irwin Feller S. Charlot ASIRPA, Paris, France; June 13, 2012.
By Bankole Ebisemiju At an Intensive & Interactive workshop on Techniques for Effective & Result Oriented Annual Operation Plan November 24th 2010 Annual.
1 Analysing the contributions of fellowships to industrial development November 2010 Johannes Dobinger, UNIDO Evaluation Group.
NETWORK STRUCTURE AND COOPERATION BETWEEN UNIVERSITIES AND INDUSTRY Prof. Ing. Tatiana Čorejová, PhD. Prof. Ing. Ján Čorej, PhD.
Mergers, Acquisitions and Export Competitiveness: Experience of Indian Manufacturing Sector Researcher: Mishra Pulak, Jaiswal Neha Publishing Year: 2012.
Methods: Pointers for good practice Ensure that the method used is adequately described Use a multi-method approach and cross-check where possible - triangulation.
Experimental Research Methods in Language Learning Chapter 1 Introduction and Overview.
GSSR Research Methodology and Methods of Social Inquiry socialinquiry.wordpress.com January 17, 2012 I. Mixed Methods Research II. Evaluation Research.
IPMA Executive Conference Value of IT September 22, 2005.
The Evaluation of Publicly Funded Research Berlin, 26/27 September 2005 Evaluation for a changing research base Paul Hubbard Head of Research Policy, HEFCE,
WIPO Pilot Project - Assisting Member States to Create an Adequate Innovation Infrastructure to Support University – Industry Collaboration.
Laura Hibbs Head of Innovation, Knowledge and Intelligence 23 rd June 2014 Introduction to NHS Improving Quality.
1 Session C: Evaluation James Farnham 3 June 2009.
Baseline Research on Executive Education in Pakistan A study for Governance Institutes Network International (GINI) Dr Pervez Tahir Dr Nadia Saleem Ms.
Quality Assurance in Staff Development Training Seminar on Implementation of Effective Quality Assurance Systems in Romanian Higher Education Institutions.
The Research Excellence Framework Expert Advisory Groups round 1 meetings February 2009 Paul Hubbard Head of Research Policy.
Third Sector Evaluation: Challenges and Opportunities Presentation to the Public Legal Education in Canada National Conference on “Making an Impact” 26.
IMPACT OF QUALITY ASSURANCE SYSTEM IN EFFECTIVENESS OF VOCATIONAL EDUCATION IN ALBANIA IMPACT OF QUALITY ASSURANCE SYSTEM IN EFFECTIVENESS OF VOCATIONAL.
Screen 1 of 22 Food Security Policies – Formulation and Implementation Policy Monitoring and Evaluation LEARNING OBJECTIVES Define the purpose of a monitoring.
María Amor Barros del Río Gender as content in research in Horizon 2020 GENDER AS CONTENT IN RESEARCH IN HORIZON 2020 CAPACITY BUILDING WORKSHOP FOR RESEARCHERS.
Office of Special Education Programs U.S. Department of Education GRANT PERFORMANCE REPORT FOR CONTINUATION FUNDING.
Advice on Data Used to Measure Outcomes Friday 20 th March 2009.
WP1: IP charter Geneva – 23rd June 2009 Contribution from CERN.
Human Resources Mobility & Training Needs Petros P. Soukoulias Head of the Regional Council for Innovation & Entrepreneurship Eastern Macedonia – Thrace.
The Roma Education Fund: Objectives, Structure and Operations Donor Conference, Paris 3 December 2004 Maureen McLaughlin Nicholas Burnett Jim Stevens Please.
PRESENTATION AT THE TECHNOLOGICAL UNIVERSITIES QUALITY FRAMEWORK Professor Sarah Moore, Chair, National Forum for the Enhancement of Teaching and Learning.
Technology transfer – The Hungarian experience Legal background Innovation Act: - Public R&D institutions are required to establish IP policy - IP created.
Influencing Policy through Research: Introduction to Principles and Tools Arnaldo Pellini:
Second International Seville Seminar on Future-Oriented Technology Analysis (FTA): Impacts on policy and decision making 28th- 29th September 2006 TOWARD.
Constructing indicators of progress/well-being with citizens/communities Dr Jonathan POTTER OECD Local Economic and Employment Development Programme.
Business and Systems Aligned. Business Empowered. TM Three Approaches to Measuring the Economic Impacts of RTD Toronto, ON. October 29, 2005 Presentation.
Quality Evaluations in Education Interventions 1 March 2016 Dr Fatima Adam Zenex Foundation.
RISK MANAGEMENT IN THE PUBLIC SECTOR CONVERGING MULTIPLE STAKEHOLDER’S EXPECTATIONS Organised by National Treasury Presented by WELEKAZI DUKUZA CEREBRO.
IDENTIFICATION OF THE SKILL NEEDS OF ENTERPRISES AND THE LABOUR MARKET AND CONTRIBUTION TO SKILL DEVELOPMENT MADE BY EDUCATION, INCLUDING FORMAL, NON -
ACF Office of Community Services (OCS) Community Services Block Grant (CSBG) Survey of Grantees Satisfaction with OCS Survey of Eligible Entities Satisfaction.
Social Sciences and Humanities in Europe: New Challenges, New Opportunities “The influence of the Framework Programmes in Social Sciences and Humanities.
Experts Workshop Brussels, 14 th Feb 2007 “Effectiveness of IST-RTD Impacts on the EU Innovation System” “Effectiveness of IST-RTD Impacts on the EU Innovation.
Taking the Road Less Travelled: Evaluating and Monitoring the Socio-Economic Impact of Investment in Research Infrastructures Dr Jelena Angelis Deputy.
Evaluating adaptation
Interpreting Foresight Process Impacts:
Questions for break-out sessions GROUP 2 messages Participants : state administrations in charge of MSFD and/or WFD, ESA and GES experts, shipping industry,
Sweden Measuring Innovation in Education 2019:
Ontario (Canada) Measuring Innovation in Education 2019:
Indonesia Measuring Innovation in Education 2019:
Presentation transcript:

The Technopolis Group Paul Simmonds Director

Introduction Private limited company Founded in 1989 A spinoff from SPRU (University of Sussex) In 2012, the Technopolis Group €8 million net sales 90 professional staff 10 offices across EU c. 50% relates to evaluation of research and innovation 2

Evaluation of research impact Research impact is not monolithic Science or co-financed applied research A research system, policy, institution or a programme Policy objectives range from improved public services to incentives for increased effort by private sector Societal impacts can be highly particular and not so readily quantified or monetised as with economic impacts Our ability to characterise and measure these different phenomena is uneven Several ‘big’ methodological challenges Diffuse and uncertain location / timing of outcomes and impacts ‘invisible’ key ingredient, knowledge spillovers Small but concentrated interventions in dynamic open systems don’t lend themselves to experimental methodologies Project fallacy Deadweight and crowding out 3

Pragmatism Theory based evaluation (logframe) to deal with context and particularities Formal assessment of available data to devise practicable methodology Use of mixed methods, looking for convergence among data streams Co-production – not a legal audit – to add to texture and insight Always endeavour to add it up, to reveal the general AND specific Precautionary principle, test findings in semi-public domain 4

Solutions Tool-box has been improving with explosion in research related data and information services (scientometrics) Bibliometrics and citation analyses Social network analyses Patent analyses These work better at higher levels, but still useful at programme or institute level used with care Use of econometric techniques is increasing for industrial R&D Control group surveys produce data to adjust for deadweight Macro-economic models being developed to cope with technical change and estimate spillovers No magic bullet however Bespoke, expert evaluators still dominate (tough to routinise and reach useful / credible conclusions) Mixed methods with choice being tuned to the particular entity 5

Qualitative research remains key however For policies and larger programmes Historical tracing of outcomes Use of longitudinal case studies to capture full extent of influences and outcomes in selected areas For industrial programmes Combine large-scale surveys to locate / weigh different types of attributable impacts with case studies of ‘known’ sample of outcomes Use of factors established through very large and exhaustive studies, applied as ‘multipliers’ to very much simpler methodologies that content themselves with researching the more immediate and visible outcomes (misses the texture of particular outcomes of course) 6

Squaring circles Evaluation remains labour intensive, and costly Contain costs by being clear about purpose Don’t try to evaluate everything and capture all impacts LFA useful to deal with context, calibrate search and agree KPIs Ambition level – non-exhaustive: have we done enough to improve things to at least the minimum desired Recruit participants to the cause from the outset, co-production Much greater use of monitoring (self-reporting) Use of primary and secondary research (converging partial indicators, normalisation) Use of standard methodology / framework for consistency Exploit multipliers / facts from selected exhaustive studies 7

Applied research centres / competence centres 8