1 www.pmn.net The Use of Surveys in Determining Outcomes of Research, Technology and Development Programs Presented at:Joint CES / AEA Conference Presented.

Slides:

Advertisements

Similar presentations

Public Private Alliances for Piloting Palm Oil Fortification with Vitamin A in Indonesia ADB-UNICEF Regional Workshop The Role of Non-State Providers in.

Advertisements

Division: EIDD WTO TBT Workshop on Good Regulatory Practice March 2008 Focus on Transparency and Consultation.

Research center for metal in the Istrian region - MET.R.IS. Istrian development agency - IDA.

Establishing a standardised methodology to measure JEREMIE impact Álvaro Navarro Innovation and Development Agency of Andalusia, Spain Brussels, 20 th.

Framework for K-Farm Green Value Chain Production of Carambola

National Agenda for CSO Capacity Development Yin Soriya, Ph.D.

And sustainable development, the example of KFUPM.

1 Stakeholder Engagement in Relu Jeremy Phillipson Assistant Director, Relu Programme.

Presentation at The Conference for Family Literacy Louisville, Kentucky By Apter & O’Connor Associates April 2013 Evaluating Our Coalition: Are We Making.

PARTNERSHIPS OF GOVERNMENTAL AND NON-GOVERNMENTAL ORGANIZATIONS (ASSOCIATIONS) IN THE SPHERE OF TOURISM: RUSSIAN AND FOREIGN EXPERIENCE Moscow, Russian.

4μ Network of Excellence Delivering the Knowledge Community in Multi- Material Micro Manufacture Rob Bell Natalie Withenshaw.

SCCD Year 4 Analysis All 32 reports received on time at end of fy Carbon Trust, Adaptation Scotland, SSN analysis CAG Consultants engaged to undertake.

THREE YEAR PLAN PRESENTATION MANDATE Tasked with the responsibility of improving productivity in all spheres of the nation’s economic and.

Presented to: AEA Evaluation Conference November 2, 2006 Presented by:

University Miguel Hernández of Elche Some GlobalStart Tools PAXIS Workshop;Salamanca, June 2005.

The Evolution of Evaluation Practices in the Canadian Federal Government Presented at CES / AEA Conference Toronto, Ontario, Canada October.

PHAB's Approach to Internal and External Evaluation Jessica Kronstadt | Director of Research and Evaluation | November 18, 2014 APHA 2014 Annual Meeting.

Networks ∙ Services ∙ People John DYER TF-MSP Video Conference Community Procurement Support Building on the SPOT-ON Proposal Smart Procurement,

Axis 3: Diversification of the rural economy and Quality of Life in rural areas Axis 4: The Leader approach DG AGRI, October 2005 Rural Development

Erica Allis United Nations Environment Programme Eleventh Caribbean Conference on Sustainable Tourism Development May 9 th -13 th St. Michael, Barbados.

1 European Development Days Brussels October 2012.

Training of National Judges INFO DAY Introduction to the new Call for Proposals 2014 Raffaella Battella - DG Competition.

The Vision Implementation Project

Lakshman Wijeyewardena Director General Industrial Services Bureau Role of Business Development Service Providers – Implementation of Technology.

Agricultural Advisory Service Department of Fisheries MSc Boris Župan Zagreb, September 2013 TAIEX, Expert Mission on Aquaculture.

Organization Mission Organizations That Use Evaluative Thinking Will Develop mission statements specific enough to provide a basis for goals and.

Attracting appropriate user funding in the context of declining public funding.

Partnership Satisfaction & Impact Survey FITT (Fosterring Interregional Exchange in ICT Technology Transfer)

Partnerships and collaboration Working together: good for business, good for research I work for business.gov.au but also thought it would be a good opportunity.

CTCN: Support implementation of NAMAs

© Dr. John T. Whiting All Rights Reserved Slide 1 The Rationale for Integrating IT Assets into the Strategic Business.

Public Participation and the Advisory Committee Process A Collaborative Partnership For Water Resources Toni M. Johnson, Chief Water Information Coordination.

Chapter 9 Developing an Effective Knowledge Service

ENERGY EFFICIENCY CENTRE GEORGIA  Support to renewable energy and energy efficiency utilization for sustainable development and as a result improve national.

Toolbox CRC programme managers – Dag Kavlie, RCN Analysis of indicators used for CRC Monitoring and Evaluation Ljubljana, 15 September 2009.

APPLICATION FORM OF ROBINWOOD SUBPROJECT SECOND STEP 1. The short listed Local Beneficiaries work together to create international partnerships and prepare.

Market entry strategies introduction. Potential determinants of the firm´s choice of foreign markets THE COMPANY Degree of internationalization and overseas.

Engaging Freight and Supply Chain Representatives in Public Sector Projects.

PROJECT MANAGEMENT. A project is one – having a specific objective to be completed within certain specifications – having defined start and end dates.

Rural Development Council 8 th December Contents Scottish Enterprise Focus Economic Downturn Implications & Actions Core Rural Objectives HIE Economic.

Characteristics of a National Innovation System

Internationalisation of Finnish Public Research Organisations Dr. Antti Pelkonen Senior Scientist, VTT Technical Research Centre of Finland

A successful proposal – in the eyes of an evaluator Tempus Information Day 14 January 2011 Annika Sundbäck, External expert,Tempus selection 2009 and 2010.

Human + Knowledge = Business² Social Innovation in SME’s Dr. Rob Gründemann Professor of Organizational Configurations and Work Relations Research Centre.

| Collaboration at Rural Business Approach.

1 Click to edit Master text styles Second level Third level Fourth level Fifth level Administrative Support for Large- Scale Funding Applications – Session.

Digital Atlas Workshop- Institutional Working Group July 26, 2006 Institutional Working Group.

The RDI Governance System Vasileios Pitsinigkos Head of Managing Authority of Eastern Macedonia - Thrace Region.

TRANSPORTATION RESEARCH BOARD WATER SCIENCE AND TECHNOLOGY BOARD TRANSPORTATION RESEARCH BOARD TRB’s Vision for Transportation Research.

Building Effective Staff Development to Support Employer Engagement Jane Timlin & Renata Eyres The University of Salford.

1 The project is financed from the European Union funds within the framework of Erasmus+, Key Action 2: Cooperation for innovation and the exchange of.

REGIONAL POLICY EUROPEAN COMMISSION The contribution of EU Regional/Cohesion programmes Corinne Hermant-de Callataÿ European Commission,

Creating Innovation through International collaboration Melanie Relton & Helen Kidd, British Council 7 April 2013, Qatar.

TPO Services An institutional perspective By: Miguel CAMACHO & Ann PENISTAN, TS/DBIS Date: 31 March 2011.

CLUSTERING PROJECT Oto Hudec Faculty of Economics Technical University of Košice.

Sales & Marketing Do’s & Don’ts For The Entrepreneur CINA October 30, 2004.

1 PRESENTATION TO THE SELECT COMMITTEE ON LABOUR AND PUBLIC ENTERPRISES (NCOP) 26 AUGUST 2009.

Version VTT TECHNOLOGY STUDIES Evaluating the societal impacts of Public research organisations: A (belated) paradigm shift in the making Kaisa.

MTT Agrifood Research Finland - strategy Target for 2015 and operating model set for achieving it.

Claire Arbon Enterprise Support Peter Christian Faculty of Art, Design and Architecture

Determining the Added-Value of Partnerships

Engaging with End–Users of Research OCTOBER 2015 Cathy Harris MBA, MInstKT Research Engagement Manager Research & Enterprise Extn 6755,

Monitoring and Evaluation Systems for NARS organizations in Papua New Guinea Day 4. Session 10. Evaluation.

UNCLASSIFIED Lift the living standards and wellbeing of all Victorians by sustainably growing Victoria’s economy and employment and by working with the.

Design of foresight-based evaluation in Tekes Activities

Alberta Livestock and Meat Agency

CIRAZ Key factor in Smart Specialization Strategy implementation

Climate finance and the role of civil society

Competition Commission Strategic Plan

SuperBIO: Evaluation Els Van de Velde

Presentation transcript:

1 The Use of Surveys in Determining Outcomes of Research, Technology and Development Programs Presented at:Joint CES / AEA Conference Presented by:Suzanne Lafortune, Partner Performance Management Network Inc. Date:October 26, 2005 Tel:(613) Fax:(613)

Results & Resources versus Reach Source: Steve Montague, The Three Rs of Performance: Core concepts for planning, measurement, and management, Performance Management Network Inc., 1997, p 120. The ‘clustering’ of services or activities by results, resources per user, and reach can provide new perspective. Intensive Problem Solving / R&D Technical Specialist Support Education Information / Advice Awareness Building Reach Results + Resources per user education and communication (workshops, speaking, trade shows) routine advisory dissemination of voluntary codes mass media Information products policy development program funding enforcement and prosecution specialized advisory investigations inspections and monitoring

Frequency and Type of Measurement Reach Results & Resources versus Reach Results + Resources per user Intensive Problem Solving / R&D Technical Specialist Support Education Information / Advice Awareness Building Source: Steve Montague, The Three Rs of Performance: Core concepts for planning measurement and management, Performance Management Network Inc., 1997, p.20 Methods OngoingForm. EvalSum. Eval Documents Databases Interviews Surveys** Case studies All Documents Databases Surveys* Interviews Surveys** Case studies All Documents Databases Surveys* Interviews Surveys** Documents Databases Surveys* Interviews Surveys** Documents Databases Surveys* Interviews 3 * Project related surveys ** Periodic surveys related to more in-depth impacts

4 Why Use Surveys for Formative / Early Evaluations of R&D Programs? u Relevance of program design to R&D needs u Satisfaction with ways in which program is designed and delivered ä Eligibility criteria ä Application process ä Use of advisory boards ä Deadlines ä Time of year (academic community versus private sector) ä Speed of decision making ä Feedback / advice / staff communications and support before project / in early phases of project ä Technical competence / expertise of staff

5 Why Use Surveys for Formative / Early Evaluations of R&D Programs? u Early outcomes – before project ä Awareness building ä Strategic alliances with partners ä Leveraging u Early outcomes – during project ä Active participation of partners ä Knowledge gained ä Job creation / maintenance within organization during project u Early outcomes – at the very end of the project ä Improved technical capability ä New / improved product / process / technology / service u Project incrementality ä Important to measure early on, particularly for longer term R&D projects

6 Why Use Surveys for Summative Evaluations of R&D Programs? u Relevance of program implementation to R&D needs u Satisfaction with ways in which program was implemented ä Feedback / advice / staff communications and support ä Technical competence / expertise of staff ä Reporting requirements ä Payment process

7 Why Use Surveys for Summative Evaluations of R&D Programs? u Intermediate outcomes ä Impact on recipient organizations Ô Revenues / sales Ô Productivity Ô Sustained / long term jobs Ô Competitiveness / increased market share Ô Increased profitability Ô Cost savings Ô Exports / new client base Ô Diversification Ô Expansion Ô Etc. ä Impact on industry Ô Spin offs Ô Technology transfer

8 Why Use Surveys for Summative Evaluations of R&D Programs? u Long term outcomes ä Impact on the economy / on society Ô Socio-economic benefits – Reduced consumer costs – Employment – Improved quality of life – Etc. Ô Environmental benefits Ô Improved energy efficiency Ô Health and safety benefits Ô Etc.

9 Types of Target Groups to Survey for R&D Evaluations u Non-recipients ä Formative evaluation – relevance to needs, satisfaction, incrementality ä Summative evaluation – relevance to needs, incrementality u Recipients u Project partners ä Formative evaluation – impact of program on participation, early project outcomes / results on partner organizations ä Summative evaluation – extent of participation, long-term collaborations, other outcomes / results on partner organizations u Beneficiaries ä Summative evaluation – impacts of projects on beneficiaries