Learning more from evaluations The use of thematic approach and impact modelling in evaluating public support measures  Innovation Policy  Tekes Technology Programmes  Thematic programme evaluation  Conclusions Jari Romanainen Executive Director, Strategy National Technology Agency, Finland Outline Research and the Knowledge Based Society - Measuring the Link Galway, 24 May, 2004

Innovation policy governance èThe role of innovation and innovation policy in other sectors èIncreasing number of stakeholders èRegional development, employment, social divide, etc. Non-technological innovation ènew business models and value chains èservice innovation, social innovation Renewal of traditional sectors  industries  public sector  universities Maintaining the balance  science - industry  economic - social  existing - new Framework conditions, creating the demand  regulatory framework, e.g. IPR, taxation of private VC investments  public procurement, lead markets, etc. From 2 nd to 3 rd Generation Innovation Policy

Information and communications technology Material technology Biotechnology Key Areas of Industrial Renewal and Welfare-promotion Sustainable development future energy solutions environmental technologies ecological effectiveness and low-emission processes life-cycle solutions Knowledge-intensive services knowledge-intensive business services product-integrated services new technology-related services Intelligent products, processes and systems adaptive and intelligent products, materials and systems navigation and identification virtual models Welfare information and communications technology for health care functional foods targeted pharmaceuticals and diagnostics healthy and safe living environment DM Copyright © Tekes Business competence in the networked economy networks with new value cluster cooperation fast commercialisation of ideas digital economy

Implementation Analysing policy needs Setting policy agenda Impact evaluation Impact evaluation Performance evaluation Performance evaluation Private interests Private interests Public interests Public interests Research National strategy National strategy Strategic intelligence Strategic intelligence Policy evaluation Policy evaluation Sector policies Sector policies Implementation strategies Implementation strategies Instrument set-up Instrument set-up Innovation Policy Cycle

Competitiveness, profitability and growth New businesses, start-ups Impact of Tekes activities Research institutes and universities Enterprises Social and environmental impacts Projects and programmes International cooperation Tekes provides expert services and R&D funding coordinates programmes DM Copyright © Tekes Boosting exports, broadening Finland’s industrial base, generating new jobs and improving wellbeing.

What are Technology Programmes? extensive programmes initiated by Tekes and consisting of numerous projects focused on a key technology sector implemented in cooperation by companies and research units companies can participate with their own projects or by joining common research projects projects and results are partially public DM Copyright © Tekes

Technology programme model Steering enterprises Tekes Grants Company R&D projects Public research projects Synergy Networking Part financing Implementation of results of public research is based on execution and networking in parallel with company projects. preparation coordination decision making DM Copyright © Tekes Grants Loans Capital loans

Key figures of technology programmes 23 ongoing programmes from the beginning of 2004 with a total cost of 1.2 billion euros Each programme typically lasts 3-5 years 2,000 company participations annually 800 research unit participations annually Tekes usually finances 60-80% of university projects 25-50% of company projects DM Copyright © Tekes

MARKETS  quality competitiveness  price competitiveness  environmental expertise F O R E S T I N D U S T R Y  Pulp and paper technology  Wood products industry  Machines, machinery and processes A11/ Securing competitiveness in the Forest Industry FORESTRY CONSTRUCTION AND WOOD TECHNOLOGY  modification of wood  construction technology and architecture  logistics, assembly MANUFACTURING TECHNOLOGY  forestry machines  process and production machinery  material technology BIOTECHNOLOGY  enzymes  rot prevention  gene technology ENERGY AND ENVIRON- MENTAL TECHNOLOGY  biofuels, combustion technology  ecobalances  closed cycles  energy-saving and emissions CHEMICAL TECHNOLOGY  paper and bleaching chemicals  surface treatment substances  pigments, adhesives INFORMATION TECHNOLOGY  sensors, measuring and control  computational intelligence, simulation and machine vision  multimedia and telecommunication  tomography

Programme evaluation at Tekes First generation –Performance evaluation –Feedback for participants –Peer reviews Second generation –Separation of performance and impact evaluation –Internal performance evaluation supporting monitoring –External impact evaluation supporting programme design Third generation –Impact modelling and thematic evaluations –Supporting renewal of technology programme concept –Better support for policy design

Knowledge transfer awareness, co-operation, mediation Market conditions customers competitors value-chains Knowledge base education science R&D Framework conditions culture, social capital, financial services, innovation services, regulation skilled labour knowledge ideas sophisticated demand competition clustering Learning governance, strategic foresight, evaluation, monitoring, assessment regulation services money Innovation process innovation capabilities complementary knowledge complementary resources Systemic approach to innovation policy

Technologies and innovations that enhance well-fare and sustainability Impact model of technology programmes The activities of the programme and the projects creates competitiveness, job, well-fare and environmental impacts Technologies and innovations that improve productivity and create new added value Technology development Strategic renewal New knowledge Operational renewal Behavioural change in organisations and clusters

Conceptual technology programme evaluations in (1/2) Programmes for new business operating models - to develop new business concepts - based on value chain analysis - 3 programmes from construction and manufacturing - evaluation report 10/2002 Programmes for innovations in changing business regulation - deregulation (electricity) and increasing regulation (environmental) - evaluation report 12/02 Industrial cluster programmes - activating companies to R&D, networking, industrial R&D strategy, strong regional and SME approach - programmes for furniture, stone and computing clusters - evaluation report 18/ “evolution of sector programmes” – construction cluster report 6/2003

… conceptual evaluations (1/2) Programmes for innovations and commercialisation - programmes with new product and new business goals - user- and exploitation oriented - programmes for pharmaceuticals, diagnostics, computing etc. - evaluation in 2003 Targeted programmes - problem-oriented focussed and well planned - “hands-on management” of projects - programmes for plastics, paper and machinery clusters - evaluation report 13/2003 Programmes looking for technology synergy - technology and knowledge transfer oriented - cross industrial programmes - evaluation report 11/2003

Commercialisation Technology Technology International competition International competition Regu- lation Regu- lation Commercialisation Evaluation of programmes during the transition from Technology orientation to Commercialisation Thematic evaluation: Programmes targeting behavioural change and clustering

Targeting behavioural change and clustering requires a more comprehensive approach Firm R&D Firm strategic behaviour TechnologyServices Target Content Business models Now Previously Source: Gaia Group

Technology programmes is a concept delivering a tailored mix of value added services – a toolbox Source: Gaia Group

Internationalisation in technology programmes Programme proposal preparation: -Focus, -Structure, -Timing Feasibility check & decision Expressed or latent need for public action. Programme launch & Call for projects Phase I: Programme promotion, project selection, network building Phase II: Joint activities, first results, intensified networking Phase III Result collection, dissemination, future orientation Midterm evaluation Programme evaluation Final reporting DefinitionImplementation Follow-up Programme output, impact & triggered changes Follow-up programmes Strategic decisions Analysis of international context(?): - Needs - Competition - Partners - Competence Joint launch, international promotion, open call? Finding of foreign partners / agencies / programmes? Foreign projects? International co- operation, market checking, application partners? International promotion of results? International business promotion, internationalisation of projects? Project co- operation? Research oriented international activities Commercially oriented international activities Source: Advansis

Rationales for internationalisation Firms Did not participate/play an important role Reaching Intrn leading-edge (upgrading) Increasing lead in relation to intrn leading-edge (upgrading) Exploiting Finnish competence Research institutes Exploiting Finnish competence (exchanging knowledge) International research programmes Intrn research & activation of business in the sector Home-base extension(?) Home-base extension + export orientation Increasing lead in relation to intrn leading- edge (upgrading) Domestic research (+networking) Knowledge-flows: Research -> business sector Domestic R&D – ”No need for others” Commercialisation of research competence Reaching Intrn leading- edge (upgrading) Technology transfer(?) ”Lakki kädessä”Knowledge flows: Business sector -> research Export orientation and activation of supporting research Did not participate/pl ay an important role No programmeDomestic activation of sectors Domestic R&D, (+networking) Strong export orientation Antares? FFusion2 GPB Kesto (turb) Kenno Presto Spin Kesto (soda) Tesla Navi iWell Source: Advansis

Findings & statements 1(2) Rationales for internationalisation: Key rationales for internationalisation relate to seeking better technological knowledge from lead developers to up-grade Finnish competence and experience on applying & delivering technological knowledge in target markets. For most companies, internationalisation aims are naturally linked to business objectives – how to get access to information on the application side of technologies. ‘Internationalisation should not be done just because the programme is building international links. The reasons must be found from each business.’ Large companies are international, almost by definition. When it comes to promoting international co-operation for commercial aims there is little reason for large companies to join. Source: Advansis

Findings & statements 2(2) Impact / added value of technology programmes: Programmes can have an important impact on the integration/synergy of the industry. This is important when trying to increase the awareness of Finnish competence abroad – one good reference speaks for the whole sector. Being part of a technology programme gives an image abroad that you are among the selected, qualified and backed-up projects. It does improve your credibility. International partners often appreciate formal qualifications, along with practical cultural & language skills. It should not be forgotten, that technology programmes are closely linked to graduate schools, which provide young, educated resources to their fields. Source: Advansis

Benefits of Thematic Evaluations Provide a much better insight into technology programmes as a concept Allows stronger conclusions and recommendations Provides understanding of sector specificity and generality of different impact mechanisms Create a learning platform for exchanging good practices Improves the design of new programmes

Conclusions Innovation policy consists of a mix of schemes targeted at several systemic failures Evaluation should recognise the role single schemes play in the policy mix targeting these failures Impact modelling and thematic approach can be useful in better understanding the role different schemes play in policy implementation, impact of mix of schemes and identifying good practices across schemes Impact modelling and thematic approach require combination of evaluation methodologies and demand methodological development Evaluation should be recognised as a continuous learning process

Measuring the link Requires in-depth understanding of the various interconnected mechanisms through which the impact of research and innovation policy are eventually realised Real understanding of the various impact mechanisms –Focus on systems, processes and policy mix instead of structures, actors, bilateral interactions and single instruments –Combinations of methodologies Focus on systemic failures Human capital –Focus on SET is not enough! –Creativeness, innovativeness, multidisciplinary and networking skills, adaptive skill, learning skills –Life-long learning, links between education and research, social innovation