Diagnosis of NIS and Development of STI Strategies in the Third Asia-Pacific NIS Forum Diagnosis of NIS and Development of STI Strategies in the Open Innovation Framework 8-9 April 2015 Bangkok, Thailand Contribution: Impact Evaluation of Innovation Policy in Slovenia with “Three Capital Model” Bojan Radej Slovenian Evaluation Society & Western Balkan Evaluation Network, bojan.radej@siol.net This presentation is licensed with the CC - Creative Commons, free share for non-commercial purposes if source is acknowledged

4C - 4 Capital Model Originally: 4C or 3C model? Ekins, Munasingh (1992, Rio de Janeiro). Broadening originally 3C model of SD (E, S, N; Brundtland commission, 1987) also with H capital. Ekins (2006), valuation of SD impacts of regional development programs in EU. Radej (2013, 2014), 4C/3C model for policy impact evaluation of synergy, integration, horizontal issues. Theory of meso level approach, puts forvard intermediation between 3 Capitals, applies hybrid categories of analysis, heterogeneous conclusions, instead of domination of one theory against the (all) other. 4C or 3C model? The difference is not between 3 and 4 capitals but between one and many: No involuntary trade-offs between capitals. We are dealing with radical diversity, asking how to provide for policy integration (in planning, programing, evaluation)? Translation of plural principles into practice?

Linear (Vertical) Evaluation Theory Source: Indicative guidelines on evaluation methods : ex ante evaluation, EC, 2007-2013

Operationalization of 4C model http://www.csdila.unimelb.edu.au/sis/Sustainability_Theories/Four_capital_model.html http://www.csdila.unimelb.edu.au/sis/Concepts_and_Theories.html http://www.forumforthefuture.org/ http://www.snam.it/en/Sustainability/responsibility_towards_everyone/generating_value.html

3C and 4C model 5

Operationalizing 3C model

KNOWLEDGE TRIANGLE (RBE) EIT‘s Knowledge Triangle From concept to policy Mid-term evaluation of selected measures in inovation policy in Slovenia, 2007-11 (MK Projekt, 2012) Level (Triangle) KNOWLEDGE TRIANGLE (RBE) 2. Level (Domains) Research & Technology (R) Entrepreneurship (B) Higher Education (E) 3. Level (Priority) Regional Development Priority 1, Measure 1 Human Dev. Progr. Priority 3, Measure 3 Priority 3, Measure 1 4. Level (Measures) Strategic research development in companies Centres of Excellence Competence Centres Young researchers Career Centres Bologna Visiting professors PhD scolarship In Higher Education 7

Evaluation of vertical and horizontal impacts Horizontal criteria of evaluation Measures Efficiency (financial data) Environm. impacts (survey) Private capital leverage (fin, data) Regional policentris (financial data) Gender equality (survey) Sustainable impacts (survey) Impact on business environment (survey) Employment (data) Wider social impacts (survey) R B E Strategic research, Centres of Excellence, Competence Centres 3,2 4,2 3,0 2,6 3,7 3,1 Young researchers 2,7 2,8 1,0 5,0 3,3 3,8 3,4 PhD scolarship n.r. 3,5 2,2 Bologna 2,5 2,9 Visiting profesors 4,0 Carreer centres 3,9

CONSISTENCY in stated purposes Consistency between broader policy scopes and specific instruments’ goals I-O matrix Scope Goals R B E 3,5 3,3 1,8 3,1 2,7 2,5 2,2 3,8 Correlation matrix 3,2 2,0 - 3,0 E (3,8) (3,0) (2,7) B (2,0) R (3,5) (3,2) 1. MIN: Najmanj povezana OP RČV RP3 z OP RR RP1 (1,8 od 5 možnih točk, Leontjeva matrika). 2. DIAG: Najbolj so povezani inštrumenti RČV RP3, vsaj gledano s stališča njunih širših družbenih vplivov (3,8 v obeh matrikah). To pomeni, da RČV RP3 najbolje dosega svoje strateške namene od vseh analiziranih RP. RČV RP1 je v pogledu doseganja svojih primarnih strateških namenov opazno najšibkejši (2,7 na diagonali obeh matrik). 3. Na diagonali > nediagonalnih polji. To pomeni, da uporabniki opazno bolje ocenjujejo povezanost med inštrumenti istih RP kot inštrumentov iz različnih RP. Od tod sledi, da je izziv integracije inštrumentov inovacijske politike na ravni njihovih strateških namenov še zelo velik.

SYNERGY of impacts Synergy between various instruments’ impacts I-O matrix Results (hor.ind.) Activity R B E 3,3 3,4 3,1 2,6 4,5 2,7 3,6 Correlation matrix 3,0 2,9 - E (3,4) (4,5) B (3,1) (2,9) R (3,3) (3,0) Odlični širši družbeni vplivi RČV RP1 na svojem ožjem področju primarnih usmeritev (upoštevaje metodološki konstrukt so to enakost spolov in trajnostni učinki). RR RP1 in RČV RP1 dosegata le dobre (3,3 in 3,4) učinke 3. Ocena navzkrižnega vpliva izvajanja. 4. Diagonalni > nediagonalni --> nizka sinergija. 5. Primerjava ocene skladnosti namenov (2,7) in ocene sinergije vplivov (3,1) priča, da instrumenti v praksi po mnenju uporabnikov delujejo opazno bolje kot so bili zastavljeni.

Further use of 4C Model Model is applicable also for planning/ programming/evaluation: Nature of overlaps between domains (such as for achieving territorial, social or any other sort of cohesion, integration) Synthesis, consensus building in conditions of deep oppositions, contradictions Setting collective priorities (local potentials vs. local needs). Simplifying and organising monitoring of multidimensional phenomena

Support for users Radej B. Apples and Oranges: Synthesis without a common denominator. Ljubljana: Slovenian Evaluation Society, Working Papers, 7/1(February 2014). http://www.sdeval.si/komisija-za-vrednotenje/publikacije/105-sde-delovni-zvezek-t-32008-aurirano-apr-2010 Radej B., M. Golobič. Divided we stand: Social integration in the middle, Ljubljana: Slovenian Evaluation Society, Working Papers, 6/1(June 2013). http://www.sdeval.si/39-objave/objave/490-divided-we-stand & Unpublished materials & Package of materials (Creative commons – for non-profit use): bradej@gmail.com