1. Ireland Evaluating the impact of enterprise/industrial policy supports: developing new methods and approaches Dr Helena Lenihan Department of Economics Kemmy Business School University of Limerick Helena.lenihan@ul.ie

2. Our world has changed!

3. Industrial Policy in Europe A stronger European Industry for Growth and Economic Recovery (10th Oct 2012) The pillars for a reinforced industrial policy –Investment in innovation –Better market conditions –Access to capital –Human capital and skills

5. Progress report from the ‘Action Plan’ (Q1-2012): Launch of a new Potential Exporters Division in Enterprise Ireland €150 million Partial Credit Guarantee Scheme.

6. Progress report from the ‘Action Plan’ (Q1 and Q3-2012): A second call under Innovation Fund Ireland Introduction of funding programmes targeting female entrepreneurs and also for business clustering Pilot Industry-led clustering programme

7. Progress Reports on the Action Plan for Jobs Departments and agencies have delivered to completion: Q2-2012 -A completion rate of 94%. Q3-2012-A completion rate of 87%

8. Old wine in new bottles?

9. In 2011, 82 Enterprise Ireland-backed entrepreneurship programmes took place Specifying Policy objectives

10. Lack of Enterprise policy evaluation nationally Compared to the UK, Ireland has been very poor in building datasets to inform policy design and evaluation (Ruane, 2012)

11. Defining Policy Evaluation Exploring the counterfactual What works best under what circumstances? Comprises analytically rigorous attempts to determine the impact of public policy initiatives. Key challenge to isolate the impact of a particular policy intervention.

12. Why Evaluate ? Impact Value for money Leads to policy improvements

13. Lots of calls recently for Policy Evaluation (1) 2012 Action Plan for Jobs – “We will complete evaluations of all agency programmes and schemes to ensure that they are adequately addressing the needs of today’s manufacturing businesses” (Action 7.1.4).

14. Policy Evaluation Let’s keep it on agenda - Let’s do it properly this time –not just another fad

16. (1) Current evaluation frameworks/metrics are generally lacking vis-à-vis the evaluation of ‘newer’ enterprise policies coupled with a move from ‘old’ towards ‘new’ business models. (2) Illustrate how ‘LOGIC’ models can be extended to account for broader ‘societal’ impacts.

17. The ‘world’ for the firm has also changed (1) Old business model characterised by: –desire for business stability –focus on core business activity –efficiency attainment via standard procedures –focus on developing in-house specific competencies

18. The ‘world’ for the firm has also changed (2) ‘Evolutionary’ approach –Multi-level perspective –No firm is an island Enabling conditions for enterprise start-up and growth. R&D; innovation, commercialisation of capital, networking and clustering, entrepreneurship and education at the heart of enterprise policy.

19. “New” enterprise/business model focuses on: Innovation and novelty and flexibility Lack of predictability Operates as part of a network Combines internal & external ‘know how’ Operates as part of supply chain models Invests in Human as opposed to physical capital Knowledge based economy and rests on intangible assets

20. What does the new approach mean for policy evaluation? (1) Evaluation metrics should include as many as possible of the following: Socio-economic fabric of locality or region Extent of additionality Opportunity cost issues Impact on ‘wider publics’ (Public Interest Evaluation’) and a concern with social impact and value

21. What does the new approach mean for policy evaluation? (2) Impact on sustainable development Impact on average local/regional wage rates within economy. Displacement and crowding out effects across firms. Capacity building Linkages between firms Multiplier impacts

22. What does the new approach mean for policy evaluation? (3) Knowledge creation through institutional diversity and pluralism. Impact of the environment shaping the economy upon the dynamics of entrepreneurship/enterprise activity.

23. Logic model-potential to make a worthwhile methodological contribution? Provide a guide as to what should be assessed and measured. Incorporate multi-level (layered) effects. Compare ideal against actual operation in a programme. Not only look at impact at level of firm but has potential to consider wider economy and even society effects.

24. Inputs Personnel Money Time Technology Partners Experience Events & Work-shops Meeting Email Co-op Interfirm Academic Agencies Change in Mindset: Increased Collaboration: Learning: Skills Risk with sharing info Increased knowledge; Innovation; Firm Profitability Firm Survival. Increased Attractiveness of the Local Economy; Jobs Figure 1: Logic model (chain of cause-and-effect) Unproductive relationships; Box 1 Box 2 Box 3 Adapted from Lenihan (2011) Activities Short-Term & Medium-Term Outcomes

25. Long-Term Outcomes Increased Network & Economic activity; Improved Knowledge Transfer Local, Regional National Figure 1: Logic model (chain of cause-and-effect) Positive on society Improved infrastructure; Negative on society increased traffic congestion; consumer prices Box 4 Box 5 Box 6 Adapted from Lenihan (2011) Unproductive Networks: Displacement Entrepreneurial activity in Society Box 7 Economic Development Innovation; Welfare Effects Box 8 Public Policy interventions

26. Source: Lenihan, H. (2011) ‘Enterprise policy evaluation: Is there a ‘new’ way of doing it?’ Evaluation and Program Planning, 34 (4), 323-332.

27. Conclusion and way forward for policy evaluation (1) Challenge: move away from evaluation methodologies that are purely concerned with narrowly defined economic impacts and measure impact purely at level of assisted firms. Bottom line: firm and economic activity more broadly do not operate in a silo but operate as part of a ‘holistic’ system.

28. Conclusion and way forward for policy evaluation (2) Greater investment should be made with regard to ex-ante evaluation. Evaluation as a ‘science’ is lacking when it comes to theory. Caution against the use of datasets originally collected for different purposes.

29. Conclusion and way forward for policy evaluation (3) Old Chestnut: ‘How’ to evaluate (single biggest challenge) Logic model-potential March 2012-Irish Government Economic and Evaluation Service (IGEES)

30. Conclusion and way forward for policy evaluation (4) Make the ‘Evaluations’ publicly available Most important specify policy objectives

31. The way we evaluate enterprise policy impact must also change!

32. Thank you Comments/Questions? Helena.Lenihan@ul.ie