1. Challenges & Strategies for Success Challenges & Strategies for Success Lewe Atkinson Manager, Evaluation and program improvement Evaluator as change agent

2. Summary What is MLA? Rationale for evaluation at MLA What is the MLA evaluation framework? Strategic framework for change Major challenges for change Successful change tactics Key points When will we know that we have been successful?

3. What is MLA? Industry-owned company Formed in 1998 Australian red meat industry Marketing, Market access & R&D H/O in Sydney others interstate 7 offices OS (US, Kor, Jap, China, Brussels, Russia, & Bahrain) 241 staff Expenditure about $160M/yr

4. Rationale for evaluation at MLA Effectiveness = doing the right things Efficiency = doing them well Benefit = doing the right things well Commonwealth stakeholders = matching $ and alignment with NRRPs Industry stakeholders = MLA accountability for levy income

5. Framework for independent program evaluation

6. What is the MLA evaluation framework ? Based on the DOFA framework. Input/output/outcome program logic framework. Reports alignment with Rural Research and Development Priorities. Consistent with the Council of Rural Research and Development Corporation “Guidelines for Evaluation”. Three major MLA programs undergo ex-post evaluation each year.

7. Strategic framework for change People-focused program logic: Jess Dart Brisbane 2005 1. MLA goal = build stakeholder confidence in MLA. 2. End-users = politicians, industry peak councils, and MLA board. 3. Next-users = MLA Staff: MD, EC, & PM’s. 4. We will be successful when; – Evaluations are meaningful to end-users – MLA staff manage for outcomes – Evaluation is core to MLA values

8. Major challenges for change 1. Making the complex simple 2. Embedding a focus on outcomes 3. Maintaining independence & consistency across programs 4. Building evaluation capability 5. Developing a learning culture

9. Successful change tactics 1. Alignment with MLA values, vision, and strategy 2. Engaging key stakeholders in setting better outcome KPIs 3. Engaging the Executive Committee 4. Creating cross functional teams, learning by doing & then reflecting 5. Creating layers of independence & transparency

10. Asking key evaluation questions Linking evaluation to strategy 1.Making the complex simple: Alignment with MLA values, vision, and strategy

11. What are the key evaluation questions…?

12. 1.Are MLA programs adding value? 2.Are our collaborations efficient and effective? 3.Do we facilitate innovation from concept through to commercialisation? 4.Do we facilitate increased industry capability and investment in innovation? 5.Are we identifying and effectively managing industry risks?

13. Linking evaluation to strategy?

14. MLA strategic plan and evaluation

15. Program evaluation strategy and schedule

16. Link to MLA strategy 2007 2008 2009 2010

17. 2.Embedding a focus on outcomes: Outcome KPI setting workshops across MLA & with key stakeholders Linking outcomes to employee performance measures Engaging key stakeholders in setting better outcome KPIs

18. Program evaluation is just the tip of the iceberg

19. Program logic encourages critical thinking Objectives Inputs Outputs Outcomes Market returns Benefit cost ratio Triple bottom line KPIs SocialEnvironmental Integrated framework

20. 3.Maintaining independence & consistency across programs: Executive learning on the job – Sign-off on the MLA Evaluation Strategy 2007 - 2010 Engaging the Executive Committee

21. Executive learning on the job 1. Annually determines a minimum of three major MLA programs to undergo an ex-post evaluation. 2. Must approve each program evaluation proposal prior to project commencement; – Scope – Methodology – Team – Budget – Timelines 3. Must approve each brochure in the evaluation series prior to publication.

22. 4.Building evaluation capability: Creating an evaluation-focused organisational culture Creating cross functional teams, learning by doing & then reflecting

23. Living & learning MLA values workshop series

24. Monthly review of ex-post program evaluation series Last Friday of each month Theme based on one of MLA’s values and an expert presentation on a related evaluation topic Panel discussion: Economic benefit analysis reinforced by real world case studies Reflection on lessons learned to encourage a culture of continuous improvement

25. 5.Developing a learning culture: Transparency – access via MLA website MLA contribution to RDC aggregate results Collaborating with evaluation experts to address the challenges for measuring MLA’s value add Creating layers of independence & transparency

26. Transparency – access via MLA website www.mla.com.au/evaluation What will you find? – A message from the Chairman – Downloadable PDF version of annual evaluation schedule – Downloadable PDF overview of MLA evaluation framework – Downloadable PDF for each brochure in impact series – Archive of all source documents and references sighted in brochure series

27. MLA contribution to RDC aggregate results INPUT FROM MLA EVALUATION FRAMEWORK JOINT INPUT INTO COLLABORATIVE RDC PROGRAMS

28. Challenges for measuring MLA’s value add Credible counterfactual case – independent verification of MLA’s value add Realistic attribution – only outcomes under control of MLA Monitoring adoption – meaningful outcome KPIs, other indicators identified early and monitored Increasing program complexity – requires consistent application of program logic

29. Independent Consistent Transparent Corporate learning Continuous improvement Key points

30. When will we know that we have been successful? Evaluations are meaningful to end- users MLA staff manage for outcomes Evaluation is core to MLA values

31. Questions