Building Capacity and Culture within a Research & Evaluation Team anzea July 2007 Research & Evaluation Team Research Division

2 Plan for session 1 Scene setting background to the Ministry’s Evaluation Strategy and its capacity building focus in the team and across the ministry 2Learning the ropes 3Trading Places 4Capability development for the future 5Discussion

3 1 Setting the scene

4 Background to the evaluation strategy Changes in public sector management framework Evaluation Strategy Development Project Capacity and capability focus 3 big outcomes: Improved prioritisation and coordination of evaluation and evaluative activity Joint understandings about evaluation and its role in organisational improvement and accountability Increased quality of evaluation

5 Ministry of Education Evaluation Strategy

6 Implementing the strategy Across the ministry Within the team Grounding strategy inside team’s practice within the ministry

7 Plausible theory of change

8 Building the capability of the Research & Evaluation Team Building knowledge, skills and expertise of the driver for change Building knowledge, skills and expertise of the R&E Team

9 Building Ministry of Education evaluation capability Organisational professional development / workshops (Kate McKegg, Jane Davidson, Patricia Rogers) Workshops on the Evaluation Standards R&E team using the “working” Evaluation Standards when advice sought

10 Effective strategy, policy advice and delivery Use of evaluative information in business planning and decision making Policy development and processes - Budget bids process

11 Building the capability of the Research & Evaluation Team Within the team, presenting examples of practice and discussing practice issues and challenges Professional development (both personal and for the team as a whole) Team participation in presentations/workshops open to all staff Shared involvement by the R&E team in developing Evaluation Standards (practice meetings and drafting the standards) Team participation in Evaluation Standards workshops delivered to key Ministry of Education staff

12 Developing evaluation practice within the Ministry Developing evaluation standards What are the evaluation standards? Using practice meetings to develop the evaluation standards Making practice explicit – from tacit, individual practice to explicit, joint practice

13 What the standards cover 1.Deciding whether to evaluate 2.Focusing the evaluation 3.Engaging an evaluator and contracting 4.Scoping and designing the evaluation 5.Evaluation implementation – data collection and analysis 6.Communicating results – information exchange, and reporting during and at the end of evaluation 7.Managing evaluation 8.Meta-evaluation

14 2 Learning the ropes

15 Request for help with an Evaluation Evaluation of Education for Sustainability - A request for assistance Education for Sustainability? Made up of three initiatives: Enviroschools School Support Services Mātauranga Tāiao Cross curriculum – used to be Environmental Education

16 So what next? New member of the Research and Evaluation Team assigned to the task New to Evaluation ►Given a pile of papers about Education for Sustainability So, I set to work using: Standard 1 – Whether to Evaluate Standard 2 – Focusing the Evaluation

17 Facilitated two activities Brainstormed key questions Basic logic modelling Lead to an improved understanding of the questions, the limitations and the opportunities Then developed RFP

18 Reflections Standards are a useful tool Provide an introduction to a ‘new language’ Gave me confidence in practice Made tacit knowledge explicit Exciting

19 3 Trading Places

20 From Scotland to NZ – my experience of working in the Scottish Executive

21 4 Capability development for the future

22 Where have we got to? Moved from being researchers to researchers and evaluators Greater clarity about our role in the organisation Receive similar/same advice from all of us Built team cohesion, community of practice

23 The future Continued Standards Development External consultation Embed the use of the Standards across the organisation Develop training opportunities for the organisation Become less administrative more strategic Develop leadership skills

24 5 Discussion

25 Thinking about your evaluation practice …