1. Building evaluation capacity – training Ben Barnes Principal Research Scientist July 2015

2. 2 Baseline evaluation skillsHigh level/ specialised skills Plan the evaluation Build evaluation thinking into all aspects of program and policy design Formulate program logics for small/medium projects Identify decisions the evaluation needs to inform Formulate clear evaluation questions Identify potential methodologies, indicators and data sources for answering evaluation questions Incorporate evaluation data requirements into program data collection and reporting systems Develop and plan for suitable timelines Formulate program logics for large scale or complex projects Choose and develop core data for monitoring and measuring progress that may be used across evaluations Implement ethical standards for evaluation Commission the evaluation Understand when to use different evaluation types and methodologies including economic evaluation Formulate tender briefs for external evaluations Select the best tender according to proposed methodology, team and other selection criteria Be a methodological/ technical expert in one or more areas Manage the contract & inform program & policy decisions Review/critique/approve the evaluation framework including methodology Critique the use of data (quant and qual) and the reporting and interpretation of findings Manage ‘actively’ to ensure CRS standards are met Use findings to inform program management Present and/or communicate findings to program delivery partners and ‘across’ the agency Evaluate the evaluation and communicate key lessons Manage complex evaluations Present and communicate findings ‘upwards’ to inform program and policy decisions Synthesise findings across individual evaluations to inform decisions Training – aligned with capability requirements

3. Co-designed and co-delivered CRS evaluation specialists (Ben Barnes and Rebecca Wilkinson) Duncan Rintoul, Rooftop Social George Argyrous, ANZSOG Presenters with reputation, and tried and tested materials Setting out internal process, working through actual upcoming evaluation projects One behaviour change, one infrastructure program People from different teams, with different perspectives Mix of presentations, facilitated discussions, small group work, homework tasks Feedback at end of each day, for each round, which led to revisions Training – two day course 3

4. Training – course content 4 Day 1 Overview of evaluation – Evaluation in the policy cycle – Key terms and concepts Introducing program logic – Key concepts – Case study workshop Day 2 Planning the evaluation – Developing an “Evaluation Plan” – Key evaluation questions Commissioning the evaluation Managing the evaluation – The Evaluation Framework – Criteria and standards – Evaluation design/methodology – Data analysis and interpretation – Data collection – Risks and ethics – Key decisions and management – Using the findings Putting it all into practice

5. Consider a full course … a short course and … and perhaps even a shorter course Tailor to how evaluation occurs in your agency Acknowledge the time investment required Achieve critical mass Training – lessons learned 5

6. 6 Questions? ben.barnes@transport.nsw.gov.au