Changing the way the New Zealand Aid Programme monitors and evaluates its Aid Ingrid van Aalst Principal Evaluation Manager Development Strategy & Effectiveness Division

Key national environmental drivers The New Zealand Government’s priorities  enhancing the operational effectiveness of ODA  focus on effective aid that supports sustainable economic development  priority to the Pacific Region and targeted approach to Asia, very small programmes in Africa and LA  focus on NZ’s comparative advantage and to address our partner’s needs  requirements for greater accountability for the use of public money and demonstrate value for money

Key international environmental drivers  the effectiveness of aid expenditure is now at the forefront of development thinking  transparency, accountability, and confidence that significant aid investments are both effective and efficient in fulfilling their objectives  commitments to Paris Declaration on Aid Effectiveness, the Accra Agenda for Action and the Millennium Development Goals, with better monitoring and evaluation for both donors and development partners

An effective and focussed monitoring and evaluation function  contribute to our understanding of whether our aid interventions are effective, efficient and provide value for money  help us improve activities, programmes, and how we deliver aid  what is working effectively (or not) & why  what progress are we making towards outcomes  be aligned to the New Zealand Aid Programme’s core strategic and operational goals, and priorities - be at the forefront and integrated into our work  support accountable requirements for spending by providing credible, independent and objective assessments of New Zealand Aid Programme’s key strategies, policies, sectors, programmes and activities - we need to better demonstrate results or “tell our story”  be integrated with the New Zealand Aid Programme’s internal and external planning and reporting

New Zealand Aid Programme’s Performance Framework

New Zealand Aid Programme’s monitoring and evaluation Development, Strategy and Effectiveness Division  New Zealand Aid Programme’s strategy and policy, work programme, planning, performance and reporting  monitoring, evaluation and research  cross cutting co-ordination  procurement  process improvement, policy coherence, development outreach  best practice programme and activity management

The Evaluation team  developing and managing a strategic evaluation work programme  building evaluation capacity in and across the New Zealand Aid Programme, including at activity level  reviewing and developing evaluation policies, strategy, plans, processes and systems

The New Zealand Aid Programme’s strategic evaluation work programme Four strands in the work programme  Sector  Programme – The New Zealand Aid Programme has 24 Programmes  Policy  Responsive  a rolling programme of 5-7 evaluations each year across a number of years  report to the Evaluation and Research Governance Group  anticipate participating in a number of joint evaluations  focus first is on developing the strategic evaluation work programme

Activity level monitoring and evaluation  simple and manageable monitoring and evaluation approach, which builds on previous monitoring and evaluation approaches  supports design of the activity at start and during implementation  greater focus on results (outputs, outcomes and goal) - away from inputs and activities  imbedded into the Activity lifecycle management processes and systems - ‘part of business as usual’  activity monitoring and evaluation is the responsibility of the Programme Teams

Activity Results Framework  Results Diagram  identifies logic supporting the Activity and intended results  Results Measurement Table  how results will be monitored and measured, including indicators and targets  Monitoring and Evaluation Work Plan  tasks and resources required to support monitoring and evaluating results

Results Measurement Table ResultsIndicatorsBaseline and TargetsMethodology & Data Sources Long-term Outcomes: Improved health of the Community 1. Cause specific morbidity rates (per 100,000)* 2. Selected infectious diseases rates (per 100,000)* Baseline: 450 (2005) and 470 (2010) Target: 460 (2012) and 400 (2015) Baseline: 460 (2005) and 480 (2010) Target: 480 (2012) and 420 (2015) Information is available each October from the MoH. There is a one year lag in available information. Medium-term Outcome: Increased use of clean water 1. % of households making use of clean water Baseline: no information currently available Target: all households using clean water (2013) Information will be collected through a specifically designed survey and captured by the MoH. Available each November. Short-term Outcome: Improved water quality 1. % of water testing samples that comply with agreed quality standards 2. Incidence water born diseases in the community* Baseline: 20% of samples comply with quality standards Target: 90% of samples comply (2012) Baseline: 60 incidence per year Target: 30 incidence (2012) and less than 10 incidence (2015) Information will be collected through a specifically designed audit process and captured by the MoH. Output: Wells Built 1. Wells built to specification to time and to budget^ Baseline: no information available Target: Compliance achieved (2012) Information will be collected through a specifically designed audit process and captured by the MoH. Information will be available in November.

M&E Work Plan TasksApproach (methods, processes) TimelineRoles and Responsibilities Deliverables and Reporting Indicative Costs Evaluation (Note: must be based on soon to be released Evaluation Policy and Guidelines) Plan and undertake year three evaluation 4.1 Develop terms of reference Identify in consultation with key stakeholders the purpose, scope, objectives / Document and confirm ToR with the Steering Committee June-July 2012Lead: MFATDocumented and agreed terms of reference TA funding: 2 days ($4k) 4.2 Consult with stakeholders and develop eval. Plan Identify and contract evaluator / Develop and agree eval. Plan June –July 2012Lead: MFATAgreed and high quality eval. Plan Funding for contract evaluator: $16k 4.3 Undertake year three evaluation Complete the evaluationJune- Aug 2013Lead: Lead EvaluatorEval report signed- off by Eval. Steering Group Funding for contract evaluator (s): $60k 4.4 Year three evaluation: Report results and develop action plan to strengthen on-going delivery a) Present results and b) Workshop to determine response to results and how services will be strengthened Sep 13Lead: EvaluatorPresentation on the results / Workshop: response to the evaluation, work plan to strengthen services Grant funding: $20k Overall Monitoring and Evaluation BudgetGrant Funding$106k TA Funding$14k Full Funding$120k

New Zealand Aid Programme’s Performance Framework

Building the monitoring and evaluation capacity and capability across the New Zealand Aid Programme  developing simple user-friendly, practice-oriented guidelines and templates covering all aspects of monitoring, and the process of commissioning and managing Activity evaluations  developing and delivering result monitoring and evaluation training  providing coaching and support for the more complex Activity evaluation challenges  re-establishing a Community of Practice for results monitoring and evaluation  supporting results monitoring and evaluation champions in each Division

Monitoring and evaluation policies, processes and systems In the next 12 months  developing activity and programme results monitoring and evaluation policy, processes, systems and guidelines, and evaluation policy and strategy  establishing the Evaluation and Research Governance Group, and a panel/pool of results monitoring and evaluation providers  developing processes for quality review of evaluation deliverables; protecting the independence of evaluators and their reports; disseminating learnings; management response and action from evaluations; link to planning, reporting and management decision-making

Any questions?