Monitoring and evaluation – lessons from the National Partnership Agreement on Remote Service Delivery 23 August 2012 Brian Gleeson, Coordinator General for Remote Indigenous Services Australian Public Service Policy Implementation Network, Canberra

The lay of the land Closing the Gap A COAG initiative – targets and building blocks Remote Service Delivery A new way of working? Accountability - the Office of the Coordinator General for Remote Service Delivery A watchful eye Strengthening communities

RSD Priority Communities

Government’s role is to create an environment where communities can improve their own circumstances empower communities in their efforts to solve their own problems respond to priorities that lead to sustainable outcomes support capacity building efforts provide ongoing support and investment to meet community priorities Insanity : doing the same thing and expecting a different result Albert Einstein

RSD National Partnership Objectives  Improve the access of Indigenous families to a full range of suitable and culturally inclusive services.  Raise the standard and range of services delivered to Indigenous families to be broadly consistent with those provided to other Australians in similar sized and located communities.  Improve the level of governance and leadership within Indigenous communities and Indigenous community organisations.  Provide simpler access and better coordinated government services for Indigenous people in identified communities.  Increase economic and social participation wherever possible, and promote personal responsibility, engagement and behaviours consistent with positive social norms.

Three core areas Improved engagement Across governments - improved collaboration and coordination Between governments and communities Better service systems Better standard and range of services Local Implementation Plans Public accountability Stronger communities Support development of governance and community leadership

National Partnership Requirements  five outcomes  twelve outputs  five reporting requirements  seven deliverables by Commonwealth, five deliverables by State and NT governments, eleven joint deliverables  Overarching service delivery principles and Closing the Gap objectives – although not recognised explicitly in the National Partnership  No clear relationship between deliverables and objectives (especially the fifth objective)  Timelines that were at times in conflict with other requirements of the National Partnership  Focus on Local Implementation Plans

Accurate and consistent tracking and reporting on progress is key to the success of Remote Service Delivery. Communities holding governments accountable through LIPs Governments being held accountable through Coordinators General Community organisations accountable to their constituents Accountability

RSD baseline mapping  Key component of National Partnership on Remote Service Delivery  Maps social and economic indicators, government investments, services and service gaps in each location  Provides evidence base from which improvements can be measured  Supports community-based local research priorities  Involves comparisons with non-Indigenous communities of similar size, location and need elsewhere in Australia  Information shared with communities and informs Local Implementation Plans  Guides future Government investment

RSD Measuring Success GatherPlanTrack Baseline mapping The collection of community data at a point in time and updated Local Implementation Plans Plans for development of the community by the community Board of Management Oversee the Implementation of the Remote Service Delivery Strategy, track the Local Implementation Plans and other community issues

Evaluation effort to date  Opportunities for formal (formative) evaluation have been missed  Informal attempts have included  Community based action research  Participatory evaluation  Sharing of promising practice  Formal evaluation in 2013

Community researchers The research conducted to date with community research teams in Ntaria (and previously in Amata and Mimili) represents a new model of generating data and analysis of value to policy development. Key elements of the model are: draws on the vast knowledge and insight of experienced local residents builds skills and knowledge locally and enables participatory processes to occur in a new and inclusive way Although the work remains at a fairly early stage, it has been possible in a short space of time to conduct rigorous research on community perspectives on road safety and on governance in Ntaria (and other topics in Amata and Mimili), generating recommendations for the Local Reference Group that are relevant to the Local Implementation Plan, as well as evaluation of progress from a community perspective. More than we might have expected at the outset, the research activity has filled gaps in knowledge essential to progressing the LIP and has built momentum for change.

Approach of CGRIS  Assessing government efforts  Sharing promising practice  Tracking services and investments  Tracking progress with key elements of Local Implementation Plans  Qualitative assessment of progress – gathering local intelligence  Guiding government efforts towards a more systematic approach

Barriers to effective evaluation  Fuzzy program logic – diffuse objectives  Difficult concept of the comparison community  Lack of minimum service standards against which to measure progress  Lack of place based data and ability to share data (privacy considerations)  Lack of performance measures in Local Implementation Plans  Diffusion of responsibility and lack of leadership authority  Multiple levels of action – across governments, between governments, at community level  Multiple objectives – new ways of working, improving service systems, improving outcomes  Difficulty in measuring improvements in social outcomes over the short term  Local Implementation Plans include in excess of 3,000 actions nationally

Lessons for future policy development  Articulate a clear program logic as part of policy development  Think carefully about purpose of baseline mapping exercises and build in replicability  Recognise the difficulty in translating high level policy into operational reality  Embed (ongoing) evaluation so formative opportunities are not lost  Recognise the context you are working within and be realistic about what is achievable  Utilise local expertise where possible – get locals to find out what people really think  Recognise the value of qualitative assessments but also insist on milestones and performance measures  Build in accountability to the community as well as to governments

Questions? Alice: would you tell me which way I ought to go from here? Cheshire Cat: That depends on where you want to get to. Alice: I don’t much care where. Cat: Then it doesn’t matter which way you go. cgris.gov.au