1. Results Based Accountability Delivering Better Outcomes

2. Key Points The delegate should have a better understanding of… Population accountability Performance accountability Outcomes, Indicators & Performance Measures How much, How well & Is anyone better off Baselines Turning the curve

3. “It can become a way of securing strategic and cultural change, moving organisations away from a focus on efficiency and process as the arbiter of value in their services, and towards making better outcomes the primary focus of their organisation.” Mark Friedman Results Based Accountability

5. What is RBA A simple framework to help us get done what needs to be done, getting from talk to action Common sense approach Focuses on Outcomes as the measure of value

6. RBA is not A silver bullet A religion A system that makes your decisions for you If you are a bad decision maker then you will still make bad decisions Remember: RBA is just a tool “If the only tool that you have is a hammer then you will treat every problem as if it is a nail” (Maslow)

7. Results Accountability is made up of two parts: Performance Accountability about CLIENT POPULATIONS For Programmes - Agencies - Service Systems Population Accountability about WHOLE POPULATIONS For Countries - Counties - Cities - Communities

8. Results Based Accountability Common Sense Common Ground Common Definitions

9. Jargon Construction Kit Benchmark Target IndicatorGoal Result Objective Outcome Measure Pick words at random and string them together Modifiers Measurable Urgent Priority Targetted Incremental Core Qualitative Performance Strategic Systemic

10. DEFINITIONS Children born healthy Safe communities Clean environment Prosperous economy Rate of low-birthweight babies Crime rate Air quality index Unemployment rate How much did we do? How well did we do it? Is anyone better off? POPULATION OUTCOME POPULATION INDICATOR PERFORMANCE MEASURE A condition of well-being for children, adults, families or communities. Data which helps quantify the achievement of an Outcome. Data to show how well a programme, agency or service system works Population Performance

11. DEFINITIONS Children born healthy Safe communities Clean environment Prosperous economy Rate of low-birthweight babies Crime rate Air quality index Unemployment rate How much did we do? How well did we do it? Is anyone better off? POPULATION OUTCOME POPULATION INDICATOR PERFORMANCE MEASURE A condition of well-being for children, adults, families or communities. Data which helps quantify the achievement of an Outcome. Data to show how well a programme, agency or service system works ENDS MEANS

12. Outcomes, Indicators, Performance Measures People in Wales live in a safe community Crime rate in Cardiff Average police response time in Wrexham People have living wage, jobs and income % of people in a job training program that get living wage jobs Unemployment rate Number of new jobs created from grants allocated Number of grant applications appraised within 20 working days Outcome Indicator Perf measure Outcome Perf measure Indicator Perf measure

13. Population Accountability

14. Some Outcome statements used in Welsh Government Children are born healthy and reach their full potential People in Wales are safe and feel safe People have a high-quality, warm, secure and energy-efficient home to live in Community is safe from the threat of deliberate fires People can travel safely & easily by train and bus to get to work and to access education & training People in Wales have access to high quality green space, clean air and an enhanced local environment People in Wales are free from poverty Encourage & support the use of the Welsh language within families Better health for all with reduced inequalities in health Ensuring our funding supports stronger and more effective Local Authority service delivery

15. Goals of the Well-being of Future Generations (Wales) Act A Prosperous Wales A Resilient Wales A Healthier Wales A More Equal Wales A Wales of Cohesive Communities A Wales of Vibrant Culture & Thriving Welsh Language A Globally Responsible Wales

16. A Prosperous Wales An innovative, productive and low carbon society that makes more efficient and proportionate use of resources (including taking action to deal with the causes of climate change); and which develops a skilled and well-educated population in an economy which generates wealth and provides employment opportunities, allowing people to take advantage of the wealth generated through securing decent work.

17. A Resilient Wales A nation which uses natural resources in a way and at a rate that maintains and enhances a biodiverse natural environment with healthy functioning ecosystems that support social, economic and ecological resilience and the capacity to adapt to change (for example climate change).

18. A Healthier Wales A society in which people’s physical and mental well-being is maximised and in which choices and behaviours that benefit future health are understood.

19. A More Equal Wales A society that enables people to fulfil their potential no matter what their background or circumstances.

20. A Wales of Cohesive Communities Attractive, viable, safe and well-connected communities.

21. A Wales of Vibrant Culture & Thriving Welsh Language A society that promotes and protects culture, heritage and the Welsh language, and which encourages people to participate in the arts and sports and recreation.

22. A Globally Responsible Wales A nation which, when doing anything to improve the economic, social and environmental well-being of Wales, takes account of the contribution it is making to global well-being.

23. A Prosperous Wales A Resilient Wales A Healthier Wales A More Equal Wales A Wales of Cohesive Communities A Wales of Vibrant Culture and thriving Welsh Language A Globally Responsible Wales An innovative, productive and low carbon society that makes more efficient and proportionate use of resources (including taking action to deal with the causes of climate change); and which develops a skilled and well-educated population in an economy which generates wealth and provides employment opportunities, allowing people to take advantage of the wealth generated through securing decent work. A nation which uses natural resources in a way and at a rate that maintains and enhances a biodiverse natural environment with healthy functioning ecosystems that support social, economic and ecological resilience and the capacity to adapt to change (for example climate change). A society in which people’s physical and mental well-being is maximised and in which choices and behaviours that benefit future health are understood. A society that enables people to fulfil their potential no matter what their background or circumstances. Attractive, viable, safe and well-connected communities. A society that promotes and protects culture, heritage and the Welsh language, and which encourages people to participate in the arts, and sports and recreation. A nation which, when doing anything to improve the economic, social and environmental well-being of Wales, takes account of the contribution it is making to global well-being.

24. The 7 Questions about Population Accountability 1.What are the conditions of wellbeing we want for people in our community? 2.What would these conditions look like if we could see them? 3.How can we measure these conditions? 4.Who are the partners that have a role to play in doing better? 5.How are we doing on the most important of these measures? 6.What works, including no-cost and low-cost ideas? 7.What do we propose to do?

25. Choosing Indicators Outcome _______________________________ Indicators Communication Power Proxy Power Data Power H M L H Indicator 1 Indicator 2 Indicator 3 Indicator 4 Indicator 5 Indicator 6 Indicator 7 H Data Development Agenda Safe Community H M L H M L H H H L

26. Three Part Indicator List for each Outcome Part 1: Primary Indicators Part 2: Secondary Indicators Part 3: Data Development Agenda 2 or 3 or 4 “Headline” Indicators What this outcome “means” to the community Meets the Public Square Test Everything else that’s good Used to explain the background New data Data in need of repair (quality, timeliness etc.)

27. Leaking Roof (Results thinking in everyday life) Experience: Measure: Story behind the baseline (causes): Partners: What Works: Action Plan: Inches of Water Fixed ? Not OK Iterative

28. Baselines Baselines have two parts: history and forecast H M L History Forecast Turning the Curve Point to Point OK?

29. DEMONSTRATING the CONTRIBUTION A Curve to Turn 2 …and it had a timely relationship to…. 3 …. a turn in the curve. Bonus: a background trend stayed the same or got worse. 1 We tried things that had a credible chance of making a difference…

30. Case Study Swansea NEET (Not in Education, Employment or Training) Link to PDF

31. NEET trends 2004-2012 NA5817710181921Position in Wales 201220112010200920082007200620052004Year

32. Actions Partnership working Identifying “at risk” pupils in Year 11 Engagement & 1-2-1 support Development of a Keeping In Touch (KIT) strategy KIT Summer Transition Programme

33. Swansea NEET Case Study 2004 21 st out of 22 local Authorities 8.3% NEET 4.2% unknown 371 NEET (including unknown) 2011 5 th out of 22 Local Authorities 3.1% NEET 0% unknown 79 NEET

34. Performance Accountability

35. Commission on Public Service Governance and Delivery (aka Williams Commission) First Minister established the Commission on Public Service Governance and Delivery in April 2013. Tasked with examining all aspects of governance, delivery and accountability in devolved public sector in Wales. Commission report published on 20 January. Serious, comprehensive and wide ranging report – 350 pages, and 62 recommendations under five themes.

36. “All Performance Measures that have ever existed for any service in the history of the universe involve answering two sets of interlocking questions.”

37. How much did we do? How well did we do it? Is anyone better off? Quantity Quality Effect Effort # %

38. How much did we do? Performance Accountability Types of performance measures found in each part of the quadrant How well did we do it? Is anyone better off? # Clients/customers served # Activities (by type of activity ) % Common measures client staff ratio, workload ratio, staff turnover rate, unit cost Skills / Knowledge Attitude / Opinion Behaviour % Circumstance # % Activity-specific measures % timely, % completing activity, % meeting standard (e.g. parenting skills) (e.g. toward drugs) (e.g. school attendance) (e.g. working, in stable housing)

39. How much did we do? Child Protection Service How well did we do it? Is anyone better off? Quantity Quality Effect Effort # investigations completed % investigations started within 24 hours of report # repeat abuse or neglect % repeat abuse or neglect

40. How much did we do? Cardiac Service How well did we do it? Is anyone better off? Quantity Quality Effect Effort # of admissions to hospital % treated within referral to treatment target # achieving stage 3 rehabilitation scheme % achieving stage 3 rehabilitation scheme

41. How much did we do? Business & Employment How well did we do it? Is anyone better off? Quantity Quality Effect Effort # Grant “X” applications received % Grant “X” applications appraised within 20 working days # new jobs created as a result of grant “X” # inward investment companies brought to Wales as a result of Grant “X” Is anyone better off? % of new jobs still in place after 2 years as a result of grant “X” % inward investment companies still in Wales after 5 years as a result of Grant “X”

42. How much did we do? Housing Association How well did we do it? Is anyone better off? Quantity Quality Effect Effort # of homes # of repairs # of tenant participation (TP) events Av. time taken to make repairs % of tenants attending TP events # of homes meeting WHQS # of tenants who feel their views are acted on % of homes meeting WHQS % of tenants who feel their views are acted on

43. How much did we do? Youth interventions (Fire Service) How well did we do it? Is anyone better off? Quantity Quality Effect Effort # of young people attending courses % young people completing course and getting Agored Cymru qualification # young people showing positive change # deliberate fires caused by reoffending young people % young people showing positive change % deliberate fires caused by reoffending young people

44. How much did we do? Welsh Women’s Aid How well did we do it? Is anyone better off? Quantity Quality Effect Effort # service providers that received advice % requests for advice that were responded to within 5 days # providers that say they are better equipped to support women % providers that say they are better equipped to support women

45. How much did we do? How well did we do it? Is anyone better off? Quantity Quality Effect Effort Most important. But data is lagging Important for judging ourselves Needed for immediate reporting Is anyone better off?

46. How much did we do? How well did we do it? Is anyone better off? Quantity Quality Effect Effort Least control Most control Is anyone better off?

47. Case Study Welsh Epilepsy Unit in Cardiff Source Ruth Jordan

48. How much did we do? Epilepsy Unit Patients with a first suspected seizure or unexplained blackout How well did we do it? Is anyone better off? Quantity Quality Effect Effort # of patients # of appointments to specialist # of DNA at clinic % patients seen by specialist within 2 weeks Average waiting time to see a specialist % that DNA their first seizure clinic Average length of time from seizure to confirmed diagnosis % patients that follow the correct pathway

49. Action Plan Develop nurse led Emergency Unit assessment service Develop nurse led first seizure clinics Enable specialist nurse referral for EEG Change the name of the Epilepsy unit

50. The Welsh Epilepsy Unit Customers: Patients with a first suspected seizure or unexplained blackout How well did we do it? Number of patients seen by a specialist within the NICE guideline of 2 weeks increased from 35% to 61% Average waiting time to see a specialist has decreased from 22 days to 11 days Is anyone better off? Average length of time from seizure to a confirmed diagnosis decreased by 81 days from 111 days to 30 days Number of admissions following a seizure has decreased from 5/month to 2/month on average

52. Impact UCL = 95.5% UCL = 104.5% Mean = 35.4% Mean = 60.9% LCL = -24.7% LCL = 17.3%

53. UCL = 42.84 LCL = 0.44 Mean = 21.64 UCL = 24.47 Mean = 11.46 LCL = -1.55

54. Outcomes UCL = 9.97 LCL = 0.13 UCL = 3.01 LCL = 0.99 Mean = 2 Mean = 5.05

55. Link to PDF Link to charts PDF

56. IndicatorsWhat other data do we need (data development agenda) Partners who can help us Outcome Population How are we doing Story behind the baseline Indicator 1Indicator 2Indicator 3Indicator 4Indicator 5 Actions

57. How much did we do? Choosing headline measures and the data development agenda How well did we do it? Is anyone better off? Quantity Quality Effect Effort # Measure 1 ---------------------------- # Measure 2 ---------------------------- # Measure 3 ---------------------------- # Measure 4 ---------------------------- # Measure 5 ---------------------------- # Measure 6 ---------------------------- # Measure 7 ---------------------------- % Measure 8 ---------------------------- % Measure 9 ----------------------------- % Measure 10 --------------------------- % Measure 11 --------------------------- % Measure 12 --------------------------- % Measure 13 --------------------------- % Measure 14 --------------------------- # Measure 15 ---------------------------- # Measure 16 ---------------------------- # Measure 17 ---------------------------- # Measure 18 ---------------------------- # Measure 19 ---------------------------- # Measure 20 ---------------------------- # Measure 21 ---------------------------- % Measure 15 ---------------------------- % Measure 16 ---------------------------- % Measure 17 ---------------------------- % Measure 18 ---------------------------- % Measure 19 ---------------------------- % Measure 20 ---------------------------- % Measure 21 ---------------------------- DDA

58. The 7 Questions about Performance Accountability (What Population Outcome do we most contribute to?) 1.Who are our customers (and what do they want from us)? 2.How can we measure if our customers are better off? 3.How can we measure if we are delivering service well? 4.Who are the partners that have a role to play in doing better? 5.How are we doing on the most important of these measures? 6.What works, or could work, including low cost and no cost ideas? 7.What do we propose to do?

59. Putting it together Performance Measures for Job Creation Programme A Prosperous Wales Gross Domestic Product (GDP) of Wales % of pupils achieving Grade A-C in core subjects # new patents filed % people aged 19-24 who are NEET Unemployment rate # of Grant “X” applications received % of Grant “X” applications appraised within 20 days # jobs created via Grant “X” # inward investment companies attracted into Wales by Grant “X” % jobs still in place after 2 years % inward investment companies still in Wales after 5 years Performance Accountability Population Accountability Outcome Indicators ENDS MEANS

60. Putting it together Population Accountability Gross Domestic Product (GDP) of Wales % of pupils achieving Grade A-C in core subjects # new patents filed % people aged 19-24 who are NEET Unemployment rate Performance Accountability Grant “X”Program 2Program 3Program 4 Performance Measures Outcome Indicator A Prosperous Wales Remember – Look for CONTRIBUTION not IMPACT ENDS MEANS

61. Every time you present your project, Use a two-part approach. Outcome: which you contribute to most directly. Indicators: Story: Partners: What would it take?: Your Role: as part of a larger strategy. Population Accountability Program: Performance measures: Story: Partners: Action plan to get better: Performance Accountability

62. The 20-60-20 RULE

63. Summary Why should you think about outcomes? Focuses us on the big picture – ends should derive means Governments have focused on inputs but ignored outcomes. (David Osborne and Ted Gaebler 1992) An outcomes mindset acknowledges complexity Because we overestimate what we can do in the short-term but underestimate what we can do in the long-term (Geoff Mulgan)

64. Turn the Curve Exercise: Population Well-being 5 min:Starting Points - timekeeper and reporter - geographic area - two hats (yours plus partner’s) 10 min:Baseline - pick an outcome, and an indicator curve to turn - forecast – OK or not OK? 15 min:Story behind the baseline - causes/forces at work - what is the reasoning behind this, what evidence is there 15 min:What works? (What would it take?) - what could work to do better? - each partners contribution - no-cost / low-cost ideas - what information do you have, or would need, going forward 10 min:Report convert notes to one page

65. Turn the Curve Exercise: Population Well-Being 5 Mins:Starting points timekeeper & reporter geographic area 10 Mins:Baseline pick an outcome and an indicator curve to turn project the trend forward and decide if this OK or not each person to wear 2 hats (your own role plus another) 15 Mins:Story behind the baseline what are the causes and forces at work? what is the reasoning behind this, what evidence is there? 15 Mins:What works? (what would it take?) what could work to do better? each partner’s contribution no cost & low cost ideas What information do you have, or would need going forward 10 Mins:Report Convert notes to one page

66. ONE PAGE Turn the Curve Report: Population Outcome: _______________ Indicator Indicator Baseline Story behind the baseline --------------------------- --------------------------- (List as many as needed) Partners --------------------------- --------------------------- (List as many as needed) Best Ideas – What would it take? 1. --------------------------- 2. --------------------------- 3. ---------No-cost / low-cost 4. --------- Off the Wall

67. Turn the Curve Exercise: Program Performance 5 min:Starting Points - timekeeper and reporter - identify a program to work on - two hats (yours plus partner’s) 10 min:Performance measure baseline - choose 1 measure to work on – from the lower right quadrant - forecast – OK or not OK? 10 min:Story behind the baseline - causes/forces at work - what is the reasoning behind this, what evidence is there 10 min:What works? (What would it take?) - what could work to do better? - each partners contribution - no-cost / low-cost ideas - what information do you have, or would need, going forward 10 min:Report Convert notes to one page

68. Service: _______________ Performance Measure Performance Measure Baseline Story behind the baseline --------------------------- --------------------------- (List as many as needed) Partners --------------------------- --------------------------- (List as many as needed) Best Ideas – What would it take? 1. --------------------------- 2. --------------------------- 3. ---------No-cost / low-cost ONE PAGE Turn the Curve Report: Performance 4. --------- Off the Wall

69. Steve Jones029 2080 1109 steve.jones@wales.gsi.gov.uk