1. I N N O V A T I O N Results-Based Accountability Discovery Communities TA Institute April 1, 2008

2. 2 2 Results Accountability Decision-making and Strategic Planning Fiscal Policy Studies Institute Santa Fe, New Mexico WEBSITES www.resultsaccountability.com www.resultsaccountability.com www.raguide.org www.raguide.orgwww.raguide.org www.charteroakgroup.com www.charteroakgroup.com BOOK ORDERS www.trafford.com www.amazon.com

3. 3 3 3 Why Are We Here?

4. 4 4 SIMPLE COMMON SENSE PLAIN LANGUAGE MINIMUM PAPER USEFUL

5. 5 5 Concepts to Take Away Today Population accountability v. performance accountability Population accountability v. performance accountability Ends v. means Ends v. means How to choose indicators and measures How to choose indicators and measures The importance of a data development agenda The importance of a data development agenda How to move from talk to action How to move from talk to action

6. 6 6 Results Accountability is made up of two parts: Performance Accountability about the well-being of CLIENT POPULATIONS For Programs – Agencies – and Service Systems Population Accountability about the well-being of WHOLE POPULATIONS For Communities – Cities – Counties – States - Nations

7. 7 7 7 Results and Performance Accountability COMMON LANGUAGE COMMON SENSE COMMON GROUND

8. 8 8 THE LANGUAGE TRAP Too many terms. Too few definitions. Too little discipline Benchmark Target Indicator Goal Result Objective Outcome Measure Modifiers Measurable Core Urgent Qualitative Priority Programmatic Targeted Performance Incremental Strategic Systemic Lewis Carroll Center for Language Disorders

9. 9 9 The Humpty Dumpty Approach to Language “When I use a word,” Humpty Dumpty said, in rather a scornful tone, “it means just what I choose it to mean—neither more nor less.” “The question is,” said Alice, “whether you can make words mean so many different things.” “The question is,” said Humpty Dumpty, “which is to be master—that’s all.” LEWIS CARROLL, Through the Looking- Glass, chapter 6, p. 205 (1934). First published in 1872.

10. 10 DEFINITIONS Children born healthy, Children succeeding in school, Safe communities, Clean Environment, Prosperous Economy Rate of low-birthweight babies, Rate of high school graduation, crime rate, air quality index, unemployment rate 1. How much did we do? 2. How well did we do it? 3. Is anyone better off? RESULT INDICATOR PERFORMANCE MEASURE A condition of well-being for children, adults, families or communities. A measure which helps quantify the achievement of a result. A measure of how well a program, agency or service system is working. Three types: = Customer Outcomes Population Performance

11. 11 From Ends to Means ENDS MEANS From Talk to Action Population Performance RESULT INDICATOR PERFORMANCE MEASURE Customer outcome = Ends Service delivery = Means

12. 12 Connecticut Glossary of RBA Terms The Appropriations Committee standardized the terms we use in Connecticut. You have a copy of the glossary. The Appropriations Committee standardized the terms we use in Connecticut. You have a copy of the glossary. We could have chosen other terms consistent with Friedman’s RBA approach so this list is somewhat arbitrary. We could have chosen other terms consistent with Friedman’s RBA approach so this list is somewhat arbitrary. Its virtue is that everyone in Connecticut doing this work – both executive branch and legislative branch – is using the same vocabulary and meaning the same thing by it. At least we should be able to understand each other. Its virtue is that everyone in Connecticut doing this work – both executive branch and legislative branch – is using the same vocabulary and meaning the same thing by it. At least we should be able to understand each other.

13. 13 Some Connecticut Early Childhood Indicators % Infants born at low birth weight % Infants born at low birth weight % Births to mothers without a high school degree % Births to mothers without a high school degree % of kindergartners with all or most pre-literacy and personal skills % of kindergartners with all or most pre-literacy and personal skills % 4 th grade reading scores at mastery or above % 4 th grade reading scores at mastery or above

14. 14 IS IT A RESULT, INDICATOR OR PERFORMANCE MEASURE? 1. Safe Community 2. Crime Rate 3. Average Police Dept response time 4. A community without graffiti 5. % of surveyed buildings without graffiti 6. People have living wage jobs and income 7. % of people with living wage jobs and income 8. % of participants in job training who get living wage jobs

15. 15 IS IT A RESULT, INDICATOR OR PERFORMANCE MEASURE? 9. % HS graduates enrolling in college 10. Traffic-related death rate 11. Clean environment 12. Air pollutants in parts per million 13. % participating cities fixing treatment plants 14. All children eat healthy 15. % students eligible for free lunch participating in free lunch

16. 16 POPULATION ACCOUNTABILITY For Whole Populations in a Geographic Area For Whole Populations in a Geographic Area

17. 17 Maryland Results for Child Well-Being ● Babies born healthy ● Healthy children ● Children enter school ready to learn ● Children successful in school ● Children safe in their families and communities ● Stable and economically independent families ● Communities that support family life

18. 18 Connecticut Results Statements A clean and healthy Long Island Sound A clean and healthy Long Island Sound All children healthy and ready to learn by age 5 All children healthy and ready to learn by age 5 All children ready by five and fine by nine All children ready by five and fine by nine

19. 19 Leaking Roof (Results thinking in everyday life) Experience: Measure: Story behind the baseline (causes): Partners: What Works: Action Plan: Inches of Water ? Fixed Not OK Turning the Curve

20. 20 Seven Population Accountability Questions  What are the quality of life conditions we want for the children, adults and families who live in our community?  What would these conditions look like if we could see them?  How can we measure these conditions?  How are we doing on the most important of these measures?  Who are the partners that have a role to play in doing better?  What works to do better, including no-cost and low-cost ideas?  What do we propose to do?

21. 21 “We haven’t got the money, so we’ve got to think.” Lord Rutherford 1871 - 1937

22. 22 Connecticut Population Template

23. 23 Criteria for Choosing Indicators as Primary vs. Secondary Measures Communication Power Proxy Power Data Power Does the indicator communicate to a broad range of audiences? Does the indicator say something of central importance about the result? Does the indicator bring along the data HERD ? Quality data available on a timely basis.

24. 24 Choosing Indicators Worksheet Outcome or Result_______________________ Candidate Indicators Communication Power Proxy Power Data Power H M L H Measure 1 Measure 2 Measure 3 Measure 4 Measure 5 Measure 6 Measure 7 Measure 8 H D ata D evelopment A genda Safe Community H M L H M L H H H L

25. 25 Three Part Indicator List for each Result Part 1: Primary Indicators Part 2: Secondary Indicators Part 3: Data Development Agenda ● 2 or 3 or 4 “Headline” Indicators ● What this result “means” to the community ● Meets the Public Square Test ● Everything else that’s any good (Nothing is wasted.) ● Used later in the story behind the baseline ● New data ● Data in need of repair (quality,timeliness etc.)

26. 26 The Matter of Baselines Baselines have two parts: history and forecast H M L History Forecast Turning the Curve Point to Point OK?

27. 27 Performance Accountability For Programs, Agencies and Service Systems For Programs, Agencies and Service Systems

28. 28 Results Accountability is made up of two parts: Performance Accountability about the well-being of CLIENT POPULATIONS For Programs – Agencies – and Service Systems Population Accountability about the well-being of WHOLE POPULATIONS For Communities – Cities – Counties – States - Nations

29. 29 “All Performance Measures that have ever existed for any program in the history of the universe involve answering two sets of interlocking questions.”

30. 30 How Much did we do? ( # ) How Well did we do it? ( % ) Quantity Quality Program Performance Measures

31. 31 Effort How hard did we try? Effect Is anyone better off? Program Performance Measures

32. 32 Effort Effect How Much How Well Program Performance Measures

33. 33 How much service did we deliver? Program Performance Measures How well did we deliver it? How much change / effect did we produce? What quality of change / effect did we produce? Quantity Quality Effect Effort Output Input

34. 34 How much did we do? Program Performance Measures How well did we do it? Is anyone better off? Quantity Quality Effect Effort # %

35. 35 How much did we do? Education How well did we do it? Is anyone better off? Quantity Quality Effect Effort Number of students Student-teacher ratio Number of high school graduates Percent of high school graduates

36. 36 How much did we do? Education How well did we do it? Is anyone better off? Quantity Quality Effect Effort Number of students Student-teacher ratio Percent of 9 th graders who graduate on time and enter college or employment after graduation Number of 9 th graders who graduate on time and enter college or employment after graduation

37. 37 How much did we do? Health Practice How well did we do it? Is anyone better off? Number of patients treated Percent of patients treated in less than 1 hour Incidence of preventable disease (in the practice) Rate of preventable disease (in the practice) Quantity Quality Effect Effort

38. 38 How much did we do? General Motors How well did we do it? Is anyone better off? # of production hrs # tons of steel # of cars produced Employees per vehicle produced # of cars sold $ amount of Profit $ car value after 2 years Quantity Quality Effect Effort Source: USA Today 9/28/98 % market share Profit per share % car value after 2 years

39. 39 ● # of people served ● % participants who got jobs ● staff turnover rate ● # participants who got jobs ● % of children reading at grade level ● cost per unit of service ● # applications processed ● % patients who fully recover What Kind of PERFORMANCE MEASURE? Upper Left Lower Right Upper Right Lower Left Lower Right Upper Right Upper Left Lower Right

40. 40 RBA Categories Account for All Performance Measures (in the history of the universe) Quantity Quality Efficiency, Admin overhead, Unit cost Staffing ratios, Staff turnover Staff morale, Access, Waiting time, Waiting lists, Worker safety Customer Satisfaction (quality service delivery & customer benefit) Cost / Benefit ratio Return on investment Client results or client outcomes Effectiveness Value-added Productivity Benefit value Product Output Impact Process Input Effect Effort Cost TQM Effectiveness Efficiency

41. 41

42. 42 How much did we do? The Matter of Control How well did we do it? Is anyone better off? Quantity Quality Effect Effort Least Control PARTNERSHIPS Most Control

43. 43 The Matter of Use 1. 1.Fundamental Purpose is to Improve Performance as a contribution to improving results 2. Avoid the Performance Measurement Equals Punishment Trap ● Acknowledge the experience as real. ● Work to create a healthy organizational environment ● Start small. ● Build bottom-up and top-down simultaneously.

44. 44 How much did we do? Program Performance Measures How well did we do it? Is anyone better off? Quantity Quality Effect Effort # %

45. 45 Connecticut Program Template

46. 46 How Population & Performance Accountability FIT TOGETHER

47. 47 POPULATION ACCOUNTABILITY Healthy Births Rate of low birth-weight babies Stable Families Rate of child abuse and neglect Children Succeeding in School Percent graduating from high school on time Contribution relationship Alignment of measures Appropriate responsibility Child Welfare Program THE LINKAGE Between POPULATION and PERFORMANCE CUSTOMER Outcomes POPULATION RESULTS # Foster Children Served % with Multiple Placements # Repeat Abuse/Neglect % Repeat Abuse/Neglect PERFORMANCE ACCOUNTABILITY

48. 48 Contribution relationship Alignment of measures Appropriate responsibility THE LINKAGE Between POPULATION and PERFORMANCE POPULATION ACCOUNTABILITY Healthy Births Rate of low birth-weight babies Children Ready for School Percent fully ready per K-entry assessment Self-sufficient Families Percent of parents earning a living wage CUSTOMER Outcomes # persons receiving training Unit cost per person trained # who get living wage jobs % who get living wage jobs PERFORMANCE ACCOUNTABILITY POPULATION RESULTS Job Training Program

49. 49

50. 50 What is Happening in the Legislature? Institutionalizing RBA in the legislature: The Appropriations Committee has created a new RBA sub- committee The Appropriations Committee has created a new RBA sub- committee Early Childhood Cabinet must develop an accountability plan and make other recommendations for changes necessary to ensure coordination, service integration, and accountability Early Childhood Cabinet must develop an accountability plan and make other recommendations for changes necessary to ensure coordination, service integration, and accountability The 2007 budget requires new and expanded programs to report to the General Assembly and OPM using an approved RBA framework, an effort lead by the Office of Fiscal Analysis (OFA) and OPM The 2007 budget requires new and expanded programs to report to the General Assembly and OPM using an approved RBA framework, an effort lead by the Office of Fiscal Analysis (OFA) and OPM The co-chairs and rankings members of the other sub- committees of Appropriations are identifying particular public policy issues (result statements) for RBA development The co-chairs and rankings members of the other sub- committees of Appropriations are identifying particular public policy issues (result statements) for RBA development All this month, the early childhood agencies that were part of the RBA Phase II initiative last session reported back to the legislature on the progress they have made All this month, the early childhood agencies that were part of the RBA Phase II initiative last session reported back to the legislature on the progress they have made

51. 51 What is Happening at State Agencies and in Communities? The Child Poverty and Prevention Council and the Juvenile Justice Policy and Operating Coordinating Committee are both using RBA to guide their implementation efforts The Child Poverty and Prevention Council and the Juvenile Justice Policy and Operating Coordinating Committee are both using RBA to guide their implementation efforts Municipalities and non-profit groups, e.g., workforce boards, are embracing RBA in areas beyond early childhood Municipalities and non-profit groups, e.g., workforce boards, are embracing RBA in areas beyond early childhood State agencies will be developing program quality and outcome measures in response to the new Appropriations Committee initiatives State agencies will be developing program quality and outcome measures in response to the new Appropriations Committee initiatives They will be passing the requirements along to their vendors They will be passing the requirements along to their vendors

52. 52 What is Happening at State Agencies and in Communities? Organizations that contract with the State will have new data collection and reporting requirements, including outcome measuresOrganizations that contract with the State will have new data collection and reporting requirements, including outcome measures Managing performance will be more complicatedManaging performance will be more complicated Likely to be greater requirements for integration of programs and collaboration with other agenciesLikely to be greater requirements for integration of programs and collaboration with other agencies Opportunity for contractors to have input on measurementOpportunity for contractors to have input on measurement Opportunity for program improvementOpportunity for program improvement

53. 53 “If you do what you always did, you will get what you always got.” Kenneth W. Jenkins President, Yonkers NY NAACP

54. 54 Thank You The Charter Oak Group, LLC www.charteroakgroup.com Ron Schack ron_schack@yahoo.com (860) 478-7847 Bennett Pudlin bpudlin@charteroakgroup.com (860) 324-3555

55. 55 Turn the Curve Exercise: Population Well-being 5 min:Starting Points - timekeeper and reporter - two hats (yours plus partner’s) 5 min:Baseline - forecast: Where is the trend line going? - turn the curve: Is forecast OK or not OK? 15 min:Story behind the baseline - causes/forces at work - information & research agenda part 1 - causes 15 min:What works? (What would it take?) - what could work to do better - each partner’s contribution - no-cost / low-cost ideas - information & research agenda part 2 – what works 10 min:Report: Convert notes to one page Two pointers to action

56. 56 ONE PAGE Turn the Curve Report Result: _______________ Indicator (Lay Definition) Indicator Baseline Story behind the baseline --------------------------- --------------------------- (List as many as needed) Partners --------------------------- --------------------------- (List as many as needed) Three Best Ideas – What Works 1. --------------------------- 2. --------------------------- 3. ---------No-cost / low-cost ---------Off the Wall Sharp Edges 4.