Achieving Family Economic Success with the People We Serve Using Results Based Accountability United Way of Greater New Haven Annie E. Casey Foundation February 16, 2010

2 Results for Today 1.Can apply the fundamentals of Results Based Accountability (RBA) 2.Know the different types of and how to use performance measures in performance accountability 3.Are equipped to make applications to the United Way of Greater New Haven and the Casey Foundation

3 Various Outcome Models Balanced Scorecard Our Own Hybrid Scales and Ladders Logic Model Results Mapping Outcome Funding Framework

4 AECF and UWGNH use Results Based Accountability Results Matter “It matters if we succeed or fail. It matters if things get better or worse.” Mark Friedman

5 a focused way of thinking and taking action that starts with ENDS (defining the change you want to make) and works backwards to the means (developing powerful strategies to bring about the change). Results Based Accountability

6 Crosswalk RBA and Logic Frameworks FeatureLogicRBA Program centric with depiction of expectations (processes) based on assumptions & “ifs & thens” Uses Factor analysis based on trend line data Results centric with course of action to improve results Population & program focusProgramBoth Data to inform decision making – trend line Shared accountability Involvement of partners Defined terms

7 1.Has consistent terms to use. take action to make it better 2.Uses a simple, step-by-step way to take action to make it better for populations of children, individuals or families. they care about 3.Brings people together on what they care about – conditions of well- being. common framework 4.Provides a common framework for human services, government, funders and communities to work together. 5.Has a dual focus – population & program level change Common Language Common Sense Common Ground Why RBA Practice

8 What is the Added Value of RBA?  Draws people together on what they care about – better results for children & families  Makes the “silo” way of working obsolete because people align their actions as partners for a shared result  Uses data to inform decisions vs. the politics of the day  Uses what works vs. favorite programs to achieve results  Shares accountability across partners for better results  Tracks progress regularly by using data to course correct or to accelerate the work

9 RBA Practice - Language Consistency To avoid - many terms, few definitions, & interchanging Benchmark Target IndicatorGoal Result Objective Outcome Measure Modifiers Measurable Core Urgent Qualitative Priority Programmatic Targeted Performance Incremental Strategic Systemic

10 Two Profoundly Different Accountabilities PerformancePopulations For Customers Whole in a place or geographic area In Jobs Agency All Families in the City

11 Common Language Population Accountability Result (Outcome) – quality of life condition Families are economically successful. Indicator – quantifies the result in the form of a percent, rate or ratio - % of residents employed

12 Population Accountability Accountability Whole Population + Desired Result Desired Result+Place = Population Accountability

13 Population Accountability Accountability Whole Population Families+ Desired Result Desired Result Have Economic Stability +Place New Haven = Population Accountability

14 Whole Populations in a Place EX Children in Low Income Families in New Haven  Result Quality of Life Change EX: Children & families achieve financial stability  Indicator Trend Line Quantifies the Result EX: 60% of families are employed in 2008 & 61% in 2009  Strategies What works to change the indicator Trend Line Multi-sector jobs pipeline  Program or Service Population Re-entry Residents  Performance Measures (How Much, How Well, Difference Made) 120 residents in training, 70% complete training, 60 (50%) residents obtain jobs  Actions to Take (Partners commit to take action) Hospital agrees to hire ex-offenders, Library sponsors a job fair, Union establishes a apprentice program POPULATIONPOPULATION RBARBA PERFORMANCEPERFORMANCE

15 7 Q’s of Population Accountability 1. Which population(s) are we concerned about? 2. What quality of life conditions do we want for the population(s)? (results) 3. What measures can tells us that these conditions are improving? (indicators) 4. How are we doing on the most important indicators for all populations? (trend line) What factors are affecting the trend line? (story behind the data) 5. Which partners are needed to change the trend line? 6. Which strategies based on evidence, practice or experience are powerful enough to change the trend line? 7. What actions are going to take as a group?

16 Shifting from Population Accountability to Performance Accountability

17 Performance Accountability Customers Recently released Offenders +Program Shea Services + Difference Made # retaining jobs for 6 months = Performance Accountability

18 An approach to help us better see the link between our work and our performance and the results achieved for the people we serve in our programs. Performance Accountability Is...

19 Performance Measures A Closer Look

20 Quantity 1. How much did we do? (Quantity) Quality 2. How well did we do it? (Quality) Impact 3. Who is better off? (Impact) Performance Measures Answer Three Questions

How many did we serve? (Customers )  # of individuals served through jobs pipeline  # of agencies did we refer to? How many activities did we provide?  # of training sessions held  # of events held in the neighborhood How much did we do?

22 1. Are our efforts T imely? % of referrals completed in 24 hours 2. What are the completion or A ttendance rates? % new workers completing training 3. Are our customers S atisfied? % of employers satisfied with job hires 4. What are the C osts per unit? TASCS $400 per jobs skills training session held 5. What are professional or organizational S tandards? Worker to Coaches ratio (10 to 1) How well did we do?

23 What Difference Made – Better Off 1. Behaviors (#/% of participants who report to work on time) 2. Attitudes (Opinions) (#/% of participants who value seeking a degree) 3. Circumstances (#/% of participants who have a job at program completion) 4. Knowledge/ Skills (#/% of individuals who have job increased skills) BACKS

24 Employment Readiness Class How much did we do? (Quantity)  # participating in the class How well did we do it? (Quality)  % satisfied with class Who is better off? (Impact)  #/% who get a job

25 How Much How Well Better Off How Performance Measures Fit Together

26 What difference did we make? Performance Measures Start with Effort Effect How hard did we try?

27 How much did we do? ( # ) How well did we do it? ( % ) QuantityQuality Performance Measures Look at Both

28 Eff ort How Much We Did How Well Here is the Fit of Performance Measures Eff ect

29 How much service did we deliver? # How well did we deliver it? % How much change / effect did we produce? # What quality of change / effect did we produce? % Quantity Quality Effect Effort Better Off Better Off The Fit In Detail

30 # of youth participating in a jobs skills class % of youth satisfied with the program Quantity Quality Effect Effort Is anyone better off? # of youth who have a job % of youth who have a job Job Skills Class

31 Performance Measures Tips

32 Performance Measures Are NOT Equal How well did we do it?How much did we do? Least Important 2 nd Most Important 3rd Most Important MOST Important Is Anyone Better Off?

job applicants in the program Numbers Tell Only Part of the Story 40% of applicants complete course % 50 applicants employed Numbers tell you the exact “number” of participants who were better off. (Q#3) Percentages tell the degree of success of the program/organization. (Q#4) # 50% of applicants employed #%

34 Measuring Improvement  Point in Time Improvement # / % of agencies showing increased results in 2008  Point to Point Over Time # / % of agencies showing results for two years

35 Performance Measures: How Many Priority Measures – Most Important Secondary Measures Data Development Work to Do  2 or 3 or 4 “Headline” Measures  Others that you may want to track...  Need to gather data to measure what you don’t have  Data that needs repair (definition, quality, timely)

36 Setting Performance Targets  Helps us understand what we are aiming for and if we don’t reach targets then we can work to discover what didn’t work and what we can do better.  Helps to answer the key questions Is it producing what it should for its customers? Is this program worth the money? Can it perform better? Can it be done less expensively?

37 What/How much did we do? # Performance Measures How well did we do it? % QuantityQuality Effect Effort # % Better Off

38 What Are The Steps To Improve Our Performance?

39 1.Who are our customers? For which program? 2.How do we measure if we are making a difference? (customers are better off) 3.How do we measure if we deliver our services well? (quality measures) 4.What is the data telling us about how we are doing on these measures? + the trend line – better, worse or the same? For all customer groups especially where disparities exist + story behind the data – factors pushing up  & pushing down  ? 5.Which partners are needed to change the trend line? 6.What actions are powerful enough to address the factors? What works based on evidence, practice or experience? 7.What actions are we going to take as a group? 7 Performance Accountability Questions

40 People participating in the program or the activities to implement a strategy  children  parents 1.Who are our customers? Which program?

41 B = Behaviors Change A = Attitudes/Opinions Shift C = Circumstances are Better K = Knowledge Increased S = Skills Increased “BACKS” 2.How do we measure if we are making a difference? Ways people are “Better off”

42 T = Timeliness of service response-delivery A = Attendance rates and programs S = Satisfaction of how well trusted C = Costs per unit or amount spent S = Standards for services, staffing, etc. “TASCS” 3.How can we measure if we deliver our services well? Quality Measures

43 Current HistoryForecast Factors increasing trend Factors decreasing trend 4. What is the data telling us about how are we doing on the performance measures? What is the story? Trend Line ? Better ? Same ? Worse

44 Shea Community Workforce Partner Number of Training Program Graduates with Jobs

45  Who is contributing?  Who is missing at the table? Who do we recruit to help move the work forward?  Which partners can expand their efforts? 5.Who are the partners?

46  What are the priority factors that are affecting the  and  of the trend line?  What do we think would be effective to make a difference – to improve the identified “better off” or “how well” performance measure?  What do we know from evidence, practice, or our experiences? 6.What actions are powerful enough to address the priority factors to improve the trend line?

47  What actions are we going to take?  Who is going to do what?  What is the timeline?  When will we collect the data to take stock of whether the actions are working? 7.What actions are we going to take?

48 Results Indicators Strategies Performance Performance Measures for Measures for Customers Performance The Intersection of Population & Performance Accountability Population

49 The Contribution of Program Measures to Population Change A Training Class One Employment Partner New Haven Partners Indicator: % of families employed Result: New Haven families achieve financial stability and security.   Program Measure: # of re-entry residents employed

50 Taking Action to Improve the Trend Line 1. Identify your Performance Measure 2. Draw your trend line 3. List the factors that increase and decrease the trend line 1. Use the priority factors to identify what actions to take to improve the trend line 2. Report out the actions to take

51 Group Report Trend Line Measure Factors affecting the trend line   Partners   Three Actions – To Take    No-cost / low-cost