Book & DVD Orders amazon.com clearimpact.com RBA Results-Based Accountability The Fiscal Policy Studies Institute raguide.org & resultsaccountability.com Book & DVD Orders amazon.com clearimpact.com SCORECARD

Teaching RBA: Approaches, Learnings and What Works Annie E. Casey Foundation Expanding the Bench in Performance Management Results-Based Accountability Summit DAY 2 Teaching RBA: Approaches, Learnings and What Works

DAY 2 AGENDA OPTIONS 1. Teaching RBA Questions 2. RBA Material I didn't get to yesterday 3. Additional Exercises & Teaching Tools 4. Review of some crucial essays in the Companion Reader

TEACHING RBA Questions & Lessons

What was your aim in integrating RBA into your course? What RBA components did you include? How did students respond to RBA? What are the learning activities used inside and outside the course?

What were your learnings? What worked? What didn't work? What would you do differently? What is your advice to your colleagues?

I didn't get to yesterday RBA MATERIAL I didn't get to yesterday

Select 3 to 5 Performance Measures ACROSS THE BOTTOM OF THE ORG CHART 3 - 5 3 - 5 This section presents instructions and reporting formats for the two turn the curve exercises, one for population accountability and one for performance accountability. And other exercises 3 - 5 3 - 5 3 - 5 3 - 5 3 - 5 3 - 5 3 - 5 20 – 60 – 20 Rule

Next Generation Contracting Contract Provisions Provision 1. Specify the 3 to 5 most important performance measures (from the How well did we do it? and Is anyone better off? categories). Provision 2. Specify that the contractor will use a continuous improvement process (the RBA 7 Questions). Provision 3. Specify how the funder and contractor will work in partnership to maximize LR customer results (quarterly meetings using the 7 questions as the agenda). We have lots of examples of well-established standards in the upper right (How well did we do it?) quadrant, because we know what good service delivery looks like. But standards in the lower right (Is anyone better off?) quadrant are almost always experimental. This is partly because of the different mixes of easy and hard cases in different caseloads or workloads. Provision 4. Specify that the funder will work with the funding community to simplify and standardize contracting and performance reporting.

Next Generation Contracting Contract Provisions Provision 5: : Clear articulation of role in population/community well-being using the language of contribution not attribution. Provision 6: 10% for quality management and administration. Provision 7: Multi-year funding using 3 year rolling contracts Provision 8: Use of targets that are fair and useful. Provision 9: Fund flexibility and virtual funding pool: transfer of up to 10% across line items and program lines. Provision 10: Request for Results: Getting past the sometimes negative effects of competitive RFP contracting or tendering.

THE LINKAGE Between POPULATION and PERFORMANCE POPULATION ACCOUNTABILITY Healthy Births Rate of low birth-weight babies Children Ready for School Percent fully ready per K-entry assessment Self-sufficient Families Percent of parents earning a living wage POPULATION RESULTS Contribution relationship PERFORMANCE ACCOUNTABILITY Alignment of measures Job Training Program # persons receiving training Unit cost per person trained The relationship is a “contribution” relationship, not a cause and effect relationship. What we do for our customers is our contribution to what we and our partners are trying to do across the community. Often the only difference between a population indicator and a lower right (Is anyone better off?) performance measure is the difference in scale between a client population and the total population. This allows us to think about how our work is aligned with what we are trying to accomplish across the community. It allows us to think about how the measures we use at the program level relate to those at the population level. And it allows us to avoid the trap of holding programs responsible for population level change. We can hold program responsible for what they do for their clients. We must hold ourselves, across the community, responsible for the well being of the population. Appropriate responsibility # who get living wage jobs % who get living wage jobs CUSTOMER RESULTS

Performance measure Population Indicator As the service population approaches the total population, the measures of client better-offness begin to play a double role. They are used as management measures for the service system and they also can be used as an indicator proxy for the well-being of the whole population. High School graduation rate is a good example. It is used by the school system as a measure of performance. And it is also used as an indicator by community collaboratives for the results “All Children Succeed in School.”

Every time you present your program, Use a two-part approach. Population Accountability Result: to which you contribute to most directly. Indicators: Story: Every time you present your program, Use a two-part approach. Partners: What would it take?: Your Role: as part of a larger strategy. Your Role Performance Accountability Program: Performance measures: Budgets of the future will have two parts: Volume I will present a picture of quality of life results and indicators and what is being done by government and its partners to improve. Volume II will present performance measures for departments and programs. Both will use the Baseline, Story, What Works and Strategy format shown above. This is a fractal… the same pattern at every level of magnification. Story: Partners: Action plan to get better:

Every time you present your program, Use a two-part approach. Population Accountability Result: to which you contribute to most directly. Indicators: Story: Every time you present your program, Use a two-part approach. Partners: What would it take?: Your Role: within the larger strategy. Your Role Performance Accountability Program: Performance measures: What happens when the population perspective is missing? Any questions about population quality of life get loaded onto the performance discussion. And the program is held unfairly responsible for producing population / community level change that it can not possibly deliver by itself. Both perspectives are needed to respond to the combination of population and performance questions you will inevitably get. Volume I will present a picture of quality of life results and indicators and what is being done by government and its partners to improve. Volume II will present performance measures for departments and programs. Both will use the Baseline, Story, What Works and Strategy format shown above. This is a fractal… the same pattern at every level of magnification. Story: Partners: Action plan to get better:

Every time you present your program, Use a two-part approach. Population Accountability Result: to which you contribute to most directly. Indicators: Shortcut Story: Every time you present your program, Use a two-part approach. Partners: What would it take?: Your Role: as part of a larger strategy. Your Role Performance Accountability Program: Performance measures: At a minimum, any program or agency ought to be able to articulate the role it plays in contributing to quality of life Budgets of the future will have two parts: Volume I will present a picture of quality of life results and indicators and what is being done by government and its partners to improve. Volume II will present performance measures for departments and programs. Both will use the Baseline, Story, What Works and Strategy format shown above. This is a fractal… the same pattern at every level of magnification. Story: Partners: Action plan to get better:

Alternative to Traditional Evaluation Methods: DEMONSTRATING the CONTRIBUTION of complex change efforts… requires 3 elements: A Curve to Turn Bonus: a background trend stayed the same or got worse. 4 2 …and it had a timely relationship to…. 3 …. a turn in the curve. 1 We tried a bunch of stuff that had a credible chance of making a difference … c FPSI

Rebound Tillamook County was successful in bringing down the teen pregnancy rate, while the rest of Oregon stayed about the same.

Additional Exercises and Teaching Tools

RBA/OBA Ends vs. Means Exercise Introduction and the difference between population and performance accountability: We are going to talk about two different kinds of accountability: Accountability for whole populations, like all children in Los Angeles, all elders in Chicago, all residents of North Carolina. This first kind of accountability is not the responsibility of any one agency or program. If we talk for example about “all children in your community being healthy,” who are some of the partners that have a role to play? Notice that the traditional answer is “It’s the health department.” It’s got the word health in it and so it must be the responsibility of the health department. And yet one of the things we have learned in the last 50 years is that the health department by itself can’t possibly produce health for all children without the active participation of many other partners. And that’s the nature of this first kind of accountability. It’s not about the health department. It’s about the kind of cross community partnerships necessary to make progress on quality of life for any population. Now the second kind of accountability, Performance Accountability, is about the health department. It’s about the programs and services we provide, and our role as managers, making sure our programs are working as well as possible. These are two profoundly different kinds of accountability. We going to talk about how to do each one well and then how they fit back together again.

On a blank sheet of paper Ends vs. Means Exercise INSTRUCTIONS On a blank sheet of paper Like this. Step 1. Draw 4 points that represent the 4 corners of a box or square, about two inches on a side. Step 2. Draw a line from the upper left point to the lower right point. Step 3. Draw a line from the upper right point to the lower left point. Step 4. Draw a horizontal line connecting the bottom two points. Step 5. Mark another  point approximately 1 inch below the middle of the horizontal line. Step 6. Connect the upper left point to this new point Step 7. Connect the upper right point to this new point.

What do you have? A Star !!!

Of course not ! So what’s the point? Question If I asked you to draw a star would you need the step-by-step instructions? Of course not ! So what’s the point?

Give people the ends and let them figure out the means. And Start Seeing Stars !

THE SORTING EXERCISE

Creating a Working Baseline from Group Knowledge Indicator or Performance Measure Not OK? 65% Forecasting Backcasting Use this method to create a working baseline when only one point of data is available or no data is available. Step 1: Estimate where you are now. Step 2: Backcasting: Where have we been over the last few years – getting better worse or about the same. Draw an increasing, decreasing or flat history. Step 3: Forecasting: Where are we headed if we don't do something more or different? Not where we want to go, where we are going. Step 4. Ask yourself I this is OK or not. If it's not OK then you can talk about turning the curve. NOTE: If you draw the improvement line make sure people don't interpret that as a promise. Now

Turn the Curve Exercise: Program Performance 5 min: Starting Points - timekeeper and reporter - identify a program to work on - two hats (yours plus partner’s) 10 min: Performance measure baseline - choose 1 measure to work on – from the lower right quadrant - forecast (to 2018) – OK or not OK? 15 min: Story behind the baseline - causes/forces at work - information & research agenda part 1 - causes Two pointers to action 15 min: What works? (What would it take?) - what could work to do better? - each partners contribution - no-cost / low-cost ideas - information & research agenda part 2 – what works Participant instructions for the performance turn the curve exercise. 10 min: Report convert notes to one page

ONE PAGE Turn the Curve Report: Performance 4. --------- Off the Wall ONE PAGE Turn the Curve Report: Performance Program: _______________ Performance Measure (Lay definition) Performance Measure Baseline Story behind the baseline --------------------------- --------------------------- (List as many as needed) Partners --------------------------- --------------------------- (List as many as needed) Group report out format for the performance turn the curve exercise. Three Best Ideas – What Works 1. --------------------------- 2. --------------------------- 3. ---------No-cost / low-cost Sharp Edges 4. --------- Off the Wall

Turn the Curve Exercise: Population Well-being 5 min: Starting Points - timekeeper and reporter - geographic area - two hats (yours plus partner’s) 10 min: Baseline - pick a result and a curve to turn - forecast (to 2020) – OK or not OK? 15 min: Story behind the baseline - causes/forces at work - information & research agenda part 1 - causes Two pointers to action 15 min: What works? (What would it take?) - what could work to do better? - each partners contribution - no-cost / low-cost ideas - information & research agenda part 2 – what works Participant instructions for the population turn the curve exercise. 10 min: Report convert notes to one page

ONE PAGE Turn the Curve Report: Population 4. --------- Off the Wall ONE PAGE Turn the Curve Report: Population Result: _______________ Indicator (Lay Definition) Indicator Baseline Story behind the baseline --------------------------- --------------------------- (List as many as needed) Partners --------------------------- --------------------------- (List as many as needed) Group report out format for the population turn the curve exercise. Three Best Ideas – What Works 1. --------------------------- 2. --------------------------- 3. ---------No-cost / low-cost Sharp Edges 4. --------- Off the Wall

20 Minute Exercise 1. What population are we concerned about? (e.g. U.S. population) Population & 1 result 2. What condition(s) do we want for this population? (Americans are free of cancer.) 3. How could we measure these conditions? ( (e.g. cancer rates) 1 indicator 4. How are we doing on the most important of these measures? (baseline history) Discuss how we’re doing 2 partners 5. Who are the partners with a role to play in doing better? 20 Minute Performance Accountability Exercise From raguide.org: Here's an exercise which allows people to experience the whole performance accountability thinking process in just 20 minutes. 1. Hand out the 7 Questions or the following instructions page. Point people to the chart  which shows the different types of performance measures in each quadrant. 2. Ask people to work in pairs, choose a program to work on, and answer the 7 Questions as described below. Point out that this allows for about 3 minutes per questions. 3. During the exercise, walk around to answer questions and check to see that people are making progress.  4. Debrief the exercise by asking  - How many people got to questions 6 or 7?  - How many people here have ever said "This is a good idea, but I don't have time for it." Point out that they just did it in 20 minutes. - What did you learn from this? 2 what works ideas (1 nc/lc) 6. What works – what would it take – to do better? (What is our role?) Discuss how to implement 7. What do we propose to do?

POPULATION ACCOUNTABILITY 20 Minute Exercise POPULATION ACCOUNTABILITY 1 population 1. What population are we concerned about? (e.g. U.S. population) 1 result 2. What condition(s) do we want for this population? (Americans are free of cancer.) 1 indicator 3. How could we measure these conditions? ( (e.g. cancer rates) Discuss how we’re doing 4. How are we doing on the most important of these measures? (baseline history) 2 partners 5. Who are the partners with a role to play in doing better? 20 Minute Performance Accountability Exercise From raguide.org: Here's an exercise which allows people to experience the whole performance accountability thinking process in just 20 minutes. 1. Hand out the 7 Questions or the following instructions page. Point people to the chart  which shows the different types of performance measures in each quadrant. 2. Ask people to work in pairs, choose a program to work on, and answer the 7 Questions as described below. Point out that this allows for about 3 minutes per questions. 3. During the exercise, walk around to answer questions and check to see that people are making progress.  4. Debrief the exercise by asking  - How many people got to questions 6 or 7?  - How many people here have ever said "This is a good idea, but I don't have time for it." Point out that they just did it in 20 minutes. - What did you learn from this? 2 what works ideas (1 nc/lc) 6. What works – what would it take – to do better? (What is our role?) Discuss how to implement 7. What do we propose to do?

If your service works really well, how are your customer's better off? School Hospital Job Training Fire Department How much did we do? How well did we do it? Primary customers Unit cost # students patients persons trained Workload ratio Primary activity % of ___x___ that happen on time # job courses alarms responded to diagnostic tests hours of instruction Is anyone better off? If your service works really well, how are your customer's better off? 1. Run this exercise after the full 4 Quadrant measures description. After each step, pause and check in with the whole group. "Did everyone get this?" Step 1: Draw the four quadrants on a blank sheet. You can do this by sectioning a page with a vertical and horizontal line. Step 2: Think of a program or service you know really well and write the name of the service at the top. For purposes of this exercise think of a program or service that is small and has a clearly defined customer group. Write the name of this program or service at the top of the page. So for example we might write "school, hospital, job training, fire department." Step 3: Who are the primary customers? All services have more than one customer group. Think of the primary customers. So for example we might write # students, patients, persons trained" Put a hash mark in front of the name to signify that we are counting these people. Step 4: What is the primary activity. All services have many activities. So for example (run through examples on slide) Put a hash mark in front to show we are counting the number of these things. Step 5: Upper right – write the words "unit cost." All services should track this. The exact definition can be figured out later. Step 6. Write down a workload ratio. For example student-teacher ratio or nurse-patient ratio. Step 7: Write down the percentage of time something happens on time, like % of applications processed in one week or % of discharge plans completed within 2 days of discharge Step 8: Consider the most important question in all of performance measurement "If your service works really well,how are your customer's better off?" So for example (Look at the screen) #/% of students who graduate; #/% of patients who recover; #/% of people who get jobs Step 9: Who wants to share. Tell us the name of the service and the lower right quadrant measure. # % fires kept to room of origin persons who get jobs patients who fully recover students who graduate

RBA/OBA Least Harm Cutback Exercise 1: POPULATION Least harm to POPULATION quality of life (Population Accountability)   1. Identify a population result/outcome to which your organization most directly contributes. (e.g. Healthy People, Sustainable Environment, Children Ready for School)   2. What is your organization's role in contributing to this result/outcome? 3. What do you do that causes the greatest contribution? 4. What do you do that causes the least contribution? (Least harm cut candidates) 5. How could you change the way you do your work so as to make the same, or close to the same, contribution with less resources? (Least harm cut candidates)   - Time: 5 minutes for each question - Record answers for each question. - Report 3 best ideas and reasoning

RBA/OBA Least Harm Cutback Exercise 2: PERFORMANCE Least harm to CUSTOMERS' quality of life (Performance Accountability) 1. Identify one service.   2. Identify the primary customer group. 3. What are you doing that is most effective in improving the lives of your customers? 4. What is least effective? (Least harm cut candidates) 5. How could you change the way you do your work so as to make the same, or close to the same, contribution with less resources? (Least harm cut candidates)   - Time: 5 minutes for each question - Record answers for each question. - Report 3 best ideas and reasoning

RBA in a Nutshell 2 – 3 - 7 2 - kinds of accountability      Population accountability      Performance accountability plus language discipline      Results & Indicators      Performance measures  3 - kinds of performance measures.      How much did we do?      How well did we do it?      Is anyone better off? 7 - questions from ends to means in less than an hour. Baselines & Turning the Curve

The Simplest Way TO IMPLEMENT RBA POPULATION ACCOUNTABILITY 1. Pick an important indicator curve 2. Run the Population Turn the Curve Exercise 3. Take action Repeat PERFORMANCE ACCOUNTABILITY 1. Pick an important performance measure curve 2. Run the Performance Turn the Curve Exercise 3. Take action Repeat

- RBA and Non-Western Cultures (p. 8) REVIEW OF SOME CRUCIAL ESSAYS in the Companion Reader - VT Accountability Compact (p. 6 & 98) - RBA and Non-Western Cultures (p. 8) - Intellectual Integrity of RBA (p. 9) - The book-club / curriculum approach (p. 26) - Misuse of targets (p. 45)

better in the next 3 years, If you could make one number better in the next 3 years, what would it be?

What’s one thing I can personally do with what I learned today? NEXT STEPS What’s one thing I can personally do with what I learned today?

Book - DVD Orders amazon.com resultsleadership.org Resources www.raguide.org www.resultsaccountability.com RBA Facebook Group Click to reveal DVD cover. Book - DVD Orders amazon.com resultsleadership.org

“If you do what you always did, you will get what you always got.” Kenneth W. Jenkins President, Yonkers NY NAACP

“We will succeed together… or fail separately.” - Winston Churchill or - Sonny and Cher |

Book - DVD Orders amazon.com resultsleadership.org THANK YOU ! Click to reveal DVD cover. Book - DVD Orders amazon.com resultsleadership.org