1. Book & DVD Orders amazon.com clearimpact.com
RBA
Results-Based Accountability
The Fiscal Policy Studies Institute raguide.org & resultsaccountability.com
Book & DVD Orders amazon.com clearimpact.com
SCORECARD

2. Teaching RBA: Approaches, Learnings and What Works
Annie E. Casey Foundation
Expanding the Bench in Performance Management
Results-Based Accountability Summit
DAY 2
Teaching RBA: Approaches, Learnings and What Works

3. DAY 2 AGENDA OPTIONS 1. Teaching RBA Questions
2. RBA Material I didn't get to yesterday
3. Additional Exercises & Teaching Tools
4. Review of some crucial essays in the Companion Reader

4. TEACHING RBA
Questions & Lessons

5. What was your aim in integrating RBA into your course?
What RBA components did you include?
How did students respond to RBA?
What are the learning activities used inside and outside the course?

6. What were your learnings? What worked? What didn't work?
What would you do differently?
What is your advice to your colleagues?

7. I didn't get to yesterday
RBA MATERIAL
I didn't get to yesterday

8. Select 3 to 5 Performance Measures ACROSS THE BOTTOM OF THE ORG CHART
3 - 5
3 - 5
This section presents instructions and reporting formats for the two turn the curve exercises, one for population accountability and one for performance accountability. And other exercises
3 - 5
3 - 5
3 - 5
3 - 5
3 - 5
3 - 5
3 - 5
20 – 60 – 20 Rule

10. Next Generation Contracting Contract Provisions
Provision 1. Specify the 3 to 5 most important performance measures (from the How well did we do it? and Is anyone better off? categories).
Provision 2. Specify that the contractor will use a continuous improvement process (the RBA 7 Questions).
Provision 3. Specify how the funder and contractor will work in partnership to maximize LR customer results (quarterly meetings using the 7 questions as the agenda).
We have lots of examples of well-established standards in the upper right (How well did we do it?) quadrant, because we know what good service delivery looks like.
But standards in the lower right (Is anyone better off?) quadrant are almost always experimental. This is partly because of the different mixes of easy and hard cases in different caseloads or workloads.
Provision 4. Specify that the funder will work with the funding community to simplify and standardize contracting and performance reporting.

11. Next Generation Contracting Contract Provisions
Provision 5: : Clear articulation of role in population/community well-being using the language of contribution not attribution.
Provision 6: 10% for quality management and administration. Provision 7: Multi-year funding using 3 year rolling contracts
Provision 8: Use of targets that are fair and useful.
Provision 9: Fund flexibility and virtual funding pool: transfer of up to 10% across line items and program lines.
Provision 10: Request for Results: Getting past the sometimes negative effects of competitive RFP contracting or tendering.

12. THE LINKAGE Between POPULATION and PERFORMANCE
POPULATION ACCOUNTABILITY
Healthy Births Rate of low birth-weight babies Children Ready for School Percent fully ready per K-entry assessment Self-sufficient Families Percent of parents earning a living wage
POPULATION RESULTS
Contribution relationship
PERFORMANCE ACCOUNTABILITY
Alignment of measures
Job Training Program
# persons receiving training
Unit cost per person trained
The relationship is a “contribution” relationship, not a cause and effect relationship. What we do for our customers is our contribution to what we and our partners are trying to do across the community.
Often the only difference between a population indicator and a lower right (Is anyone better off?) performance measure is the difference in scale between a client population and the total population.
This allows us to think about how our work is aligned with what we are trying to accomplish across the community. It allows us to think about how the measures we use at the program level relate to those at the population level. And it allows us to avoid the trap of holding programs responsible for population level change. We can hold program responsible for what they do for their clients. We must hold ourselves, across the community, responsible for the well being of the population.
Appropriate responsibility
# who get living wage jobs
% who get living wage jobs
CUSTOMER RESULTS

13. Performance measure
Population Indicator
As the service population approaches the total population, the measures of client better-offness begin to play a double role. They are used as management measures for the service system and they also can be used as an indicator proxy for the well-being of the whole population.
High School graduation rate is a good example. It is used by the school system as a measure of performance. And it is also used as an indicator by community collaboratives for the results “All Children Succeed in School.”

14. Every time you present your program, Use a two-part approach.
Population Accountability
Result: to which you contribute to most directly.
Indicators:
Story:
Every time you present your program,
Use a two-part approach.
Partners:
What would it take?:
Your Role: as part of a larger strategy.
Your Role
Performance Accountability
Program:
Performance measures:
Budgets of the future will have two parts:
Volume I will present a picture of quality of life results and indicators and what is being done by government and its partners to improve.
Volume II will present performance measures for departments and programs.
Both will use the Baseline, Story, What Works and Strategy format shown above.
This is a fractal… the same pattern at every level of magnification.
Story:
Partners:
Action plan to get better:

15. Every time you present your program, Use a two-part approach.
Population Accountability
Result: to which you contribute to most directly.
Indicators:
Story:
Every time you present your program,
Use a two-part approach.
Partners:
What would it take?:
Your Role: within the larger strategy.
Your Role
Performance Accountability
Program:
Performance measures:
What happens when the population perspective is missing? Any questions about population quality of life get loaded onto the performance discussion. And the program is held unfairly responsible for producing population / community level change that it can not possibly deliver by itself. Both perspectives are needed to respond to the combination of population and performance questions you will inevitably get.
Volume I will present a picture of quality of life results and indicators and what is being done by government and its partners to improve.
Volume II will present performance measures for departments and programs.
Both will use the Baseline, Story, What Works and Strategy format shown above.
This is a fractal… the same pattern at every level of magnification.
Story:
Partners:
Action plan to get better:

16. Every time you present your program, Use a two-part approach.
Population Accountability
Result: to which you contribute to most directly.
Indicators:
Shortcut
Story:
Every time you present your program,
Use a two-part approach.
Partners:
What would it take?:
Your Role: as part of a larger strategy.
Your Role
Performance Accountability
Program:
Performance measures:
At a minimum, any program or agency ought to be able to articulate the role it plays in contributing to quality of life
Budgets of the future will have two parts:
Volume I will present a picture of quality of life results and indicators and what is being done by government and its partners to improve.
Volume II will present performance measures for departments and programs.
Both will use the Baseline, Story, What Works and Strategy format shown above.
This is a fractal… the same pattern at every level of magnification.
Story:
Partners:
Action plan to get better:

17. Alternative to Traditional Evaluation Methods: DEMONSTRATING the CONTRIBUTION of complex change efforts… requires 3 elements:
A Curve to Turn
Bonus: a background trend stayed the same or got worse. 4
2 …and it had a timely relationship to….
3 …. a turn in the curve.
1 We tried a bunch of stuff that had a credible chance of making a difference …
c FPSI

18. Rebound
Tillamook County was successful in bringing down the teen pregnancy rate, while the rest of Oregon stayed about the same.

19. Additional Exercises
and Teaching Tools

20. RBA/OBA Ends vs. Means Exercise
Introduction and the difference between population and performance accountability: We are going to talk about two different kinds of accountability: Accountability for whole populations, like all children in Los Angeles, all elders in Chicago, all residents of North Carolina. This first kind of accountability is not the responsibility of any one agency or program. If we talk for example about “all children in your community being healthy,” who are some of the partners that have a role to play? Notice that the traditional answer is “It’s the health department.” It’s got the word health in it and so it must be the responsibility of the health department. And yet one of the things we have learned in the last 50 years is that the health department by itself can’t possibly produce health for all children without the active participation of many other partners. And that’s the nature of this first kind of accountability. It’s not about the health department. It’s about the kind of cross community partnerships necessary to make progress on quality of life for any population. Now the second kind of accountability, Performance Accountability, is about the health department. It’s about the programs and services we provide, and our role as managers, making sure our programs are working as well as possible. These are two profoundly different kinds of accountability. We going to talk about how to do each one well and then how they fit back together again.

21. On a blank sheet of paper
Ends vs. Means Exercise
INSTRUCTIONS
On a blank sheet of paper
Like this.
Step 1. Draw 4 points that represent the 4 corners of a box or square, about two inches on a side.
Step 2. Draw a line from the upper left point to the lower right point.
Step 3. Draw a line from the upper right point to the lower left point.
Step 4. Draw a horizontal line connecting the bottom two points.
Step 5. Mark another  point approximately 1 inch below the middle of the horizontal line.
Step 6. Connect the upper left point to this new point
Step 7. Connect the upper right point to this new point.

22. What do you have?
A Star !!!

23. Of course not ! So what’s the point?
Question If I asked you to draw a star would you need the step-by-step instructions?
Of course not !
So what’s the point?

24. Give people the ends and let them figure out the means.
And Start Seeing Stars !

27. THE SORTING EXERCISE

28. Creating a Working Baseline from Group Knowledge
Indicator or Performance Measure
Not OK?
65%
Forecasting
Backcasting
Use this method to create a working baseline when only one point of data is available or no data is available.
Step 1: Estimate where you are now.
Step 2: Backcasting: Where have we been over the last few years – getting better worse or about the same. Draw an increasing, decreasing or flat history.
Step 3: Forecasting: Where are we headed if we don't do something more or different? Not where we want to go, where we are going.
Step 4. Ask yourself I this is OK or not. If it's not OK then you can talk about turning the curve.
NOTE: If you draw the improvement line make sure people don't interpret that as a promise.
Now

29. Turn the Curve Exercise: Program Performance
5 min: Starting Points - timekeeper and reporter - identify a program to work on - two hats (yours plus partner’s)
10 min: Performance measure baseline - choose 1 measure to work on – from the lower right quadrant - forecast (to 2018) – OK or not OK?
15 min: Story behind the baseline - causes/forces at work - information & research agenda part 1 - causes
Two pointers to action
15 min: What works? (What would it take?) - what could work to do better? - each partners contribution - no-cost / low-cost ideas - information & research agenda part 2 – what works
Participant instructions for the performance turn the curve exercise.
10 min: Report convert notes to one page

30. ONE PAGE Turn the Curve Report: Performance
Off the Wall
ONE PAGE Turn the Curve Report: Performance
Program: _______________
Performance Measure (Lay definition)
Performance Measure Baseline
Story behind the baseline (List as many as needed)
Partners (List as many as needed)
Group report out format for the performance turn the curve exercise.
Three Best Ideas – What Works No-cost / low-cost
Sharp Edges
Off the Wall

31. Turn the Curve Exercise: Population Well-being
5 min: Starting Points - timekeeper and reporter - geographic area - two hats (yours plus partner’s)
10 min: Baseline - pick a result and a curve to turn - forecast (to 2020) – OK or not OK?
15 min: Story behind the baseline - causes/forces at work - information & research agenda part 1 - causes
Two pointers to action
15 min: What works? (What would it take?) - what could work to do better? - each partners contribution - no-cost / low-cost ideas - information & research agenda part 2 – what works
Participant instructions for the population turn the curve exercise.
10 min: Report convert notes to one page

32. ONE PAGE Turn the Curve Report: Population
Off the Wall
ONE PAGE Turn the Curve Report: Population
Result: _______________
Indicator (Lay Definition)
Indicator Baseline
Story behind the baseline (List as many as needed)
Partners (List as many as needed)
Group report out format for the population turn the curve exercise.
Three Best Ideas – What Works No-cost / low-cost
Sharp Edges
Off the Wall

33. 20 Minute Exercise
1. What population are we concerned about? (e.g. U.S. population)
Population & 1 result
2. What condition(s) do we want for this population? (Americans are free of cancer.)
3. How could we measure these conditions? ( (e.g. cancer rates)
1 indicator
4. How are we doing on the most important of these measures? (baseline history)
Discuss how we’re doing
2 partners
5. Who are the partners with a role to play in doing better?
20 Minute Performance Accountability Exercise
From raguide.org:
Here's an exercise which allows people to experience the whole performance accountability thinking process in just 20 minutes.
1. Hand out the 7 Questions or the following instructions page. Point people to the chart  which shows the different types of performance measures in each quadrant.
2. Ask people to work in pairs, choose a program to work on, and answer the 7 Questions as described below. Point out that this allows for about 3 minutes per questions.
3. During the exercise, walk around to answer questions and check to see that people are making progress. 
4. Debrief the exercise by asking 
- How many people got to questions 6 or 7?
 - How many people here have ever said "This is a good idea, but I don't have time for it." Point out that they just did it in 20 minutes.
- What did you learn from this?
2 what works ideas (1 nc/lc)
6. What works – what would it take – to do better? (What is our role?)
Discuss how to implement
7. What do we propose to do?

34. POPULATION ACCOUNTABILITY
20 Minute Exercise
POPULATION ACCOUNTABILITY
1 population
1. What population are we concerned about? (e.g. U.S. population)
1 result
2. What condition(s) do we want for this population? (Americans are free of cancer.)
1 indicator
3. How could we measure these conditions? ( (e.g. cancer rates)
Discuss how we’re doing
4. How are we doing on the most important of these measures? (baseline history)
2 partners
5. Who are the partners with a role to play in doing better?
20 Minute Performance Accountability Exercise
From raguide.org:
Here's an exercise which allows people to experience the whole performance accountability thinking process in just 20 minutes.
1. Hand out the 7 Questions or the following instructions page. Point people to the chart  which shows the different types of performance measures in each quadrant.
2. Ask people to work in pairs, choose a program to work on, and answer the 7 Questions as described below. Point out that this allows for about 3 minutes per questions.
3. During the exercise, walk around to answer questions and check to see that people are making progress. 
4. Debrief the exercise by asking 
- How many people got to questions 6 or 7?
 - How many people here have ever said "This is a good idea, but I don't have time for it." Point out that they just did it in 20 minutes.
- What did you learn from this?
2 what works ideas (1 nc/lc)
6. What works – what would it take – to do better? (What is our role?)
Discuss how to implement
7. What do we propose to do?

35. If your service works really well, how are your customer's better off?
School
Hospital
Job Training
Fire Department
How much did we do?
How well did we do it?
Primary customers
Unit cost #
students
patients
persons trained
Workload ratio
Primary activity
% of ___x___ that happen on time #
job courses
alarms responded to
diagnostic tests
hours of instruction
Is anyone better off?
If your service works really well, how are your customer's better off?
1. Run this exercise after the full 4 Quadrant measures description.
After each step, pause and check in with the whole group. "Did everyone get this?"
Step 1: Draw the four quadrants on a blank sheet. You can do this by sectioning a page with a vertical and horizontal line.
Step 2: Think of a program or service you know really well and write the name of the service at the top. For purposes of this exercise think of a program or service that is small and has a clearly defined customer group. Write the name of this program or service at the top of the page. So for example we might write "school, hospital, job training, fire department."
Step 3: Who are the primary customers? All services have more than one customer group. Think of the primary customers. So for example we might write # students, patients, persons trained" Put a hash mark in front of the name to signify that we are counting these people.
Step 4: What is the primary activity. All services have many activities. So for example (run through examples on slide) Put a hash mark in front to show we are counting the number of these things.
Step 5: Upper right – write the words "unit cost." All services should track this. The exact definition can be figured out later.
Step 6. Write down a workload ratio. For example student-teacher ratio or nurse-patient ratio.
Step 7: Write down the percentage of time something happens on time, like % of applications processed in one week or % of discharge plans completed within 2 days of discharge
Step 8: Consider the most important question in all of performance measurement "If your service works really well,how are your customer's better off?" So for example (Look at the screen) #/% of students who graduate; #/% of patients who recover; #/% of people who get jobs
Step 9: Who wants to share. Tell us the name of the service and the lower right quadrant measure. # %
fires kept to room of origin
persons who get jobs
patients who fully recover
students who graduate

36. RBA/OBA Least Harm Cutback Exercise 1: POPULATION
Least harm to POPULATION quality of life (Population Accountability)
1. Identify a population result/outcome to which your organization most directly contributes. (e.g. Healthy People, Sustainable Environment, Children Ready for School)
2. What is your organization's role in contributing to this result/outcome?
3. What do you do that causes the greatest contribution?
4. What do you do that causes the least contribution? (Least harm cut candidates)
5. How could you change the way you do your work so as to make the same, or close to the same, contribution with less resources? (Least harm cut candidates)  
- Time: 5 minutes for each question
- Record answers for each question.
- Report 3 best ideas and reasoning

37. RBA/OBA Least Harm Cutback Exercise 2: PERFORMANCE
Least harm to CUSTOMERS' quality of life (Performance Accountability)
1. Identify one service.
2. Identify the primary customer group.
3. What are you doing that is most effective in improving the lives of your customers?
4. What is least effective? (Least harm cut candidates)
5. How could you change the way you do your work so as to make the same, or close to the same, contribution with less resources? (Least harm cut candidates)  
- Time: 5 minutes for each question
- Record answers for each question.
- Report 3 best ideas and reasoning

38. RBA in a Nutshell 2 – 3 - 7 2 - kinds of accountability
     Population accountability
     Performance accountability
plus language discipline
     Results & Indicators
     Performance measures
 3 - kinds of performance measures.
     How much did we do?
     How well did we do it?
     Is anyone better off?
7 - questions from ends to means in less than an hour.
Baselines & Turning the Curve

39. The Simplest Way TO IMPLEMENT RBA
POPULATION ACCOUNTABILITY
1. Pick an important indicator curve
2. Run the Population Turn the Curve Exercise
3. Take action
Repeat
PERFORMANCE ACCOUNTABILITY
1. Pick an important performance measure curve
2. Run the Performance Turn the Curve Exercise
3. Take action
Repeat

40. - RBA and Non-Western Cultures (p. 8)
REVIEW OF SOME
CRUCIAL ESSAYS
in the Companion Reader
- VT Accountability Compact (p. 6 & 98)
- RBA and Non-Western Cultures (p. 8)
- Intellectual Integrity of RBA (p. 9)
- The book-club / curriculum approach (p. 26)
- Misuse of targets (p. 45)

41. better in the next 3 years,
If you could
make one number
better in the next 3 years,
what would it be?

42. What’s one thing I can personally do with what I learned today?
NEXT STEPS
What’s one thing I can personally do with what I learned today?

43. Book - DVD Orders amazon.com resultsleadership.org
Resources
RBA Facebook Group
Click to reveal DVD cover.
Book - DVD Orders amazon.com resultsleadership.org

44. “If you do what you always did, you will get what you always got.”
Kenneth W. Jenkins President, Yonkers NY NAACP

45. “We will succeed together… or fail separately.”
- Winston Churchill or
- Sonny and Cher |

46. Book - DVD Orders amazon.com resultsleadership.org
THANK YOU !
Click to reveal DVD cover.
Book - DVD Orders amazon.com resultsleadership.org