1. Performance Accountability Types of Measures found in each Quadrant
How much did we do?
How well did we do it?
% Common measures
e.g. client staff ratio, workload ratio, staff turnover rate, staff morale, % staff fully trained, % clients seen in their own language, worker safety, unit cost
# Clients/customers served
% Activity-specific measures
e.g. % timely, % clients completing activity, % correct and complete, % meeting standard
# Activities (by type of activity)
Is anyone better off?
% Skills / Knowledge (e.g. parenting skills) #
This chart shows in detail the different types of measure we typically find in each quadrant, and the measures that go with the three basic categories of performance measurement:
How much did we do?
How well did we do it?
Is anyone better off?
In the upper left, How much did we do? Quadrant, we typically count customers and activities.
In the upper right, How well did we do it? Quadrant, there are a set of common measures that apply to many different programs. And there is a set of activity specific measures. For each activity in the upper left, there is one or more measures that tell how well that particular activity was performed, usually having to do with timeliness or correctness.
In the lower quadrants, Is anyone better off? We usually have # and % pairs of the same measure. And these measures usually have to do with one of these four dimensions of better-offness: Skills/knowledge, Attitude, Behavior and Circumstance.
For each of these measures, we can use point in time measures or point to point improvement measures.
Point in Time vs. 2 Point Comparison
% Attitude / Opinion (e.g. toward drugs) # # %
% Behavior (e.g.school attendance) #
% Circumstance (e.g. working, in stable housing) #

2. If your service works really well, how are your customer's better off?
School
Hospital
Job Training
Fire Department
How much did we do?
How well did we do it?
Primary customers
Unit cost #
students
patients
persons trained
Workload ratio
Primary activity
% of ___x___ that happen on time #
job courses
alarms responded to
diagnostic tests
hours of instruction
Is anyone better off?
If your service works really well, how are your customer's better off?
1. Run this exercise after the full 4 Quadrant measures description.
After each step, pause and check in with the whole group. "Did everyone get this?"
Step 1: Draw the four quadrants on a blank sheet. You can do this by sectioning a page with a vertical and horizontal line.
Step 2: Think of a program or service you know really well and write the name of the service at the top. For purposes of this exercise think of a program or service that is small and has a clearly defined customer group. Write the name of this program or service at the top of the page. So for example we might write "school, hospital, job training, fire department."
Step 3: Who are the primary customers? All services have more than one customer group. Think of the primary customers. So for example we might write # students, patients, persons trained" Put a hash mark in front of the name to signify that we are counting these people.
Step 4: What is the primary activity. All services have many activities. So for example (run through examples on slide) Put a hash mark in front to show we are counting the number of these things.
Step 5: Upper right – write the words "unit cost." All services should track this. The exact definition can be figured out later.
Step 6. Write down a workload ratio. For example student-teacher ratio or nurse-patient ratio.
Step 7: Write down the percentage of time something happens on time, like % of applications processed in one week or % of discharge plans completed within 2 days of discharge
Step 8: Consider the most important question in all of performance measurement "If your service works really well,how are your customer's better off?" So for example (Look at the screen) #/% of students who graduate; #/% of patients who recover; #/% of people who get jobs
Step 9: Who wants to share. Tell us the name of the service and the lower right quadrant measure. # %
fires kept to room of origin
persons who get jobs
patients who fully recover
students who graduate

3. Choosing Headline Measures and the Data Development Agenda
Quantity
Quality
How much did we do?
How well did we do it?
#3 DDA
# Measure
# Measure
# Measure
# Measure
# Measure
# Measure
# Measure
% Measure
% Measure
% Measure
% Measure
% Measure
% Measure
% Measure
#2 Headline
Effect Effort
Is anyone better off?
Examples of measures for the Jim Casey Youth Opportunity Initiative
# Measure
# Measure
# Measure
# Measure
# Measure
# Measure
# Measure
#2 DDA
% Measure
% Measure
% Measure
% Measure
% Measure
% Measure
% Measure
#3 Headline
#1 Headline
#1 DDA

4. Primary v. Secondary Direct v. Indirect Internal v. ExternalLR UR
Baseline & Story
Here is the thinking process in the form of 7 plain language common sense questions.
These questions should be asked an answered periodically (monthly, quarterly) at every intersection of supervision from the top to the bottom of the organization.
This is the most important take-away page for performance measurement. It can be used immediately without any further training.

6. 73KB

8. Welsh Epilepsy Unit case study was also posted on the same page and that's the one showing benefits such as:"
• The average length of time from seizure to a confirmed diagnosis has decreased by 81 days from 111 days to 30 days
• The number of patients who have been seen by a specialist within the NICE guideline of two weeks has increased from 35% to 61%
• The average waiting time to see a specialist has decreased from 22 days to 11 days
• The number of admissions following a seizure have decreased from 5 a month to 2 a month on average"

10. Watermain Breaks per year London Ontario

11. Select 3 to 5 Performance Measures ACROSS THE BOTTOM OF THE ORG CHART
3 - 5
3 - 5
This section presents instructions and reporting formats for the two turn the curve exercises, one for population accountability and one for performance accountability. And other exercises
3 - 5
3 - 5
3 - 5
3 - 5
3 - 5
3 - 5
3 - 5
20 – 60 – 20 Rule

12. HOW Population and Performance FIT TOGETHER

13. THE LINKAGE Between POPULATION and PERFORMANCE
POPULATION ACCOUNTABILITY
Healthy Births Rate of low birth-weight babies Children Ready for School Percent fully ready per K-entry assessment Self-sufficient Families Percent of parents earning a living wage
POPULATION RESULTS
Contribution relationship
PERFORMANCE ACCOUNTABILITY
Alignment of measures
Job Training Program
# persons receiving training
Unit cost per person trained
The relationship is a “contribution” relationship, not a cause and effect relationship. What we do for our customers is our contribution to what we and our partners are trying to do across the community.
Often the only difference between a population indicator and a lower right (Is anyone better off?) performance measure is the difference in scale between a client population and the total population.
This allows us to think about how our work is aligned with what we are trying to accomplish across the community. It allows us to think about how the measures we use at the program level relate to those at the population level. And it allows us to avoid the trap of holding programs responsible for population level change. We can hold program responsible for what they do for their clients. We must hold ourselves, across the community, responsible for the well being of the population.
Appropriate responsibility
# who get living wage jobs
% who get living wage jobs
CUSTOMER RESULTS

14. Curve Turn Evaluation Methodology (CTEM) DEMONSTRATING the CONTRIBUTION of complex change efforts… requires 3 elements:
A Curve to Turn
Bonus: a background trend stayed the same or got worse. 4
2 …and it had a timely relationship to….
3 …. a turn in the curve.
1 We tried a bunch of stuff that had a credible chance of making a difference …
c FPSI

15. Tillamook County was successful in bringing down the teen pregnancy rate, while the rest of Oregon stayed about the same.

16. Performance measure
Population Indicator
As the service population approaches the total population, the measures of client better-offness begin to play a double role. They are used as management measures for the service system and they also can be used as an indicator proxy for the well-being of the whole population.
High School graduation rate is a good example. It is used by the school system as a measure of performance. And it is also used as an indicator by community collaboratives for the results “All Children Succeed in School.”

17. Every time you present your program, Use a two-part approach.
Population Accountability
Result: to which you contribute to most directly.
Indicators:
Story:
Every time you present your program,
Use a two-part approach.
Partners:
What would it take?:
Your Role: as part of a larger strategy.
Your Role
Performance Accountability
Program:
Budgets of the future will have two parts:
Volume I will present a picture of quality of life results and indicators and what is being done by government and its partners to improve.
Volume II will present performance measures for departments and programs.
Both will use the Baseline, Story, What Works and Strategy format shown above.
This is a fractal… the same pattern at every level of magnification.
Performance measures:
Story:
Partners:
Action plan to get better:

18. Every time you present your program, Use a two-part approach.
Population Accountability
Result: to which you contribute to most directly.
Indicators:
Shortcut
Story:
Every time you present your program,
Use a two-part approach.
Partners:
What would it take?:
Your Role: as part of a larger strategy.
Your Role
Performance Accountability
Program:
At a minimum, any program or agency ought to be able to articulate the role it plays in contributing to quality of life
Budgets of the future will have two parts:
Volume I will present a picture of quality of life results and indicators and what is being done by government and its partners to improve.
Volume II will present performance measures for departments and programs.
Both will use the Baseline, Story, What Works and Strategy format shown above.
This is a fractal… the same pattern at every level of magnification.
Performance measures:
Story:
Partners:
Action plan to get better:

19. Different Kinds of Progress
1. Data
a. Population indicators Actual turned curves: movement for the better away from the baseline.
b. Program performance measures: customer progress and better service: How much did we do? How well did we do it? Is anyone better off?
There are four kinds of progress which can be reported. The first at the population level, and 2 3 and 4 at the program agency or service system level.
2. Accomplishments: Positive activities, not included above.
3. Stories behind the statistics that show how individuals are better off.

20. Board of Directors Meeting AGENDA
1. New data
1. New data
2. New story behind the curves
2. New story behind the curves
3. New partners
3. New partners
4. New information on what works.
4. New information on what works.
Community collaborative groups and programs and agencies could use this as the agenda for their meetings. The meeting would be aligned with the thinking process that produced the action plan. Each iteration of this thinking process will improve the action plan.
5. New information on financing
5. New information on financing
6. Changes to action plan and budget
6. Changes to action plan and budget
7. Adjourn
7. Adjourn

21. SUMMARY

23. RBA in a Nutshell 2 – 3 - 7 2 - kinds of accountability
     Population accountability
     Performance accountability
plus language discipline
     Results & Indicators
     Performance measures
 3 - kinds of performance measures.
     How much did we do?
     How well did we do it?
     Is anyone better off?
7 - questions from ends to means in less than an hour.
Baselines & Turning the Curve

24. “If you do what you always did, you will get what you always got.”
Kenneth W. Jenkins President, Yonkers NY NAACP

25. EXERCISES
This section presents instructions and reporting formats for the two turn the curve exercises, one for population accountability and one for performance accountability. And other exercises

26. Source: Kids Count Data Center

32. AECF Kids Count Data Center
AECF Kids Count Data Center

33. Creating a Working Baseline from Group Knowledge
Indicator or Performance Measure
Not OK?
65%
Forecasting
Backcasting
Use this method to create a working baseline when only one point of data is available or no data is available.
Step 1: Estimate where you are now.
Step 2: Backcasting: Where have we been over the last few years – getting better worse or about the same. Draw an increasing, decreasing or flat history.
Step 3: Forecasting: Where are we headed if we don't do something more or different? Not where we want to go, where we are going.
Step 4. Ask yourself I this is OK or not. If it's not OK then you can talk about turning the curve.
NOTE: If you draw the improvement line make sure people don't interpret that as a promise.
Now

34. Turn the Curve Exercise: Population Well-being
5 min: Starting Points - timekeeper and reporter - geographic area - two hats (yours plus partner’s)
10 min: Baseline - pick a result and a curve to turn - forecast (to 2022) – OK or not OK?
15 min: Story behind the baseline - causes/forces at work - information & research agenda part 1 - causes
Two pointers to action
Participant instructions for the population turn the curve exercise.
15 min: What works? (What would it take?) - what could work to do better? - each partners contribution - no-cost / low-cost ideas - information & research agenda part 2 – what works
10 min: Report convert notes to one page

35. Turn the Curve Exercise: Program Performance
5 min: Starting Points - timekeeper and reporter - identify a program to work on - two hats (yours plus partner’s)
10 min: Performance measure baseline - choose 1 measure to work on – from the lower right quadrant - forecast (to 2020) – OK or not OK?
15 min: Story behind the baseline - causes/forces at work - information & research agenda part 1 - causes
Two pointers to action
15 min: What works? (What would it take?) - what could work to do better? - each partners contribution - no-cost / low-cost ideas - information & research agenda part 2 – what works
Participant instructions for the performance turn the curve exercise.
10 min: Report convert notes to one page

36. ONE PAGE Turn the Curve Report: Performance
Off the Wall
ONE PAGE Turn the Curve Report: Performance
Program: _______________
Performance Measure (Lay definition)
Performance Measure Baseline
Story behind the baseline (List as many as needed)
Partners (List as many as needed)
Group report out format for the performance turn the curve exercise.
Three Best Ideas – What Works No-cost / low-cost
Sharp Edges
Off the Wall

37. ONE PAGE Turn the Curve Report: Population
Off the Wall
ONE PAGE Turn the Curve Report: Population
Result: _______________
Indicator (Lay Definition)
Indicator Baseline
Story behind the baseline (List as many as needed)
Partners (List as many as needed)
Group report out format for the population turn the curve exercise.
Three Best Ideas – What Works No-cost / low-cost
Sharp Edges
Off the Wall

38. Turn the Curve Exercise – Lessons Talk to Action in an hour
1. How was this different from other processes? What worked and what didn’t work?
2. Why did we ask for:
a. Results before indicators? b. Forecast? c. Story? d. No cost / low cost? e. Two hats? f. Crazy idea?
g. Only 3 best ideas?
3. Do you think a lay audience could understand the reports?
4. How many think you could lead this exercise with a small group? (2+ curves at the same time)

39. The Simplest Way TO IMPLEMENT RBA
POPULATION ACCOUNTABILITY
1. Pick an important indicator curve
2. Run the Population Turn the Curve Exercise
3. Take action
Repeat
PERFORMANCE ACCOUNTABILITY
1. Pick an important performance measure curve
2. Run the Performance Turn the Curve Exercise
3. Take action
Repeat

40. What’s one thing I can personally do with what I learned today?
Next Steps
What’s one thing I can personally do with what I learned today?

41. Book - DVD Orders amazon.com clearimpact.org
Resources
RBA Facebook Group
Book - DVD Orders amazon.com clearimpact.org

42. Book - DVD Orders amazon.com resultsleadership.org
THANK YOU !
Book - DVD Orders amazon.com resultsleadership.org

44. Next Generation Contracting Contract Provisions
Provision 1. Specify the 3 to 5 most important performance measures (from the How well did we do it? and Is anyone better off? categories).
Provision 2. Specify that the contractor will use a continuous improvement process (the RBA 7 Questions).
Provision 3. Specify how the funder and contractor will work in partnership to maximize LR customer results (quarterly meetings using the 7 questions as the agenda).
We have lots of examples of well-established standards in the upper right (How well did we do it?) quadrant, because we know what good service delivery looks like.
But standards in the lower right (Is anyone better off?) quadrant are almost always experimental. This is partly because of the different mixes of easy and hard cases in different caseloads or workloads.
Provision 4. Specify that the funder will work with the funding community to simplify and standardize contracting and performance reporting.

45. Next Generation Contracting Contract Provisions
Provision 5: : Clear articulation of role in population/community well-being using the language of contribution not attribution.
Provision 6: 10% for quality management and administration. Provision 7: Multi-year funding using 3 year rolling contracts
Provision 8: Use of targets that are fair and useful.
Provision 9: Fund flexibility and virtual funding pool: transfer of up to 10% across line items and program lines.
Provision 10: Request for Results: Getting past the sometimes negative effects of competitive RFP contracting or tendering.

47. The Architecture of Accountability
Population Accountability
Hall
Performance Accountability
Hall

48. The Architecture of Accountability
Population Accountability
Hall
Performance Accountability
Hall

49. The Architecture of Accountability

50. The Architecture of Accountability
Population Accountability
Hall
Population Accountability
Hall

51. The Architecture of Accountability
Population Accountability
Hall
Population Accountability
Hall