Performance Accountability Types of Measures found in each Quadrant How much did we do? How well did we do it? % Common measures e.g. client staff ratio, workload ratio, staff turnover rate, staff morale, % staff fully trained, % clients seen in their own language, worker safety, unit cost # Clients/customers served % Activity-specific measures e.g. % timely, % clients completing activity, % correct and complete, % meeting standard # Activities (by type of activity) Is anyone better off? % Skills / Knowledge (e.g. parenting skills) # This chart shows in detail the different types of measure we typically find in each quadrant, and the measures that go with the three basic categories of performance measurement: How much did we do? How well did we do it? Is anyone better off? In the upper left, How much did we do? Quadrant, we typically count customers and activities. In the upper right, How well did we do it? Quadrant, there are a set of common measures that apply to many different programs. And there is a set of activity specific measures. For each activity in the upper left, there is one or more measures that tell how well that particular activity was performed, usually having to do with timeliness or correctness. In the lower quadrants, Is anyone better off? We usually have # and % pairs of the same measure. And these measures usually have to do with one of these four dimensions of better-offness: Skills/knowledge, Attitude, Behavior and Circumstance. For each of these measures, we can use point in time measures or point to point improvement measures. Point in Time vs. 2 Point Comparison % Attitude / Opinion (e.g. toward drugs) # # % % Behavior (e.g.school attendance) # % Circumstance (e.g. working, in stable housing) #

If your service works really well, how are your customer's better off? School Hospital Job Training Fire Department How much did we do? How well did we do it? Primary customers Unit cost # students patients persons trained Workload ratio Primary activity % of ___x___ that happen on time # job courses alarms responded to diagnostic tests hours of instruction Is anyone better off? If your service works really well, how are your customer's better off? 1. Run this exercise after the full 4 Quadrant measures description. After each step, pause and check in with the whole group. "Did everyone get this?" Step 1: Draw the four quadrants on a blank sheet. You can do this by sectioning a page with a vertical and horizontal line. Step 2: Think of a program or service you know really well and write the name of the service at the top. For purposes of this exercise think of a program or service that is small and has a clearly defined customer group. Write the name of this program or service at the top of the page. So for example we might write "school, hospital, job training, fire department." Step 3: Who are the primary customers? All services have more than one customer group. Think of the primary customers. So for example we might write # students, patients, persons trained" Put a hash mark in front of the name to signify that we are counting these people. Step 4: What is the primary activity. All services have many activities. So for example (run through examples on slide) Put a hash mark in front to show we are counting the number of these things. Step 5: Upper right – write the words "unit cost." All services should track this. The exact definition can be figured out later. Step 6. Write down a workload ratio. For example student-teacher ratio or nurse-patient ratio. Step 7: Write down the percentage of time something happens on time, like % of applications processed in one week or % of discharge plans completed within 2 days of discharge Step 8: Consider the most important question in all of performance measurement "If your service works really well,how are your customer's better off?" So for example (Look at the screen) #/% of students who graduate; #/% of patients who recover; #/% of people who get jobs Step 9: Who wants to share. Tell us the name of the service and the lower right quadrant measure. # % fires kept to room of origin persons who get jobs patients who fully recover students who graduate

Choosing Headline Measures and the Data Development Agenda Quantity Quality How much did we do? How well did we do it? #3 DDA # Measure 1 ---------------------------- # Measure 2 ---------------------------- # Measure 3 ---------------------------- # Measure 4 ---------------------------- # Measure 5 ---------------------------- # Measure 6 ---------------------------- # Measure 7 ---------------------------- % Measure 8 ---------------------------- % Measure 9 ----------------------------- % Measure 10 --------------------------- % Measure 11 --------------------------- % Measure 12 --------------------------- % Measure 13 --------------------------- % Measure 14 --------------------------- #2 Headline Effect Effort Is anyone better off? Examples of measures for the Jim Casey Youth Opportunity Initiative # Measure 15 ---------------------------- # Measure 16 ---------------------------- # Measure 17 ---------------------------- # Measure 18 ---------------------------- # Measure 19 ---------------------------- # Measure 20 ---------------------------- # Measure 21 ---------------------------- #2 DDA % Measure 15 ---------------------------- % Measure 16 ---------------------------- % Measure 17 ---------------------------- % Measure 18 ---------------------------- % Measure 19 ---------------------------- % Measure 20 ---------------------------- % Measure 21 ---------------------------- #3 Headline #1 Headline #1 DDA

Primary v. Secondary Direct v. Indirect Internal v. External LR UR Baseline & Story Here is the thinking process in the form of 7 plain language common sense questions. These questions should be asked an answered periodically (monthly, quarterly) at every intersection of supervision from the top to the bottom of the organization. This is the most important take-away page for performance measurement. It can be used immediately without any further training.

73KB

Welsh Epilepsy Unit case study was also posted on the same page and that's the one showing benefits such as:" • The average length of time from seizure to a confirmed diagnosis has decreased by 81 days from 111 days to 30 days • The number of patients who have been seen by a specialist within the NICE guideline of two weeks has increased from 35% to 61% • The average waiting time to see a specialist has decreased from 22 days to 11 days • The number of admissions following a seizure have decreased from 5 a month to 2 a month on average"

Watermain Breaks per year London Ontario

Select 3 to 5 Performance Measures ACROSS THE BOTTOM OF THE ORG CHART 3 - 5 3 - 5 This section presents instructions and reporting formats for the two turn the curve exercises, one for population accountability and one for performance accountability. And other exercises 3 - 5 3 - 5 3 - 5 3 - 5 3 - 5 3 - 5 3 - 5 20 – 60 – 20 Rule

HOW Population and Performance FIT TOGETHER

THE LINKAGE Between POPULATION and PERFORMANCE POPULATION ACCOUNTABILITY Healthy Births Rate of low birth-weight babies Children Ready for School Percent fully ready per K-entry assessment Self-sufficient Families Percent of parents earning a living wage POPULATION RESULTS Contribution relationship PERFORMANCE ACCOUNTABILITY Alignment of measures Job Training Program # persons receiving training Unit cost per person trained The relationship is a “contribution” relationship, not a cause and effect relationship. What we do for our customers is our contribution to what we and our partners are trying to do across the community. Often the only difference between a population indicator and a lower right (Is anyone better off?) performance measure is the difference in scale between a client population and the total population. This allows us to think about how our work is aligned with what we are trying to accomplish across the community. It allows us to think about how the measures we use at the program level relate to those at the population level. And it allows us to avoid the trap of holding programs responsible for population level change. We can hold program responsible for what they do for their clients. We must hold ourselves, across the community, responsible for the well being of the population. Appropriate responsibility # who get living wage jobs % who get living wage jobs CUSTOMER RESULTS

Curve Turn Evaluation Methodology (CTEM) DEMONSTRATING the CONTRIBUTION of complex change efforts… requires 3 elements: A Curve to Turn Bonus: a background trend stayed the same or got worse. 4 2 …and it had a timely relationship to…. 3 …. a turn in the curve. 1 We tried a bunch of stuff that had a credible chance of making a difference … c FPSI

Tillamook County was successful in bringing down the teen pregnancy rate, while the rest of Oregon stayed about the same.

Performance measure Population Indicator As the service population approaches the total population, the measures of client better-offness begin to play a double role. They are used as management measures for the service system and they also can be used as an indicator proxy for the well-being of the whole population. High School graduation rate is a good example. It is used by the school system as a measure of performance. And it is also used as an indicator by community collaboratives for the results “All Children Succeed in School.”

Every time you present your program, Use a two-part approach. Population Accountability Result: to which you contribute to most directly. Indicators: Story: Every time you present your program, Use a two-part approach. Partners: What would it take?: Your Role: as part of a larger strategy. Your Role Performance Accountability Program: Budgets of the future will have two parts: Volume I will present a picture of quality of life results and indicators and what is being done by government and its partners to improve. Volume II will present performance measures for departments and programs. Both will use the Baseline, Story, What Works and Strategy format shown above. This is a fractal… the same pattern at every level of magnification. Performance measures: Story: Partners: Action plan to get better:

Every time you present your program, Use a two-part approach. Population Accountability Result: to which you contribute to most directly. Indicators: Shortcut Story: Every time you present your program, Use a two-part approach. Partners: What would it take?: Your Role: as part of a larger strategy. Your Role Performance Accountability Program: At a minimum, any program or agency ought to be able to articulate the role it plays in contributing to quality of life Budgets of the future will have two parts: Volume I will present a picture of quality of life results and indicators and what is being done by government and its partners to improve. Volume II will present performance measures for departments and programs. Both will use the Baseline, Story, What Works and Strategy format shown above. This is a fractal… the same pattern at every level of magnification. Performance measures: Story: Partners: Action plan to get better:

Different Kinds of Progress 1. Data a. Population indicators Actual turned curves: movement for the better away from the baseline. b. Program performance measures: customer progress and better service: How much did we do? How well did we do it? Is anyone better off? There are four kinds of progress which can be reported. The first at the population level, and 2 3 and 4 at the program agency or service system level. 2. Accomplishments: Positive activities, not included above. 3. Stories behind the statistics that show how individuals are better off.

Board of Directors Meeting AGENDA 1. New data 1. New data 2. New story behind the curves 2. New story behind the curves 3. New partners 3. New partners 4. New information on what works. 4. New information on what works. Community collaborative groups and programs and agencies could use this as the agenda for their meetings. The meeting would be aligned with the thinking process that produced the action plan. Each iteration of this thinking process will improve the action plan. 5. New information on financing 5. New information on financing 6. Changes to action plan and budget 6. Changes to action plan and budget 7. Adjourn 7. Adjourn

SUMMARY

RBA in a Nutshell 2 – 3 - 7 2 - kinds of accountability      Population accountability      Performance accountability plus language discipline      Results & Indicators      Performance measures  3 - kinds of performance measures.      How much did we do?      How well did we do it?      Is anyone better off? 7 - questions from ends to means in less than an hour. Baselines & Turning the Curve

“If you do what you always did, you will get what you always got.” Kenneth W. Jenkins President, Yonkers NY NAACP

EXERCISES This section presents instructions and reporting formats for the two turn the curve exercises, one for population accountability and one for performance accountability. And other exercises

Source: Kids Count Data Center

2014 2016 2018 2020 2022

2014 2016 2018 2020 2022

2014 2016 2018 2020 2022

2013 2014 2016 2018 2020 2022

2014 2016 2018 2020 2022

AECF Kids Count Data Center 2018 2020 2022 AECF Kids Count Data Center

Creating a Working Baseline from Group Knowledge Indicator or Performance Measure Not OK? 65% Forecasting Backcasting Use this method to create a working baseline when only one point of data is available or no data is available. Step 1: Estimate where you are now. Step 2: Backcasting: Where have we been over the last few years – getting better worse or about the same. Draw an increasing, decreasing or flat history. Step 3: Forecasting: Where are we headed if we don't do something more or different? Not where we want to go, where we are going. Step 4. Ask yourself I this is OK or not. If it's not OK then you can talk about turning the curve. NOTE: If you draw the improvement line make sure people don't interpret that as a promise. Now

Turn the Curve Exercise: Population Well-being 5 min: Starting Points - timekeeper and reporter - geographic area - two hats (yours plus partner’s) 10 min: Baseline - pick a result and a curve to turn - forecast (to 2022) – OK or not OK? 15 min: Story behind the baseline - causes/forces at work - information & research agenda part 1 - causes Two pointers to action Participant instructions for the population turn the curve exercise. 15 min: What works? (What would it take?) - what could work to do better? - each partners contribution - no-cost / low-cost ideas - information & research agenda part 2 – what works 10 min: Report convert notes to one page

Turn the Curve Exercise: Program Performance 5 min: Starting Points - timekeeper and reporter - identify a program to work on - two hats (yours plus partner’s) 10 min: Performance measure baseline - choose 1 measure to work on – from the lower right quadrant - forecast (to 2020) – OK or not OK? 15 min: Story behind the baseline - causes/forces at work - information & research agenda part 1 - causes Two pointers to action 15 min: What works? (What would it take?) - what could work to do better? - each partners contribution - no-cost / low-cost ideas - information & research agenda part 2 – what works Participant instructions for the performance turn the curve exercise. 10 min: Report convert notes to one page

ONE PAGE Turn the Curve Report: Performance 4. --------- Off the Wall ONE PAGE Turn the Curve Report: Performance Program: _______________ Performance Measure (Lay definition) Performance Measure Baseline Story behind the baseline --------------------------- --------------------------- (List as many as needed) Partners --------------------------- --------------------------- (List as many as needed) Group report out format for the performance turn the curve exercise. Three Best Ideas – What Works 1. --------------------------- 2. --------------------------- 3. ---------No-cost / low-cost Sharp Edges 4. --------- Off the Wall

ONE PAGE Turn the Curve Report: Population 4. --------- Off the Wall ONE PAGE Turn the Curve Report: Population Result: _______________ Indicator (Lay Definition) Indicator Baseline Story behind the baseline --------------------------- --------------------------- (List as many as needed) Partners --------------------------- --------------------------- (List as many as needed) Group report out format for the population turn the curve exercise. Three Best Ideas – What Works 1. --------------------------- 2. --------------------------- 3. ---------No-cost / low-cost Sharp Edges 4. --------- Off the Wall

Turn the Curve Exercise – Lessons Talk to Action in an hour 1. How was this different from other processes? What worked and what didn’t work? 2. Why did we ask for: a. Results before indicators? b. Forecast? c. Story? d. No cost / low cost? e. Two hats? f. Crazy idea? g. Only 3 best ideas? 3. Do you think a lay audience could understand the reports? 4. How many think you could lead this exercise with a small group? (2+ curves at the same time)

The Simplest Way TO IMPLEMENT RBA POPULATION ACCOUNTABILITY 1. Pick an important indicator curve 2. Run the Population Turn the Curve Exercise 3. Take action Repeat PERFORMANCE ACCOUNTABILITY 1. Pick an important performance measure curve 2. Run the Performance Turn the Curve Exercise 3. Take action Repeat

What’s one thing I can personally do with what I learned today? Next Steps What’s one thing I can personally do with what I learned today?

Book - DVD Orders amazon.com clearimpact.org Resources www.raguide.org www.resultsaccountability.com RBA Facebook Group Book - DVD Orders amazon.com clearimpact.org

Book - DVD Orders amazon.com resultsleadership.org THANK YOU ! Book - DVD Orders amazon.com resultsleadership.org

Next Generation Contracting Contract Provisions Provision 1. Specify the 3 to 5 most important performance measures (from the How well did we do it? and Is anyone better off? categories). Provision 2. Specify that the contractor will use a continuous improvement process (the RBA 7 Questions). Provision 3. Specify how the funder and contractor will work in partnership to maximize LR customer results (quarterly meetings using the 7 questions as the agenda). We have lots of examples of well-established standards in the upper right (How well did we do it?) quadrant, because we know what good service delivery looks like. But standards in the lower right (Is anyone better off?) quadrant are almost always experimental. This is partly because of the different mixes of easy and hard cases in different caseloads or workloads. Provision 4. Specify that the funder will work with the funding community to simplify and standardize contracting and performance reporting.

Next Generation Contracting Contract Provisions Provision 5: : Clear articulation of role in population/community well-being using the language of contribution not attribution. Provision 6: 10% for quality management and administration. Provision 7: Multi-year funding using 3 year rolling contracts Provision 8: Use of targets that are fair and useful. Provision 9: Fund flexibility and virtual funding pool: transfer of up to 10% across line items and program lines. Provision 10: Request for Results: Getting past the sometimes negative effects of competitive RFP contracting or tendering.

The Architecture of Accountability Population Accountability Hall Performance Accountability Hall

The Architecture of Accountability Population Accountability Hall Performance Accountability Hall

The Architecture of Accountability

The Architecture of Accountability Population Accountability Hall Population Accountability Hall

The Architecture of Accountability Population Accountability Hall Population Accountability Hall