Measuring & Managing Health Services: The Balanced Scorecard David Peters, Director, Future Health Systems Research Consortium Sept 28, 2006

“What gets measured gets managed. What gets managed gets done.” Tom Peters

Would This Measurement Describe Your Child?

Why Have a Balanced Scorecard? Health sector is complex, has many components Need an efficient way to assess multiple objectives Overloaded with different types of reports Stakeholders demand vigilance Poor measurement can lead to crisis

Balanced Scorecard – Original Definition “A multi-dimensional framework for describing, implementing and managing strategy at all levels of an enterprise by linking objectives, initiatives, and measures to an organization’s strategy.” Kaplan & Norton, 1996

Which dimensions should we measure? Scorecard Domains Which dimensions should we measure?

Balanced Scorecard: General Domains Customer Results Internal Processes Staff and Organizational Growth Financial Results Vision & Strategy

Qualities of Good Scorecard Measures Valid Reliable Balanced Easily Understood Intermediable Agreed Upon Limited Specific Measurable Achievable Targets Relevant Timely

Presenting Scorecards: Performance Dashboards Simple to see & interpret Multi-dimensional measures of critical indicators of overall performance Allows for the early detection of problems and successes Can be benchmarked against other data

Afghanistan National Health Services Performance Assessment (NHSPA)

Developing the Balanced Scorecard Frontline providers, NGOs, MOPH, donors to agree on: Purpose of BSC Domains to measure Unit of analysis Process & frequency of review/decisions Principles for benchmarking Short-listing indicators based on face validity, importance, reliability

Afghanistan Balanced Scorecard Domains Patient & Community Results Staff Results Capacity for Service Provision Service Provision Results Financial System Results Overall Vision Results

Measuring perceptions The Naanogram Many survey questions sought perceptions. But how to ask? Likert scale couldn’t be understood Money? Excellent Good Fair Poor

Afghanistan Sample for Balanced Scorecard   2004 NHSPA 2005 Provinces 33 30 Facilities 617 628 Basic Health Centers 323 368 Comprehensive HCs 243 217 District Hospitals 51 43 Observations 5719 5863 Exit Interviews 5597 Health worker Interviews 1553 1458 CHW Interviews 167 306

Afghanistan NHSPA Provincial Balanced Scorecard

Patients & Community Perspectives Mean Score (29 Provinces) 2004 2005 Difference 1. Overall patient satisfaction 81.3 84.1 2.8 2. Patient perceptions of quality 74.1 76.5 2.4 3. Written Shura-e-Sehie Activities 39.9 52.4 12.5

Staff Perspectives Mean Score (29 Provinces) 2004 2005 Difference 61.9 4. Health worker satisfaction 61.9 64.0 2.1 5. Salary payments current 68.9 83.9 15.0

Service Provision Mean Score (29 Provinces) 2004 2005 Difference 18. Patient history and physical exam 69.5 74.7 5.1 19. Patient counseling 36.7 38.1 1.4 20. Proper sharps disposal 58.5 48.9 -9.6 21. New outpatient visits (>750/month) 27.6 42.8 15.2

Overall Vision Mean Score (29 Provinces) 2004 2005 Difference 53.7 27. Females as % of new outpatients 53.7 56.3 2.6 28. Outpatient visit concentration index -0.008 -0.021 -0.013 29. Pt. satisfaction concentration index -0.001 0.004 0.005

Identifying Exceptional Provincial Performance Top Performers   Change 2004-05 Wardak 18.0 Balkh 16.0 Ghor 15.5 Faryab 12.0 Saripul 11.1 Bottom Performers   Change 2004-05 Herat -2.2 Paktika -3.0 Nangarhar -4.7 Nimroz -5.5 Nuristan -13.7

Balanced Scorecard: Improvements Progress has been made in most areas Improvements in 22 out of 29 indicators Areas of largest improvement include: Shura-e-Sehie activity Salary payments current Laboratory functionality Drug availability Meeting minimum staffing guidelines

Balanced Scorecard: Problems Improvement not consistent across all provinces or aspects of service delivery Areas of concern: Time spent with patients Facility infrastructure Sharps disposal User fee guidelines Patient records

Balanced Scorecard on Contract Performance: Cumulative Ratings 2004 2005 Difference Performance Based Contracts (WB) 52.3 64.4 12.1 Contracts with Technical Assistance (USAID) 53.4 63.6 10.2 Ministry of Health Strengthening (WB) 53.7 61.8 8.1 Contracting Without Performance (EU) 54.2 57.7 3.5 Usual Management 51.5 48.5 -3.0

Some Final Thoughts

Successful Scorecards Focus on significant, success- determining measures of organization’s total performance Based on timely information Predictive (leading) indicators more useful than trailing indicators

“You can’t fatten a cow by weighing it”