1. Measuring & Managing Health Services: The Balanced Scorecard
David Peters, Director, Future Health Systems Research Consortium
Sept 28, 2006

2. “What gets measured gets managed. What gets managed gets done.”
Tom Peters

3. Would This Measurement Describe Your Child?

4. Why Have a Balanced Scorecard?
Health sector is complex, has many components
Need an efficient way to assess multiple objectives
Overloaded with different types of reports
Stakeholders demand vigilance
Poor measurement can lead to crisis

5. Balanced Scorecard – Original Definition
“A multi-dimensional framework for describing, implementing and managing strategy at all levels of an enterprise by linking objectives, initiatives, and measures to an organization’s strategy.”
Kaplan & Norton, 1996

6. Which dimensions should we measure?
Scorecard Domains
Which dimensions should we measure?

7. Balanced Scorecard: General Domains
Customer Results
Internal Processes
Staff and Organizational Growth
Financial Results
Vision & Strategy

8. Qualities of Good Scorecard Measures
Valid
Reliable
Balanced
Easily Understood
Intermediable
Agreed Upon
Limited
Specific
Measurable
Achievable Targets
Relevant
Timely

9. Presenting Scorecards: Performance Dashboards
Simple to see & interpret
Multi-dimensional measures of critical indicators of overall performance
Allows for the early detection of problems and successes
Can be benchmarked against other data

10. Afghanistan National Health Services Performance Assessment (NHSPA)

11. Developing the Balanced Scorecard
Frontline providers, NGOs, MOPH, donors to
agree on:
Purpose of BSC
Domains to measure
Unit of analysis
Process & frequency of review/decisions
Principles for benchmarking
Short-listing indicators based on face validity, importance, reliability

12. Afghanistan Balanced Scorecard Domains
Patient & Community Results
Staff Results
Capacity for Service Provision
Service Provision Results
Financial System Results
Overall Vision Results

13. Measuring perceptions
The Naanogram
Many survey questions sought perceptions. But how to ask?
Likert scale couldn’t be understood
Money?
Excellent
Good
Fair
Poor

14. Afghanistan Sample for Balanced Scorecard
2004
NHSPA
2005
Provinces 33 30
Facilities
617
628
Basic Health Centers
323
368
Comprehensive HCs
243
217
District Hospitals 51 43
Observations
5719
5863
Exit Interviews
5597
Health worker Interviews
1553
1458
CHW Interviews
167
306

15. Afghanistan NHSPA Provincial Balanced Scorecard

16. Patients & Community Perspectives
Mean Score
(29 Provinces)
2004
2005
Difference
1. Overall patient satisfaction
81.3
84.1
2.8
2. Patient perceptions of quality
74.1
76.5
2.4
3. Written Shura-e-Sehie Activities
39.9
52.4
12.5

17. Staff Perspectives Mean Score (29 Provinces) 2004 2005 Difference 61.9
4. Health worker satisfaction
61.9
64.0
2.1
5. Salary payments current
68.9
83.9
15.0

18. Service Provision Mean Score (29 Provinces) 2004 2005 Difference
18. Patient history and physical exam
69.5
74.7
5.1
19. Patient counseling
36.7
38.1
1.4
20. Proper sharps disposal
58.5
48.9
-9.6
21. New outpatient visits (>750/month)
27.6
42.8
15.2

19. Overall Vision Mean Score (29 Provinces) 2004 2005 Difference 53.7
27. Females as % of new outpatients
53.7
56.3
2.6
28. Outpatient visit concentration index
-0.008
-0.021
-0.013
29. Pt. satisfaction concentration index
-0.001
0.004
0.005

20. Identifying Exceptional Provincial Performance
Top Performers
Change
Wardak
18.0
Balkh
16.0
Ghor
15.5
Faryab
12.0
Saripul
11.1
Bottom Performers
Change
Herat
-2.2
Paktika
-3.0
Nangarhar
-4.7
Nimroz
-5.5
Nuristan
-13.7

21. Balanced Scorecard: Improvements
Progress has been made in most areas
Improvements in 22 out of 29 indicators
Areas of largest improvement include:
Shura-e-Sehie activity
Salary payments current
Laboratory functionality
Drug availability
Meeting minimum staffing guidelines

22. Balanced Scorecard: Problems
Improvement not consistent across all provinces or aspects of service delivery
Areas of concern:
Time spent with patients
Facility infrastructure
Sharps disposal
User fee guidelines
Patient records

23. Balanced Scorecard on Contract Performance: Cumulative Ratings
2004
2005
Difference
Performance Based Contracts (WB)
52.3
64.4
12.1
Contracts with Technical Assistance (USAID)
53.4
63.6
10.2
Ministry of Health Strengthening (WB)
53.7
61.8
8.1
Contracting Without Performance (EU)
54.2
57.7
3.5
Usual Management
51.5
48.5
-3.0

24. Some Final Thoughts

25. Successful Scorecards
Focus on significant, success- determining measures of organization’s total performance
Based on timely information
Predictive (leading) indicators more useful than trailing indicators

26. “You can’t fatten a cow by weighing it”