1. From Silos to Systems: Performance Management in Public Health Turning Point Performance Management Collaborative October 2002

2. Learning Objectives
Gain understanding of the components of a performance management system
Identify at least three benefits of performance management

3. Turning Point’s National Excellence Collaboratives, 2000-2004
Funded by Robt. Wood Johnson Foundation
- States, communities, national partners
Combine collective experience, skills
Take next steps in transforming public health
Review of literature & current practice; analysis
Development of innovative models
Testing and disseminating innovation
Evaluation
The expectations of National Excellence Collaboratives

4. Turning Point: National Excellence Collaboratives
Public Health Statute Modernization
Performance Management
Information Technology
Social Marketing
Leadership Development
The five Collaboratives, each working on tools to improve public health systems.

5. Performance Management Collaborative (PMC)
7 Turning Point States
Illinois*
New York
Montana
Alaska
New Hampshire
Missouri
West Virginia
* Lead State
The Performance Management Collaborative’s participating states

6. More PMC Members
TP National Program Office at Univ. of WA/School of Public Health
National Partners
ASTHO
NACCHO
CDC
HRSA
ASTHLHLO
The Performance Management Collaborative’s national partners

7. PMC Vision
Widespread use of dynamic and accountable public health performance management
We would like to see the field move in this direction in the next five years

8. PMC Goals
To develop useful and feasible performance management models for states
To stimulate national dialogue and consensus on performance management in public health
To support the application of performance management as a core discipline of public health practice
We believe these goals, if achieved, have significant implications for public health practice and workforce development.

9. What Is Performance Management?
The practice of actively using performance data to improve the public’s health.
Performance management can be carried out at the program, organization, community and state levels.
The key words here for us are not “performance data”; they are “actively using”. Data mean nothing if they are not applied and used effectively.
We’re working towards a very flexible model—a balanced management model for public health, not just measurement.
The Collaborative’s preference is to carry out performance management at the systems level.

10. Four Components of Performance Management
Performance Standards
Performance Measures
Reporting of Progress
Quality Improvement

11. Why Develop a PM System?
To maximize public health’s effectiveness. This requires
More than measurement alone
More than standards alone
All four PM components to be continuously integrated into a system of performance management
Why have we spent four years on this? What is the compelling reason for thinking about and changing the practice of performance management?
The key is to integrate the four components into a continuous system.

12. Using Data to Achieve Results
Quality improvement efforts
Policy change
Resource allocation change
Program change
Managerial Action
Public health, like other sectors, needs management results in – READ ALL FOUR ALOUD
These are the outputs of managerial action—what we are keeping our eyes on.

13. Survey of Performance Mgmt. Practices in States
Baseline Assessment
Conducted by PHF
47 of 50 States Responded
Survey Asks About:
Use of Performance Targets, Reports
Impact on Program and Policy
Need for New Tools
We wanted to increase our knowledge and better understand the scope and usefulness of current state practices.
The baseline assessment conducted by the Public Health Foundation achieved an extremely high response rate.

14. Nearly All SHAs Have Some Performance Management Efforts However, only about half apply performance management efforts statewide beyond categorical programs
Figure 1. Agencies or programs to which SHAs apply performance management efforts (N=47)
What we learned is that nearly all states have some performance management efforts, but only about half of these are statewide efforts. We defined statewide very broadly, to encompass state-level efforts, state plus local health department efforts and local health department efforts only.
The other (approximately) half are targeted at categorical programs only or had no PM effort at all.

15. Performance Management Efforts Result in Improved Performance for Three-Quarters of SHAs
Figure 19. Percentage of SHAs that report their performance management efforts resulted in improved performance (N=41)
Reported Positive Outcomes:
Improved delivery of services—program services, clinical preventive services, essential services
Improved administration/management— contracting, tracking/reporting, coordination
Legislation or policy changes
Yes: 76% said PM improved their performance
This positive view of the usefulness of PM efforts, combined with the fact that most state conduct some form of PM, is a very strong base to build on.
A state may have a few of the pieces in place that can be the starting points to transition into a system-wide PM effort.

16. SHAs Most Likely to Have Components of Performance Management for Health Status; Least Likely for Human Resource Development
Figure 8. Areas most and least likely to have performance targets, measures or standards, reports, and processes for quality improvement (QI)/change, of SHAs that apply performance management efforts SHA wide, SHA wide and to local public health agencies, or to local public health agencies only (N=25)
To what areas of public health practice is performance management applied?
We found the components of performance management were most likely to be in place for health status (not surprising, since there are a number of tools available to assist users in health status assessment)
More interesting, perhaps, is that our four components were least likely to be in place for human resource development or management of public health capacity.

17. Top Three Models/Frameworks Explicitly Incorporated by SHAs Into Their Performance Management Efforts
Healthy People Objectives
Core Public Health Functions
Ten Essential Public Health Services
States use a variety of
performance management
models/frameworks, in a
variety of combinations

18. Most SHAs Have Performance Measures, Targets, and Reports, While Fewer States Have Process for Quality Improvement or Change*
Figure 15. Percentage of SHAs that have specified components of performance management for public health capacity (N=25)
State health agencies were very different in the array of performance management components they have in place.
Overall, they were least likely to have a process for quality improvement or change.
They were most likely to have performance measures or standards.
*Correlation analysis revealed that there is a comparatively weak relationship between having performance targets, performance measures or performance reports and process for quality improvement (QI)/change. That is, in general, fewer states indicated that they did have a process for change, even though they indicated having performance targets, performance measures, or performance reports. This was the case for all areas of performance management studied (Human Resource Development, Data & Information Systems, Customer Focus and Satisfaction, Financial Systems, Management Practices, Public Health Capacity, and Health Status). Figure 15 illustrates this finding.

19. Most States Use Neither Incentives nor Disincentives to Improve Performance
Figure 18. Percentage of SHA performance efforts that include incentives or disincentives to improve performance (N=40)
Sixty-three (63) percent don’t use incentives or disincentives to manage performance.
Very small percentage of states use incentives and disincentives use PM in outside contracts, while even less use incentives or disincentives with staff.
Note: Respondents could choose more than one response, so total does not equal 100

20. Funding for Performance Management Chosen as Number One Way to Improve States’ Efforts
Figure 5. Types of aid identified as most useful to SHAs to improve SHA performance management efforts, in rank order (N=47)
We asked what would help states the most to improve their PM efforts.
Not surprisingly, the top “vote-getter” for assistance needed to improve states’ performance management efforts was funding.
It’s interesting to look at other factors, too. For example,
A number of states want to understand what their colleagues in other states are doing in this area and would like technical consultations.
Also receiving a large number of votes were a “toolkit” and national public health performance standards to ground their work.

21. What Did We Learn?
SHA performance management practices are widespread, although often not system-wide or with processes leading to quality improvement or changes.
SHAs report their efforts result in improved performance, with positive outcomes broadly defined.
No single framework is used by most SHAs, and there are insufficient data to inform leaders’ choices in performance management approach.

22. Learning Projects Nat’l PH Performance Standards
Florida Quality Improvement Program
Balanced Scorecard
State-Based Standards in WA
Workforce Preparedness Centers
In addition to the learning from our baseline assessment, the Collaborative has used more qualitative learning approaches that have really enhanced our knowledge base.
We have had technical presentations, as well as conducting site visits in several states, to learn more about performance management practices.

23. More Learning Projects
WI Performance-Based Contracting
MI Accreditation Program
Foundation on Accountability Health Care Quality Measures
Professional Credentialing

24. Conceptual Framework of the Public Health System as a Basis for
Measuring Public Health System Performance P U B
Structural Capacity L M I
Information Resources A C
Organizational Resources C
Physical Resources R H
Human Resources O E
Fiscal Resources A
PHS Mission C L
and Purpose O T N H
Philosophy T
Goals
"Core Functions" E S
Processes
Outcomes X Y
This slide shows a conceptual framework of the public health system. To our Collaborative, this chart represents the big picture of how performance measurement fits into the larger system.
This framework was published in the APHA Journal in 2001 in an article by Handler and Turnock. T S
The 10 Essential
Effectiveness T
Public Health
Efficiency
Services
Equity E M
Source: Handler A, Issel M, and Turnock B A conceptual framework to measure performance of the public health system. Am J Public Health, 91:

25. Conceptual Framework of the Public Health System as a Basis for
Measuring Public Health System Performance M A C R O N T E X P U B L I C H E A T S Y M
Structural Capacity
Information Resources
Organizational Resources
Physical Resources
Human Resources
Fiscal Resources
PHS Mission
and Purpose
Philosophy
Goals
"Core Functions"
Processes
The 10 Essential
Public Health
Services
Outcomes
Effectiveness
Efficiency
Equity
We’ll go through this carefully so you can understand the flow of logic.
CLICK: The macro context is open-ended. Issues arise or emerge, and become defined or redefined as public health issues. An example is violence, which 20 years ago was not a public health issue. Today, we would have trouble not thinking of it as a public health concern.
CLICK: In the macro context, changes in law or funding are ways in which issues become defined as public health issues CLICK and become incorporated into the public health system, which is also open-ended to accommodate these emerging public health issues.
CLICK: Within the public health system, the public health mission and purpose is our core. It is stable and solid, regardless of issues that are being addressed, and it drives program or policy.
TWO CLICKS: The public health system mission and purpose is operationalized by structure, process and outcome. Structural capacity, or infrastructure, includes organizational resources like partnerships. List the other elements of structural capacity.
CLICK: The elements of structural capacity allow us to perform the 10 essential public health services, now well recognized as the processes of public health practice.
CLICK: Performance of these 10 essential services leads to outputs in program and policy, not shown on the chart, and hopefully to outcomes.
Structure, process and outcome are shaded in the chart because they are the targets of effective system-wide performance management.
Source: Handler A, Issel M, and Turnock B A conceptual framework to measure performance of the public health system. Am J Public Health, 91:

26. Four components of a performance management system
Four components of our Performance Management Model:
performance standards – the basic direction-setting activities of an organization or a system. Involve the four elements (read them aloud)
performance measurement – related to performance standards, but a very different set of activities - read them aloud - HEDIS is an example of a process where a huge amount of time has been spent on refining indicators and defining numerators and denominators
reporting of progress – this is about accountability - read them aloud – this is an often forgotten part of a coherent process that is really about converting data into useable information
quality improvement process – this is about managing change - read them aloud - creating a learning organization that can use information dynamically and continuously to make decisions and create change
Source: Turning Point Performance Management Collaborative, From Silos to Systems: Performance Management in Public Health (in press).

27. In a performance management system...
All components should be driven by the public health mission and organizational strategy
Activities should be integrated into routine public health practices
The goal is continuous performance and quality improvement
This is another view of our model. Note that the mission and strategy drive into all components of the performance management system. Each component of our framework interacts together as an integrated process to improve performance.
Source: Turning Point Performance Management Collaborative.

28. The Management Challenge 9 of 10 companies fail to execute strategy
Vision Barrier: Only 5% of the workforce understands the strategy
People Barrier: Only 25% of managers have incentives linked to strategy
Management Barrier: 85% of executive teams spend <1 hour per month discussing strategy
Resource Barrier: 60% of organizations don’t link budgets to strategy
Why is it important for mission and strategy to drive performance?
The “Balanced Scorecard” folks have helped us to understand this better. The Balanced Scorecard is a strategic organizational development model, marketed primarily to business. They believe 9 out of 10 companies fail to execute strategy.
CLICK and READ EACH ONE ALOUD
Performance management gives us an opportunity to ALIGN public health mission and strategy with human resources, management direction and resources.
Source:
Balance Scorecard Collaborative,

29. Future Performance Management Issues
How do we move from “silos to systems”?
How do we create a model quality improvement process that converts performance data into information then action?
How can we inform and influence federal initiatives to improve performance of public health?
These three issues definitely form a leadership challenge to us.

30. What’s the Transformation?
Widespread Use shifts from a measurement to a more balanced and cohesive management model
Shifts from categorical to a systems- wide scope targeting capacity, process and outcomes
Back to the Collaborative’s vision, we hope our work will create two major transformations with a systems approach to performance management.

31. Performance Management Series of Reports
Literature Review
Baseline Assessment Survey of States
From Silos to Systems: Performance Management in Public Health
Toolkit
This is where the Collaborative’s products stand right now.
The literature review and baseline assessment are currently available through the Turning Point web site (
From Silos to Systems is a Guide describing our conceptual framework along with examples and will be available by early 2003.
We hope the Toolkit will be more a how-to guide and will help managers shift from a measurement to a balanced and coherent management model.