From Silos to Systems: Performance Management in Public Health Turning Point Performance Management Collaborative October 2002

Learning Objectives Gain understanding of the components of a performance management system Identify at least three benefits of performance management

Turning Point’s National Excellence Collaboratives, 2000-2004 Funded by Robt. Wood Johnson Foundation - States, communities, national partners Combine collective experience, skills Take next steps in transforming public health Review of literature & current practice; analysis Development of innovative models Testing and disseminating innovation Evaluation The expectations of National Excellence Collaboratives

Turning Point: National Excellence Collaboratives Public Health Statute Modernization Performance Management Information Technology Social Marketing Leadership Development The five Collaboratives, each working on tools to improve public health systems.

Performance Management Collaborative (PMC) 7 Turning Point States Illinois* New York Montana Alaska New Hampshire Missouri West Virginia * Lead State The Performance Management Collaborative’s participating states

More PMC Members TP National Program Office at Univ. of WA/School of Public Health National Partners ASTHO NACCHO CDC HRSA ASTHLHLO The Performance Management Collaborative’s national partners

PMC Vision Widespread use of dynamic and accountable public health performance management We would like to see the field move in this direction in the next five years

PMC Goals To develop useful and feasible performance management models for states To stimulate national dialogue and consensus on performance management in public health To support the application of performance management as a core discipline of public health practice We believe these goals, if achieved, have significant implications for public health practice and workforce development.

What Is Performance Management? The practice of actively using performance data to improve the public’s health. Performance management can be carried out at the program, organization, community and state levels. The key words here for us are not “performance data”; they are “actively using”. Data mean nothing if they are not applied and used effectively. We’re working towards a very flexible model—a balanced management model for public health, not just measurement. The Collaborative’s preference is to carry out performance management at the systems level.

Four Components of Performance Management Performance Standards Performance Measures Reporting of Progress Quality Improvement

Why Develop a PM System? To maximize public health’s effectiveness. This requires More than measurement alone More than standards alone All four PM components to be continuously integrated into a system of performance management Why have we spent four years on this? What is the compelling reason for thinking about and changing the practice of performance management? The key is to integrate the four components into a continuous system.

Using Data to Achieve Results Quality improvement efforts Policy change Resource allocation change Program change Managerial Action Public health, like other sectors, needs management results in – READ ALL FOUR ALOUD These are the outputs of managerial action—what we are keeping our eyes on.

Survey of Performance Mgmt. Practices in States Baseline Assessment Conducted by PHF 47 of 50 States Responded Survey Asks About: Use of Performance Targets, Reports Impact on Program and Policy Need for New Tools We wanted to increase our knowledge and better understand the scope and usefulness of current state practices. The baseline assessment conducted by the Public Health Foundation achieved an extremely high response rate.

Nearly All SHAs Have Some Performance Management Efforts However, only about half apply performance management efforts statewide beyond categorical programs Figure 1. Agencies or programs to which SHAs apply performance management efforts (N=47) What we learned is that nearly all states have some performance management efforts, but only about half of these are statewide efforts. We defined statewide very broadly, to encompass state-level efforts, state plus local health department efforts and local health department efforts only. The other (approximately) half are targeted at categorical programs only or had no PM effort at all.

Performance Management Efforts Result in Improved Performance for Three-Quarters of SHAs Figure 19. Percentage of SHAs that report their performance management efforts resulted in improved performance (N=41) Reported Positive Outcomes: Improved delivery of services—program services, clinical preventive services, essential services Improved administration/management— contracting, tracking/reporting, coordination Legislation or policy changes Yes: 76% said PM improved their performance This positive view of the usefulness of PM efforts, combined with the fact that most state conduct some form of PM, is a very strong base to build on. A state may have a few of the pieces in place that can be the starting points to transition into a system-wide PM effort.

SHAs Most Likely to Have Components of Performance Management for Health Status; Least Likely for Human Resource Development Figure 8. Areas most and least likely to have performance targets, measures or standards, reports, and processes for quality improvement (QI)/change, of SHAs that apply performance management efforts SHA wide, SHA wide and to local public health agencies, or to local public health agencies only (N=25) To what areas of public health practice is performance management applied? We found the components of performance management were most likely to be in place for health status (not surprising, since there are a number of tools available to assist users in health status assessment) More interesting, perhaps, is that our four components were least likely to be in place for human resource development or management of public health capacity.

Top Three Models/Frameworks Explicitly Incorporated by SHAs Into Their Performance Management Efforts Healthy People Objectives Core Public Health Functions Ten Essential Public Health Services States use a variety of performance management models/frameworks, in a variety of combinations

Most SHAs Have Performance Measures, Targets, and Reports, While Fewer States Have Process for Quality Improvement or Change* Figure 15. Percentage of SHAs that have specified components of performance management for public health capacity (N=25) State health agencies were very different in the array of performance management components they have in place. Overall, they were least likely to have a process for quality improvement or change. They were most likely to have performance measures or standards. *Correlation analysis revealed that there is a comparatively weak relationship between having performance targets, performance measures or performance reports and process for quality improvement (QI)/change. That is, in general, fewer states indicated that they did have a process for change, even though they indicated having performance targets, performance measures, or performance reports. This was the case for all areas of performance management studied (Human Resource Development, Data & Information Systems, Customer Focus and Satisfaction, Financial Systems, Management Practices, Public Health Capacity, and Health Status). Figure 15 illustrates this finding.

Most States Use Neither Incentives nor Disincentives to Improve Performance Figure 18. Percentage of SHA performance efforts that include incentives or disincentives to improve performance (N=40) Sixty-three (63) percent don’t use incentives or disincentives to manage performance. Very small percentage of states use incentives and disincentives use PM in outside contracts, while even less use incentives or disincentives with staff. Note: Respondents could choose more than one response, so total does not equal 100

Funding for Performance Management Chosen as Number One Way to Improve States’ Efforts Figure 5. Types of aid identified as most useful to SHAs to improve SHA performance management efforts, in rank order (N=47) We asked what would help states the most to improve their PM efforts. Not surprisingly, the top “vote-getter” for assistance needed to improve states’ performance management efforts was funding. It’s interesting to look at other factors, too. For example, A number of states want to understand what their colleagues in other states are doing in this area and would like technical consultations. Also receiving a large number of votes were a “toolkit” and national public health performance standards to ground their work.

What Did We Learn? SHA performance management practices are widespread, although often not system-wide or with processes leading to quality improvement or changes. SHAs report their efforts result in improved performance, with positive outcomes broadly defined. No single framework is used by most SHAs, and there are insufficient data to inform leaders’ choices in performance management approach.

Learning Projects Nat’l PH Performance Standards Florida Quality Improvement Program Balanced Scorecard State-Based Standards in WA Workforce Preparedness Centers In addition to the learning from our baseline assessment, the Collaborative has used more qualitative learning approaches that have really enhanced our knowledge base. We have had technical presentations, as well as conducting site visits in several states, to learn more about performance management practices.

More Learning Projects WI Performance-Based Contracting MI Accreditation Program Foundation on Accountability Health Care Quality Measures Professional Credentialing

Conceptual Framework of the Public Health System as a Basis for Measuring Public Health System Performance P U B Structural Capacity L ----------------- M I Information Resources A C Organizational Resources C Physical Resources R H Human Resources O E Fiscal Resources A PHS Mission C L and Purpose O T ---------------- N H Philosophy T Goals "Core Functions" E S Processes Outcomes X Y ------------------- --------------- This slide shows a conceptual framework of the public health system. To our Collaborative, this chart represents the big picture of how performance measurement fits into the larger system. This framework was published in the APHA Journal in 2001 in an article by Handler and Turnock. T S The 10 Essential Effectiveness T Public Health Efficiency Services Equity E M Source: Handler A, Issel M, and Turnock B. 2001. A conceptual framework to measure performance of the public health system. Am J Public Health, 91:1235-1239.

Conceptual Framework of the Public Health System as a Basis for Measuring Public Health System Performance M A C R O N T E X P U B L I C H E A T S Y M Structural Capacity ----------------- Information Resources Organizational Resources Physical Resources Human Resources Fiscal Resources PHS Mission and Purpose Philosophy Goals "Core Functions" Processes ------------------- The 10 Essential Public Health Services Outcomes --------------- Effectiveness Efficiency Equity We’ll go through this carefully so you can understand the flow of logic. CLICK: The macro context is open-ended. Issues arise or emerge, and become defined or redefined as public health issues. An example is violence, which 20 years ago was not a public health issue. Today, we would have trouble not thinking of it as a public health concern. CLICK: In the macro context, changes in law or funding are ways in which issues become defined as public health issues CLICK and become incorporated into the public health system, which is also open-ended to accommodate these emerging public health issues. CLICK: Within the public health system, the public health mission and purpose is our core. It is stable and solid, regardless of issues that are being addressed, and it drives program or policy. TWO CLICKS: The public health system mission and purpose is operationalized by structure, process and outcome. Structural capacity, or infrastructure, includes organizational resources like partnerships. List the other elements of structural capacity. CLICK: The elements of structural capacity allow us to perform the 10 essential public health services, now well recognized as the processes of public health practice. CLICK: Performance of these 10 essential services leads to outputs in program and policy, not shown on the chart, and hopefully to outcomes. Structure, process and outcome are shaded in the chart because they are the targets of effective system-wide performance management. Source: Handler A, Issel M, and Turnock B. 2001. A conceptual framework to measure performance of the public health system. Am J Public Health, 91:1235-1239.

Four components of a performance management system Four components of our Performance Management Model: performance standards – the basic direction-setting activities of an organization or a system. Involve the four elements (read them aloud) performance measurement – related to performance standards, but a very different set of activities - read them aloud - HEDIS is an example of a process where a huge amount of time has been spent on refining indicators and defining numerators and denominators reporting of progress – this is about accountability - read them aloud – this is an often forgotten part of a coherent process that is really about converting data into useable information quality improvement process – this is about managing change - read them aloud - creating a learning organization that can use information dynamically and continuously to make decisions and create change Source: Turning Point Performance Management Collaborative, From Silos to Systems: Performance Management in Public Health (in press).

In a performance management system... All components should be driven by the public health mission and organizational strategy Activities should be integrated into routine public health practices The goal is continuous performance and quality improvement This is another view of our model. Note that the mission and strategy drive into all components of the performance management system. Each component of our framework interacts together as an integrated process to improve performance. Source: Turning Point Performance Management Collaborative.

The Management Challenge 9 of 10 companies fail to execute strategy Vision Barrier: Only 5% of the workforce understands the strategy People Barrier: Only 25% of managers have incentives linked to strategy Management Barrier: 85% of executive teams spend <1 hour per month discussing strategy Resource Barrier: 60% of organizations don’t link budgets to strategy Why is it important for mission and strategy to drive performance? The “Balanced Scorecard” folks have helped us to understand this better. The Balanced Scorecard is a strategic organizational development model, marketed primarily to business. They believe 9 out of 10 companies fail to execute strategy. CLICK and READ EACH ONE ALOUD Performance management gives us an opportunity to ALIGN public health mission and strategy with human resources, management direction and resources. Source: Balance Scorecard Collaborative, www.bscol.com

Future Performance Management Issues How do we move from “silos to systems”? How do we create a model quality improvement process that converts performance data into information then action? How can we inform and influence federal initiatives to improve performance of public health? These three issues definitely form a leadership challenge to us.

What’s the Transformation? Widespread Use shifts from a measurement to a more balanced and cohesive management model Shifts from categorical to a systems- wide scope targeting capacity, process and outcomes Back to the Collaborative’s vision, we hope our work will create two major transformations with a systems approach to performance management.

Performance Management Series of Reports Literature Review Baseline Assessment Survey of States From Silos to Systems: Performance Management in Public Health Toolkit This is where the Collaborative’s products stand right now. The literature review and baseline assessment are currently available through the Turning Point web site (www.turningpointprogram.org) From Silos to Systems is a Guide describing our conceptual framework along with examples and will be available by early 2003. We hope the Toolkit will be more a how-to guide and will help managers shift from a measurement to a balanced and coherent management model.