1. 1 Canadian Institute for Health Information

2. Developing a Model for Measuring the Efficiency of the Health System in Canada Released July 5, 2012 2

3. Outline Purpose, Project Goals and Audience Study Approach and Methods Components of an Efficiency Model: Health System Objectives, Health System Boundaries, Decision- Making Units Methods for Estimating Efficiency Proposed Efficiency Model Challenges and Next Steps 3

4. Purpose, Project Goals and Audience 4

5. Why Are We Interested in Measuring Health System Efficiency? Health system policy- and decision-makers are responsible for ensuring that health system resources are used judiciously A key policy challenge is to improve public services while containing costs Information about variations in efficiency within Canada could support provincial/territorial efforts to improve system performance 5

6. Project Goals and Audience The goal of this project is to develop a model for measuring health system efficiency in order to then identify factors that may improve efficiency This report is the first phase of this project: it explains the process for developing the model and reviews the available data The target audience includes policy-makers and health system managers at regional, provincial, territorial and federal levels 6

7. Study Approach and Methods 7

8. Study Approach and Methods (1) Common approaches to measuring the efficiency of a health system include –A disease-based approach –A sub-sector approach –A system-level approach This project starts with a system-level approach, recognizing that the other approaches could be complementary to this work 8

9. Study Approach and Methods (2) Measuring efficiency requires defining four components –Health system objectives (what the system is meant to achieve) –Health system boundaries (inputs, environmental factors or constraints) –Health system decision-making units (DMUs) that convert resources (inputs) into outputs and outcomes –Estimation techniques to measure efficiency scores 9

10. Study Approach and Methods (3) The process of defining these components requires making assumptions, and choices often depend on the values of system stakeholders This project used multiple methods and sources to guide decisions about the model components 10

11. Study Approach and Methods (4) Qualitative studies Literature reviews Data reviews 11

12. Qualitative Studies to Collect Stakeholders’ Opinions Analysis of Government Documents Stakeholder Interviews Facilitated Stakeholder Dialogue Goal To identify the stated objectives (that is, goals, mandates or visions) for the health system To identify, explore and better understand health policy-makers’ views regarding the desired inputs to and outcomes of the health system To solicit stakeholder views on health system objectives, boundaries and methods by giving stakeholders an opportunity to express their views in a frank, off-the-record facilitated deliberation Selection Criteria Publicly available documents produced by Canadian federal, provincial and territorial governments that address health system administration, functions and delivery (for example, annual reports, legislation and strategic documents) Current or former senior health ministry officials of Canadian provincial and territorial governments (that is, deputy ministers, associate deputy ministers, assistant deputy ministers, executive directors or directors) Current or former senior decision- makers from Canadian federal, provincial and territorial governments, health system consultants, and senior executives from technical-, administrator- and practitioner- oriented groups Sample Size/Frame A number of websites and publicly available resources from all provinces and territories, and selected federal sources 17 interviewees from 9 provinces and 2 territories 16 participants from 6 provinces,1 territory and the federal government There was no overlap between the participants in the 17 elite interviews and the 16 participants in the facilitated stakeholder dialogue 12

13. Components of an Efficiency Model 13

14. Health System Objectives Stakeholders agreed that, as a starting point, health system efficiency should be assessed against the objective of providing care to Canadians when they are sick (as measured by an indicator of premature mortality) Stakeholders also agreed that in future work, efficiency could be assessed against the objective of preventing illness and promoting overall population health There was recognition among stakeholders of the importance of considering the objectives of reducing health inequalities and ensuring equitable and quality services in the model 14

15. Health System Boundaries Stakeholders agreed that health system expenditures should be the principal inputs to the analysis Stakeholders suggested that a range of health system performance indicators could be useful when exploring factors that could encourage or hinder efficiency –Environmental factors that lie outside the responsibility of the health system, such as levels of socio-economic deprivation and characteristics of the physical environment –Factors that could be considered to come under the responsibility of the health system that could affect efficiency 15

16. Decision-Making Units A decision-making unit (DMU) is an entity that designs and controls the process (or the rules for the processes) that converts inputs into outputs and outcomes Health regions exhibit enough differences in system- level inputs and outcomes to effectively compare health system efficiency across Canada Health system stakeholders supported measuring efficiency at the regional level and using health regions as DMUs 16

17. Methods for Estimating Efficiency 17

18. Two Common Methods for Estimating Efficiency 1.Parametric approaches (stochastic frontier analysis or SFA) 2.Non-parametric approaches (data envelopment analysis or DEA) Each method has strengths and weaknesses in its ability to deal with –Outliers –Underperformers –Multiple outputs –Sample size and other issues Stakeholders recommended using both methods of estimating efficiency if possible 18

19. Proposed Efficiency Model 19

20. The Proposed Efficiency Model Health region DMU Inputs Public spending on Hospitals Other institutions Physicians Nurses outside hospital Drugs Public health Outcomes Measures of premature mortality (e.g. Potential Years of Life Lost)

21. Additional Factors to Consider in the Analysis Environmental Factors –Population density –Unemployment rate –Proportion of people age 65 and older –Proportion of Aboriginal peoples –Proportion of immigrants –Income inequality –Gender distribution Lifestyle-Related Factors –Smoking –Obesity –Physical activity –Diet Inequalities –Self-assessed health –Self-reported use of hospital and physician services Health System Performance –Quality indicators (e.g. 30-day stroke in-hospital mortality; readmission rate for various conditions) –Access indicators (e.g. wait times for various procedures) 21

22. Public spending on Hospitals Other institutions Physicians Nurses in community Drugs Public health Population density Unemployment rate Average income % of people age 65 and older % of Aboriginal peoples % of immigrants Income inequality (GINI) Gender distribution Premature mortality Preventable mortality Treatable mortality Avoidable mortality Stochastic Frontier Analysis (SFA) Inputs Outcomes Environmental factors DMU’s Efficiency Score DMU’s Efficiency Score Estimating Efficiency Scores With SFA

23. Estimating Efficiency Scores With DEA 23 Population density Unemployment rate Average income % of people age 65 and older % of Aboriginal peoples % of immigrants Income inequality (GINI) Gender distribution Outcomes Account for Environmental Factors DMU’s Efficiency Score DMU’s Efficiency Score DMU’s Efficiency Score DMU’s Efficiency Score Data Envelopment Analysis (DEA) Premature mortality Preventable mortality Treatable mortality Avoidable mortality Public spending on Hospitals Other institutions Physicians Nurses in community Drugs Public health Inputs

24. Challenges and Next Steps 24

25. Challenges Data is missing for some variables and health regions There is a lack of real-time data The roles and responsibilities of health regions vary across provinces The proposed model does not account for possible time lags—the time between the use of resources and the effect these resources have on outcomes 25

26. Next Steps Produce preliminary results on health system efficiency at the regional level in Canada During the process of producing these results, CIHI will continue its engagement with stakeholders to explore possible factors leading to inefficiency Preliminary analyses may lead to model refinement, additional data collection and inclusion of new indicators 26

27. Bibliography Canadian Institute for Health Information. Developing a Model for Measuring the Efficiency of the Health System in Canada. Ottawa, Ont.: CIHI; 2012. Canadian Institute for Health Information. Developing a Model for Measuring the Efficiency of the Health System in Canada – Data Availability. Ottawa, Ont.: CIHI; 2012. Retzlaff-Roberts D, Chang C, Rubin R. Technical efficiency in the use of health care resources: a comparison of OECD countries. Health Policy. 2004;69(1):55-72. Verhoeven M, Gunnarsson V, and Lugaresi S. The Health Sector in the Slovak Republic: Efficiency and Reform. Washington, D.C.: International Monetary Fund, Fiscal Affairs Department; 2007. 27

28. 28 Thank You