1. Strategies for health system performance comparison: some international experience Peter C. Smith Emeritus Professor of Health Policy Imperial College London

2. Health system performance comparison Why comparison? Some European efforts to date What does HSPA look like? Challenges and barriers Conclusions

3. Tallinn Charter, WHO Europe, 2008  “We, the member states, commit ourselves to: o Promote shared values of solidarity, equity and participation... o Invest in health systems, and foster investment across sectors that influence health... o Promote transparency and be accountable... o Make health systems more responsive... o Engage stakeholders... o Foster cross-country learning and cooperation... o Ensure that health systems are prepared and able to respond to crises...”

4. A European Union priority Letter from Jean-Claude Juncker, President of the European Commission, to incoming Commissioner for Health and Food Safety: “… developing expertise on performance assessments of health systems, drawing lessons from recent experience, and from EU- funded research projects to build up country-specific and cross- country knowledge which can inform policies at national and European level.” Brussels, 1 November 2014

5. The universal role of performance information … to enable actors throughout the system to make better decisions … measuring and reporting performance offers one of the most powerful instruments for incentivizing and facilitating performance improvement.

6. Three stages of HSPA What aspects of performance to include: – Reporting framework – Selection criteria – Practical limitations How to report that performance: – At what level of organization – How much aggregation / detail – How to adjust for contextual differences – Media and dissemination tools What to do with the results: – Who is intended to take action? – What incentives do they have to take action? – How is such action facilitated (eg benchmarking clubs)?

7. Conceptual Framework for Portuguese HSPA Report

12. The various stated goals of HSPA Armenia: Enhance stewardship; Accountability; Transparency; Identify policy priorities. Belgium: Transparency and accountability; Comparisons with other countries; Performance monitoring over time. England: Performance management of public sector organizations. Estonia: Enhance accountability; Enhance stewardship; Provide a monitoring scheme for the National Health Plan. Kyrgyzstan: Monitor progress and impact of health sector programmes; Accountability to donors; Identify potential policy problem areas. Portugal: Accountability; Inform policy. Turkey: Provide a monitoring and evaluation scheme for the Health Transformation Program; Transparency and accountability; Support the development of evidence-based policy-making; Guide governmental policy development; Identify policy priority areas. World Health Organization, Case studies on health system performance assessment. A long-standing development in Europe, 2012, Copenhagen: World Health Organization Regional Office for Europe.

13. Some differences in focus Health (PT) vs health services (NL) Between country (EE) vs within country (SE) Aggregate outcomes (PT) vs distribution (BE) Policymakers (planning) (GB) vs accountability and transparency (NL) Trends over time (NL) or cross-sectional international comparison (BE)

14. Belgian Health Care Knowledge Centre (KCE), Performance of the Belgian health system report 2012.

15. Performance of the Belgian health system 2012: continuity of care

16. Eight enemies of valid comparison ‘You cannot measure what we are trying to achieve.’ (eg mental health) ‘Our objectives go beyond what you are trying to measure.’(eg quality of life) ‘The data you are using are of poor quality and cannot be relied on.’ ‘There are external factors that influence our performance that you have not taken account of.’ (eg low income population) ‘The risk adjustment methods you have used are inadequate.’ ‘There is huge uncertainty in the reported measures.’ ‘The data you are using are out of date.’ ‘We are unique and cannot be compared with other institutions.’

17. EQ-5D: A Generic Quality of Life Measure

18. Towards patient-reported outcome measures (PROMS) Mandated in England since 2009 – Hip replacement – Knee replacement – Hernia repair – Varicose veins Health status measured just before surgery and 3/6 months afterwards Independent data collection Also relevant for routine population health monitoring

19. HSPA: some leadership responsibilities Development of a clear conceptual framework and a clear vision of the purpose of performance measurement; Seizing information technology possibilities; Prompting and sustaining clinical involvement and leadership; Mandating data collection mechanisms; Information assurance and governance; Development of analytic devices and capacity to help understand the data; Development of appropriate data presentational methods; Dissemination and securing attention; Stimulating action in response to performance measures; Proper evaluation of performance measurement instruments.

20. Health System Performance Comparison Examines rationale for international comparison State of the art in six performance domains Current practice and resources Future prospects 20 http://www.euro.who.int/en/who-we-are/partners/observatory/studies/health-system-performance-comparison.-an-agenda-for- policy,-information-and-research