What we Measure Matters Sharon Johnston, MD, LLM, CCFP University of Ottawa William Hogg, Hons. BSc, MSc, MClSc, MD, CCFP, FCFPC University of Ottawa

Performance Measurement in PHC

UK: Quality and Outcomes Framework

Performance Measurement in PHC Dear Dr. Johnston Thank you for your interest in the Primary Care Physician Practice (PCP) Report. The PCP Report provides primary care physicians with data on practice demographics, health service use (e.g., emergency room visits, hospital admissions, and specialist referrals), and chronic disease prevention and management. These confidential reports come directly to the physician and include data comparisons of the physician with other relevant practices, his/her LHIN, and the province. This information can enable you to better plan and monitor the impact of your improvement work and the services you provide. Health Quality Ontario (HQO) has now developed a secure, web-based portal where you can register to receive your confidential PCP Report, as well as download your most up-to-date Report and access support tools. Reports with updated data (up to March 2013) will be available on the portal in August 2014.

Transforming CBPHC Delivery Through Comprehensive Performance Measurement and Reporting Overall goal: demonstrate the feasibility and usefulness of comparative and comprehensive CBPHC performance measurement and reporting to inform innovation of the Canadian PHC system.

Performance Measurement How do we get a more complete picture of primary health care in Canada? Uses: –Practice QI –Pay for Performance –System level reporting and accountability –Research to improve understanding of best practices

Project Team Principal Investigators/Leads Nominated PI: Sabrina Wong (University of BC) Co-PI: Bill Hogg (University of Ottawa) Co-I: Fred Burge (Dalhousie University) Co-Investigators Julia Abelson (McMaster University) Antoine Boivin (U of Sherbrooke) John Campbell (University of Exeter, UK) Khaled El Emam (University of Ottawa) Rick Glazier (Institute for Evaluative Sciences) Jeannie Haggerty (McGill) Sharon Johnston (Bruyère Research Institute) Alan Katz (University of Manitoba) Jean-Fred Levesque (Bureau of Health Info, Australia) Kim McGrail (UBC Centre for Health Services & Policy Research) Walter Wodchis (University of Toronto) Decision-Makers John Abbott (Health Council Canada) Heather Davidson (BC Ministry of Health) Victoria Lee (Fraser Health Authority) Michelle Rey & Wissam Haj-Ali (Health Quality Ontario) Paul Roumeliotis (Eastern Ontario Health Unit) Collaborators Annette Browne (University of BC) Marilyn Ford Gilboe (University of Western Ontario) Bev Holmes (Michael Smith Foundation for Health Research) Anthony Mar (Cliniconex) Steve Morgan (UBC Centre for Health Services & Policy Research) Monica Taljaard (Ottawa Hospital Research Institute) Colleen Varcoe (University of BC)

Funders, Stakeholders and Partners EQUIP Healthcare

1 2 3 The Study Regions 3 Sites in 3 Provinces 1.Fraser East (in Fraser Health) 2.Eastern Ontario Health Unit 3.Capital Health *All regions were selected from Peer Group A, groups identified by Statistics Canada as having similar social and economic make up

Study 1: Regional PHC System Performance Compare comprehensive measures of CBPHC performance and healthcare equity between regions, including comparative performance for complex vulnerable patients Practice-based performance measurement

Data Sources –Linked Patient, Provider and Organizational Surveys –Health Administrative Data

Sampling In each of 3 health regions: –40 practices –2 clinicians providing general medical care –20 patients per organization

Study 2: Policy Context Case Study Objective: –To examine contextual facilitators and barriers that may explain regional variation in CBPHC performance

Research Questions What are the characteristics of: 1.The populations served by the CBPHC practices in each of the 3 study regions and 2.The CBPHC practices themselves? In the past five years, what CBPHC innovations have been implemented in each of the 3 study regions and how have they impacted practices’ performance?

Research Questions What features of a performance measurement and reporting system in CBPHC are important to key stakeholders?

Methods Document and literature review Interviews with decision makers, clinician leaders, community groups Patient focus groups Clinician focus groups

Study 3:Deliberative Dialogues on PHC performance Reporting Research Questions: What dimensions of CBPHC are of greatest priority for measurement and public reporting to promote accountability and public engagement in health system decision-making? How is public information on CBPHC performance used by individuals; acting as patients and/or acting as citizens? What is the best way to report performance results to different stakeholders: governance level (clinic, region, province), format, mode?

Study 3:Deliberative Dialogues on PHC performance Reporting Deliberative Dialogues is a methodology for directly involving the public and other stakeholders in value-based discussions about the health system

Deliberative Dialogues Full day Deliberative Dialogue (DD) sessions in BC, ON, NS Each region: one DD with complex patients and one with low health system user patients –10-15 patients per group –All participants sent a pre-meeting brief about PHC performance measurement, core dimensions of PHC and examples of PHC performance reporting approaches

Deliberative Dialogues Small group and large group discussions on: – Priority dimensions and indicators to measure and report? –How should/would people use publicly reported PHC performance data? Discuss different formats for reporting on PHC performance.

Performance reporting Regional PHC performance reports will be prepared for different audiences for each TRANSFORM study region Content (indicators and relevant context) and format informed by deliberative dialogues Follow up interviews with policy-makers to understand the impact and use of regional publicly reported data

Pilot study: Automating the Patient Survey A primary goal of TRANSFORM is to advance the science of performance measurement –Better understanding of important measures –Better methods for measuring and reporting performance

Automated Patient Survey Traditional patient survey in waiting room –Captures views of attenders –Long –Less flexibility to adjust centrally and redistribute when paper-based –Tablet-based more expensive and still requires survey administrator –Harder to link to ER –Follow-up timing options are limited

Automated Patient Survey Automated interactive voice response surveys linked to EMR eg. after hospital discharge Existing primary care practice-based platform for automated patient phone and reminders for appointments, practice notifications, etc. This is an opportunity to harness emerging technology to create a more flexible and sustainable patient survey method which can be scaled up or limited to a single practice.

Automated Patient Survey Is an automated practice-based patient survey feasible? Is an automated practice-based patient survey acceptable to patients? Is an automated practice-based patient survey acceptable to providers?

Automated Patient Survey Pilot study within TRANSFORM patient survey data collection Patients who fill out the waiting room survey will be asked to consent to a follow up automated survey of 3-6 questions within 72 hours –Telephone, , text message preference Analyse degree of reachabilty by this method and respondent profile

Automated Patient Survey Based on your experience, how easy is it for you to get health advice from your clinic over the phone? Not at all easy Not very easy A bit easy Moderately easy Very easy

Automated Patient Survey Long term potential would be linked to EMR and deployed by preset parameters Automated survey questions could be linked to phone call reminders about flu shot clinics, appointment reminders, preventive care reminders, change of clinic hours notices, etc. Survey question burden per patient minimised as spread over large practice roster

Thank you