ePREM & YVM Data Information Session November 1, 2016 Cancer Analytics

Background and Objective Electronic Patient Reported Experience Measures (ePREM) data are pulled from the vendor database (National Research Corporation Canada (NRCC)) to Cancer Care Ontario (CCO), making data accessible to Cancer Analytics in support of the Patient & Family Engagement and Experience Program and Person-Centred Care Portfolio at CCO The availability of provincial ePREM data to Cancer Analytics enables the conduct of analyses and reporting of the ePREM Your Voice Matters (YVM) tool

Data Flow Application Programming Interface (API)

Data Flow (cont’d) ePREM YVM Data Moves Across 3 Main Locations at CCO: Data Quarantine Area (DQA) ePREM YVM database and ISAAC source tables Data is loaded in raw format with no applied exclusion(s) Stage Preparation Layer (EDW) The data that were loaded into the DQA are validated and cleaned Data transformations are applied Presentation Layer (EDW) Provides data for end-users – CCO Cancer Analytics

Linkage with Other Administrative Data Sources The flow of ePREM YVM data includes linkage with other administrative data sources including (i) Your Symptoms Matter (YSM) database previously known as ESAS from the Interactive Symptom Assessment and Collection (ISAAC) and (ii) Activity Level Reporting (ALR) To enable these linkages for unique survey records, an ISAAC-specific patient identifier (Patient Site ID) and hospital identifier (Master Number) must be available along with the survey data. Records without these identifiers are not able to be linked The ISAAC specific ID randomly generated All identifiable data will remain confidential and will not be shared with the respondents’ health care team. Data shared by CCO is aggregated, and not at the patient level.

Why Link ePREM YVM Data to Other Data Sources? For the first time in Ontario, the ability to link ePREM YVM data to other administrative databases enables a holistic view of the cancer patient experience, thereby identifying ways to improve the cancer system

Data Availability ePREM YVM data are available to Cancer Analytics in the EDW on a monthly basis, coinciding with when ALR and YSM data is made available YSM data is collected through ISAAC and is refreshed monthly in the EDW although more real-time data is available through the YSM Reporting database ALR has a two month lag in availability to Cancer Analytics (i.e. October cancer activity data will be available for reporting in December) based on the hospital reporting schedule for this data

Frequency of Information Patient & Family Engagement and Experience Program will be sharing monthly Excel based analytics report to Patient & Family Engagement and Experience leads (also known as PCC Leads). Please note: Frequency to vary in early months due to data stabilization Data stabilization is a diligent process to ensure the information is accurate and ensure relevant fixes to known issues are in place prior to dissemination.

Confidential Vs Anonymous What does it mean when we say YVM is confidential? Confidential: All responses are private Data is identifiable and linkable between YSM database and ePREM database at CCO level through a unique ID (for example a #) but it is not the same as HIN. This type of linkage cannot be done by hospital staff. Anonymous: All responses are private No identifying information is collected or stored

Monthly Indicator Rates Screening Rate: Measuring the spread of YVM Completion Rate: Measuring the uptake of YVM Total number of unique patients seen at all live RCCs who completed an ePREM YVM questionnaire at least once (n≥1) in the same reporting time period (month) as the denominator cohort X 100% = Provincial YVM Screening Rate The sum of cancer cases seen at all live RCCs in a given reporting period (month) Total number of unique patients seen at all live RCCs who completed ePREM YVM questionnaire at least once (n≥1) in the same reporting time period (month) as the denominator cohort X 100% = Provincial YVM Completion Rate Total number of unique patients seen at all live RCCs who were prompted for an ePREM YVM questionnaire at least once (n≥1) in a given reporting period (month) Considerations: The screening rate reflects the percentage of patients who completed a YVM, regardless if a patient was prompted to respond to the questionnaire The two indicators should be used in combination to best understand the spread and uptake of YVM Record volumes may differ between the ePREM dashboard and the CCO EDW

Examples of how record volumes may differ between the EDW and the NRCC Tool Source of Discrepancy Description Impact YVM records with missing Patient Site ID and Master Number These records can be created if the YVM assessment URL is copied and pasted in a separate browser window. Sites may see higher volumes of YVM assessments on the tool than what is reported by CCO “Quit” button functionality If the “Quit” button is pressed, a partial record will be generated that is separate from the original YVM response record. This record will have a PUUID that is not null and tsq=quit.

Field Report - Completions tab: Click - Through The number of respondents that entered the survey/opened the first page of the survey Available as hyperlink in Field Report, “Completions” tab: Click to open the partial data in tab-delimited form in a new browser window; does not contain the patient answers To save the partial data you can either copy and paste the contents into a text editor, or select "save page as" for the browser window and save the contents to your computer in a .txt file. You can then import the file into Excel using the import wizard; simply select the tab-delimited .txt file and import the partial data into Excel. Partial data example and data dictionary:

Future Planned use of the ePREM data Improving Patient Care Informs clinicians and administrators of patient reported information over time and across care settings for use in care Improve regional and clinical practice Regional monthly progress reports Demographic reports such as: gender, disease, provider, age. Performance Management (future options) CCO’s quarterly reviews with regions (for discussion in future) CCO’s performance scorecard (for discussion in future) Public Reporting and Accountability (future options) Cancer System Quality Index (CSQI) Inform planning and impact on system outcomes Researchers accessing database Advanced analytics capacity to drive forward 13

Thank you! Questions?