Patient Experience Body of Knowledge Metrics and Measurement Domain Team Week 3 Check-in Call Aug 17, 2012

Metrics and Measurement Domain Team NameRoleOrganization John Murray (Team Leader) Director, Patient ExperienceMemorial Hermann Jan AlthousePatient Satisfaction CoordinatorCook Children's Health Care System Melissa AndersonDirector, Patient ExperienceThe Nebraska Medical Center Jason MacedoniaVice President, Training and DevelopmentAvatar International Linda MelilloDirector of Patient ExperienceSpaulding Rehabilitation Network Julie O'ShaughnessyExecutive Healthcare Service ConsultantAvatar International Pam PrisselDirector of Customer ServiceMayo Clinic Health System, in the NW Region

Domain Development Key Learning Components Concepts Sections Domain Learning Objectives (3 – 5) Domain Objective Understanding the components of an effective measurement process, including the identification of key metrics and the use of data. Data ElementsSect. 1Concept 1 Key Learning 1 Key Learning 2 Concept 2Sect. 2Sect. 3 Data Sources Analyzing Data Applying Data Sharing Data

Week 3 WWW Plan Review WhatWhoWhe Post Slides and Recording of week 2 check-in call to Forum for future reference JohnMon, Aug 13 Create Forum discussion thread to discuss the Concepts and Key Learning Components/Take aways for the Data Elements and Data Sources Sections JohnMon, Aug 13 discuss the Concepts and Key Learning Components/Take aways for the Data Elements and Data Sources Sections All Team Members Mon, Aug 13 – Thu, Aug 16 (Ongoing throughout the week) Check-in Webex/Conference Call to discuss Forum input and consider the Concepts and Key Learning Components/Take awaysfor the Data Elements and Data Sources Sections All Team MembersFri, Aug 11am CST WeekGoal Forum Collaboration & Discussion Check-in Call/Webex (Friday at 11am CST) 3 Continue to refine Sections for all 5 Learning Objectives Develop Section Material (Part 1 of 2) Consider the Concepts and Key Learning Components/Take aways for the Understanding Data and Gathering Data Sections Aug 13 – 16 Aug 17

Sections Data ElementsData SourcesAnalyzing DataApplying DataSharing Data 1.Why do we need data? 2.Types of data (qualitative v quantitative) 3.How to use data for improvement 4.Basic Statistical Concepts (variation, std deviation, statistical significance, central tendency) 5.Patient Experience data ( Scales used, mean score, Top Box, Percentile Rank) 6.Data limitations (small sample size) 7.Understanding measurement of inputs (actions/activity) and outcomes (survey scores) 1.Survey Collection modes (phone, mail, online) 2.Internal sources of information (surveys, rounding logs, post discharge callbacks, grievances/complaint s, focus groups) 3.External sources (Hospital Compare, Why Not the Best) 4.Research Articles – best practices 1.Aggregating data from multiple sources (how to pull data and bring it together) 2.Types of information collected (HCAHPS v vendor survey, Kano model) 3.Finding useful/actionable information (correlation, vendor analysis on key drivers) 4.Statistical Analysis (regression, modeling etc. examples where used and relevant) 5.Looking for trends 6.Understanding VBP performance calculations (contribution of HCAHPS) 1.Using data for performance improvement 2.Tracking and trending qualitative data (survey comments, complaints) 3.Analysis of HCAHPS scores and application to VBP 4.Performance Improvement Analysis Tools (Radar charts, affinity diagrams) 1.Transparency 2.Dashboards and Report Cards (format, detail level, process and frequency, audience specific) 3.How to display data (trends, control charts etc.) 4.Helping others to understand what data is telling us 5.Hardwiring distribution and discussion of metrics (push or pull or mix of both)

Data Elements Sections/ConceptsLearning Objectives/Take Aways 1.Why do we need data? ~ Value of Evidence-based practice ~ Research Philosophy/Methodology Why data is important 2.Types of data ~how the data can (or cannot) be used (Nominal Data, Ordinal Data, Continuous Data) ~ qualitative (patient comments and stories) ~ quantitative To understand uses and limitations of data (the right tool for the right job) 3.Ways to look at data ~Baseline data (starting point) ~Trending (your performance over time) ~ Benchmarking (how you compare to others) What the data is telling you 4.Basic Statistical Concepts ~ populations and sampling (N/n) ~ normal variation (within groups-between groups) ~ regression to the mean/data smoothing ~ extraneous variable/confounds (lack of controls) ~ std deviation ~ statistical significance (power, error) ~ central tendency (mean, median, mode) Understanding of basic statistics and how to apply them 5.Patient Experience data ~ Understanding Scales: Mean score, Top Box, Percentile Rank, Likert scale, clustering towards top of scale ~ Net Promoter score (where useful and where not) Understanding of how data is used and applied in our particular environment 6.Understanding measurement of inputs actions/activity outcomes (survey scores) ~ Predictive Indicators (activities aimed at driving the score – e.g. response to call light, rounding, bedside shift report, employee engagement) ~ Outcomes Measures (survey returns) Measuring both process (inputs) and outcomes (scores)

Data Sources Sections/ConceptsLearning Objectives/Take Aways 1.Survey Collection modes ~ phone, mail, online, kiosk/in-house ~ compare and contrast methods ~ impact on outcomes Understand various survey methodologies and differences 2.Internal sources of information ~ surveys, rounding logs, post discharge callbacks, grievances/complaints, focus groups, Patient Advisory Councils, Patients on Committees, mystery shoppers, interactive media (skylight), drop boxes/comment cards ~ compare and contrast methods ~ quantification of outcomes Understanding internal or informal ways or collecting information from patients/family members/employees 3.External sources ~ public reporting (Hospital Compare, Why Not the Best) ~ public forums ~ professional forums/advocacy groups ~ social media Understanding external sources of information 4.Research Articles - Best practices ~ research articles ~ replication studies ~ meta analyses Learning from others and using directional information to improve 5. Data Sources and Management ~ Availability - vendor websites for access to information ~ Ethical Data management

Reminder: Suggested Content – from Public Consultation METRICS & MEASUREMENT Understanding the components of an effective measurement process, including the identification of key metrics and the use of data. Summary of Content Suggestions  Listening to patients/family - Focus Groups, Mystery Shopping, etc.  Understanding, managing and coordinating patient survey administration - survey methodologies/vendors.  Data analysis & validity – segmentation.  Goal-setting.  Results analysis, reporting and action planning – providing data in a way that is accessible, usable, and actionable.  Market analysis.  Quantitative and qualitative research methods & data analysis.  Data management.  Determining ROI.  Methods for keeping staff informed of Patient Experience scores on ongoing basis.  Real time data opportunities such as Patient and Family Advisory Councils, Patients on hospital committees, Patient family shadowing, etc.  Use of Patient and Family Stories to drive change. Full ‘raw data’ file from BOK data collection process is available on the Beryl Institute Forum:

Work Plan/Schedule WeekGoal Forum Collaboration & Discussion Check-in Call/Webex (Friday at 11am CST) 1 Review Raw Data Identify Domain Learning Objectives (3-5) Identify possible Outside Resources/Supporting Materials July 30 – Aug 2Aug 3 2 Identify possible Sections of the learning module For each Section, specify the Learning Objective Aug 6 – 9Aug 10 3 Develop Section Material (Part 1 of 2) Consider the Concepts and Key Learning Components/Take aways for the Data Elements and Data Sources Sections Aug 13 – 16Aug 17 4 Develop Section Material (Part 2 of 2) Consider the Concepts and Key Learning Components/Take aways for the Data Analysis, Application of Data and Sharing Data Sections Aug 20 – 23Aug 24 5 Suggested Assessment Questions Final Review Aug 27 – 30Aug 31

Week 4 WWW Plan Review WhatWhoWhe Post Slides and Recording of week 3 check-in call to Forum for future reference JohnMon, Aug 20 Create Forum discussion thread to discuss the Concepts and Key Learning Components/Take aways for the Data Analysis, Application of Data and Sharing Data Sections JohnMon, Aug 20 discuss the Concepts and Key Learning Components/Take awaysfor the Data Analysis, Application of Data and Sharing Data Sections All Team Members Mon, Aug 20 – Thu, Aug 23 (Ongoing throughout the week) Check-in Webex/Conference Call to discuss Forum input and consider the Concepts and Key Learning Components/Take awaysfor the Data Analysis, Application of Data and Sharing Data Sections All Team MembersFri, Aug 11am CST WeekGoal Forum Collaboration & Discussion Check-in Call/Webex (Friday at 11am CST) 3 Develop Section Material (Part 2 of 2) Consider Concepts and Key Learning Components/Take aways for the Data Analysis, Application of Data and Sharing Data Sections Aug 20 – 23Aug 24

THANK YOU & QUESTIONS John Murray, MBA Director, Patient Experience Tel: