Clearly Connecting Performance Measures to Learning Interventions in Performance Improvement Activities Natalie Lavelle, M.Ed. Andy Hicken, PhD Director for Learning Technologies Director of Product Development National Board of Osteopathic Medical Examiners Web Courseworks May 16, 2016 Tech Demo Medbiquitous

Natalie Lavelle, M.Ed. Andy Hicken, PhD Nothing to disclose. DISCLOSURES Natalie Lavelle, M.Ed. Nothing to disclose. Andy Hicken, PhD Employee of Web Courseworks, a vendor of educational technologies (including some technology discussed in this presentation).

NBOME OPAIM A platform for on-demand, self-paced performance improvement (OCC Part IV ~ MOC Part IV) activities Competency-based education Stage A Provides immediate feedback on performance gaps and successes Compares performance to national quality benchmarks Automated, individualized reporting: no extra administrative work Stage B Directs participants to education based on performance gaps Various media used for the educational intervention Stage C NBOME has been able to successfully document that the educational activities are improving physician performance and having a positive impact on patient care.

Defining our Terms PI CME (Performance Improvement Continuing Medical Education): “A process by which evidence-based performance measures and quality improvement (QI) interventions are used to help physicians identify patient care areas for improvement and change their performance.” --AMA MOC Part IV (Maintenance of Certification Part IV): “These standards contribute to improved patient care through ongoing assessment and improvement in the quality of care provided by diplomates in their individual practices and/or in the larger hospital, health system, or community setting in which the diplomates practice medicine.” --ABMS OCC Part IV (Osteopathic Continuous Certification Part IV): “Requires that you engage in continuous quality improvement through comparison of personal practice performance measured against national standards for your medical specialty.” --AOA

Use of competency framework Pre-development Use of competency framework Mapped learning interventions to performance measures based in recognized best practices and current research

Three-Stage Model for PI CME Stage A Assess Current Performance “How well am I doing?” “What can I do to improve?” “Have I improved as a result?” Stage B Implement Interventions Stage C Reassess Performance

Assess Current Performance Three-Stage Model Stage A Assess Current Performance Performance assessment

“Infographic” Instantly available after completion of performance assessment Compares performance data to benchmarks Recommends learning based on performance data Key: NBOME staff can create intelligent recommendations based on performance data

Stage B: Individual Action Plan Practitioners reflect on Stage A, select learning interventions, and set goals for improvement Mention action plan?

Stage B: Learning Interventions Chunking: Learning Interventions are short (<5 min) videos grouped by performance measure Mention action plan?

Stage B: Practice Interventions Takeaway tools (quick references, performance support) to help learners implement interventions

Stage C Practitioners re-assess their performance after learning and practice interventions Infographic compares Stage A performance measures to Stage C

Results Completion rates have increased. Level 1 evaluation data: Learners rankings of the activity are improved. Level 3 evaluation data (performance assessment data) shows a significant improvement in performance. Average learner performance has improved on all performance measures – 10%. Statistically significant improvement in 2 performance measures – Potential side effects (16%) and interactions with other medications (23%)

Live Demo Participant view Administrator view Mention action plan?

Non-Clinically Active and Limited Scope Physicians OPAIM Futures Non-Clinically Active and Limited Scope Physicians OPAIM Simulation replaces data entry How could we draw in performance data from other sources? EHRs CDRs Elsewhere?