1. Clearly Connecting Performance Measures to Learning Interventions in Performance Improvement Activities
Natalie Lavelle, M.Ed.
Andy Hicken, PhD
Director for Learning Technologies
Director of Product Development
National Board of Osteopathic Medical Examiners
Web Courseworks
May 16, 2016
Tech Demo
Medbiquitous

2. Natalie Lavelle, M.Ed. Andy Hicken, PhD Nothing to disclose.
DISCLOSURES
Natalie Lavelle, M.Ed.
Nothing to disclose.
Andy Hicken, PhD
Employee of Web Courseworks, a vendor of educational technologies (including some technology discussed in this presentation).

3. NBOME OPAIM
A platform for on-demand, self-paced performance improvement (OCC Part IV ~ MOC Part IV) activities
Competency-based education
Stage A
Provides immediate feedback on performance gaps and successes
Compares performance to national quality benchmarks
Automated, individualized reporting: no extra administrative work
Stage B
Directs participants to education based on performance gaps
Various media used for the educational intervention
Stage C
NBOME has been able to successfully document that the educational activities are improving physician performance and having a positive impact on patient care.

4. Defining our Terms
PI CME (Performance Improvement Continuing Medical Education): “A process by which evidence-based performance measures and quality improvement (QI) interventions are used to help physicians identify patient care areas for improvement and change their performance.” --AMA
MOC Part IV (Maintenance of Certification Part IV): “These standards contribute to improved patient care through ongoing assessment and improvement in the quality of care provided by diplomates in their individual practices and/or in the larger hospital, health system, or community setting in which the diplomates practice medicine.” --ABMS
OCC Part IV (Osteopathic Continuous Certification Part IV): “Requires that you engage in continuous quality improvement through comparison of personal practice performance measured against national standards for your medical specialty.” --AOA

5. Use of competency framework
Pre-development
Use of competency framework
Mapped learning interventions to performance measures based in recognized best practices and current research

6. Three-Stage Model for PI CME
Stage A
Assess Current Performance
“How well am I doing?” “What can I do to improve?” “Have I improved as a result?”
Stage B
Implement Interventions
Stage C
Reassess Performance

7. Assess Current Performance
Three-Stage Model
Stage A
Assess Current Performance
Performance assessment

8. “Infographic”
Instantly available after completion of performance assessment
Compares performance data to benchmarks
Recommends learning based on performance data
Key: NBOME staff can create intelligent recommendations based on performance data

9. Stage B: Individual Action Plan
Practitioners reflect on Stage A, select learning interventions, and set goals for improvement
Mention action plan?

10. Stage B: Learning Interventions
Chunking: Learning Interventions are short (<5 min) videos grouped by performance measure
Mention action plan?

11. Stage B: Practice Interventions
Takeaway tools (quick references, performance support) to help learners implement interventions

12. Stage C
Practitioners re-assess their performance after learning and practice interventions
Infographic compares Stage A performance measures to Stage C

13. Results Completion rates have increased.
Level 1 evaluation data: Learners rankings of the activity are improved.
Level 3 evaluation data (performance assessment data) shows a significant improvement in performance.
Average learner performance has improved on all performance measures – 10%.
Statistically significant improvement in 2 performance measures – Potential side effects (16%) and interactions with other medications (23%)

14. Live Demo
Participant view
Administrator view
Mention action plan?

15. Non-Clinically Active and Limited Scope Physicians OPAIM
Futures
Non-Clinically Active and Limited Scope Physicians OPAIM
Simulation replaces data entry
How could we draw in performance data from other sources?
EHRs
CDRs
Elsewhere?