How to Measure Impact of Health Leadership Development Speakers: Dr. Graham Dickson, CHLNet Suzanne Schell, ROI Institute Canada Moderator: Kelly Grimes October 26, 2017 CHLNet/LEADS Collaborative Webinar Series

Dr. Graham Dickson CHLNet Senior Policy Advisor

CHLNet ROI Study Partners 15 Partners: ROI Institute Canada, Manitoba Centre for Healthcare Innovation, Canadian College of Health Leaders, HealthCareCAN, Canadian Institute for Health Information, Canadian Medical Protective Association, Canadian Nurses Association, Canadian Society of Physician Leaders, Joule/CMA, Accreditation Canada, University of Ottawa, Ontario Hospital Association, Alberta Health Services, Alberta Innovates – Health Solutions, and Canadian Foundation for Healthcare Improvement. Looking for More Partners!

ROI Steering Group To provide stewardship of the entire Return on Investment (ROI) project over its duration to ensure it achieves its goals and is conducted in an efficient manner (i.e. within budget). Core Group: CHLNet Senior Policy Advisor, CHLNet Executive Director, Network Partner ROI Funder Reps, ROI Experts

LDI Assessment Toolkit Principles Evidence-based Simple to use Customizable Economical

LDI Study Phases Pre-Phase Funding (May to Dec. 2015) Phase I Scoping Review (Jan. to May 2016) Phase II Design Instrument and Pilot Test (September 2017 to May 2018) Phase III Finalize Instrument and Implement (June 2018)

ROI Scoping Review Three objectives: ROI determinants associated with healthcare leadership quality/style. ROI determinants associated with leadership development programs/tactics. ROI determinants associated with existing ROI tools. Evidence Brief and Journal Publication www.chlnet.ca/tools-resources/research

Rethink of Approach Staged approach LD defined: “Deliberate, structured learning interventions aimed a growing or enhancing individual and organizational leadership capability.” LD Investment goals: Improve individual leadership capability Collective leadership capacity

Three LDI User Options Automated Partnership with ROI Institute Canada Companion Document on Designing an Impactful LDI

Suzanne Schell ROI Institute Canada Proving value, impact and ROI

Isolate the Effects of the Program Reaction & Planned Action Learning Application & Implementation Isolate the Effects of the Program Intangible Benefits ROI Impact Chain of Impact ROI Institute Canada

1. Reaction To and Satisfaction with the Program ROI Institute Canada

2. New Skills, Knowledge and Competencies Acquired - “Capability” ROI Institute Canada

3. Application and Implementation Barriers Enablers ROI Institute Canada

4. Impact – Tangible and Intangible ROI Institute Canada

The finance and accounting calculation ROI – The Calculation The finance and accounting calculation Net Program Benefits Program Costs ROI (%) = X 100 Example Net Program Benefits = $38,776 Program Costs = $32,984 ROI =117% ROI Institute Canada

6. Intangible Measures, Not Converted to Money ROI Institute Canada

Isolate the Effects of the Program Reaction & Planned Action Learning Application & Implementation Isolate the Effects of the Program Intangible Benefits ROI Impact Chain of Impact ROI Institute Canada

Measuring Impact What level will you evaluate your leadership program? Level 1 – Short programs, briefings, policy introductions Level 2 – When participants need to acquire specific knowledge or skills – for example a CPR course Level 3 – Essential to ensure the behaviour change in the workplace is occurring to create the desired impact Level 4 – Proves the value the program is adding to the organization ROI Institute Canada

Criteria for Level 4 or ROI Expected life cycle Linkage to strategic objectives Cost Time commitment Visibility Management/executive interest Client requirements ROI Institute Canada

LDI Tools to Evaluate Tool kit gives you access to: Online tools Downloadable tools How to use sections ROI Institute Canada

Developing Objectives ROI Institute Canada

Evaluation Planning ROI Institute Canada

Data Collection ROI Institute Canada

Isolating the Effects of the Program, Project, Initiative Case Application ROI Institute Canada

Convert Data to Monetary Values ROI Institute Canada

Measuring ROI ROI Institute Canada

Pilot Test: A Fantastic Opportunity!! Diversity of program, i.e. geography, size of organization, type of organization, interprofessional vs. profession specific, and executive vs. front line leaders Timing of program, i.e. fall 2017 User option desired, i.e. LD outcomes, LD impact, ROI Overall LD investment in last three years, i.e. robust vs. new endeavour In-kind contribution, i.e. willingness to devote staff time and resources to pilot test including write-up as a case study

Your Feedback Required Does the new phased approach make sense and would it be useful for your organization to generate evidence? Any issues or red flags we should consider as you think about it for your organization? Any advice for us in pilot testing? And would you want to be part of this fantastic opportunity?

Next Steps and Thank You! For more information: www.chlnet.ca

Please take 2 minutes to complete a survey at www.questionpro.com/t/AEeuVZayv5 Next Webinar: February 2018 on Cultural Transformation chlnet.ca leadscanada.net