1. How to Measure Impact of Health Leadership Development
Speakers: Dr. Graham Dickson, CHLNet Suzanne Schell, ROI Institute Canada
Moderator: Kelly Grimes
October 26, 2017
CHLNet/LEADS Collaborative Webinar Series

2. Dr. Graham Dickson CHLNet Senior Policy Advisor

3. CHLNet ROI Study Partners
15 Partners: ROI Institute Canada, Manitoba Centre for Healthcare Innovation, Canadian College of Health Leaders, HealthCareCAN, Canadian Institute for Health Information, Canadian Medical Protective Association, Canadian Nurses Association, Canadian Society of Physician Leaders, Joule/CMA, Accreditation Canada, University of Ottawa, Ontario Hospital Association, Alberta Health Services, Alberta Innovates – Health Solutions, and Canadian Foundation for Healthcare Improvement.
Looking for More Partners!

4. ROI Steering Group
To provide stewardship of the entire Return on Investment (ROI) project over its duration to ensure it achieves its goals and is conducted in an efficient manner (i.e. within budget).
Core Group: CHLNet Senior Policy Advisor, CHLNet Executive Director, Network Partner ROI Funder Reps, ROI Experts

5. LDI Assessment Toolkit Principles
Evidence-based
Simple to use
Customizable
Economical

6. LDI Study Phases Pre-Phase Funding (May to Dec. 2015) Phase I
Scoping Review (Jan. to May 2016)
Phase II
Design Instrument and Pilot Test (September 2017 to May 2018)
Phase III
Finalize Instrument and Implement (June 2018)

7. ROI Scoping Review Three objectives:
ROI determinants associated with healthcare leadership quality/style.
ROI determinants associated with leadership development programs/tactics.
ROI determinants associated with existing ROI tools.
Evidence Brief and Journal Publication

8. Rethink of Approach Staged approach LD defined:
“Deliberate, structured learning interventions aimed a growing or enhancing individual and organizational leadership capability.”
LD Investment goals:
Improve individual leadership capability
Collective leadership capacity

9. Three LDI User Options Automated Partnership with ROI Institute Canada
Companion Document on Designing an Impactful LDI

10. Suzanne Schell ROI Institute Canada
Proving value, impact and ROI

11. Isolate the Effects of the Program
Reaction & Planned Action
Learning
Application & Implementation
Isolate the Effects of the Program
Intangible Benefits
ROI
Impact
Chain of Impact
ROI Institute Canada

12. 1. Reaction To and Satisfaction with the Program
ROI Institute Canada

13. 2. New Skills, Knowledge and Competencies Acquired - “Capability”
ROI Institute Canada

14. 3. Application and Implementation
Barriers
Enablers
ROI Institute Canada

15. 4. Impact – Tangible and Intangible
ROI Institute Canada

16. The finance and accounting calculation
ROI – The Calculation
The finance and accounting calculation
Net Program Benefits
Program Costs
ROI (%) =
X 100
Example
Net Program Benefits = $38,776
Program Costs = $32,984
ROI =117%
ROI Institute Canada

17. 6. Intangible Measures, Not Converted to Money
ROI Institute Canada

18. Isolate the Effects of the Program
Reaction & Planned Action
Learning
Application & Implementation
Isolate the Effects of the Program
Intangible Benefits
ROI
Impact
Chain of Impact
ROI Institute Canada

19. Measuring Impact What level will you evaluate your leadership program?
Level 1 – Short programs, briefings, policy introductions Level 2 – When participants need to acquire specific knowledge or skills – for example a CPR course Level 3 – Essential to ensure the behaviour change in the workplace is occurring to create the desired impact Level 4 – Proves the value the program is adding to the organization
ROI Institute Canada

20. Criteria for Level 4 or ROI
Expected life cycle
Linkage to strategic objectives
Cost
Time commitment
Visibility
Management/executive interest
Client requirements
ROI Institute Canada

21. LDI Tools to Evaluate Tool kit gives you access to: Online tools
Downloadable tools
How to use sections
ROI Institute Canada

22. Developing Objectives
ROI Institute Canada

23. Evaluation Planning
ROI Institute Canada

24. Data Collection
ROI Institute Canada

25. Isolating the Effects of the Program, Project, Initiative
Case Application
ROI Institute Canada

26. Convert Data to Monetary Values
ROI Institute Canada

27. Measuring ROI
ROI Institute Canada

28. Pilot Test: A Fantastic Opportunity!!
Diversity of program, i.e. geography, size of organization, type of organization, interprofessional vs. profession specific, and executive vs. front line leaders
Timing of program, i.e. fall 2017
User option desired, i.e. LD outcomes, LD impact, ROI
Overall LD investment in last three years, i.e. robust vs. new endeavour
In-kind contribution, i.e. willingness to devote staff time and resources to pilot test including write-up as a case study

29. Your Feedback Required
Does the new phased approach make sense and would it be useful for your organization to generate evidence?
Any issues or red flags we should consider as you think about it for your organization?
Any advice for us in pilot testing? And would you want to be part of this fantastic opportunity?

31. Next Steps and Thank You!
For more information:

32. Please take 2 minutes to complete a survey at
Next Webinar: February 2018 on Cultural Transformation
chlnet.ca
leadscanada.net