Assessment of IPE Initiative: Structure, Process, Outcome Part 2 Moderated by: Abby A. Kahaleh, BPharm, MS, PhD, MPH January 21, 2016

Contact Information Abby A. Kahaleh, BPharm, MS, PhD, MPH Curriculum SIG Chair 847-330-4537 (Phone) Akahaleh@Roosevelt.Edu

Presenters 1. Christine K O'Neil, PharmD, BCPS, CPG, FCCP Professor, Director of Curricular Development and IPE Mylan School of Pharmacy Duquesne University 2. Elena Umland, PharmD Associate Dean for Academic Affairs Professor of Pharmacy Practice Jefferson College of Pharmacy Thomas Jefferson University 3. Michelle Z. Farland, PharmD, BCPS, CDE Clinical Associate Professor University of Florida College of Pharmacy 4. Jennifer Danielson, PharmD, MBA, CDE Director of Interprofessional and Experiential Education & Assistant Professor University of Washington School of Pharmacy

Objectives Describe key structure of IPE programs Review current IPE assessment tools Explain critical steps in assessing IPE outcomes Share "lessons learned” in developing, implementing, and assessing IPE

Presenter I Christine K O'Neil, PharmD, BCPS, CPG, FCCP Professor Duquesne University Mylan School of Pharmacy

Creating Interprofessional Education Experiences Interprofessional education (IPE): “When students from two or more professions learn about, from and with each other to enable effective collaboration and improve health outcomes. “ World Health Organization. Learning Together to Work Together for Health. Report of a WHO study group on multiprofessional education for health personnel: the team approach. http://whqlibdoc.who.int/hq/2010/WHO_HRH_HPN_10.3_eng.pdf. Accessed August 25, 2014. The CAPE 2013 Outcomes are embedded in ACPE Standards 2016 in Standards 1-4. These require programs to prepare students for collaborative roles. Also Standards 2016 include a separate standard addressing IPE within the curriculum. Specifically, Standard 11

Evidence for IPE IPE is well-received and is a conduit for enabling knowledge and skills necessary for collaborative work IPE is less able to positively influence attitudes and perceptions towards others Evidence of IPE effectiveness is limited Evidence does suggest that an interprofessional approach improves quality and decreases cost of care. Hammick M, Freeth D, Koppel I et al. A best evidence systematic review of interprofessional education. Med Teach 2007;29:735-51. Reeves S, Zwarenstein M, Goldman J et al. Interprofessional education: effects on professional practice and health care outcomes. Cochrane Database of Systematic Reviews 2008, Issue1. Art. No.:CD002213. DOI:10.1002/14651858.CD002213.pub2. While IPE is well-received and is a pedagogical method for enabling knowledge and skills, it may be less able to positively inlfuence attitudes and behaviors. Evidence of the impact of IPE on patient outcomes is limited, however there are 4 studies with positive impact on patient satisfaction, teamwork, error rates, mental health competencies or care outcomes. Evidence does suggest that an interprofessional approach to care improves quality and decreases cost of care.

Interprofessional Education Collaborative (IPEC) Expert panel consisting of representatives from: Association of Colleges of Nursing, American Association of Colleges of Osteopathic Medicine, American Association of Colleges of Pharmacy, American Dental Education Association, Association of American Medical Colleges, and Association of Schools of Public Health. Competencies are general and flexible enough to be used by any profession: Domain 1 – Value and Ethics for Interprofessional Practice Domain 2 – Roles and Responsibilities Domain 3 – Interprofessional Communication Domain 4 – Teams and Teamwork IPE learning experiences should be linked to one or more of these competencies. Interprofessional Education Collaborative Expert Panel. Core Competencies for Interprofessional Collaborative Practice: Report of an Expert Panel. Washington, D.C: Interprofessional Education Collaborative; 2011. The development of core competencies approved by IPEC – an expert panel – will facilitate implementation of IPE across programs. This is an essential tool for curriculum development of IPE experiences. Of note, these competencies focus on development of IPE skills. The focus of IPE learning experiences is not primarily content.

Key Elements of IPE Courses & Experiences Planning and teaching by an interprofessional mix of faculty Link to a minimum of one learning objective (IPEC Competencies) Content relating to interprofessional competence is included and preferably threaded, throughout the course At least one assignment that necessitates interprofessional group work  Experiential courses may be designated as interprofessional. Student participants in the course/experience represent at least two health professions. Match students with equivalent levels of education for IPE experiences IPE experience should be reflective of real practice connections between disciplines IPE learning experiences should be optimized to achieve IPE outcomes for each of the programs involved and do not necessarily need to involve every health profession Includes an assessment of growth in interprofessional competence. IPEC domains – link assures that that students, upon completion of the course/experience, will demonstrate an increase in level of interprofessional competence

Curricular Considerations Ideally, IPE experiences should be present in didactic, experiential, and co-curricular formats. Present in each year of the professional curriculum to facilitate development of pharmacist as collaborators. Incorporation of early IPE exposure in pre-professional programs is also an opportunity. When planning for IPE, there are several considerations for implementation…..see list

Curricular Themes Communication skills Health care ethics Treatment adherence Public health Emergency preparedness Medication errors; patient safety topics Care of the patient with chronic illnesses or those in special populations. Buring SM, Bhushan A, Brazeau G, Conway S, Hansen L, Westberg S. Keys to successful implementation of interprofessional education: learning location, faculty development, and curricular themes. Am J Pharm Educ. 2009;73(4): Article 60. Meyer SM, Garr DR, Evans C. Advancing Interprofessional Clinical Prevention and Population Health Education. Curriculum Development Guide for Health Professions Faculty. http://c.ymcdn.com/sites/www.aptrweb.org/resource/collection/245F0E9A-CA95-47CB-BCC6-3CB02C0E2EB4/APTR-HPCTF_IPE_Crosswalk_2013.pdf. Accessed August 25, 2014. There are some curricular themes that are very conducive to implementation of IPE experiences for students.

Implementing IPE: Models IPE is possible with any model of pharmacy education: Fully integrated academic health center Partially co-located program with pharmacy and other professions under a common university ownership Partially co-located with pharmacy and other professions under different university components Pharmacy with other health professions but no medical school Pharmacy with no other health education programs on campus More readily implemented in programs that have co-existing health professions schools Partnerships with programs outside the school and creative teaching strategies with technology may facilitate IPE experiences in distant partnerships. IPE is possible with any model of pharmacy education. 5 models have been identified – see list. While IPE is more readily implemented in programs with co-existing health professions schools, distant partnership may be created with programs outside of the school utilizing creative teaching strategies and technology.

Assessment Tools for IPE Elena Umland, PharmD Jefferson School of Pharmacy Michelle Farland, PharmD, BCPS CDE University of Florida College of Pharmacy Reference: https://nexusipe.org/advancing/measurement-instruments

Kirkpatrick Assessment Model Level 4b: Benefits to patients/clients Improvements in health or well being of patients/clients Level 4a: Change in organizational practice Wider changes in the organization and delivery of care Level 3: Behavioral change Identifies individuals’ transfer of IP learning to their practice and setting Level 2b: Acquisition of knowledge & skills Including knowledge and skills linked to IP collaboration Level 2a: Modification of perceptions & attitudes Changes in reciprocal attitudes or perceptions between participant groups, Changes in perception or attitude towards the value and/or use of team approaches to caring for a client Level 1: Reaction Learners’ views on the learning experience and its IP nature Academia Practice Culture Change Need to be here We are here Students and/or faculty? What are you evaluating Danielson J, Willgerodt M. University of Washington Health Sciences. 2015

Tools to Assess Reaction Tool Name Brief Description Individual/ Team Assessment Curricular Location Readiness for interprofessional learning scale (RIPLS) Evaluates the readiness of health professions students for IPE. Individual Early; prior to the start of any IPE programming. Potential to repeat it to see improvement in readiness as initial distribution may serve as a baseline. Interdisciplinary education perception scale (IEPS) Assess student perceptions of interprofessional experiences. Early with inaugural IPE programming. Consider repeating to evaluate changes in perceptions. Note that 12- , 17- and 18-item scales may serve different purposes.

Tools to Assess Modifications of Perceptions & Attitudes Tool Name Brief Description Individual/ Team Assessment Curricular Location Student perceptions of physician-pharmacist interprofessional clinical education (SPICE) Assesses pharmacy and medical student attitudes toward interprofessional clinical education. Individual As early as possible where interprofessional clinical education occurs. First year students following a required IPE activity or course. Attitudes toward healthcare teams scale (ATHCT) Assesses team member perceptions of the quality of care delivered by healthcare team; team member attitudes towards physician authority in teams and their control over information about patients. Individual/team As used to determine effect of interprofessional interventions, its placement at beginning and end of program. 20-item scale (versus 14-item scale) has benefit of evaluating domain of roles/responsibilities.

Tools to Assess Behavior Change Tool Name Brief Description Individual/ Team Assessment Curricular Location Interprofessional collaborator assessment rubric (ICAR) Observational tool to assess learner achievement of interprofessional competency domains Individual APPEs. Faculty or preceptors observe students in interprofessional activities over time. Consider use at end of week 1, towards middle and at end of clinical rotation. Interprofessional collaborative competency attainment survey (ICCAS) Self-assessment of achievement of interprofessional care competencies APPEs. At start and end of clinical rotation. Also consider pre- and post- other IPE activities as evaluates all 4 IPEC domains.

Tools to Assess Behavior Change Tool Name Brief Description Individual/Team Assessment Curricular Location IPEC competency survey instrument Self-assessment of the achievement of the competencies defined by the Interprofessional Education Collaborative expert panel Individual Conclusion of APPE. May be introduced early as formative assessment for pre-/post-assessment. Collaborative practice assessment tool (CPAT) Assesses the views of team members in a collaborative care team on elements of collaboration Team Following a longitudinal experience (Pre-APPE or APPE). Performance assessment for communication and teamwork tool set (PACT) Observational tool to assess teams during a live simulated scenario Team OSCE (pre-APPE or APPE)

Tools to Assess Change in Organizational Practice Tool Name Brief Description Individual/Team Assessment Curricular Location Index for interdisciplinary collaboration (IIC) Assesses aspects and levels of interprofessional collaboration within an organization Individual & Team Designed for a longitudinal work environment may be helpful with students completing numerous APPEs at the same institution or residency programs Survey of organizational attributes of primary care (SOAPC) Assesses healthcare providers’ perceptions of resources available to make changes in the patient care process in primary care settings Team Designed for longitudinal teams, may be helpful with students completing numerous APPEs at the same institution or residency programs

IPE Assessment Tool Application Identify the outcomes you need/want to measure. Review tools to identify those that best measure the outcomes and are feasible to implement. Multiple evaluation methods will be needed Quantitative & qualitative Self-assessment & observation of behavior Identification of outcomes will likely vary based on how far into implementation your IPE program is. Initially, assessing reaction may be the most important component, while more developed programs may not feel a need to assess reaction if IP collaboration is part of the organizational culture already.

Presenter IV Jennifer Danielson, PharmD, MBA, CDE Director of Interprofessional Education and Experiential Education Assistant Professor University of Washington School of Pharmacy

Kahaleh A, Danielson J, Franson K, et al. AJPE 2015; 79 (1): Article 6.

UW SOP IPE Curriculum Framework Exposure Immersion Practice Integration Student Growth Patient Care Outcomes Population Health Outcomes Exposure Immersion Practice   Values Roles Communication Teamwork Shadowing Simulations Clinical Placements/Experiences Service Learning Project-Based Learning Case Exercises Active Learning Danielson J, Willgerodt M. UW Health Sciences. 2015

UW SOP Assessment Framework   UW SOP Assessment Framework   Teamwork Attitudes Knowledge Skills Behaviors Practice Change Graduation = Team-based practice ready Integration Communication Immersion Roles Values Exposure Reaction Danielson J. University of Washington School of Pharmacy. 2015

Q&A

THANK YOU!

Discussion Questions What types of IPE programs have you implemented at your college/school of pharmacy? What are some of the challenges/opportunities of your IPE program? Which assessment tools have you used to evaluate students’ performance outcomes? What are your future plans for CQI?