Andrea Weeks, PharmD Elora Hilmas, PharmD, BCPS May 2, 2017 Are we asking the right questions? Optimizing PharmAcademic™ to obtain valuable feedback from preceptors and residents 15min including questions, shoot for 7-8 slides San sarif 24ct Elora to begin Andrea to end Nemours logo slides Andrea Weeks, PharmD Elora Hilmas, PharmD, BCPS May 2, 2017

Disclosures There is no commercial support associated with this educational activity The presenters have no relationship with any commercial company

Knowledge Level Objectives: Recognize evaluation questions that will successfully obtain valuable feedback that can be used to improve resident performance Describe two PharmAcademic™ customized evaluations that have been used to achieve resident self-evaluation goals and to facilitate preceptor responsibility to ensure resident success in the program

Polling Question How many of you are currently using PharmAcademic™ to incorporate custom resident and preceptor evaluations into your residency programs?

ASHP Residency Objectives Competency Area R3: Leadership and Management Objective R3.1.2: (Applying) Apply a process of ongoing self-evaluation and personal performance improvement Objective R3.2.4: (Applying) Manage one’s own practice effectively Why are you doing this? What is the setting for presentation. ASHP added additional guidance for obj R3.1.2 in April 2016 requiring programs to incorporate resident self-evaluation in at least 3 learning experiences OR 1 learning experience and 2 times in a longitud. Learning experience OR required 3 times in a longitud.learning experience. Both are applying objectives which are required competency areas and are surveyable. Educational Objectives: Observable, measurable statements describing what residents will be able to do as a result of participating in the residency program.

Is this a Simple or Critical Question? Do you feel you are progressing in your acute care rotations? How do you rate your ability to accurately identify medication problems on patient profiles? How many presentations have you completed? What strategies have worked in your ability to provide education to patients in clinic? This slide is an audience assessment question

Developing Effective Questions Critical vs. Simple Questions Simple Can be answered with a “yes” or “no” Contain the answers within themselves Can only be answered by a fact, or a series of facts Critical Leads to more questions Provokes discussion Considers the audience and authorial intent Derives from a critical or careful review Yes or no (this is not helpful when trying to elicit further questions, discussion, or analysis) Do you feel you are progressing in your acute care rotations? How do you rate your ability to accurately identify medication problems on patient profiles? How many presentations have you completed? What strategies have worked in your ability to provide education to patients in clinic?

Resident Progress Report

Resident Progress Report Prior to PharmAcademic™ at our Institution Word document Resident submits 2-3 days before monthly residency steering committee Reviewed at committee Resident receives feedback Enhancement through PharmAcademic™ Customized evaluation form More critical questions Increased value of the feedback Automatically scheduled for resident Storage of form facilitates progress review and resident development plans

Paper Residency Progress Update Form

New Resident Progress Report Print out form to share remove screenshots

New Resident Progress Report

How can Preceptors Facilitate an Effective Rotation Handoff? 13 How can Preceptors Facilitate an Effective Rotation Handoff? This is the transition slide from Elora to Andrea.

Preceptor Responsibilities as Part of ASHP Residency Standards 14 Preceptor Responsibilities as Part of ASHP Residency Standards 3.4.b.(2) Formative Evaluation. Preceptors must make appropriate adjustments to residents’ learning activities in response to information obtained through day-to-day informal observations, interactions, and assessments. 4.7 Preceptors’ Responsibilities. Preceptors serve as role models for learning experiences. They must: 4.7.a. contribute to the success of residents and the program. Standard 3.4 refers to evaluation of the resident’s progression towards achieving the program’s required educational goals and objectives. 3.4.b(2) refers direction to formative ongoing regular assessments of the resident. In the guidance document from ASHP it mentions preceptors should ensure residents’ responsibilities and/or activities align with residents’ progress within each learning experience. What better way than informing the future preceptor of the resident’s progress and needs.

Effective Preceptor to Preceptor Communication Share feedback on residents’ performance at preceptor meetings Distribute residents’ customized learning plans Review resident feedback and evaluations in PharmAcademic™ Custom-build end of rotation handoffs for subsequent preceptor

Example Rotation Handoff Questions 16 Example Rotation Handoff Questions How is the resident progressing on: Their own individual goals for the residency (from their entering interests form)? Handling drug information (questions, journal clubs, etc.)? Interdisciplinary communication? Level of enthusiasm and professional growth? Assignment or project deadlines, meeting attendance, evaluations, and sought after learning opportunities? Questions for handoff should be used to help update the new preceptor on resident progress and areas of improvement needed to work on for the next rotation. For example, if a resident had trouble handling deadlines what could you as a the new preceptor do to help with this?

17 Assessment Question What other preceptor rotation hand- off information would be helpful to you as the incoming preceptor for a resident about to start a rotation with you?

18 Let’s Explore How to Create a Customized Resident Rotation Handoff using PharmAcademicTM is a registered trademark of the McCreadieGroup

How to Get Started Need to have access to edit program RPD or Designee 19 How to Get Started Need to have access to edit program RPD or Designee

PharmAcademicTM Evaluations 20 PharmAcademicTM Evaluations Helpful step by step guides Once in the edit mode custom evaluations can be created Suggest while you are experimenting with creating new evaluations that you add a mock resident. I added myself with my home email so that I could sign on PharmAcademic and practice taking the evaluations. I also could sign on as a preceptor reviewing the mock resident to see that the evaluation was assigned properly. Help button provides you with actual PDFs of how to build custom evaluation scales and evaluations.

Creating Custom Evaluations 21 Creating Custom Evaluations When adding questions you can choose the question type, answers and even a comments box. The alternative is to use a custom evaluation scale here instead of manually entering answer choices. This is how you create the actual evaluation in PharmAcademic. You build the questions and responses along with comments box.

Scheduling Custom Evaluations 22 Scheduling Custom Evaluations Preceptors who have access to edit their rotation can build a custom evaluation from here Click here to add an evaluation The steps to get to this screen begin with getting to the actual learning experience you would like to view. Then it will give you the option to edit the learning experience. That is where you can choose evaluations and it gives you the option to build a new custom evaluation here or add a new evaluation. You can choose here from any existing ASHP evaluation or any evaluations you have already built. By selecting the evaluation from the drop down. Here I chose rotation notes for clinical rotations. Next you can choose the frequency of the evaluation. If you want it to occur let’s say at the beginning and midpoint you would have to add the evaluation more than once with each percentage of the rotation you want it assigned. You cannot choose an actual date in PharmAcademic. For example if you had a rotation that was 4 weeks beginning May 1st and ending May 30th and you chose to have a scheduled resident self evaluation at the 50% time frame PharmAcademic would calculate the actual date based on the start and stop date of the rotation. You can always reschedule the evaluation to a specific date once it is assigned by tweaking the actual date from what PharmAcademic calculates. These evaluations do show up in the resident’s email as a reminder they are due and also on both the resident and preceptor tasks due section in PharmAcademic as reminders. This drop down box allows you to choose an existing evaluation Pick frequency of evaluation

23 Key Takeaways Open the lines of communication in your residency program Use critical thinking questions to challenge your residents to reflect upon their own performance Consider adding more formalized preceptor communication during rotation transitions Implement PharmAcademic™ to schedule custom evaluations for your program Opening the lines of communication in your residency program will help ensure that you are meeting the needs of your residents and taking responsibility as a preceptor to enhance your program. Critical thinking questions help elicit more meaningful responses from your residents to help them grow as professionals and realize the goals and objectives for the program Preceptors need to make time to review resident recent progress to make their own rotations more customized to the needs of their resident. PharmAcademic has unlimited capabilities to incorporate customized evaluations that fit the needs of each program.

24 References American Society of Health-System Pharmacists, 2014 Accreditation Standard for PGY1 Pharmacy Residency Programs. Available at: http://www.ashp.org/DocLibrary/Residents/Guidance-PGY1-Standard-2015.aspx. Accessed February 26, 2017 American Society of Health-System Pharmacists, 2014 Accreditation Standard for PGY1 Pharmacy Residency Programs. Available at: http://www.ashp.org/DocLibrary/Accreditation/Regulations-Standards/Newly- Approved-Required-Competency-Areas-Goals-and-Objectives.pdf. Accessed February 26, 2017. 2016. PharmAcademic™, McCreadie Group, Inc. Copyright 2016. Available at: http://www.mccreadiegroup.com/. Accessed February 26, 2017. Reading, Writing, and Researching for History. Patrick Rael, Bowdoin College 2004. http://academic.Bowdoin.edu/WritingGuides

25 Thank you! Please feel free to contact either of us for more personal assistance Elora Hilmas, PharmD, BCPS Alfred I. duPont Hospital for Children Wilmington, Delaware 302-651-5791en Elora.Hilmas@nemours.org Andrea Weeks, PharmD Paoli Hospital Paoli, Pennsylvania 484-565-1144 weeksa@mlhs.org