Patrick Huffer MD, Thomas Hahn MD, Jensena Carlson MD University of Wisconsin, Madison Department of Family Medicine and Community Health May 2, 2016 GOAL-ORIENTED ASSESSMENT OF LEARNING: CREATION OF AN INDIVIDUALIZED LEARNING PLAN FOR RESIDENTS

At the end of this presentation the participants will be able to: 1. Appreciate background of individualized learning plans (ILPs) in medical education 2. Introduce the Goal-Oriented Assessment of Learning (GOAL) or similar ILP to learners 3. Assess the efficacy of GOAL in aiding residents with future planning using results of a pilot study 4. Apply the GOAL to the Practice-Based Learning and Improvement-2 milestone regarding self-directed learning. OBJECTIVES

Self-directed learning, self-assessment, and goal-setting are important skills for lifelong improvement as a physician ILPs:  Encourage users to develop measurable goals and track progress (Li et al., 2010)  Increase motivation and sharing of ideas among learners (Chan & Chein, 2000) BACKGROUND: ILPS

Thinking of concrete goals can be difficult (Stuart et al., 2005) Difficult to apply ILP over a short time period (like a clinical rotation)  Greater utility over the span of residency? CHALLENGES OF ILPS

Little research in family medicine Most research involves pediatric residents, nursing and medical students  Pedialink ILPS IN FAMILY MEDICINE

Goal-Oriented Assessment of Learning  ILP for residents to help plan/organize residency and future learning through self-assessment and goal setting  Created using guidance from learning plans described in the literature  For use by University of Wisconsin Madison Family Medicine residents (n=43) WHAT IS THE GOAL?

 Complete GOAL at start of residency and update every 6 months (Supplement form)  Upload/store in New Innovations  Review every 6 months with APD and clinic- based faculty advisor GOAL

Tailor residency learning to long-term career goals o Set learning goals every 6 months to shape residency experience to help reach career goals Identify strengths and deficiencies Establish plans for developing areas of interest Select a "team" of mentors and others to help meet goals Provide a uniform framework to track educational development Help facilitate development of curriculum vitae SPECIFIC AIMS OF THE GOAL

7 main sections  A. Career goals  B. Goals to meet by the end of residency  C. Self-assessment of areas of confidence and challenge  D. Pursuits (residency activities)  E. 3 SMART (Specific, Measurable, Attainable, Realistic, Time- bound) goals for the next 6 months  F. 3 personal goals for the next 6 months  G. Mentorship team WHAT DOES THE GOAL LOOK LIKE?

Summer 2015: IRB application, exempted as not research September 2015: Introduced the GOAL form via seminar, o Pre-survey (Qualtrics) o Residents review GOAL with APDs March 2016: Residents update the GOAL form using supplement o Post-survey (Qualtrics) o GOALS reviewed with APD during Spring meeting Ongoing: Every 6 months, residents update GOAL using the GOAL supplement and save on online platform STUDY TIMELINE/METHODS

RESPONSE DEMOGRAPHICS  Total residents: 43  Pre-survey participants: 33 (77%)  Post-survey participants: 14 (33%)  Residents who completed both surveys: 10 (23%)

RESIDENCY GOALS

MOTIVATION

CONFIDENCE

POST-RESIDENCY CAREER

 The following questions showed no significant difference between pre- and post-test:  I have goals for my residency experience.  I have an organized plan for my learning throughout residency.  I am motivated to reach my learning goals.  I am confident that I am reaching my residency learning goals.  I believe my residency program has a system in place to help me reach my learning goals.  I know what I want to do in my career after residency.  I am aware of my strengths and weaknesses. SURVEY QUESTIONS AND RESPONSES

 Though not statistically significant, there was a potential trend showing that use of the GOAL may help with reaching residency goals  I am confident I am reaching my residency goals (p = 0.081) SURVEY QUESTIONS AND RESPONSES

To what extent did completing the GOAL tool help you set residency goals?2.82 (0.40) To what extent did completing the GOAL tool help you design/complete a resume/curriculum vita? 2.40 (0.84) To what extent did completing the GOAL tool help you identify mentors?2.64 (0.67) To what extent did completing the GOAL tool help you plan post-residency career plans? 2.55 (0.52) To what extent did completing the GOAL tool help you feel that you are reaching your goals? 2.27 (0.47) How useful was discussing the GOAL with your APD/faculty mentor?2.00 (0.50) POSTTEST SPECIFIC QUESTIONS Responses based on Likert Scale, with 1 = Not at all, 2 = Neutral/Not Sure, 3 = A little, 4 = Very much/a lot

 Difficult to draw meaningful conclusions between residency class averages  5-point Likert Scale not optimal  Low n, standard deviations overlap Small sample size (total n=43) Low survey response rate (77% for pre-survey, 33% for post-survey, and only 23% filled out both) Short time period between assessments (6 months) LIMITATIONS

Adapt GOAL for use by faculty, medical students Solicit subjective feedback on utility of the tool Make the tool shorter/easier vs. longer/more comprehensive Compare pre-residency responses with post- residency responses FUTURE DIRECTIONS

 Chan SW, Chien WT. Implementing contract learning in a clinical context: report on a study. Journal of Advanced Nursing. 2000;3:7.  Li ST, Burke AE. Individualized learning plans: basics and beyond. Academic Pediatrics. 2010;10:  Li ST, et al. (2011). Is resident’s progress on individualized learning plans related to type of learning goal set? Academic Medicine, 86,  Li ST, et al. Factors associated with successful self-directed learning using individualized learning plans during pediatric residency. Academic Pediatrics. 2010;10:  Lockspeiser TM, et al. (2013). Assessing residents’ written learning goals and goal writing skill: Validity evidence for the learning goal scoring rubric. Academic Medicine, 88.  McDermott MM, Curry RH, Stille FC, Martin GJ. Use of learning contracts in an office-based primary care clerkship. Medical education. 1999;33:  Sectish T, Floriani V, Badat MC, Perelman R, Bernstein H. Continuous professional development: raising the bar for pediatricians. Pediatrics. 2002; 110:  Shebard ME. (2012). Use of individualized learning plans among fourth-year sub-interns in pediatrics and internal medicine. Medical Teacher, 34,  Stuart E, Sectish T, Huffman L. Are residents ready for self directed learning? A pilot program of individualized learning plans in continuity clinic. Ambulatory Pediatrics. 2005;5: REFERENCES