1. Compass Learning Management System An Online Environment that Facilitates Learning and Assessment in Residency Education Molly Cohen-Osher, MD, Greg Sawin, MD, MPH, Allen Shaughnessy, PharmD, M.Med.Ed.

2. Compass Learning Management System An online learning management system developed to: – Define and measure competency – Foster life-long learning skills Reflection Self assessment – Learning plans Foster point of care learning – Learning resources – Information mastery – Capture and record structured assessment over time

3. Skills for life-long learning Self-directed learning Reflective ability Development of higher order thinking skills An ability to self-assess A system to know what you have mastered, and what needs to be learned An ability to develop learning plans and master new concepts Information mastery skills

4. The ACGME Milestone Project Implementation by 2013 Every 6 month assessment of residents’ progress toward competency Competencies determined by expert panels

6. Learning Objectives Describe the benefits (for learners, faculty and patients) of a learning management system. Describe the major educational theories that Compass LMS is based on. Explain the major components of Compass LMS including competency modules, learning resources, clinical blogs, learning plans, and various assessment forms including cognitive skills and procedure assessment forms. Explain how they might implement a similar approach to competency-based education in their own programs.

7. Active vs. Passive Learners Active Learners – How do I learn what I need to know? – Learn it when you need it – Reflect on it Passive Learners – What are you going to teach me today? – Learn it at 8:00 Thursday – Learning ends there

8. Components of Lifelong Learning Experience – Feedback (external) Reflection – Internally triggered (how am I getting on?) – Externally triggered New Information – Should I do something different?

9. What is “Reflection”? “An important human activity in which people re-capture their experience, think about it, mull it over, and evaluate it.” – (Boud 1985) “... A process where individuals think about and evaluate their experience in order to come to new understandings and appreciations.” – (Williams 2002) “A conscious and deliberate reinvestment of mental energy aimed at exploring and elaborating one’s understanding of the problem one has faced (or is facing) rather than aimed simply at trying to solve it.” – (Eva 2008) Sorting through the information received during a clinical encounter and selecting the pertinent details for further study. – (Kjaer 2008) “A mental process of trying to structure or restructure an experience, problem, or existing knowledge.” – (Driessen 2007) Thinking through an experience to make sense of it.

10. The Kolb Learning Cycle Kolb, D. A. (1984). Experiential Learning: Experience as the Source of Learning and Development. Upper Saddle River, NJ, Prentiss Hall.

11. Experience Doesn’t Always Lead to Learning “He’s had 10 years of experience.” “No, he’s had 1 year of experience, repeated 9 times.” “Experts don’t make better decisions. Experts remember more past decisions.”

12. The development of Expertise Ericsson, 2004

13. DEFINING COMPETENCE Compass Learning Management System

14. Competency? Competent in what? – Areas of Competence How do you define it? – Competency Standards How do you assess it? – Assessment tools

15. Document Competence Old – Letter from program director or dean stating “This learner is competent” New – An electronic portfolio demonstrating multiple facets of a learner’s competence

16. Define Competence Old – No explicit criteria “We’ll let you know when you’re competent” New – Competency standards are set and attainment is documented

17. Measure Competence Old – Rotation evaluations – Board Exams New – Detailed measurement of pre-defined knowledge, decision-making, skills throughout training

18. Defining Competence Identified topics that all family medicine residents need to know by the end of residency Organized into clinical areas where learning occurs – inpatient, outpatient, etc Categorized into system topics – cardiovascular, ENT, GI Divided into common patient concerns/diseases – The patient with chest pain **This has been named a “Competency Module”**

19. Defining Competence – For Each Competency Module there is a written learner/assessor guide: Describes learning outcomes consistent with competency Outcomes are the standard by which both the faculty and the individual could measure the resident’s progress or competency Defines what needs to be learned – Need to know vs. nice to know

20. Defining Competence Competency module objectives: – Categorized by Primary Prevention/ Preventative Measures Diagnosis/ Diagnostic Criteria Secondary Prevention/ Monitoring Parameters Comorbidities Treatment Quality improvement/ Systems – Behavioral objectives “Describe, be able to demonstrate, explain, know…” – Linked to ACGME competencies and rotations

21. Defining Competence Included in 114 Competency Modules are – Information management – Communication and cultural competency – Office practice of medicine – Out-of-office care – Professionalism/ethics – Wider context of medicine

22. Learner/Assessor Guide

23. Measuring Competency

24. LEARNING COMPONENTS Compass Learning Management System

25. Compass Learning Components Current components – Competency Standards – Learning Resources – Clinical Blogs Future components – Self Assessment – Learning Plans

26. Compass Learning Components Current components – Competency Standards – Learning Resources – Clinical Blogs Future components – Self Assessment – Learning Plans

27. Competency Module

28. Compass Learning Components Current components – Competency Standards – Learning Resources – Clinical Blogs Future components – Self Assessment – Learning Plans

29. Learning resources Suggested resources attached to each module Able to input, store and share your favorite resources in Compass

30. Compass Learning Components Current components – Competency Standards – Learning Resources – Clinical Blogs Future components – Self Assessment – Learning Plans

31. Clinical Blogging We have integrated a reflective exercise into our curriculum at Tufts Named “clinical blogging” Specific time allotted Format: – Process vs. product – Privacy honored Linked to competency you are reflecting about

32. Clinical Blog

33. Why Add Reflection to Residency Training? Facilitating the switch from passive to active learners Fosters adult self-directed learning – Identify learning needs – Decide what to learn and how to learn it Gives residents time to process their educational “tasks” to learn from them Changes the culture to one where it is “ok not to know” Hopefully create life-long learners

34. Reflection Exercise Describe an episode in your practice, in your day-to-day activities, or in your personal life that surprised you. – What happened? – Why did it surprise you? – What are you going to do now?

35. Reflection Exercise Reflect on a recent clinical question you encountered in your practice or learning. – What do you need to learn? – Why is this question difficult for you? – How would you learn it? Go do it – What answers did you find?

36. Compass Learning Components Current components – Competency modules – Learning Resources – Clinical Blogs Future components – Self Assessment – Learning Plans

37. Learning Plan Develop learning plan Refine with advisor during advising meeting Track progress of learning plan on Compass Use self assessment to guide learning plan

38. Learning Plans

40. ASSESSMENT TOOLS Compass Learning Management System

41. Measuring Competency

42. Compass Assessment Tools Current components – Cognitive Skills Forms – Basic Skills Qualifications (BSQ’s) – Procedure Assessment – Shadow Precepting/videotaping – 360 Evaluation

43. Compass Assessment Tools Current components – Cognitive Skills Forms – Basic Skills Qualifications (BSQ’s) – Procedure Assessment – Shadow Precepting/videotaping – 360 Evaluation

44. Cognitive Skills Form Assessment and document of knowledge, understanding, and application during precepting Moves assessment up bloom’s taxonomy Using the one-point-five minute preceptor model – Adding assessment to the precepting encounter

45. Bloom’s Taxonomy Encourage higher order learning

46. Resident Demonstration of Cognitive Skills

48. Let’s Practice

49. Overview Cognitive Skills

50. Compass Assessment Tools Current components – Cognitive Skills Forms – Procedure Assessment – Basic Skills Qualifications (BSQ’s) – Shadow Precepting/videotaping – 360 Evaluation

52. Compass Assessment Tools Current components – Cognitive Skills Forms – Procedure Assessment – Basic Skills Qualifications (BSQs) – Shadow Precepting/videotaping – 360 Evaluation

53. Points collected..

54. Start to reveal...

55. The full picture of a learner

56. Go to compasslms.net

57. FUTURE DIRECTIONS AND IMPLEMENTATION AT OTHER SITES Compass Learning Management System

58. Large Group Discussion