You can look at the same data through different lenses… EPAs/ Competencies Curricula Educational Trajectory Assessment Data Educational Achievement Data

Specification available Competency Framework Curriculum Report Competency Object Competency Object Competency Object Competency Object Competency Object Other competency framework Competency Object Competency Object Competency Object 1 2 3 1 2 3 1 2 3 Mapping (SKOS) Performance Framework Educational Achievement data Educational Trajectory data Specification available Development pending Existing resources

My Educational Achievement Data * Available Data Available Actions My Competencies CanMeds ACGME My Entrustable Activities and Skills My Trajectories My Curricula University of Toronto University of Maryland NBME ABP Other Report my data To residency program director To certifying board Share with mentor NOTE: reports to external reviewers and sharing with mentor are shown as options here to get the idea across that the data may be used for multiple purposes. Special interfaces would allow the learner to report data or select data for reporting to a mentor. This presentation does not detail what those interfaces might look like but acknowledges that they would be necessary. The rest of this presentation focuses on the learner reviewing their own data for self assessment and improvement. Date of report: October 5, 2011

My Competencies: CanMeds Performance levels 1 – Novice 2 – Intermediate 3 – Expert Related Activity: University of Toronto MD Program Date of report: Oct 5, 2011 Date of achievement: May 30, 2008 Source: University of Toronto

My Competencies: CanMeds As Communicators, physicians effectively facilitate the doctor-patient relationship and the dynamic exchanges that occur before, during, and after the medical encounter. See full framework Performance levels 1 – Novice 2 – Intermediate 3 – Expert Related Activity: University of Toronto MD Program Date of report: Oct 5, 2011 Date of achievement: May 30, 2008 Source: University of Toronto

My Competenciess: CanMeds Function effectively as consultants, integrating all of the CanMEDS Roles to provide optimal, ethical and patient-centered medical care See full framework Performance levels 1 – Novice 2 – Intermediate 3 – Expert Related Activity: University of Toronto MD Program Date of report: Oct 5, 2011 Date of achievement: May 30, 2008 Source: University of Toronto

My Competencies: CanMeds Performance levels Novice: demonstrates behaviors expected of a first year trainee. Requires close supervision. Intermediate: demonstrates behaviors expected of a trainee after 2 years. Requires some supervision. Expert: demonstrates behaviors expected of a trainee at the completion of training. No supervision required. Performance levels 1 – Novice 2 – Intermediate 3 – Expert Related Activity: University of Toronto MD Program Date of report: Oct 5, 2011 Date of achievement: May 30, 2008 Source: University of Toronto

My Competencies: ACGME Performance levels 1 – Beginner 2 – Adv Beginner 3 – Competent 4 – Adv Competent 5 – Mastered Related Activity: University of Maryland Pediatrics Residency Date of report: Oct 5, 2011 Date of achievement: Oct 5, 2011 Source: University of Maryland

My Competency-based Achievements: Patient Care Performance levels 1 – Beginner 2 – Adv Beginner 3 – Competent 4 – Adv Competent 5 – Mastered Learner provided: Spanish asthma brochure Notes My notes (private): Review procedure list and discuss action plan with mentor (Private comment) CB: There are some additional procedures I’d like you to work on – let’s discuss Date of report: Oct 5, 2011 Date of achievement: Oct 5, 2011 Source: University of Maryland

My Competencies: Gathering Essential Information Details Peer group: Institution cohort Context: University of Maryland Pediatrics Residency Date of report: Oct 5, 2011 Date of achievement: Oct 5, 2011 Source: University of Maryland

My Competency-based Achievements: Gathering Essential Information Benchmark to Peers Details Date of report: Oct 5, 2011 Date of achievement: Sep 10, 2011 Source: University of Maryland

My Competency-based Achievements: Gathering Essential Information – Benchmark to Peers Peer group: Institution Date of report: Oct 5, 2011 Date of achievement: Sep 10, 2011 Source: University of Maryland

My Competencies: Gathering Essential Information – Details* Title: Pediatric Level 3 360 Degree Feedback Duration: 2 Hours Description: Residents receive feedback from supervisors, peers, nurses, and patients. Keywords: Inservice training Interprofessional? No Associated Competencies: Gathering Essential Information, Professionalism Resource: Real Patient Assessment Method: Multisource feedback Purpose: Formative assessment Assessment results: Professionalism – 75, Communication – 70, Patient care – 78. View associated course Date of report: Oct 5, 2011 Date of achievement: Sep 10, 2011 Source: University of Maryland

My Competencies: Gathering Essential Information – Details* Title: Pediatric Level 3 360 Degree Feedback Duration: 2 Hours Description: Residents receive feedback from supervisors, peers, nurses, and patients. Keywords: Inservice training Interprofessional? No Associated Competencies: Gathering Essential Information, Professionalism Resource: Real Patient Assessment Method: Multisource feedback Purpose: Formative assessment Assessment results: Professionalism – 75, Communication – 70, Patient care – 78. View associated course Title: Pediatric Residency level 3 Description: The resident begins this sequence midway through their training. Duration: 12 months Academic level: 3 Competencies: Patient Care, Professionalism Includes: (list of instruction and assessment events or sub-courses) Date of report: Oct 5, 2011 Date of achievement: Sep 10, 2011 Source: University of Maryland

My Competency-based Achievements: Medical Knowledge  Identifies evidence-based resources to answer basic and clinical science questions. Met  With Concerns □ Did Not Meet Insuff Evid Evidence: Brief Summary:  Applies core concepts of pathophysiology to new problems in the basic and clinical sciences relevant to medicine.  Identifies and acknowledges gaps in knowledge and develops and implements plans to correct.  Achieves breadth and depth of knowledge in the curricular threads (eg. physiology, pharmacology, etc.). Justify your decision Medical Knowledge Competency From Elaine Dannefer: Under evidence, reviewers of the portfolio note the important pieces of evidence related to the competency standard that have informed their judgment.  For example if they note in their summary  that student has excelled in research they might put under evidence –published in Nature, or see research preceptor Ref #10.    The  overall decision is made for the competency, not the standard.  However, not all standards are of equal importance, so in some competencies, not meeting a particular standard might result in not meeting the competency even though they meet the other 3 standards, so it’s important to make judgments about each standard based on a careful review of the evidence.  Although this may seem very subjective, there are multiple pieces of evidence that are triangulated to inform judgments by trained reviewers. This documentation is also used by the reviewers of a portfolio to write a letter to the student that provides very specific feedback –  and if I have questions when I review letters, it’s helpful. Date of report: Oct 5, 2011 Date of achievement: Oct 5, 2011 Source: University of Maryland Cleveland Clinic Lerner College of Medicine of Case Western Reserve University

My Entrustable Activities and Skills* Status Activity Summary Related Curricula Arterial puncture for blood gas analysis Entrustment Details Clinical skills OSCE, U Toronto Intravenous catheter insertion Skills and thrills, U Toronto . . . Manage patients with common, single system diagnoses who require a hospital setting In progress Infant/Toddler Rotation, U Maryland Care of a healthy newborn Nursery rotation, U Maryland We probably need to add some sort of flag for entrustment to the data model. Referring to a level of supervision is inadequate as people may come up with differing schemes for describing supervisory levels. Date of report: Oct 5, 2011 Date of achievement: Various Source: Various

My Entrustable Activities and Skills* Intravenous Catheter Insertion Exam: Clinical Skills OSCE Exam 1, May 1, 2008 Passed Additional Criteria: Satisfactory completion of 4 intravenous catheter insertions as determined by supervising faculty member. Description: Learners are required to demonstrate satisfactory skills on simulators and real patients. Methodology: Clinical performance rating/checklist Context: University of Toronto MD Program Date of report: Oct 5, 2011 Date of achievement: Jun 1, 2008 Source: University of Toronto

My Entrustable Activities and Skills: Activity Details October 5, 2011 Peer group: Institution cohort Subcompetencies Supervision Context: University of Maryland Pediatrics Residency Manage patients with common, single system diagnoses who require a hospital setting: Assessment Date of report: Oct 5, 2011 Date of achievement: Oct 5, 2011 Source: University of Maryland

My Entrustable Activities and Skills : Activity Details October 5, 2011 Peer group: Institution cohort Subcompetencies Supervision Description Methodology August 2011 Details Context: University of Maryland Pediatrics Residency Manage patients with common, single system diagnoses who require a hospital setting: Assessment Date of report: Oct 5, 2011 Date of achievement: Oct 5, 2011 Source: University of Maryland

My Entrustable Activities and Skills : Activity Details Level 2: Expected Elements: Tries to approach the physical examination with attention to the developmental and emotional state of the child by altering the flow to attend to those areas causing most distress towards the end, with some lapses due to: a) inexperience with specific examination maneuvers, b) strict adherence to an examination template for fear of errors of omission, and c) inexperience in judging triggers for distress based on developmental age. (continue) October 5, 2011 Peer group: Institution cohort Subcompetencies Supervision Context: University of Maryland Pediatrics Residency Manage patients with common, single system diagnoses who require a hospital setting: Assessment Date of report: Oct 5, 2011 Date of achievement: Oct 5, 2011 Source: University of Maryland

My Activity-specific Subcompetencies Manage patients with common, single system diagnoses who require a hospital setting PC: Perform complete and accurate physical examinations (psychomotor performance) Perform complete and accurate physical examinations (approach) Perform complete and accurate physical examinations (approach to focused exam) (continued) Context: University of Maryland Pediatrics Residency Date of report: Oct 5, 2011 Date of achievement: Oct 5, 2011 Source: University of Maryland

My Activity-specific Subcompetencies Manage patients with common, single system diagnoses who require a hospital setting PC: Make informed diagnostic & therapeutic decisions that result in optimal clinical judgment PC: Develop & carry out management plans ICS: Communicate effectively with patients, families, & the public, as appropriate, across a broad range of socioeconomic & cultural backgrounds Context: University of Maryland Pediatrics Residency Date of report: Oct 5, 2011 Date of achievement: Oct 5, 2011 Source: University of Maryland

My Entrustable Activities and Skills: Activity Details October 5, 2011 Peer group: Institution cohort Assessment Context: University of Maryland Pediatrics Residency Manage patients with common, single system diagnoses who require a hospital setting: Level of Supervision Date of report: Oct 5, 2011 Date of achievement: Oct 5, 2011 Source: University of Maryland

My Entrustable Activities and Skills: Activity Details Care of a healthy newborn, Awarded August 18, 2011 by Thelonius Smith October 5, 2011 Details Do we need to know who awarded responsibility? Peer group: Institution cohort Subcompetencies Supervision Trajectory view Assessment Context: University of Maryland Pediatrics Residency Date of report: Oct 5, 2011 Date of achievement: Oct 5, 2011 Source: University of Maryland

My Entrustable Activities and Skills: Activity Details Care of a healthy newborn, Awarded August 18, 2011 by Thelonius Smith October 5, 2011 Details Peer group: Institution cohort Assessment Level of Supervision Context: University of Maryland Pediatrics Residency Date of report: Oct 5, 2011 Date of achievement: Oct 5, 2011 Source: University of Maryland

My Educational Trajectory John Doe, December 8, 2009 http://www.example.edu/portfolio/1234 Year Quarter 1 (Jun - Aug) Quarter 2 (Sep - Nov) Quarter 3 (Dec – Feb) Quarter 4 (Mar – May) 05 – 06 06 – 07 07 – 08 08 – 09 02 03 01 02 Intravenous Catheter Insertion 01 02 01 02 01 02 Coursework leading to MD/primary degree Coursework beyond MD/primary degree Enrichment, priority Enrichment Date of report: Oct 5, 2011 Date of achievement: Dec 8, 2009 Source: University of Toronto

My Educational Trajectory John Doe, December 8, 2009 http://www.example.edu/portfolio/1234 Year Quarter 1 (Jun - Aug) Quarter 2 (Sep - Nov) Quarter 3 (Dec – Feb) Quarter 4 (Mar – May) 05 – 06 06 – 07 07 – 08 08 – 09 02 03 01 University of Toronto MD Program Institution: University of Toronto Degree: M.D. Length of program: 4 years Jun 2005 – May 2009 The MD program consists of a preclerkship and clerkship curriculum. The basic goal of the Preclerkship is to help students learn the necessary biomedical and humanistic knowledge and skills, and to develop the appropriate professional attitudes. In Clerkship, students participate as members of a working health care team. Author: University of Toronto ID: utoronto.ca/23456 Updated: 12/8/2009 See Curriculum Details 02 Intravenous Catheter Insertion 01 02 01 02 01 02 ADD slide showing what happens when you click on black bar,: description, then curriculum details ala bank statement Coursework leading to MD/primary degree Coursework beyond MD/primary degree Enrichment, priority Enrichment Date of report: Oct 5, 2011 Date of achievement: Dec 8, 2009 Source: University of Toronto

My Educational Trajectory John Doe, December 8, 2009 http://www.example.edu/portfolio/1234 Year Quarter 1 (Jun - Aug) Quarter 2 (Sep - Nov) Quarter 3 (Dec – Feb) Quarter 4 (Mar – May) 05 – 06 06 – 07 07 – 08 08 – 09 My Curriculum: University of Toronto Year 1 The Art and Science of Clinical Medicine 1 Brain and behavior Determinants of Community Health Metabolism and Nutrition Structure and Function Previous Next 02 03 01 University of Toronto MD Program Institution: University of Toronto Degree: M.D. Length of program: 4 years Jun 2005 – May 2009 The MD program consists of a preclerkship and clerkship curriculum. The basic goal of the Preclerkship is to help students learn the necessary biomedical and humanistic knowledge and skills, and to develop the appropriate professional attitudes. In Clerkship, students participate as members of a working health care team. Author: University of Toronto ID: utoronto.ca/23456 Updated: 12/8/2009 See Curriculum Details 02 Intravenous Catheter Insertion 01 02 01 02 01 02 Because both trajectory and curriculum are dated, the system displaying the data can bring that data together, provided the curriculum for the medical school is linked to the trajectory for the medical school. Coursework leading to MD/primary degree Coursework beyond MD/primary degree Enrichment, priority Enrichment Date of report: Oct 5, 2011 Date of achievement: Dec 8, 2009 Source: University of Toronto

My Educational Trajectory & Awarded Responsibilities John Doe, October 5, 2011 http://www.example.edu/portfolio/2234 Year Quarter 1 (Jun - Aug) Quarter 2 (Sep - Nov) Quarter 3 (Dec – Feb) Quarter 4 (Mar – May) 09 – 10 10 – 11 11 – 12 01 02 03 Care of a Healthy Newborn 01 01 Residency Training Additional Coursework beyond MD/primary degree Leave of absence Date of report: Oct 5, 2011 Date of achievement: Oct 5, 2011 Source: University of Maryland

My Curricula: University of Toronto* Year 1 The Art and Science of Clinical Medicine 1 Brain and behavior Determinants of Community Health Metabolism and Nutrition Structure and Function Year 2 The Art and Science of Clinical Medicine 2 Determinants of Community Health 2 Mechanisms, Manifestations, and Management of Disease Family Medicine Longitudinal Experience Date of report: October 5, 2011

My Curricula: University of Toronto (cont’d)* Year 3 Transition to Clerkship Surgery Clerkship Medicine Clerkship Emergency Medicine Clerkship Anesthesia Clerkship Ophthalmology Clerkship Otolaryngology Clerkship Year 3 cont’d Psychiatry Clerkship Paediatrics Clerkship Obstetrics and Gynecology Clerkship Family and Community Medicine Clerkship Portfolio course Integrated OSCE 1 Integrated OSCE 2 Date of report: October 5, 2011

The Art and Science of Clinical Medicine 1* Description: This course is organized in hospital-based groups of 5 to 6 students, and will take place 1/2 day each week of the academic year. Timing: September 1, 2005 – July 30, 2006 Academic Level: 1 Competencies: Medical Experts, Communicator, Manager, Scholar, Professional Post-conditions: Students deemed to have failed the course by the Board of Examiners will be required to repeat the course in the following academic year… Events: Lecture 1, Group clinic: Listening, Group Clinic: questioning techniques… Assessment results: Passed Date of report: October 5, 2011

The Art and Science of Clinical Medicine 1* Description: This course is organized in hospital-based groups of 5 to 6 students, and will take place 1/2 day each week of the academic year. Timing: September 1, 2005 – July 30, 2006 Academic Level: 1 Competencies: Medical Experts, Communicator, Manager, Scholar, Professional Post-conditions: Students deemed to have failed the course by the Board of Examiners will be required to repeat the course in the following academic year… Events: Lecture 1, Group clinic: Listening, Group Clinic: questioning techniques… Assessment results: Passed As Communicators, physicians effectively facilitate the doctor-patient relationship and the dynamic exchanges that occur before, during, and after the medical encounter. See full framework See assessment data Date of report: October 5, 2011

The Art and Science of Clinical Medicine 1* Description: This course is organized in hospital-based groups of 5 to 6 students, and will take place 1/2 day each week of the academic year. Timing: September 1, 2005 – July 30, 2006 Academic Level: 1 Competencies: Medical Experts, Communicator, Manager, Scholar, Professional Post-conditions: Students deemed to have failed the course by the Board of Examiners will be required to repeat the course in the following academic year… Events: Lecture 1, Group clinic: Listening, Group Clinic: questioning techniques… Assessment results: Passed My Competencies: CanMeds As Communicators, physicians effectively facilitate the doctor-patient relationship and the dynamic exchanges that occur before, during, and after the medical encounter. See full framework See assessment data Date of report: October 5, 2011

Date of report: Oct 5, 2011 Source: University of Toronto Medicine Clerkship* Description: Timing: September 25, 2007 – December 15, 2007 Academic Level: 3 Competencies: Medical Expert, Professional Events: Shadowing, History taking, Medical skills Assessment Results: Grade: Proficient Comments: Interacted exceptionally well with patients and staff. Benchmark to peers Date of report: Oct 5, 2011 Source: University of Toronto

Medicine Clerkship: Benchmark to Peers Date of report: Oct 5, 2011 Date of achievement: Dec 15, 2007

My Curricula: NBME* USMLE Step 1, May 15, 2007 Score: 230 Mean: 225 Standard deviation: 20 Source: NBME USMLE Step 2 CK, March 15, 2009 Score: 210 Mean: 225 Standard deviation: 20 Source: NBME Removed institutional OSCE data and subject exam as I think they would belong in u Toronto curriculum. NBME Obstetrics and Gynecology Subject Exam, April 15, 2008 Score: 76 Mean: 72.8 Standard Deviation: 7.9 Percentile: 69 Source: University of Toronto Date of report: Oct 5, 2011

Date of report: October 5, 2011 My Curricula: NBME* USMLE Step 2 CS, April 15, 2009 Pass Source: NBME USMLE Step 3, May 15, 2010 Score: 230 Mean: 225 Standard deviation: 20 Source: NBME Date of report: October 5, 2011

Date of report: October 5, 2011 My Curriculum: ABP* American Board of Pediatrics In Training Exam, July 14, 2009 Score: 60 Mean: 58 Cohort: US Level 1 trainees Standard deviation: 8 American Board of Pediatrics In Training Exam, July 14, 2010 Score: 72 Mean: 67 Cohort: US Level 2 trainees Standard deviation: 8 Date of report: October 5, 2011

Date of report: October 5, 2011 Other* Undergraduate transcript Undergraduate portfolio Links to undergraduate achievements are shown here. There are existing specifications for undergraduate transcript data that can be used, so we don’t need to go into details on that. Existing specifications for portfolios, like Leap2A, can be used for the undergraduate portfolio. The schema will be structured so that it can be extended with additional data. Date of report: October 5, 2011