1. You can look at the same data through different lenses…
EPAs/ Competencies
Curricula
Educational Trajectory
Assessment Data
Educational Achievement Data

2. Specification available
Competency Framework
Curriculum Report
Competency Object
Competency Object
Competency Object
Competency Object
Competency Object
Other competency framework
Competency Object
Competency Object
Competency Object 1 2 3 1 2 3 1 2 3
Mapping (SKOS)
Performance Framework
Educational Achievement data
Educational Trajectory data
Specification available
Development pending
Existing resources

3. My Educational Achievement Data *
Available Data
Available Actions
My Competencies
CanMeds ACGME
My Entrustable Activities and Skills
My Trajectories
My Curricula
University of Toronto University of Maryland NBME ABP
Other
Report my data
To residency program director To certifying board
Share with mentor
NOTE: reports to external reviewers and sharing with mentor are shown as options here to get the idea across that the data may be used for multiple purposes. Special interfaces would allow the learner to report data or select data for reporting to a mentor. This presentation does not detail what those interfaces might look like but acknowledges that they would be necessary. The rest of this presentation focuses on the learner reviewing their own data for self assessment and improvement.
Date of report: October 5, 2011

4. My Competencies: CanMeds
Performance levels
1 – Novice
2 – Intermediate
3 – Expert
Related Activity:
University of Toronto MD Program
Date of report: Oct 5, 2011 Date of achievement: May 30, 2008 Source: University of Toronto

5. My Competencies: CanMeds
As Communicators, physicians effectively facilitate the doctor-patient relationship and the dynamic exchanges that occur before, during, and after the medical encounter.
See full framework
Performance levels
1 – Novice
2 – Intermediate
3 – Expert
Related Activity:
University of Toronto MD Program
Date of report: Oct 5, 2011 Date of achievement: May 30, 2008 Source: University of Toronto

6. My Competenciess: CanMeds
Function effectively as consultants, integrating all of the CanMEDS Roles to provide optimal, ethical and patient-centered medical care See full framework
Performance levels
1 – Novice
2 – Intermediate
3 – Expert
Related Activity:
University of Toronto MD Program
Date of report: Oct 5, 2011 Date of achievement: May 30, 2008 Source: University of Toronto

7. My Competencies: CanMeds
Performance levels
Novice: demonstrates behaviors
expected of a first year trainee. Requires close supervision.
Intermediate: demonstrates behaviors expected of a trainee after 2 years. Requires some supervision.
Expert: demonstrates behaviors expected of a trainee at the completion of training. No supervision required.
Performance levels
1 – Novice
2 – Intermediate
3 – Expert
Related Activity:
University of Toronto MD Program
Date of report: Oct 5, 2011 Date of achievement: May 30, 2008 Source: University of Toronto

8. My Competencies: ACGME
Performance levels
1 – Beginner
2 – Adv Beginner
3 – Competent
4 – Adv Competent
5 – Mastered
Related Activity:
University of Maryland Pediatrics Residency
Date of report: Oct 5, 2011 Date of achievement: Oct 5, 2011 Source: University of Maryland

9. My Competency-based Achievements: Patient Care
Performance levels
1 – Beginner
2 – Adv Beginner
3 – Competent
4 – Adv Competent
5 – Mastered
Learner provided:
Spanish asthma brochure
Notes
My notes (private): Review procedure list and discuss action plan with mentor
(Private comment) CB: There are some additional procedures I’d like you to work on – let’s discuss
Date of report: Oct 5, 2011 Date of achievement: Oct 5, 2011 Source: University of Maryland

10. My Competencies: Gathering Essential Information
Details
Peer group: Institution cohort
Context: University of Maryland Pediatrics Residency
Date of report: Oct 5, 2011 Date of achievement: Oct 5, 2011 Source: University of Maryland

11. My Competency-based Achievements: Gathering Essential Information
Benchmark to Peers
Details
Date of report: Oct 5, 2011 Date of achievement: Sep 10, 2011 Source: University of Maryland

12. My Competency-based Achievements: Gathering Essential Information – Benchmark to Peers
Peer group: Institution
Date of report: Oct 5, 2011 Date of achievement: Sep 10, 2011 Source: University of Maryland

13. My Competencies: Gathering Essential Information – Details*
Title: Pediatric Level Degree Feedback
Duration: 2 Hours
Description: Residents receive feedback from supervisors, peers, nurses, and patients.
Keywords: Inservice training
Interprofessional? No
Associated Competencies: Gathering Essential Information, Professionalism
Resource: Real Patient
Assessment Method: Multisource feedback
Purpose: Formative assessment
Assessment results: Professionalism – 75, Communication – 70, Patient care – 78.
View associated course
Date of report: Oct 5, 2011 Date of achievement: Sep 10, 2011 Source: University of Maryland

14. My Competencies: Gathering Essential Information – Details*
Title: Pediatric Level Degree Feedback
Duration: 2 Hours
Description: Residents receive feedback from supervisors, peers, nurses, and patients.
Keywords: Inservice training
Interprofessional? No
Associated Competencies: Gathering Essential Information, Professionalism
Resource: Real Patient
Assessment Method: Multisource feedback
Purpose: Formative assessment
Assessment results: Professionalism – 75, Communication – 70, Patient care – 78.
View associated course
Title: Pediatric Residency level 3
Description: The resident begins this sequence midway through their training.
Duration: 12 months
Academic level: 3
Competencies: Patient Care, Professionalism
Includes: (list of instruction and assessment events or sub-courses)
Date of report: Oct 5, 2011 Date of achievement: Sep 10, 2011 Source: University of Maryland

15. My Competency-based Achievements: Medical Knowledge
 Identifies evidence-based resources to answer basic and clinical science questions.
Met 
With
Concerns □
Did Not Meet
Insuff Evid
Evidence:
Brief Summary:
 Applies core concepts of pathophysiology to new problems in the basic and clinical sciences relevant to medicine.
 Identifies and acknowledges gaps in knowledge and develops and implements plans to correct.
 Achieves breadth and depth of knowledge in the curricular threads (eg. physiology, pharmacology, etc.).
Justify your decision
Medical Knowledge Competency
From Elaine Dannefer:
Under evidence, reviewers of the portfolio note the important pieces of evidence related to the competency standard that have informed their judgment.  For example if they note in their summary  that student has excelled in research they might put under evidence –published in Nature, or see research preceptor Ref #10. 
The  overall decision is made for the competency, not the standard.  However, not all standards are of equal importance, so in some competencies, not meeting a particular standard might result in not meeting the competency even though they meet the other 3 standards, so it’s important to make judgments about each standard based on a careful review of the evidence.  Although this may seem very subjective, there are multiple pieces of evidence that are triangulated to inform judgments by trained reviewers.
This documentation is also used by the reviewers of a portfolio to write a letter to the student that provides very specific feedback –  and if I have questions when I review letters, it’s helpful.
Date of report: Oct 5, 2011 Date of achievement: Oct 5, 2011 Source: University of Maryland
Cleveland Clinic Lerner College of Medicine of Case Western Reserve University

16. My Entrustable Activities and Skills*
Status
Activity
Summary
Related Curricula
Arterial puncture for blood gas analysis
Entrustment
Details
Clinical skills OSCE, U Toronto
Intravenous catheter insertion
Skills and thrills, U Toronto
. . .
Manage patients with common, single system diagnoses who require a hospital setting
In progress
Infant/Toddler Rotation, U Maryland
Care of a healthy newborn
Nursery rotation, U Maryland
We probably need to add some sort of flag for entrustment to the data model. Referring to a level of supervision is inadequate as people may come up with differing schemes for describing supervisory levels.
Date of report: Oct 5, 2011 Date of achievement: Various Source: Various

17. My Entrustable Activities and Skills*
Intravenous Catheter Insertion
Exam: Clinical Skills OSCE Exam 1, May 1, Passed
Additional Criteria: Satisfactory completion of 4 intravenous catheter insertions as determined by supervising faculty member.
Description: Learners are required to demonstrate satisfactory skills on simulators and real patients.
Methodology: Clinical performance rating/checklist
Context: University of Toronto MD Program
Date of report: Oct 5, 2011 Date of achievement: Jun 1, 2008 Source: University of Toronto

18. My Entrustable Activities and Skills: Activity Details
October 5, 2011
Peer group: Institution cohort
Subcompetencies
Supervision
Context: University of Maryland Pediatrics Residency
Manage patients with common, single system diagnoses who require a hospital setting: Assessment
Date of report: Oct 5, 2011 Date of achievement: Oct 5, 2011 Source: University of Maryland

19. My Entrustable Activities and Skills : Activity Details
October 5, 2011
Peer group: Institution cohort
Subcompetencies
Supervision
Description
Methodology
August 2011 Details
Context: University of Maryland Pediatrics Residency
Manage patients with common, single system diagnoses who require a hospital setting: Assessment
Date of report: Oct 5, 2011 Date of achievement: Oct 5, 2011 Source: University of Maryland

20. My Entrustable Activities and Skills : Activity Details
Level 2:
Expected Elements:
Tries to approach the physical examination with attention to the developmental and emotional state of the child by altering the flow to attend to those areas causing most distress towards the end, with some lapses due to: a) inexperience with specific examination maneuvers, b) strict adherence to an examination template for fear of errors of omission, and c) inexperience in judging triggers for distress based on developmental age. (continue)
October 5, 2011
Peer group: Institution cohort
Subcompetencies
Supervision
Context: University of Maryland Pediatrics Residency
Manage patients with common, single system diagnoses who require a hospital setting: Assessment
Date of report: Oct 5, 2011 Date of achievement: Oct 5, 2011 Source: University of Maryland

21. My Activity-specific Subcompetencies Manage patients with common, single system diagnoses who require a hospital setting
PC: Perform complete and accurate physical examinations (psychomotor performance)
Perform complete and accurate physical examinations (approach)
Perform complete and accurate physical examinations (approach to focused exam) (continued)
Context: University of Maryland Pediatrics Residency
Date of report: Oct 5, 2011 Date of achievement: Oct 5, 2011 Source: University of Maryland

22. My Activity-specific Subcompetencies Manage patients with common, single system diagnoses who require a hospital setting
PC: Make informed diagnostic & therapeutic decisions that result in optimal clinical judgment
PC: Develop & carry out management plans
ICS: Communicate effectively with patients, families, & the public, as
appropriate, across a broad range of socioeconomic & cultural backgrounds
Context: University of Maryland Pediatrics Residency
Date of report: Oct 5, 2011 Date of achievement: Oct 5, 2011 Source: University of Maryland

23. My Entrustable Activities and Skills: Activity Details
October 5, 2011
Peer group: Institution cohort
Assessment
Context: University of Maryland Pediatrics Residency
Manage patients with common, single system diagnoses who require a hospital setting: Level of Supervision
Date of report: Oct 5, 2011 Date of achievement: Oct 5, 2011 Source: University of Maryland

24. My Entrustable Activities and Skills: Activity Details
Care of a healthy newborn, Awarded August 18, 2011 by Thelonius Smith
October 5, 2011
Details
Do we need to know who awarded responsibility?
Peer group: Institution cohort
Subcompetencies
Supervision
Trajectory view
Assessment
Context: University of Maryland Pediatrics Residency
Date of report: Oct 5, 2011 Date of achievement: Oct 5, 2011 Source: University of Maryland

25. My Entrustable Activities and Skills: Activity Details
Care of a healthy newborn, Awarded August 18, 2011 by Thelonius Smith
October 5, 2011
Details
Peer group: Institution cohort
Assessment
Level of Supervision
Context: University of Maryland Pediatrics Residency
Date of report: Oct 5, 2011 Date of achievement: Oct 5, 2011 Source: University of Maryland

26. My Educational Trajectory
John Doe, December 8, 2009
Year
Quarter 1 (Jun - Aug)
Quarter 2 (Sep - Nov)
Quarter 3 (Dec – Feb)
Quarter 4 (Mar – May)
05 – 06
06 – 07
07 – 08
08 – 09 02 03 01 02
Intravenous Catheter Insertion 01 02 01 02 01 02
Coursework leading to MD/primary degree
Coursework beyond MD/primary degree
Enrichment, priority
Enrichment
Date of report: Oct 5, 2011 Date of achievement: Dec 8, 2009 Source: University of Toronto

27. My Educational Trajectory
John Doe, December 8, 2009
Year
Quarter 1 (Jun - Aug)
Quarter 2 (Sep - Nov)
Quarter 3 (Dec – Feb)
Quarter 4 (Mar – May)
05 – 06
06 – 07
07 – 08
08 – 09 02 03 01
University of Toronto MD Program
Institution: University of Toronto
Degree: M.D. Length of program: 4 years
Jun 2005 – May 2009
The MD program consists of a preclerkship and clerkship curriculum. The basic goal of the Preclerkship is to help students learn the necessary biomedical and humanistic knowledge and skills, and to develop the appropriate professional attitudes. In Clerkship, students participate as members of a working health care team.
Author: University of Toronto
ID: utoronto.ca/23456
Updated: 12/8/2009
See Curriculum Details 02
Intravenous Catheter Insertion 01 02 01 02 01 02
ADD slide showing what happens when you click on black bar,: description, then curriculum details ala bank statement
Coursework leading to MD/primary degree
Coursework beyond MD/primary degree
Enrichment, priority
Enrichment
Date of report: Oct 5, 2011 Date of achievement: Dec 8, 2009 Source: University of Toronto

28. My Educational Trajectory
John Doe, December 8, 2009
Year
Quarter 1 (Jun - Aug)
Quarter 2 (Sep - Nov)
Quarter 3 (Dec – Feb)
Quarter 4 (Mar – May)
05 – 06
06 – 07
07 – 08
08 – 09
My Curriculum: University of Toronto
Year 1
The Art and Science of Clinical Medicine 1
Brain and behavior
Determinants of Community Health
Metabolism and Nutrition
Structure and Function
Previous Next 02 03 01
University of Toronto MD Program
Institution: University of Toronto
Degree: M.D. Length of program: 4 years
Jun 2005 – May 2009
The MD program consists of a preclerkship and clerkship curriculum. The basic goal of the Preclerkship is to help students learn the necessary biomedical and humanistic knowledge and skills, and to develop the appropriate professional attitudes. In Clerkship, students participate as members of a working health care team.
Author: University of Toronto
ID: utoronto.ca/23456
Updated: 12/8/2009
See Curriculum Details 02
Intravenous Catheter Insertion 01 02 01 02 01 02
Because both trajectory and curriculum are dated, the system displaying the data can bring that data together, provided the curriculum for the medical school is linked to the trajectory for the medical school.
Coursework leading to MD/primary degree
Coursework beyond MD/primary degree
Enrichment, priority
Enrichment
Date of report: Oct 5, 2011 Date of achievement: Dec 8, 2009 Source: University of Toronto

29. My Educational Trajectory & Awarded Responsibilities
John Doe, October 5, 2011
Year
Quarter 1 (Jun - Aug)
Quarter 2 (Sep - Nov)
Quarter 3 (Dec – Feb)
Quarter 4 (Mar – May)
09 – 10
10 – 11
11 – 12 01 02 03
Care of a Healthy Newborn 01 01
Residency Training
Additional Coursework beyond MD/primary degree
Leave of absence
Date of report: Oct 5, 2011 Date of achievement: Oct 5, 2011 Source: University of Maryland

30. My Curricula: University of Toronto*
Year 1
The Art and Science of Clinical Medicine 1
Brain and behavior
Determinants of Community Health
Metabolism and Nutrition
Structure and Function
Year 2
The Art and Science of Clinical Medicine 2
Determinants of Community Health 2
Mechanisms, Manifestations, and Management of Disease
Family Medicine Longitudinal Experience
Date of report: October 5, 2011

31. My Curricula: University of Toronto (cont’d)*
Year 3
Transition to Clerkship
Surgery Clerkship
Medicine Clerkship
Emergency Medicine Clerkship
Anesthesia Clerkship
Ophthalmology Clerkship
Otolaryngology Clerkship
Year 3 cont’d
Psychiatry Clerkship
Paediatrics Clerkship
Obstetrics and Gynecology Clerkship
Family and Community Medicine Clerkship
Portfolio course
Integrated OSCE 1
Integrated OSCE 2
Date of report: October 5, 2011

32. The Art and Science of Clinical Medicine 1*
Description: This course is organized in hospital-based groups of 5 to 6 students, and will take place 1/2 day each week of the academic year.
Timing: September 1, 2005 – July 30, 2006
Academic Level: 1
Competencies: Medical Experts, Communicator, Manager, Scholar, Professional
Post-conditions: Students deemed to have failed the course by the Board of Examiners will be required to repeat the course in the following academic year…
Events: Lecture 1, Group clinic: Listening, Group Clinic: questioning techniques…
Assessment results: Passed
Date of report: October 5, 2011

33. The Art and Science of Clinical Medicine 1*
Description: This course is organized in hospital-based groups of 5 to 6 students, and will take place 1/2 day each week of the academic year.
Timing: September 1, 2005 – July 30, 2006
Academic Level: 1
Competencies: Medical Experts, Communicator, Manager, Scholar, Professional
Post-conditions: Students deemed to have failed the course by the Board of Examiners will be required to repeat the course in the following academic year…
Events: Lecture 1, Group clinic: Listening, Group Clinic: questioning techniques…
Assessment results: Passed
As Communicators, physicians effectively facilitate the doctor-patient relationship and the dynamic exchanges that occur before, during, and after the medical encounter.
See full framework
See assessment data
Date of report: October 5, 2011

34. The Art and Science of Clinical Medicine 1*
Description: This course is organized in hospital-based groups of 5 to 6 students, and will take place 1/2 day each week of the academic year.
Timing: September 1, 2005 – July 30, 2006
Academic Level: 1
Competencies: Medical Experts, Communicator, Manager, Scholar, Professional
Post-conditions: Students deemed to have failed the course by the Board of Examiners will be required to repeat the course in the following academic year…
Events: Lecture 1, Group clinic: Listening, Group Clinic: questioning techniques…
Assessment results: Passed
My Competencies: CanMeds
As Communicators, physicians effectively facilitate the doctor-patient relationship and the dynamic exchanges that occur before, during, and after the medical encounter.
See full framework
See assessment data
Date of report: October 5, 2011

35. Date of report: Oct 5, 2011 Source: University of Toronto
Medicine Clerkship*
Description: Timing: September 25, 2007 – December 15, 2007 Academic Level: 3 Competencies: Medical Expert, Professional Events: Shadowing, History taking, Medical skills Assessment Results: Grade: Proficient Comments: Interacted exceptionally well with patients and staff. Benchmark to peers
Date of report: Oct 5, 2011 Source: University of Toronto

36. Medicine Clerkship: Benchmark to Peers
Date of report: Oct 5, 2011 Date of achievement: Dec 15, 2007

37. My Curricula: NBME*
USMLE Step 1, May 15, 2007 Score: 230 Mean: 225 Standard deviation: 20 Source: NBME
USMLE Step 2 CK, March 15, 2009 Score: 210 Mean: 225 Standard deviation: 20 Source: NBME
Removed institutional OSCE data and subject exam as I think they would belong in u Toronto curriculum.
NBME Obstetrics and Gynecology Subject Exam, April 15, 2008 Score: 76 Mean: 72.8 Standard Deviation: 7.9 Percentile: 69 Source: University of Toronto
Date of report: Oct 5, 2011

38. Date of report: October 5, 2011
My Curricula: NBME*
USMLE Step 2 CS, April 15, 2009 Pass Source: NBME
USMLE Step 3, May 15, 2010 Score: 230 Mean: 225 Standard deviation: 20 Source: NBME
Date of report: October 5, 2011

39. Date of report: October 5, 2011
My Curriculum: ABP*
American Board of Pediatrics In Training Exam, July 14, 2009 Score: 60 Mean: 58 Cohort: US Level 1 trainees Standard deviation: 8
American Board of Pediatrics In Training Exam, July 14, 2010 Score: 72 Mean: 67 Cohort: US Level 2 trainees Standard deviation: 8
Date of report: October 5, 2011

40. Date of report: October 5, 2011
Other*
Undergraduate transcript
Undergraduate portfolio
Links to undergraduate achievements are shown here. There are existing specifications for undergraduate transcript data that can be used, so we don’t need to go into details on that. Existing specifications for portfolios, like Leap2A, can be used for the undergraduate portfolio. The schema will be structured so that it can be extended with additional data.
Date of report: October 5, 2011