1. Assessing the student experience: creating a culture of assessment in undergraduate and professional education.”
Presented at the Annual NEASC Meeting
December 9, 2016
Deborah Harmon Hines, PhD
Vice Provost for School Services
University of Massachusetts Medical School

2. UMMS Competencies for Medical Education
We strives to produce graduates who will become caring healers both by assuring that they possess the requisite knowledge and skills and by strengthening their natural talents and desire to care for others. The multiple roles of the physician as healer, and the associated competencies that graduating medical students must demonstrate, form the basis for a new way of organizing what is taught, how it is taught and the methods for evaluating student performance at UMMS. They embody the Medical School’s educational philosophy and the distinctive attributes of its faculty and students.
These six attributes or roles, which form the cornerstone for a redefining of the Medical School’s educational objectives, are Physician as Professional, Scientist, Communicator, Clinical Problem Solver, Patient & Community Advocate, and Person.

3. Student Assessment
Begins with performance on internal/course examinations
Evaluations also done of small group work
Foundations of Medicine 1&2 (Y1&2) on CR/NCR
Monthly Evaluation Board meetings
Core Clinical Experiences (Y3) and Advanced Studies (Y4) on BEP, EP, AEP, O, F, I
Clinical courses are also assessed using OSCEs (objective structured clinical examinations)
Monthly Evaluation Boards meetings
Both Academic Evaluation Board meetings attended by ALL course coordinators, Associate and Assistant Deans for SA, FA, Registrar and Learning Specialist,
2016 NEASC ANNUAL MEETING

4. Continues with Student Assessment of lectures, lecturers and courses
25 students are randomly selected to assess each lecture and lecturer
Information compiled by IREA* and given to Course Coordinators and individual faculty members
IREA = Institutional Research, Evaluation and Assessment
2016 NEASC ANNUAL MEETING

5. Student Performance on Shelf and USMLE
“Shelf” exams are provided by the US Medical Licensure Exam (USMLE) in subject areas
USMLE is administered in three steps
Step 1, at the end of the FOM2
Step 2, at the end of CCE
Clinical skills (2CS)
Clinical knowledge (2CK)
Needed for conditional license to practice
Step 3, between residency Y1-Y2
Needed for full licensure
Outcomes normed against all who take each section of the USMLE
2016 NEASC ANNUAL MEETING

6. All topics, titles, key words and placed in the curriculum data base
Able to see where and when a topic, title or key word is in the curriculum
Allows adjustments for too much redundancy or for missing information
2016 NEASC ANNUAL MEETING

7. Graduate Questionnaire (GC)
Administered by the AAMC
Each medical school graduate is given this questionnaire to assess the entire medical school experience
Covers: didactic information in all subject areas and clinical care. i.e.
Do you feel you learned what was needed to provide competent care about HIV/AIDS?
How to treat LGBTQ patients?
Also covers: appropriate treatment of learners, FA, discrimination, etc.
Were you ever made to feel belittled?
Was FA easy to access? Were all of your questions answered?
Normed against all medical schools.
UMMS is above the mean on 95% of the questions. We still have work to do.
2016 NEASC ANNUAL MEETING

8. Residency Program Director’s Questionnaire
Administered by the AAMC
Given to Residency Program Directors each year.
Asked to evaluate preparation of each resident and correlated with the resident’s medical school.
2016 NEASC ANNUAL MEETING

9. Student Support Associate Dean for Student Affairs
Collects all grades and presents to Academic Evaluation Boards
Assistant Dean for Student Advising assigns each student to a faculty advisor
Learning Community House mentors
Resident Learning Specialist
Course review sessions
USMLE Review Sessions
Learning Communities
Physician mentors
Peer mentors
Upperclassmen mentors
Student ADA services for those who need it
2016 ANNUAL NEASC MEETING

10. Final thoughts on 360 degree evaluation
This system of evaluation has taken years to put into place. It was built one ‘brick’ at a time. The cost/individual and the impact on patients’ lives made a compelling rationale for this system of evaluation.
Student Support is key.
Internal assessments
- Of students be examinations and OSCEs
- Of courses and faculty by students
- Curriculum data base
- Everyone must be at the table
External assessments
- Shelf exams
- USMLE
- Graduate Questionnaire
- Residency Program Directory Questionnaire