Implementing and Evaluating an Early Clinical Experience Longitudinal Block for Medical Students Lia Pierson Bruner, M.D. Betsy Goebel Jones, Ed.D. Tommie W. Farrell, M.D. Kathryn McMahon, Ph.D.

Implementing A New Early Clinical Experience Previous Course – typical history/physical course mostly at end of 2nd year with hospitalized patients Partially in response to AAMC Dean’s report July, 2004, wanted experience that started earlier and was more ambulatory 2005-2006, started ambulatory participatory clinics in the first weeks of first year 2006-2007, added the second year experience and absorbed a course on ethics, professionalism and cultural competency (added monthly lectures/small group activities)

Early Clinical Experience Year 1 Monthly half-day clinics with a Master Clinical Teacher in an HSC clinic (in groups of 3-4 students). Monthly lectures (two hours) Monthly small group sessions with faculty facilitators (one to two hours) Portfolio assignments related to lecture & small group projects Two OSCEs: Fall and Spring

Early Clinical Experience Year 2 Monthly half-day clinics with a community preceptor (one student with one preceptor) Monthly lectures (two hours) Monthly small group sessions with faculty facilitators and student-led discussions (two hours) Portfolio assignments related to course topics and clinical experiences Workshops to teach advanced physical exam skills Two OSCEs: Fall and Spring Involves similar activities as the first year, but with a community mentor instead of a master clinical teacher. Students practice and expand skills from first year.

Early Clinical Experience II: Adjustments for 2007-2008 Divided labor: Separate block directors for ECE I and ECE II and increased support staff Increased relevance of content to Step I Small group discussions led by students rather than faculty Emphasized verbal presentations as a skill (and had community mentors give feedback on verbal and written notes) Used workshops to teach advanced physical exam skills

ECE II Program Evaluation Mid-Year student satisfaction data 2006-07: 68 responses (49%) 2007-08: 139 responses (97%) End-of-Year student satisfaction data 2006-07: 24 responses (17%) Small-Group faculty satisfaction data 2007-08: 13 responses (93%) Note the marked increased in response rate this year—this was a result of adding professionalism criteria to the grading that included completion of course evaluations

Community Mentor Specialty Practice Location: Lubbock: 54% TTUHSC 32% Outlying 11% Other 3% 30% family medicine mentors. Student recruited mentors were a result of having difficulty getting 135 community mentors Mentor Assignment: Student recruited: 22% Requested specialty 36% Random assignment 38% Other 4%

Rating of Community Mentor Experiences

Assessment of Lectures Highest Rated There were some differences between what they liked and what they thought would be useful. In general, the more practical topics were rated as more useful

Assessment of Lectures Lowest Rated Lowest ones tended to be the cultural competency curriculum

Physical Examination Workshops Pretty uniformly liked and highly rated (second most popular part of the course second only to the community mentor experience)

Small Group Topics Student vs. Faculty Ratings Faculty had higher rating for both materials and usefulness to practice for every category (?more perspective)

Small Group Topics Student vs. Faculty Ratings Faculty had higher rating for both materials and usefulness to practice for every category (?more perspective)

Small Group Evaluation Student vs. Faculty Ratings Students not all as keen on student leaders as the faculty were (possibly for obvious reasons—more work for students and less work for faculty) Widely discordant on value of portfolios—students largely felt they were not very valuable, while faculty thought they were quite valuable

Portfolio Assignments Student vs. Faculty Ratings Again, students seemed to like the more practical topics slightly better

Overall ECE II Small Group Evaluation Student vs. Faculty Ratings Faculty in general had a more favorable impression. Student estimation of the categories decreased slightly in all areas in the second part of the year (closer to Step I)

Overall ECE II Evaluation Relationship to Other Variables Variables that correlate most positively with students’ overall course assessment (mid-yr) The ECE II provides an engaging opportunity to learn (r=.697) The ECE II is relevant to my future career (r=.677) The ECE II addresses the needs of a diverse student population (r=.660) The ECE II promotes critical thinking (r=.638) Overall assessment of the small group experience (r=.586) Overall assessment of physical exam workshops (r=.541) Year-end correlation between assessment of the community mentor experience & ECEII overall: r=.346 (p<.01) We tried to look for predictors of overall course satisfaction. Somewhat surprisingly there was less correlation between the community mentor experience and the overall ECE II evaluation

ECE II Typical Student Comments: Small-Group Experiences I truly enjoy my small group members and facilitator. Everyone is open to others opinions and sadly that is not found in all small groups. The ECE experience as a whole truly sets Tech apart, and allows me to remember WHY I am working so hard. We like to hear what other students have to say, but we are starving for the advice of physicians, also. Thanks for the opportunity to do both. I like the experience, and the group stuff. Our group has really bonded and I like practicing and talking in that safe environment

ECE II Typical Student Comments: Physical Exam Workshops The workshops are so useful in learning clinical skills, especially the cardio workshop because it put students on the spot and simulated year 3 situations. The Cardiology Workshop designed by Dr. B was amazing. It was one of those days when I felt in love with TTUHSC! I actually learned what the heart murmurs actually were and what a systolic vs. diastolic is. I felt that the workshops have been really valuable and have made me feel more confident during clinic. Overall very positive feedback

ECE II Typical Student Comments: Lectures Most of the lectures are informative & relative (abuse, law, sexual history) Dr. C is GREAT! For the most part, the topics in lectures are very helpful. I don't like having the lectures sprinkled in here and there, I can't focus on the material because I'm worried about my basic sciences courses The lectures that directly address the physical exam are very useful, but lectures focusing on more esoteric things are useless, and basically a waste of time. Lecture feedback more mixed

ECE II Typical Student Comments: Challenges and Complaints I think there would be more attendance to class lectures if they were not scheduled the week of our tests. Lose the portfolios. I understand the need for reflection, but I don't think portfolios are the way. Portfolios promote writing the "right" thing. Small groups are much better at helping us reflect, especially listening to other people’s viewpoints. I feel ECE should schedule meeting times (exams/small groups/lectures) on weeks we do not have an exam in another block and I don’t feel the portfolios are useful I struggle with the relevance of some of the lecture topics to board exam preparation and find the class distracts from basic science study. Consider making the class seminar-based with required attendance but no grade.

ECE II Typical Student Comments: Advice from students If you go with clinical topics, or can tie social justice to a care setting, you'll get a better response. We want to be doctors, not social workers. This has worked well in small groups, where the students generally will address common interests. You may get some criticism about the course, but stay with it. This is very important. There are still a lot of doctors (and students) out there who are really bad at dealing with people.

End-Year Evaluation Overall Ratings Overall, this showed the relative popularity of the community mentor experiences and physical exam workshops with lectures less appreciated.

My overall assessment of the Early Clinical Experience II Block Mid-Year Evaluation End-Year Evaluation

What’s next for ECE II Try to connect better with community mentors—to show appreciation and work on retention (administrator to visit each) Add abdominal exam workshop Think about portfolios Try to improve scheduling with other blocks