1. Deborah Jones MD MPH, Michael Devlin MD, Michelle Hall MA
A Collaborative Wikispace to Teach Effective Socio-cultural Inquiry to Third Year Students
Describe background of clerkship – mix of states/pt populations/failicities/specialties
Deborah Jones MD MPH, Michael Devlin MD, Michelle Hall MA
Society of Teachers of Family Medicine
New Orleans, LA May 1, 2011 Annual Spring Conference 1

2. Goals of this session
Describe changes made in how students engage in a task to reflect in writing on the socio-cultural factors that impact patient care and the impact of those changes on outcomes
Share lessons learned, challenges, and questions unanswered

3. Clerkship goal: teach culturally competent care
Improve patient care through enhanced student understanding of the socio-cultural factors that affect patient care1,2
Reflect on improving communication between patients and doctors
Demonstrate awareness of the socio-cultural factors that affect patient care
Explore patients’ beliefs about health and illness, values and behaviors
How should we do this without meeting as a group?
Students at this medical school journey through their major clinical (third) year together in the same group of students (n=15) throughout the year. How can we tap into this experiential diversity and still ensure standardization about two of our important course objectives,
1) Address relevant psychosocial and cultural issues that impact on care.
2) Counsel and educate patients and their families in a culturally sensitive manner
1Wiener SJ et al, Contextual Errors and Failues in Individualizing Patient Care Ann Intern Med 2010;153:69-75
2Street RL, Haidet P. How well do doctors know their patients? Factors affecting physician understanding of patients’ health beliefs.JGIM 2011;26:21-7

4. Socio-cultural exercise
After doing background reading1 and interviewing a patient with a structured questionnaire to probe patient about their experience of their illness, what should they do?
Submit individual, private reflection to faculty?
Straightforward task
Literature supported2
Mixed student reviews
Heavy lifting by faculty
Sometimes sub-par student effort anecdotally
No peer learning or interaction
Collaborate with student at another site on wikispace?
Blogs have been tried successfully3
Wiki too complicated?
Not private enough?
Would tap into variety in experiences?
Raise bar for expectations?
Blogs have been utilized in similar settings, studies I am aware with involve reflection on profressionalism
1Carrillo JE et al.Cross-cultural primary care: a patient-based approach. Ann Intern Med 1999;130:
2Wald HS et al. Reflecting on reflections: enhancement of medical education curriculum with structured field notes and guided feedback. Acad Med 2009;84:
3Fischer MA et al. Comparison of blogged and written reflections in two medicine clerkships. Med Educ 2011;45:

5. Our new idea in Fall of 2009
Increase depth of student reflection on their patient’s model of disease and experience of chronic illness through the use of a collaborative wikispace.

6. 2009-2010 academic year intervention:
July-December 2009:
January-June 2010: Wiki
Students in clerkship 77 82
Post-interview assignment
Record findings, and reflect on your own, send to faculty
Record findings, reflect and collaborate with your partner in wikispace
Feedback from reader
Via , lengthy
Via wikispace; concise
Viewed by
Faculty
Rotation group (password protected)
Survey responses 61
Submissions reviewed 30 39
Describe limitations to old system

7. Analysis methods Student reaction to the exercise:
Anonymous end-of-clerkship electronic survey
Descriptive data: what did they write about?
Themes discussed in submissions recorded and tabulated
Qualitative data: How reflective were the submissions?
Submissions coded with a peer reviewed, previously published reflective analysis tool.
IRB approval obtained

8. Student reaction – no difference
No difference by Fisher’s exact tests
If you will remember, these are the three fundamental objectives we wanted to achieve through this exercise. The students answered on a likert scale how much they agreed with each of these statements. Blue and red are strongly agree and agree, so those are the most important colors to look at. As you can see, there isn’t much difference between these two modalities – wiki is on the left, /old system on the right. No difference was detected by the Fischer’s exact test.
The exercise helped me to reflect on improving communication between my patients and me.
The exercise increased my awareness of the sociocultural factors that affect patient care.
The exercise allowed me to explore patient's beliefs about health and illness, values and behavior.

9. Student reaction – additional

10. Qualitative assessment of writing: The Reflective Skills Worksheet
A Semi-Structured Approach for Critiquing, Framing and Delivering Feedback in Response to Learners' Narrative Clinical Reflections1
Modified slightly for our purposes
Scored 0, 1 or 2 in each category:
Description: clarity in building a story
Comparative: inclusion of multiple perspectives
Critical: explicit recognition of learning outcomes
Personal: personal engagement
1Mutnick A et al. MedEdPORTAL ID#: 8270

11. Reflection scoring
Faculty simultaneously scored batches of 5 submissions and reached agreement on any discrepancies
Scoring reached 75% agreement after first 15 submissions
Thereafter, each submission scored by one of two faculty members
Convenience sample of 30 anonymous individual essays, 37 anonymous wiki entries reviewed
This was clearly not a blinded study – no way to mask which modality a submission was

12. Results – student reflection in all categories
The chisquare test for difference among the total scores in all four categories found that there was a difference between the wiki and ed submissions.
p=0.047

13. Results – Student Reflection*
The chisuare test for difference It is not surprising they are comparing more in the wikispace – that is what they are kind of forced to do in the wikispace.
*chi-sq, p=0.022

14. On average, 1.3 more themes raised in wikispace (p=0.038)

15. Summary of Findings: Limitations:
Setting up the wiki and inviting members is fairly straightforward!
Students were equally enthusiastic in both modalities about the value of the exercise
Student descriptions of patients’ stories in wikispace tended to be more in depth, and involved more comparing and contrasting models of disease
Student reaction to this exercise may not be congruent with their performance or skill demonstrated.
Students read each other’s work, even if they don’t have to
A small % of students do not feel comfortable writing in a wikispace
Faculty happy with change
Limitations:
Un-blinded study
Not controlled for time of year or type of submission
Qualitative tool is not a perfect measure of reflection
They are just doing what we told them to do

16. Challenges Locating partners to create and maintain a wikispace
Ensuring faculty give timely feedback
Balancing a sense of privacy and safety vs openness and sharing

17. Questions
Is it the public nature, the collaboration, or both in a wikispace that may change student writing?
Are we sure that better writing = better reflection?
If personal engagement, and commitment to change are important goals of this writing assignment, how can we increase scores in these domains within a wikispace?
How can we get faculty and students to interact more on the wikispace?
How can we fit this wikispace into a longitudinal student portfolio to encompass all four years?

18. Thank you
Pablo Joo, Albert Einstein College of Medicine, for initial design and direction of this exercise
Columbia’s Center for Education Research and Evaluation: Dorene Balmer, Boyd Richards and Aileen Becker
PDDI 2009