1. The relation between a first impression and a station rating when examinee performance changes Timothy J. Wood, Debra Pugh, Claire Touchie, James Chan, Susan Humphrey-Murto 12 22 2 1 = Department of Innovation in Medical Education, University of Ottawa 2 = Department of Medicine, University of Ottawa Contact Information: twood@uottawa.ca Background Every time we interact with someone we have never met, we form a first impression of that person. First impressions are judgments that are formed quickly (within 60 seconds) based on very little information made with little conscious awareness. Are these first impressions accurate and to what degree do they influence subsequent judgments? Studies have found that first impressions can be accurate and that there is a relationship with subsequent judgments (Ambady, Bernieri & Richeson, 2000; Wood 2014). But what happens if your first impression about someone is wrong? Can you adjust your subsequent ratings? Macon & Diploye (1990) : interviewers adjust for negative first impressions but not for positive first impressions Wyer (2010): Raters easily adjust for first impressions that are explicit (e.g., what is your rating of this person) but find it more difficult to adjust for impressions that are implicit (e.g., does your impression of a person in one situation influence a judgment in an unrelated situation). Research question The purpose of this study was to determine if a an examiner’s first impression determines the final station score despite a change in examinee performance. Design 29 physician examiners viewed a series of seven videos of examinees (i.e. actors) performing a physical examination of a patient with heart failure Videos were in three conditions  Baseline : three videos in which performance stayed consistent throughout the video.  Strong to weak: two videos in which performance changed from initially strong to weak  Weak to strong: two videos in which the performance changed from initially weak to strong Participants made a rating of the examinees ‘abilities after 60 seconds (first impression). Participants viewed the rest of the video and completed a global rating of the examinees’ abilities (final rating) Analysis As a function of condition (baseline, strong to weak, weak to strong)  Differences between the first impression and final rating  Correlations between the first impression and final rating Methods Mean Ratings BaselineStrong to Weak The final rating is lower than the First Impression in the Baseline condition (p<.001) and in the Strong to Weak Condition (p<.001) The drop in mean ratings is bigger in Strong to Weak (p<.001) The final rating is higher than the First Impression in Weak to Strong condition Discussion Summary Raters were willing to adjust their ratings when their First Impressions are wrong but it depends on the change. If the initial impression was positive, raters lowered their subsequent ratings if the performance of the examinee declined, however, there was a relationship between a First Impression and a final judgment. If the First Impression was poor, raters increased their subsequent ratings if examinee performance improved but it is not done in a consistent manner. Implications First Impressions are thought to be a type of bias that can influence subsequent judgements that raters make and therefore raters should avoid them. Raters in this study demonstrated that they are willing to correct their judgments if the first impression is wrong but how they do so it depends on the initial impression. In an OSCE setting, if an examinee starts strong then weakens, examiners will take that change in performance into account but are still influenced by their first impression (i.e. systematically shift their ratings lower). If an examinee starts off poorly but finishes well, examiners take that change into account and increase their ratings but the increase is not done systematically and there is little relation to the first impression Ambady N, Bernieri F, Richeson J. Toward a histology of social behavior: Judgmental accuracy from thin slices of the behavioral stream. Advances in Experimental Social Psychology. 2000. p. 201–71. Macan TM, Dipbloye RL. The relationship of interviewers preinterview impressions to selection and recruitment outcomes. Personnel Psychology. 1990, 43:745-767 Wood TJ. Exploring the role of first impressions in rater-based assessments. Adv Health Sci Educ Theory Pract. 2014. 9:409–27. Wyer N. You never get a second chance to make a first (implicit) impression: the role of elaboration in the formation and revision of implicit impressions. Social Cognition. 2010. 28: 1-19 References Acknowledgements The authors would like to acknowledge the support of the Department of Innovation in Medical Education (DIME), University of Ottawa Funding for this project came from the Medical Council of Canada Research in Clinical Assessment Grant. Imagine you are an OSCE examiner on a history taking station for heart failure. Dr. Smith enters the room. She walks toward the patient then halts momentarily then walks back to a sink to wash her hands. Although she confirms the patients name, she forgets to introduce herself to the standardized patient. Her shoulders slumped, arms are crossed and she has minimal eye contact with the patient. She starts the interview by saying “So…” and makes a hand motion to the patient to start talking. What is your rating of this examinee’s clinical abilities? As the interview continues, Dr. Smith starts making increasing eye contact with the patient. Her body language becomes less withdrawn. Her history questions are focused on the issue. Toward the end of the station, she suggests the correct diagnosis to the patient and discusses management options in a language the patient can understand. What is your rating of this examinee’s clinical abilities? Dr Smith enters the room. He immediately washes his hands and starts a conversation with the standardized patient by introducing himself and confirming their name. The whole time he is smiling, stands with his shoulders back, makes good eye contact with the patient and seems genuinely interested in the patient as he asks why the patient is visiting the clinic. What is your rating of this examinee’s clinical abilities? As the interview continues, Dr. Smith’s body language is still open and confident and he clearly has a good rapport with the standardize patient. Unfortunately many of his questions are not related to the case and are not asked in an orderly manner. At one point, you even have to remind him to reread the station instructions. He ends the station by listing several diagnoses, some that are clearly irrelevant and does not give any management options. What is your rating of this examinee’s clinical abilities? Weak to Strong Baseline Strong to WeakWeak to Strong r0.360.49-0.13 Correlation between First Impression Rating and the final rating for each condition UnsatisfactorySatisfactory InferiorPoorBorderline GoodExcellent UnsatisfactorySatisfactory InferiorPoorBorderline GoodExcellent UnsatisfactorySatisfactory InferiorPoorBorderline GoodExcellent UnsatisfactorySatisfactory InferiorPoorBorderline GoodExcellent