Recognition Memory of Visual, Audio & Written Dialogue in Film Derek M. Ellis and Donald Homa Arizona State University Introduction Results We recently demonstrated that recognition memory for individual, visual frames extracted from a cinema clip was accurate even when foils were taken from the same movie. Surprisingly, recognition accuracy was unaffected if the film was viewed in a jumbled, rather than coherent, order1. In the present study, we explored recognition accuracy for spoken dialogue and written narrative as well, following presentation of a cinema clip in a coherent or jumbled order. Half the participants listened only to the audio of the same film. Following presentation, participants made recognition judgments to audio segments only, narrative segments only and, in the audio with video condition, visual frames. The results showed enhanced recognition of audio and written narrative in full condition relative to audio-only, but again with little impact of jumbling in any condition. Results are addressed in terms of narrative processing generally and event boundary processing for naturalistic, temporally coherent events. Linear Vs. Jumbled With: No significant differences were found between Linear and Jumble formats on the number of correct responses for frame (p=.63), audio (p=.44), and narrative (p=.59) items. Without: No significant differences were found between Linear and Jumble formats on the number of correct responses for audio (p=.95) and narrative (p=.59) items. Methods & Materials Participants Data is from Arizona State University students, predominately psychology undergraduates (N = 146, 46 females, mean age = 19.40). Final group sizes were Linear With (n = 34), Linear Without (n= 40), Jumbled With (n = 39), and Jumbled Without (n= 33). Video Vs. Audio Only Groups Display Format Video (With) Audio Only (Without) Order Linear Linear With Linear Without Jumble Jumbled With Jumbled Without Film: Touch of Evil (1958) The first 15 minutes of the film were used. The clip was broken up into the length between two camera changes (mean length = 6.47 seconds). Participants viewed middle portion of the clip before being tested on recognition items. Content from the first (Before) and last (After) five minutes of the film were used to create foils. Audio: Groups that saw audio with video performed significantly better than audio only, F(1, 144)=44.94, p<.001, η 𝑝 2 =.24. Narrative: Groups that saw audio with video performed significantly better than audio only, F(1, 144)=34.67, p<.001, η 𝑝 2 =.19. Minutes 0-5 Minutes 5-10 Minutes 10-15 Before After Seen Before Vs. After Recognition Test and Items Items consisted of frames, narrative, and audio items. Frames were individual screen captures obtained using Adobe Premier. Narrative and Audio items were based upon the spoken dialogue in the film. Narrative items are the dialogue written in text format, while Audio is dialogue played via audio clip on a blank screen (mean # of words per item = 7.35). “New” items were evenly distributed from Before and After sections. Participants made remember (rem), know, and new judgements2. Item presentation order was random. # of Recognition Items Format Presentation Order Frames Audio Narrative Audio & Video Linear 40 Old/ 40 New 20 Old/ 20 New Jumble Only - Audio: Groups that saw audio with video performed significantly better than audio only on Before, F(1, 144)=22.37, p<.001, η 𝑝 2 =.13, and After, F(1, 144)=9.41, p<.01, η 𝑝 2 =.06. Audio Before vs After: Items from Before the seen clip were more often correctly identified than After items, t(1,145)=6.99, p<.001, d=.37. Narrative: Groups that saw audio with video performed significantly better than audio only on Before, F(1, 144)=21.50, p<.001, η 𝑝 2 =.13, and After, F(1, 144)=24.95, p<.001, η 𝑝 2 =.15. Narrative Before vs After: No significant differences were found between Before and After, p=.77, d=.01. Conclusions Acknowledgements Our results replicate our previous findings that there is no effect of jumbling on recognition performance. Separate tests between display format and collapsing between display formats (only testing narrative and audio) were non-significant. Further testing however between display formats (With vs Without) revealed performance was better when the audio was paired with video. This scaffolding effect appeared in performance on both audio and narrative items. Some of this effect may be accounted for by one’s ability to better reject Before and After items. While false alarms were significantly reduced on Before Audio items (i.e. more correct responses), narrative items showed a slight but nonsignificant mean difference. Thank you to Gene Brewer. Chris Blais, Fernando Coronado, and Yinyin Lu for helping immensely on this project. Derek Ellis is supported by NSF Grant 1632291. References Ferguson, R., Homa, D., & Ellis D (2016). Memory for Temporally Dynamic Scenes. Quarterly Journal of Experimental Psychology, 1-42. McCabe & Geraci (2009). The influence of instructions and terminology on the accuracy of remember-know judgments. Consciousness and Cognition, 18(2), 401-413.