Assessment feedback: More modality matters Michael Henderson, Tracii Ryan, Michael Phillips, Phillip Dawson, David Boud, Elizabeth Molloy, Paige Mahoney

Introduction Digital recordings – audio, video, and screencasts – can be effective ways of providing assessment feedback to students, particularly in comparison with text-based feedback. Benefits in comparison with text-based feedback include: easier to understand (Bourgault, Mundy, & Joshua, 2013; Henderson & Phillips, 2015) more supportive (Borup, West, & Thomas, 2015) more personal (Knauf, 2016; West & Turner, 2016) Most digitally recorded feedback studies are limited to comparisons with text- based or hand written feedback comments. Educators also provide comments via electronic annotations, face-to-face conversations, digital recordings, and marking sheets/rubrics. We propose that affordances of the feedback modality may influence how it is perceived by students. Assessment feedback: more modality matters

Method Participants Key demographics for N = 4514 students Gender 67% Women 32% Men 1% Other / unspecified Enrolment 70% Domestic 82% Full-time 82% On-campus Course type 67% Undergraduate 33% Postgraduate Discipline 28% Arts and Education 25% Business and Law 23% Health 22% STEM 2% Art, Design, Architecture Large four phase study, including an online survey of 4515 students Data are drawn from four key survey items relating to the most recent feedback comments received by student: Modality of the feedback comments Level of agreement that comments were: Detailed Usable Personalised Assessment feedback: more modality matters

Results: Hypothesis 1 Students will provide more positive ratings when they receive digitally recorded feedback compared to when they receive any other modality Students who received a digital recording as their only form of feedback had higher ranked mean scores than students who received any other single modality of feedback comments. A Kruskal Wallis test revealed that there were significant differences (p <.001) between groups for ratings of detail, usability, and personalisation.   Assessment feedback: more modality matters

Results: Hypothesis 2 Students will provide more positive ratings if they received multiple feedback modalities rather than a single modality Students who received multiple forms of feedback had consistently higher ranked means than students who did not. A Mann Whitney U test revealed that the differences between the two groups were significant for all three questions (p <.001).   Assessment feedback: more modality matters

Results: Hypothesis 3 Students who received multiple modes of feedback will provide more positive ratings when one of those modes was a digital recording Students who received multiple forms of feedback, including at least one digital recording, had consistently higher ranked means that those who did not receive a digital recording. The differences in ranked means between the two groups were also significant (p <.001) for the two questions relating to detail and intention to use to improve subsequent work, but not for personalisation  Assessment feedback: more modality matters

Conclusions All three hypotheses were supported Digitally recorded feedback is perceived as more detailed, personalised, and usable when compared to alternate feedback modes, such as handwritten comments, electronic annotations, rubrics, and even face-to-face conversations. It is also likely that multiple modes of feedback are more useful for students, especially if at least one form of digital recording is included. These findings add to our understanding of effective feedback design, indicating that we need to consider the importance of media richness theory, and the value of offering multiple channels or modes of feedback. Assessment feedback: more modality matters

Acknowledgement References Assessment feedback: more modality matters Support for this publication has been provided by the Australian Government Department of Education and Training. The views expressed in this publication do not necessarily reflect the views of the Australian Government Department of Education and Training. Borup, J., West, R. E., & Thomas, R. (2015). The impact of text versus video communication on instructor feedback in blended courses. Educational Technology Research and Development, 63(2), 161-184. Bourgault, A. M., Mundy, C., & Joshua, T. (2013). Comparison of Audio vs. Written Feedback on Clinical Assignments of Nursing Students. Nursing Education Perspectives, 34(1), 43-46. doi:10.5480/1536-5026-34.1.43 Evans, C. (2013). Making sense of Assessment Feedback in Higher Education. Review of Educational Research, 83(1), 70-120. Henderson, M., & Phillips, M. (2014). Technology enhanced feedback on assessment. Paper presented at the Australian Computers in Education Conference 2014, Adelaide, SA. http://acec2014.acce.edu.au Henderson, M., & Phillips, M. (2015). Video-based feedback on student assessment: scarily personal. Australasian Journal of Educational Technology, 31(1), 51-66. Knauf, H. (2016). Reading, listening and feeling: audio feedback as a component of an inclusive learning culture at universities. Assessment & Evaluation in Higher Education, 41(3), 442-449. doi:10.1080/02602938.2015.1021664 Morris, C., & Chikwa, G. (2016). Audio versus written feedback: Exploring learners' preference and the impact of feedback format on students' academic performance. Active Learning in Higher Education, 17, 125-137. West, J., & Turner, W. (2016). Enhancing the assessment experience: improving student perceptions, engagement and understanding using online video feedback. Innovations in Education and Teaching International, 53(4), 400 - 410. Assessment feedback: more modality matters