14:30 Introduction and news/ updates Ross Ward and Mari Cruz García 14:35 Multimedia assessment and feedback: why it does matter Mari Cruz García 15:00 Case study: Use of Media Hopper at the School of Education Andrew Manches 15:35 Tea and coffee   15:50 Designing a video assessment activity Mari Cruz García and Ross Word 16:25 Q&A Close

Multimedia assessment and feedback: why it does matter Mari-Cruz García EDE Assessment and Feedback advisor

“If students can escape, with difficulty, from the effects of poor teaching, they cannot escape the effects of poor assessment” David Boud with difficulty, from the effects of poor teaching, they cannot escape the effects of poor assessment. Assessment design should be aimed at fostering learners’ creativity and gaining a deeper understanding of the academic subject. If we want to promote deep learning, we need to embed innovation in our methods of assessment, which are currently more focused on awarding a grade than on fostering creativity. Technology can help us to foster this innovation by transforming traditional written essays into multimedia assignments, which provide students with the digital and soft skills that the Digital Society demands.

West, J. & Turner, W. (2015): Enhancing the assessment experience using online video feedback Sally Brown’s work on Improving assessment and feedback JISC “Sounds good” project

Pros of using multimedia assessment Improving student engagement by reaching students who have a predominant visual/ auditory learning style Students perceive multimedia feedback as “more individualised” and “easier to comprehend and act upon” (West and Turner 2015) Video assessment can be used in a variety of academic disciplines, not only Medicine: Law, Languages, Multimedia, Engineering, etc. It can be used to assess generic skills, such as communications, leadership, problem-solving, team work, etc., as well as subject-based specific knowledge. There is enough empirical evidence to support the benefits of using video (and audio) in assessment and feedback: research from Susan Rhind, JISC Sounds good project: http://www.webarchive.org.uk/wayback/archive/20140614072955/http://www.jisc.ac.uk/whatwedo/programmes/usersandinnovation/soundsgood.aspx Video assessment improves student engagement by reaching those students who may feel alienated or disenfranchised when using traditional assignment methods. In education, emphasis has been put on producing diverse learning materials that can suit different learning styles, yet the majority of the assessment methods only support a verbal-linguistic learning style.   Formative assignments based on video submissions can increase student participation. Some authors consider that formative feedback is better internalised by students than summative feedback because it does not include the judgement and “punishment” of summative assignment (Norcini and Burch 2007) (Epstein (2007). Yet some students miss this opportunity of receiving this type of feedback because they see formative assignments as time-consuming and non-compulsory. Formative video assignments introduce an element of novelty and creativity that spur students’ curiosity contributing, therefore, to increase the submission rates. As an evidence of this, I can mention one of the modules of our MSc in Diabetes Care, Education and Management in which we replaced the traditional formative draft (students had to submit the first draft of their work-based project) by a video assignment in which students were asked to submit a 5 minutes video relating to their work-based projects. For distance learning and blended learning courses we discovered that replacing traditional ice-breaking activities with video submissions improved students’ online engagement. For example, most of our modules, delivered via the School’s VLE, included a “Saying Hello” forum hosted during the induction week. Replacing that forum with an “Introductory selfie” activity hosted in Kaltura increased students’ online participation for the rest of the activities in the induction week.

Pros of using multimedia assessment Tutors use video feedback to establish a more personalised relationship with their students Video assessment and feedback help engage students in peer assessment and group work Formative assignments based on video submissions can increase student participation For distance learning and blended learning courses we discovered that replacing traditional ice-breaking activities with video submissions improved students’ online engagement. For example, most of our modules, delivered via the School’s VLE, included a “Saying Hello” forum hosted during the induction week. Replacing that forum with an “Introductory selfie” activity hosted in Kaltura increased students’ online participation for the rest of the activities in the induction week.   Those tutors who used video feedback (using Kaltura or Voicethread, another web platform for supporting video assessment and feedback) were able to establish a more personalised relationship with their students. Research suggests that students see video and audio feedback as more “individualised” and “easier to comprehend and act upon” (West and Turner 2015). Video assessment and feedback helped engage students in collaborative working and peer assessment. In postgraduate part time programmes, an obstacle to overcome is how to engage students in team work and peer assessment when both activities do not contribute to their final marks. Likewise, many peer assessment activities do not have the expected student turnouts as students find the peermarking software confusing or obsolete-looking. The Kaltura Media Gallery, integrated into the online activities supported by most of the VLEs (e.g. forum or assignment) can be used as a peer assessment tool that students find easy to use and in which they can provide video feedback to their peers.

Cons of multimedia assessment Video assignment requires more time than a traditional written assignment How much “more time”? Logistics and technical support Time allocation: Video assignment requires more time in comparison with a traditional written assignment. Tutors and students need to familiarise themselves with the video management software and the way in which the software is integrated intop the VLE. Students need to know how to record and submit their videos and tutors need to know how to review the videos and provide feedback.   It is important that the tutor consider the amount of time that students will spend preparing the video assignment to allocate enough time during their teaching week. Likewise, it is important that the workload model for academics also recognise the time spent taking part in innovative teaching practices, such as video assignment. Logistics and technical support: Some tutors panic that creating a video assignment activity may end in sorting the logistics of providing video cameras to students or liaising with complex multimedia queries. Other tutors assume that every student has a smartphone and know how to record a video from the smartphone and submit the video using the Kaltura Native Mobile app. Both assumptions are dangerous and the reality is somewhere half way: supporting a video assignment activity is not as time consuming as it looks like, and some students may need to book recording equipment and may need addition help submitting their videos to Kaltura. It is important to count with video or FLIP cameras that students can borrow as well as to have technical support for the activities. At the University of Edinburgh, this support is provided locally for some Schools or via the IS. How to grade a video? Some tutors may find difficult to mark and provide feedback for a video, in particular for summative assignment. When using video assignments, tutors should be upfront with the students about the marking process, since video assignment is also a new medium for most of the students. I always advise to attach a marking grid/rubric to the video assignment to make clear to students what is being assessed and how is being assessed. It is a good practice that tutor and students discuss the marking grid before the submission to make sure that students understand what is expected from them.

Cons of multimedia assessment How to grade a video/audio?: Rubrics and marking grid How to include video assessment in a programme’s assessment strategy Time allocation: Video assignment requires more time in comparison with a traditional written assignment. Tutors and students need to familiarise themselves with the video management software and the way in which the software is integrated intop the VLE. Students need to know how to record and submit their videos and tutors need to know how to review the videos and provide feedback.   It is important that the tutor consider the amount of time that students will spend preparing the video assignment to allocate enough time during their teaching week. Likewise, it is important that the workload model for academics also recognise the time spent taking part in innovative teaching practices, such as video assignment. Logistics and technical support: Some tutors panic that creating a video assignment activity may end in sorting the logistics of providing video cameras to students or liaising with complex multimedia queries. Other tutors assume that every student has a smartphone and know how to record a video from the smartphone and submit the video using the Kaltura Native Mobile app. Both assumptions are dangerous and the reality is somewhere half way: supporting a video assignment activity is not as time consuming as it looks like, and some students may need to book recording equipment and may need addition help submitting their videos to Kaltura. It is important to count with video or FLIP cameras that students can borrow as well as to have technical support for the activities. At the University of Edinburgh, this support is provided locally for some Schools or via the IS. How to grade a video? Some tutors may find difficult to mark and provide feedback for a video, in particular for summative assignment. When using video assignments, tutors should be upfront with the students about the marking process, since video assignment is also a new medium for most of the students. I always advise to attach a marking grid/rubric to the video assignment to make clear to students what is being assessed and how is being assessed. It is a good practice that tutor and students discuss the marking grid before the submission to make sure that students understand what is expected from them.

Taught Assessment Regulations Academic Year 2016/2017 Feedback Standards and Guiding Principles 2016/2017