1. UCL Arena Exchange Seminar Improving assessment and feedback scores

2. Improving assessment and feedback scores Andrew Gardner, Senior Lecturer in the Archaeology of the Roman Empire Departmental Tutor, UCL Institute of Archaeology Rachel Rees, Programme Tutor for BSc Speech Sciences 2009-13 (followed by Christina Smith) Programme Tutor for MSc Speech and Language Sciences 2013-present Division of Psychology and Language Sciences

3. Assessment and feedback scores, UCL and the sector 2015 UCL Sector avgRG avg Assessment and feedback 617369 5. The criteria used in marking have been clear in advance. 637772 6. Assessment arrangements and marking have been fair. 747877 7. Feedback on my work has been prompt. 587067 8. I have received detailed comments on my work. 557365 9. Feedback on my work has helped me clarify things I did not understand. 566863

4. ArchaeologySector avg 5. The criteria used in marking have been clear in advance. 7977 6. Assessment arrangements and marking have been fair. 9278 7. Feedback on my work has been prompt. 6570 8. I have received detailed comments on my work. 73 9. Feedback on my work has helped me clarify things I did not understand. 7368 Archaeology and the sector

5. Speech Sciences Sector avg 5. The criteria used in marking have been clear in advance. 9277 6. Assessment arrangements and marking have been fair. 8178 7. Feedback on my work has been prompt. 7570 8. I have received detailed comments on my work. 9273 9. Feedback on my work has helped me clarify things I did not understand. 6468 Speech Sciences and the sector

6. Andrew Gardner Departmental Tutor Senior Lecturer in the Archaeology of the Roman Empire UCL Institute of Archaeology

7. Assessment & Feedback in context

8. BSc ArchBA ArchBA Arch & Anth 5. The criteria used in marking have been clear in advance. 808470 6. Assessment arrangements and marking have been fair. 909580 7. Feedback on my work has been prompt. 806860 8. I have received detailed comments on my work. 908470 9. Feedback on my work has helped me clarify things I did not understand. 907480 ARCLG_SHS response rate = 77% BSc Arch response rate = 83% BA Arch response rate = 83% BA Arch & Anth response rate = 71%

9. Assessment & Feedback mechanisms

12. Student Progress “I would also like to comment on the outstanding system of monitoring student progress […]. This system of student progress monitoring is relevant for all of the taught programmes in the Institute of Archaeology. I was also provided with the template used to monitor student progress, the UNDERGRADUATE STUDENT PROGRESS FORM. This system is a model of excellent practice that should be flagged to UCL. The activities and methods of monitoring have been meticulously planned and provide fantastic opportunities for students to engage in critical self-reflection, discuss performance with a member of staff and for year tutors to carefully monitor progression of students through the degree programme. I have no doubt that it is these kinds of activities that not only nurture student progression but are reflected in the attainment levels on modules and in the NSS results.” External Examiner 2015

13. These mechanisms support feedback which is: Timely Detailed Constructive Contextualised

14. Rachel Rees Programme Tutor for BSc Speech Sciences 2009- 13 (followed by Christina Smith) Programme Tutor for MSc Speech and Language Sciences 2013-present Division of Psychology and Language Sciences

15. BSc Speech Sciences NumberStatement% Agreeing 5The criteria used in marking have been clear in advance. 92 8I have received detailed comments on my work.92 9Feedback on my work has helped me clarify things I did not understand. 64

16. BSc Speech Sciences: OPINIO surveys for individual modules for SAME cohort Yrs 2-4 “ I received constructive feedback on my coursework.” Rated on a 5 point scale: Strongly Agree Agree Neither Agree nor Disagree Disagree Strongly Disagree How did responses vary ACROSS modules in terms of the %s students that agreed?

17. “I received constructive feedback on my coursework.” Year of Module% Strongly Agree or Agree 494 493 392 281 362 235 232 329 224 314 What is surprising about these figures? What could explain it?

18. What have students said? Feedback is “too vague”, “too broad”. “It’s a huge boost to just to be told ‘the way you did X was really good’. ” What have we learnt? Be specific Be positive and be specific about those remarks too

19. 5 What have students said? “okay, you didn’t do blah, you didn’t consider whatever,” and we get told, ‘this is what you didn’t do,’ but not what to do to fix the problem. ‘So, this is the problem with your coursework,’ but there’s no kind of action plan or suggestions.” “ As opposed to saying, ‘oh, you didn’t do enough with this,’ more kind of,…. ‘in order to do it next time, you could think about X, Y and Z.” “I want to know how to get a better grade!” “I also think that one of the major skills that was taught us in the course is self-reflection, so, to give ourselves feedback.” What have we learnt? Provide specific suggestions for how to improve Provide feedback that can be turned into an internal checklist

20. Hattie and Timperley (2007): Effective learners have self-regulation strategies. These are cognitive routines that they can draw on while preparing work. If feedback (from others) could more easily be converted into self-regulation strategies there should be more carryover to future coursework.

21. Examples of feedback that ticks the boxes Rather than: “Good answer to first viva question” “Q1: You considered how factors for prioritisation (e.g. severity, type of communication difficulty, practical resources) can interact and gave a clear practical example of this (relating to the need for intensive therapy for severe speech disorders). Positive remark Specific

22. Examples of feedback that ticks the boxes Rather than: “You are clearly confused about the X speech processing model” “If you revise the difference between stored motor programs and motor execution, you could improve the third hypotheses, which is currently inaccurate” Specific More positive tone Suggestion for action to take to improve

23. Examples of feedback that ticks the boxes Rather than: “The explanation of your intervention programme was patchy” “You teased out most of the clinical decisions you made but remember to provide clear rationale for each one” Specific Suggestion that could be converted to internal checklist.

24. Link between internal checklist and marking criteria? A+ Each clinical decision is explicitly described and backed up with rationale

25. Marking Criteria Specific Tried and tested (if possible, marker completes the coursework and notes what makes a good piece of work) Can be also use as an “internal checklist”

26. Any questions?

27. Useful resources Hattie, J., & Timperley, H. (2007). The Power of Feedback. Review of Educational Research, 77(1), 81–112.