How can we encourage and enable students to use feedback to improve their work? Sarah Cork, Senior Lecturer Social Marketing, Marketing Ethics & Responsibility s.cork@brighton.ac.uk

Introduction An insight-driven intervention to improve perception of feedback and student outcomes on MK285 – 2nd year core 20 credits, year through marketing module. - 140 students - 4 staff - 2 assignments: 1500 word report (40%), 2500 lit review (60%) Will introduce you to: Formative research (literature and student voice) findings Design and implementation of feedback intervention Results Learnings and Recommendations

Background & Context Across higher education, improving the quality of feedback is an area of focus with national student survey results revealing poor student satisfaction with feedback (hefce, 2016). BBS survey identified particular area of improvement needed - feedback that helps students improve their work. In addition, a questionnaire to students at the beginning of the year identified gaps between what students perceived as good feedback and what was being delivered.

The starting point... Pedagogical literature Need to encourage students to move beyond skim-reading of feedback ....and take joint responsibility for their learning (Gibbs & Simpson, 2004) Need for ipsative feedback, use across modules (Hughes et al, 2011) Encourage self-regulation with students assessing their own work against criteria (Nicol, 2010) Identify strengths and clear areas for improvement with constructive feedback to show how to improve as well as what areas to improve (Nicol 2010) Much research focusing on nature of feedback rather than how to get students to engage mindfully with it (Handley et al, 2011)

Led to Formative Research An anonymous survey for all module participants to understand current perceptions of feedback and the barriers and enablers to engagement with it and to establish a baseline for measuring impact of the intervention. (32 responses). Interviews x 6: Deputy Head for Learning and Teaching & member of CLT to understand good practice already happening & other members of the module team to ensure delivery of the intervention was feasible. Post-intervention survey to measure student’s perception of feedback on this module and recommendations for improvement. Analysis of a sample of assignments (2 seminar groups) to measure correlation between engagement with intervention and improvement in grades between assignment 1 and 2.

Findings Barriers and Facilitators to Use of Feedback

Barriers to Using Feedback According to the Literature: Students can have lack of understanding of terminology and language used. (Winstone et al, 2016) Lack of motivation to go through feedback and see tutor and re-visit feedback where necessary – know they would benefit from it but don’t do it. (Winstone et al, 2016) Students are not proactive in seeking discussions with tutors about feedback – they ‘need more than just an invitation’ (Price et al, 2010) Don’t see relevance for other work because modular approach means lack of chance to implement in similar assignment (Hughes 2011) Students feel there is nothing they can do with writing style and other academic skills, don’t feel they are told how to improve it (Winstone et al, 2016)

Facilitators to Using Feedback According to Students: ‘Being unclear’…’vague… ambiguous’… ‘just putting ‘what???’ next to a sentence.’. ..’very brief with one word. e.g. good, adequate’.. ‘not clear what to improve’ ‘I can’t be bothered to read through it properly and take it on board’ ‘Isn’t relevant for other topics’ ‘too specific for this assignment’… ‘hard to generalise across other modules’ ‘when it just says what was good and bad without saying why… vague factors that I have difficulty applying’

Facilitators to Using Feedback According to the Literature: Motivate students to want feedback, communicate the importance of it (Race, 2005) Need to give students more chance to increase their skills in self-regulation (Nicol & Macfarlane Dick, 2010) Ensure feedback clearly related to assessment criteria (Parkin et al, 2012) & demonstrates to student what good performance looks like (Nicol & Macfarlane, Dick 2010) Feedback as a two-way process, ‘dialogue’, encouraging dialogue rather than simple transmission of information from lecturer to student (Nicol & Macfarlane-Dick 2010). Encourage positive emotions (Gibbs, 2004) Provide feedback that can be fed forward into other modules, across their university career, moving away from emphasis on grade and achievement for one particular assignment and marker (Hughes 2011)

Facilitators to Using Feedback According to Students: ‘if it is proclaimed and emphasised that the implementation of feedback will improve my work and my grade’ ‘explanation of why I received a specific grade especially when others in the class received different for similar work’ ‘clear what to do next time…what are the issues and what to work on…specific recommendations’ ‘encourage questions on feedback from student…face to face… meetings with tutor… workshops to exercise new techniques or methods suggested ‘ ‘Be a bit more positive and driven… all negativity demotivated me’ ‘repost feedback when appropriate for another assignment as a reminder to look at comments…more about how can use strengths in future assignments…’general essay / structure comments… ‘section on student central for all personal feedback that’s easily accessible and visible’

Key Insights informing the Intervention Ensure students understand the role and benefits of using their feedback Get over the inertia – intervene against the ‘can’t be bothered’ Be clear on strengths and weaknesses of the work and recommend strategies for how to improve, as well as what to improve Deliver feedback that can be fed forward into other modules Be clear on why the mark was awarded and that students are clear on the assessment criteria and can self-review against it Provide opportunities for face to face feedback, discussion and clarification Use technology to support students in accessing and understanding feedback given

The Intervention What we did and the results...

So What Did We Do? Enable as well as Encourage behaviour Designed the ‘Act on Feedback’ intervention, as part of wider changes to feedback practice on the module: All feedback included 3 good things and 3 areas to improve with recommended strategies and resources students could use to improve their work. A detailed scoring rubric was developed to ensure consistency of marking and feedback across the module team and to enable students to understand their mark, with feedback regarding strengths and weaknesses clearly aligned to those scoring high / low and language in the feedback reflecting that used in the rubric. Students briefed to use the rubric, to self-assess against the marking criteria to increase their self-regulation skills A variety of feedback delivery methods were incorporated into the teaching and learning activity schedule to increase inclusivity and answer students’ clear requests for the option for face to face feedback, mixed with online grade and feedback.

ACT on Feedback Template Feed forward that can be used across modules Engaged with assessment criteria Deeper engagement with the literature

As a Result.... Students perception of feedback improved, with more understanding how to improve their work. Qualitative comments revealed that students benefitted from one to one and group feedback tutorials and this was borne out in grades. Students valued the detailed rubric and used this to self-assess their work for assignment 2 to make improvements and ensure they were meeting the brief. Many students improved in areas that could be transferred across modules – structure, referencing, writing style, discussion and application of theory, analysis and quality and use of sources. There were however comments regarding the consistency of feedback across markers which will be addressed further next year. In addition to an initial review of the literature, which identified the need to overcome psychological as well as physical barriers to using feedback (Winstone et al, 2016), a range of research methods were then used to to inform the design of the intervention including: - An anonymous survey for all module participants to understand current perceptions of feedback and the barriers and enablers to engagement with it and to establish a baseline for measuring impact of the intervention. (32 responses). - 5 semi-structured interviews; with the School’s Deputy Head for Learning and Teaching to understand good practice already happening and with the 4 members of the module team to ensure delivery of the intervention was feasible. - A post-intervention survey to measure student’s perception of feedback on this module and recommendations for improvement. Analysis of a sample of assignments (2 seminar groups) to measure correlation between engagement with intervention and improvement in grades between assignment 1 and 2.

Impact on Student Perception of Feedback Pre Intervention Post Intervention

Impact on grades

Going Forward Learnings & Recommendations for the Future

Key things to take-away Enable as well as encourage change - build use of feedback into the process – make it unavoidable and reward its use (students and staff!) Encourage joint responsibility - ensure students can assses their own work and enter into dialogue with peers and tutors – clarify expectations and their own performance against these. Make it clear how to apply feedback on this and other modules. Motivate students to use it – sell them the benefits of using / risks of not using it.

Recommendations for Future Practice Show results of engagement with feedback to next year’s students to ‘sell the benefits’ of students taking responsibility to use feedback to improve their work. Include evidence of use of feedback from this and other modules as part of explicit assessment criteria to encourage greater number to benefit from the process. Ensure face to face meetings are scheduled element of module learning and teaching activities to fit within lecturer’s existing workload and encourage implementation. Implement peer / self review against assessment criteria (rubric) for assignment 1, identifying strengths and areas for improvement as formative assessment. Use adaptive release – feedback first then grades...

Thank You for Listening Any Questions?

References Gibbs, G. & Simpson, C. 2004. Conditions under Which Assessment Supports Students’ Learning’ Learning and Teaching in Higher Education 1:3-31 Handley, K. , Price M. & Millar J, 2011. Beyond ‘Doing Time’; investigating the concept of student engagement with feedback. Oxford Review of Education, 27;4, 543-560.  Hughes, G. 2011 ’Towards a Personal Best: A Case for Introducing Ipsative Assessment in Higher Education. ‘ Studies in Higher Education. 36:353-67 Nicol, D. and Macfarlane-Dick D., 2006. ‘Formative Assessment and Self-Regulated Learning: A Model and Seven Principles of Good Feedback Practice’ Studies in Higher Education 31:199- 218 Parkin, H., Hepplestone S., Holden G., Irwin B., & Thorpe L., 2012. ‘A Role for Technology in enhancing students’ engagement with feedback’, Assessment & Evaluation in Higher Education, 27:8, 963-973.  Price, M., Handley K., Millar J., & O’Donavan B., 2010. ‘Feedback: All That Effort, But What is the Effect?’ Assessment & Evaluation in Higher Education 35:277-89.  Race, P., Making Feedback Work: A Compendium of extracts from ‘Making Learning Happen’ (2005), ‘500 Tips for Tutors(with Sally Brown, 2005), ‘The Lecturer’s Toolkit; 3rd edition (2006) and ‘Making Teaching Work’ (with Ruth Pickford, 2007).  Winstone, N., Nash R., Rowntree J. & Parker M 2016 ‘It’d be useful, but I wouldn’t use it’: barriers to university students’ feedback seeking and recipience, Studies in Higher Education