Business School Beyond 'scribbles in the margin': Engaging students with assessment feedback Student Learning Experience Conference, 16th May 2007 Dr. Karen Handley Business School khandley@brookes.ac.uk

The problem … Business School “it is not inevitable that students will read and pay attention to feedback even when that feedback is lovingly crafted and provided promptly” (Gibbs and Simpson, 2002, p. 20). Business School

Purpose of assessment and feedback Feedback is arguably the most important part of the assessment process in its potential to affect future learning and student achievement (Hattie, 1987; Black and Wiliam, 1998; Gibbs and Simpson, 2002) Students appreciate and want good feedback (Hyland 2000; O’Donovan et al 2001; Higgins et al 2002;) But … students are dissatisfied with assessment feedback and lack engagement Judging achievement Maintaining standards Promoting Learning NSS: In the 2006 survey, 49% of respondents said that feedback was slow and unhelpful, prompting a response from the Higher Education Minister, Bill Rammell, that he hoped institutions would ‘look long and hard at assessment and feedback’ (Shepherd, THES, 2006). Does it matter? Yes, if you believe that students learn and develop not only by listening in a passive way but also through an iterative process involving assessment and feedback - as Laurillard (1993, p. 61) has said, 'action without feedback is completely unproductive for a learner'. Carless et al., 2006 Business School

National Student Survey 2006 Categories: The teaching on my course Assessment and feedback Academic support Organisation and management Learning resources Personal development Overall satisfaction Assessment and feedback is generally graded 0.5 below that of other categories Business School

National Student Survey 2006 Assessment and feedback Categories: The criteria used in marking have been clear in advance Assessment arrangements and marking have been fair Feedback on my work has been prompt I have received detailed comments on my work Feedback on my work has helped me clarify things I did not understand Generally, categories 1 and 2 were rated highest - these relate to student perceptions of assessment. The lowest ratings were for the last 3. Of those the worst was the question asking whether the feedback had helped students clarify things they did not understand > issues around learning from feedback Business School

So why doesn’t feedback work? Students sometimes don’t read feedback (Hounsell, 1987) or can't read the 'scribbles in the margin' Oral feedback is more effective (McCune, 2004) Feedback is sometimes not understood (Lea and Street, 1998); 30% of students never found the feedback helpful (Maclellan, 2001) Student concerns that the process is too subjective (Holmes & Smith, 2003) Feedback may have an emotional impact on students (Layder, 1997); Feedback may damage students ‘self efficacy’ when the emphasis is on the mark or grade (Wotjas, 1998) Feedback may be ineffectual in changing future performance (Fritz et al, 2000); timing issues Perhaps because Timing (Juwah et al., 2004) Language – difference in academic discourses; complexity, jargon, interpretation (Lea and Street, 1998; Ridsdale 2003) Negativity – overly critical, tone, judgemental not developmental (Hyland, 2000; Ridsdale, 2003) Vagueness (Higgins, 2000) Inconsistency (MacKenzie, 1974) Summative marks obscure assessment feedback (Black & Wiliam (1998) Lack of understanding of assessment processes and 'criteria-related feedback' (Holmes & Smith, 2003; Gibbs & Simpson, 2002; Dunbar-Goddet, 2006) Lack of student engagement >> FDTL5 ESWAF Project >> Lack of student engagement >> FDTL5 ESWAF Project Business School

FDTL5: Engaging Students with Assessment Feedback Who are we? A networked project … Brookes, Bedfordshire, Bradford (project partners) UWE, Sunderland, Bournemouth, Northumbria, London Met. (cascade partners) Our objective To understand how to engage students in feedback How are we doing this? Case studies (completed 2006) Cascade partner mini-cases (in progress) Generic student and staff interviews (completed 2007) Questionnaire across the 3 project partner institutions (completed 2006) Business School

The case studies – challenging the traditional assessment > feedback model Feedback on draft assignments (two cases) Student self-assessment (two cases) Peer review Student evaluation of peer vs. tutor feedback Feedback before/after grade Business School

Case study 1: the problem Develop an assessment/feedback approach with the following stages for the student - C.R.A.F.T. Collect feedback Reflect on feedback Apply feedback to next assignment in same module (collect) Follow-up feedback Transfer learning to next module Undergraduate; 2nd and 3rd year students; n=111 Business School

Case study 1: the experiment/solution The discussions themselves also followed a set format in which the students were asked: what mark are you expecting; what went well; what could you improve; and what should be re-written and re-submitted. The whole process thus started from asking student to reflect on their performance, before requiring them to apply the feedback and resubmit their work, an important element of a warmer style of feedback. Business School

Case study 1: what did students/tutors think? Findings from module evaluation forms: 50% said they liked the F2F meeting ('able to ask questions'; 'personal meeting') 35% said they liked being able to improve on their work, based on the feedback given 25% mentioned their dislike of hand-written feedback ('circles without explanations'; 'scribbling in the margins') Comments from tutors: Quality of the coursework improved after draft-plus-rewrite Business School

Some interpretations and possible implications … Provide more opportunities for draft-plus-rework, perhaps where only a summative grade is given on the final submission Ask students (when submitting assignments) to indicate the areas on which they'd like feedback Reduce emphasis on (hand)written feedback; facilitate student engagement and understanding through dialogue; consider 'safety' and relationship issues Use class time for feedback (e.g. in the peer review case) Give generic (in-class) feedback as well as individual feedback; Feedforward Consider whether feedback can do much more than 'diagnose' development issues and then direct students to other resources Improve the linkage of assessment strategies between modules and across programmes Prepare students to receive feedback (within year 1 orientation) Business School

… Ongoing research on improving student engagement over time at Brookes Business School