Preparing your students for feedback Helping students understand ‘the rules of the assessment game’ Dr Graham Holden (Director of Learning and Teaching) g.j.holden@shu.ac.uk @GrahamJHolden Dr Helen Kay (Senior Lecturer Academic Development) helen.kay@shu.ac.uk Directorate of Learning Enhancement and Academic Development (LEAD)

Session Outcomes Participants will have an understanding of the importance of student assessment literacy, and in particular assessment standards, to their positive engagement with learning acquired a range of techniques to enable them to support student understanding of standards and criteria applied these techniques in their own context and identified actions that they will take in their own practice.

Evans Assessment Tool (EAT) (Evans 2016) The EAT framework has evolved from extensive research on assessment feedback and is based on three core dimensions of practice: Assessment Literacy Assessment Feedback Assessment Design

Activity 1 - Assessment Literacy In order for learners to be able to fully engage in their learning in higher education, they need to have a good understanding of the requirements of assessment. These requirements need to be clear to both students and tutors and such understanding is helped if there are clear principles underpinning assessment practice that are shared and owned by all. Working individually use the Evans Assessment Tool to reflect on your current practice scoring AL1 to AL4 between 0 (do not meet to 5 fully meets)

Contention 1 Higher education teachers are often frustrated by the modest impact feedback has in improving learning. The status of feedback deserves to be challenged on the grounds that it is essentially about telling. For students to become self-sustaining producers of high quality intellectual and professional outcomes, they must be equipped to take control of their own learning and performance. (Sadler, D. R. 2013, 'Opening up feedback: Teaching learners to see'. In Merry, S., Price, M., Carless, D., & Taras, M. (Eds.) Reconceptualising Feedback in Higher Education: developing dialogue with students. (Ch. 5, 54-63).London: Routledge. )

Conditions for students to benefit from feedback Possess a concept of the goal/standard or reference level being aimed for Compare the actual (or current) level of performance with that goal or standard Engage in appropriate action which leads to some closure of the gap Sadler, D.R. (1989). Formative assessment and the design of instructional systems. Instructional Science. 18, 119-144. Theoretically, our ability to investigate this problem may be helped by drawing on Sadler’s (1989:121) ‘indispensible conditions for improvement’. These are: that students grasp their tutors’ standards, they can ‘judge the quality of what they are producing’ against those standards and they are able to take action to improve their work. In particular, we should note his emphasis on students developing an understanding of their tutors’ concept of quality as well as understanding the feedback they receive. This concept, which we might reasonably refer to as a tutor’s standards, provides the context for feedback: ‘If students cannot grasp the meaning of assessment criteria/ standards, feedback intended to guide them on achieving those standards may be incomprehensible and therefore ineffectual’ (Handley et al, 2011; 96). This argument regarding students’ need to understand ‘the rules of the game’ is now well rehearsed in the research literature (Northedge check which one, Rust et al, Bloxham and West) with Hunter and Docherty (2011, A&E, 1: 112) arguing that students who hold the same beliefs as staff about work fare well in assessment. However, a contention of this chapter is that our commonplace approaches to guidance and feedback do not help students to understand the required standard and their achievement in relation to that standard. Indeed, in our drive to transparency, perhaps we have created a mismatch between tutors’ conceptions of quality as expressed in guidance and feedback to students with the conceptions used in marking their work.

Learning Outcome Assessment Criterion Pass/threshold standards descriptor Differentiated standards descriptors A statement of what a learner should know, do or understand. A statement of which aspects of learner work will be judged, in relation to the learning outcomes. A statement of the typical pass requirement, in relation to the learning. outcomes. A description of what the learner must do to achieve a particular grade/mark against the assessment criterion

Contention 2 Assessment criteria, rubrics, grade descriptors and statements of standards have been partially developed with the admirable intentions of making assessment more transparent to students. However, the use of such artefacts to guide students in their assignments is simply not enough and are often poorly matched to the nature of the learning assessed at this level and our knowledge of professional judgement in grading.

Why standards matter Video created by The University of Hong Kong for the course "University Teaching". 2017 https://www.coursera.org/learn/university-teaching/lecture/TD6pE/interview-with-prof-royce-sadler-on-understanding-standards

Activity 2: Marking practices Working in pairs, preferably with someone you don’t know and is from a distinctly different subject discipline, discuss your marking practice and in doing so answer the following questions: How do you approach marking? How do you respond to an assignment that is poorly formatted and written? How do you benchmark/standardise your marking? Do the marking / standards descriptors accurately reflect the basis on which you make judgements? If you mark as part of a team how do you ensure you have a shared understanding of what is good? How are new staff inducted into the Note concerns that emerge from this research about the potential for different standards as in notes below leading to our research on understanding and use of standards in practice, the focus on external examiners was taken as these are the people who are given a key role in safeguarding standards. assessment criteria, grade descriptors, statements of standards and marking schemes: multiple terms used interchangeably , muddled as concepts Emphasis on ‘internalised’ standards: ‘internalised’, ‘absorbed’, ‘instinctively’, ‘got a sense of’, ‘in my mind’, ‘subliminal’, ‘rooted in my mind’, ‘got a mind set’, ‘implicit’, ‘have things in our heads’, ‘feel’, ‘familiar’ and ‘an understanding’.

What our assessment guidance implies Reality of academic judgement Bloxham, S & Boyd P (2012) Accountability in grading student work: British Educational Research Journal, 38(4): 615-634 What our assessment guidance implies Reality of academic judgement Explicit documentation of standards Internalised, tacit standards Criterion-referenced grading Norm-referencing of judgement needed/ interpretation of criteria Analytical judgement Holistic professional judgement Broad consensus on standards possible Individualised standards or localised consensus Note concerns that emerge from this research about the potential for different standards as in notes below leading to our research on understanding and use of standards in practice, the focus on external examiners was taken as these are the people who are given a key role in safeguarding standards. assessment criteria, grade descriptors, statements of standards and marking schemes: multiple terms used interchangeably , muddled as concepts Emphasis on ‘internalised’ standards: ‘internalised’, ‘absorbed’, ‘instinctively’, ‘got a sense of’, ‘in my mind’, ‘subliminal’, ‘rooted in my mind’, ‘got a mind set’, ‘implicit’, ‘have things in our heads’, ‘feel’, ‘familiar’ and ‘an understanding’.

The challenge How do we communicate the reality of our professional judgement in a way which is understandable and credible to students? Commonplace approaches to guidance and feedback do not help students to understand the required standard and their achievement in relation to that standard. Note concerns that emerge from this research about the potential for different standards as in notes below leading to our research on understanding and use of standards in practice, the focus on external examiners was taken as these are the people who are given a key role in safeguarding standards. assessment criteria, grade descriptors, statements of standards and marking schemes: multiple terms used interchangeably , muddled as concepts Emphasis on ‘internalised’ standards: ‘internalised’, ‘absorbed’, ‘instinctively’, ‘got a sense of’, ‘in my mind’, ‘subliminal’, ‘rooted in my mind’, ‘got a mind set’, ‘implicit’, ‘have things in our heads’, ‘feel’, ‘familiar’ and ‘an understanding’.

How can we support students to gain an understanding of standards? Video created by The University of Hong Kong for the course "University Teaching". 2017 https://www.coursera.org/learn/university-teaching/lecture/TD6pE/interview-with-prof-royce-sadler-on-understanding-standards

Activity 3: Your practice Looking at your own teaching / module – reflect on the score you derived for AL1 and consider the following questions: Are your marking/grading criteria fit for purpose ? How effectively do you support / enable students understanding of standards and criteria? How effectively do you ensure that your module team has a shared understanding of what constitutes good? Thinking about 1,2 and 3 what changes will you make to the delivery of your module? Note concerns that emerge from this research about the potential for different standards as in notes below leading to our research on understanding and use of standards in practice, the focus on external examiners was taken as these are the people who are given a key role in safeguarding standards. assessment criteria, grade descriptors, statements of standards and marking schemes: multiple terms used interchangeably , muddled as concepts Emphasis on ‘internalised’ standards: ‘internalised’, ‘absorbed’, ‘instinctively’, ‘got a sense of’, ‘in my mind’, ‘subliminal’, ‘rooted in my mind’, ‘got a mind set’, ‘implicit’, ‘have things in our heads’, ‘feel’, ‘familiar’ and ‘an understanding’.

And finally...... Please spend a few minutes to reflect on the session, your key learning points and the actions you will take as a result. Note concerns that emerge from this research about the potential for different standards as in notes below leading to our research on understanding and use of standards in practice, the focus on external examiners was taken as these are the people who are given a key role in safeguarding standards. assessment criteria, grade descriptors, statements of standards and marking schemes: multiple terms used interchangeably , muddled as concepts Emphasis on ‘internalised’ standards: ‘internalised’, ‘absorbed’, ‘instinctively’, ‘got a sense of’, ‘in my mind’, ‘subliminal’, ‘rooted in my mind’, ‘got a mind set’, ‘implicit’, ‘have things in our heads’, ‘feel’, ‘familiar’ and ‘an understanding’.

Thank you Dr Graham Holden, Director of Learning and Teaching, Directorate of Learning Enhancement and Academic Development https://blogs.shu.ac.uk/lead/ g.j.holden@shu.ac.uk @GrahamJHolden