Looking at assessment dilemmas What factors do we need to consider in assessment design? Professor Margaret Price NTF Dr Erica J. Morris PFHEA ARU TeachMeet January 2017: 25 January, Cambridge; 27 January, Chelmsford

Overview Why assessment dilemmas? Four key dilemmas On balance considerations Conclusions ARU’s Assessment Protocol

Why assessment dilemmas? The detail beyond principles Assessment design considerations Subject, discipline or professional context Your students ARU Assessment Protocol Example principles: Assessment and feedback are integral to learning as part of personal and professional development Assessment is a complex process and, consequently, the development of assessment literacy is essential for students and staff Moving from principles to decisions Assessment design considerations: Purpose of assessment Validity Reliability Transparency Overall assessment strategy for module and course (programme) e.g. practical implications, such as scheduling Image from: http://www.inspirationforthespirit.com/the-road- not-taken/

Assessment dilemma one: enabling Independent learning Scaffolding for learners Independent learning Arguments in favour: More challenging for students Encourages creativity and originality Aligned with nature of higher education Makes marking more interesting Arguments against: Too risky for students Might not meet ‘service’ expectations – students expect a ‘good service’ Does not necessarily support students’ need to have an understanding of elements of quality Scaffolding for learners Models approach to assessment for students (illustrates to students how to prepare for an assignment) Develop students’ learning techniques Marking can be more straightforward Convergence of student work (guided therefore could be similar work) Minimises element of discovery for students Increases student dependency on staff Photos: Erica Morris

On balance: independence vs scaffolding The decision for assessment design depends on: the place in the course – considering student transition, level of study, expected standard of work learning outcomes relating to expectation of independence extent of student preparation for the assessment method students’ understanding of the purpose, nature and techniques of assessment (assessment literacy) Practice and rehearsal of assessment tasks (including opportunities to develop skills) are emphasised by assessment for learning perspectives (e.g. Sambell et al, 2013) a key consideration in terms of employing inclusive assessment practices Extent of student preparation - have students already completed this type of assessment before? Consideration of assessment strategy of the programme (course) or module Standard of work – e.g. dissertation work is guided as unlikely student would reach required standards without guidance See the Anglia assessment album: snapshots on Developing assessment literacy

Assessment dilemma two: employing Conventional method Innovative assessment Conventional assessment method Arguments in favour: Reliability Can be resource efficient Meet student expectations (‘collusion’ with staff) Fits with the discipline (e.g. lab reports) Arguments against: No evidence of greater reliability Often not designed for learning Do not cover range of learning outcomes Some methods may ‘favour’ some groups of students (links with inclusive assessment practices) Innovative assessment More likely to be authentic assessment Interests and motivates students Can align fully with learning outcomes More interesting for staff Can produce efficiency gains – e.g. includes peer assessment More variety is not always a good thing (Gibbs and Dunbar-Goddet, 2007) (‘over diversifying’ assessment) Can increase student stress if lack of preparation opportunities (Gibbs and Dunbar-Goddet, 2007) Up front development and persuading colleagues can take time Image from: http://blog.cengage.com/tips-for-students-how- to-focus-listen-attentively-to-college-lectures/

On balance: conventional vs innovative methods Each method needs to be evaluated in relation to: learning outcomes resources available other methods in use as part of the course time and opportunities for student preparation ‘fit’ with the discipline (although this may need to be challenged) student preference for authentic assessment Examples of (more) innovative assessments writing a journal, blog or reflective diary producing a briefing paper, newspaper article or action plan working on a group presentation or conference poster undertaking a case study analysis organising and giving student-led seminars developing information leaflets or learning packages (e.g. for students in schools or related to health promotion) From: snapshot on Inclusive assessment See the Anglia assessment album: snapshot on Inclusive assessment

Assessment dilemma three: using Expert judgement Expert judgement Arguments in favour: Meets (traditional) student expectations Fits easily with quality systems Staff are experienced markers and know their subjects Staff teams can compare marks Arguments against: Little evidence of expert judgment being more reliable Does not encourage self regulation Encourages dependence on staff Self- and peer-assessment Develops students’ understanding of standards Develops students’ assessment literacy Encourages development of independent learning skills Can reduce staff marking time Develops graduate skills (including communicating and working with others, and evaluation of self and others) Requires time for necessary preparation and set up Students can be worried about showing their work to other students Students may perceive use as staff ‘laziness’ Students may think peer-assessment is ‘unfair’ Self- and peer- assessment Image from: http://www.sunseed.org.uk/projectpack/2014/05/16/sunseed- library-and-study-space/ Image from: http://tradin.co.uk/

On balance: expert judgement vs self- and peer-assessment Consideration needs to be given to: students’ concerns in using self- and peer-assessment through briefing opportunities with discussion preparation of students to enable them to undertake self- and peer- assessment well and with confidence emphasising to students the benefits of self- and peer-assessment (e.g. in terms of graduate attributes and employability) The assessment strategy with regard to self- and peer- assessment opportunities on the course Designing authentic or work integrated assessment that involves varied audiences (see Collaborate project): peers, clients, experts, employers See the Anglia assessment album: snapshot on Assessing groupwork

Assessment dilemma four: assessment criteria Providing developed criteria Students creating criteria Providing developed criteria Arguments in favour: Aims to provide clarity and transparency of assessment and marking for students (e.g. see NUS, n.d.) Belief that helps to improve consistency in marking process Can be used as a basis for discussion with students about what is expected in their work Arguments against: It is difficult to fully specify assessment standards (i.e. in terms of student attainment) Students may become over-reliant on marking guidance and criteria, which may have implications for developing as independent learners Providing detailed assessment criteria do not reflect practices in the real world or work place (relates to designing authentic assessment) Students creating criteria Engages students as they are motivated do well on assessments Enables students to see more fully the issues related to achieving consistency in marking (and the limitations of using criteria) Helps students to understand what is expected of them in terms of quality work (can use criteria with exemplars, for instance) (relates to assessment literacy) Could create confusion for some students who may not see that it is their role to be involved in creating criteria (as they are not the expert) Unlikely to be effective if students are not prepared Students may become ‘bogged down’ in the detail of devising criteria (as a ‘science’) Photo from assessment workshop at ARU: Erica Morris

On balance: assessment criteria Carefully consider when students might have opportunities to be involved in creating criteria through working with other students in class To fully understand what is expected students need opportunities for activities involving the use of assessment criteria Is there sufficient opportunity for staff and students to share and understand the criteria? See the Anglia assessment album: snapshots on Developing assessment literacy and Marking matters

Conclusions There is no one answer to suit all contexts Take an evidence-informed approach for complex decision making in assessment design Once informed, discuss your ideas with colleagues Validity matters: learning outcomes are the key guide to resolving dilemmas

References Collaborate Project on designing work integrated assessment: http://www.exeter.ac.uk/staff/development/academic/resources/assessment/wia/ Gibbs, G. and Dunbar-Goddet, H. (2007) The effect of programme assessment environments on student Learning. York: Higher Education Academy. Higher Education Funding Council for England (HEFCE) (2016) A new National Student Survey for 2017. Annex A: Final list of NSS 2017 questions. National Union of Students (NUS) (n.d.) Assessment and feedback benchmarking tool. Price, M., Rust, C., O’Donovan, B., Handley, K. and Bryant, R. (2012) Assessment Literacy: The Foundation for Improving Student Learning. Assessment Standards Knowledge exchange (ASKe), Oxford Centre for Staff and Learning Development, Oxford Brookes University. Sambell, K., McDowell, L. and Montgomery, C. (2013) Assessment for Learning in Higher Education. Abingdon, Oxon, UK: Routledge.