Assessment for learning: the benefits of generating feedback David Nicol Professor of Higher Education Centre for Academic Practice and Learning Enhancement (CAPLE] Director, Peer project (www.reap.ac.uk) University of Strathclyde, Scotland University of Cadiz, March 9-11, 2011

Plan Background Re-engineering Assessment Practices (REAP) project www.reap.ac.uk/reap/index.html Concepts and example of practice Institutional considerations PEER project www.reap.ac.uk/peer.aspx Discussion

Background Departments and faculties: educational improvement projects, including REAP project Policy/strategy: development of educational policy and principles of assessment & feedback (based on REAP) Students: ‘Feedback as dialogue’ campaign Quality procedures:– redesigning course documentation and review procedures PEER project – developing students ability to evaluate the quality of each other’s work See www.reap.ac.uk

REAP : Re-engineering Assessment Practices Scottish Funding Council for Universities (£1m) 3 Universities - Strathclyde, Glasgow & Glasgow Caledonian Large 1st year classes (160-600 students) A range of disciplines (19 modules ~6000 students) Many technologies: online tests, simulations, discussion boards, e-portfolios, e-voting, peer/feedback software, VLE, online-offline Learning quality and teaching efficiencies Assessment for learner self-regulation www.reap.ac.uk

Background (1) Gibbs, G. & Simpson, C (2004) Conditions under which assessment supports students’ learning, Learning and Teaching in Higher Education, 1, 3-31. See: Formative Assessment in Science Teaching (FAST) project at: http://www.open.ac.uk/science/fdtl

Gibbs and Simpson (2004) Assessment tasks [Conditions 1-4] Capture sufficient study time (in and out of class) Are spread out evenly across timeline of study Lead to productive activity (deep vs surface) Communicate clear and high expectations i.e concern here is with ‘time on task’ how much work students do - their active engagement in study

Background (2) Literature Review Nicol, D. & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 34 (1), 199-218 Background Student Enhanced Learning through Effective Feedback [SENLEF] project funded by HE Academy REAP project: www.reap.ac.uk

Rethinking assessment and feedback 1. Consider self and peers as much as the teacher as sources of assessment and feedback Tap into different qualities than teacher can provide Saves time Provides considerable learning benefits (lifelong learning) 2. Focus on every step of the cycle: Understanding the task criteria Applying what was learned in action 3. Not just written feedback: Also verbal, computer, vicarious, formal and informal

Seven principles of good feedback Clarifies what good performance is (goals, criteria, standards). Facilitates the development of reflection and self-assessment in learning Delivers high quality information to students: that enables them to self-correct Encourages student-teacher and peer dialogue around learning Encourages positive motivational beliefs & self esteem Provides opportunities to act on feedback Provides information to teachers that can be used to help shape their teaching (making learning visible) Source: Nicol and Macfarlane-Dick (2006)

Principle 1: Clarify what good performance is (the context of dialogue) EMPOWERMENT/ SELF-REGULATION Students create criteria Students add own criteria Students identify criteria from samples of work Exemplars of different performance levels provided Students rephrase criteria in own words Provide document with criteria ENGAGEMENT 10

Principle 2: Facilitate reflection and self-assessment in learning EMPOWERMENT/ SELF-REGULATION Students create MCQs including feedback for right and wrong (distractors) answers Students self-assess using MCQs and confidence ratings Students self-assess own performance using online MCQs ENGAGEMENT 11

Two meta principles 1. Meta-PRINCIPLE 1: time and effort on task (structured engagement) i.e. steers on how much work to do and when – Gibbs and Simpson 4 conditions 2. Meta-PRINCIPLE 2: developing learner self-regulation (empowerment/self-regulation) i.e steers to encourage ownership of learning – the seven principles discussed above. Key task for teacher is to balance 1 and 2

Example: Psychology

Psychology 560 first year students 6 topic areas (e.g. personality, classical conditioning), 48 lectures, 4 tutorials, 12 practicals Assessment; 2 x MCQs (25%), tutorial attendance (4%), taking part in experiment (5%), essay exam (66%)

Problems identified No practice in writing skills but required in the exam More detail provided in lectures than mentioned in exams (not enough independent reading) No feedback except on Multiple Choice Questions (percent correct) Didn’t want to increase staff workload

Psychology Redesign Discussion board in Learning Management System Students in 85 discussion groups of 7-8, same groups throughout year Also open discussion board for class Friday lectures cancelled – discover material themselves Series of online tasks

Structure of group tasks 6 cycles of 3 weeks (one cycle x major course topic) First week: ‘light’ written task (e.g. define terms) = 7 short answers (all answer) Second week = guided reading Week three: ‘deep’ written task: students collaborate in writing a 700-800 word essay on the same topic. Within each week: The Monday lecture – introducing material Immediately after lecture, task posted online – for delivery the following Monday Model answers (selected from students) posted for previous week’s task

The teaching role Participation in the discussions was compulsory but not marked (in subsequent years there was 2% mark for participation) Course leader provided general feedback to the whole class – often motivational He encouraged students to give each other feedback The group discussions were not moderated but monitored for participation

An example of ‘deep’ task The Task – 800 word essay: Assess the strengths and weaknesses of Freud’s and Eysenck’s theories of personality. Are the theories incompatible? readings suggested questions provided which all students should try 19

Relation to the Gibbs & Simpson’s four assessment conditions Tasks require significant study out of class (condition 1) Tasks are distributed across topics and weeks (condition 2) They move students progressively to deeper levels of understanding (condition 3) There are explicit goals and progressive increase in challenge (condition 4)

Relation to 7 feedback principles Standard format and model answers provide progressive clarification of expectations (principle 1) Students encouraged to self-assess against model answer (principle 2) Course leader provides motivational and meta-level feedback and selects model answers (principle 3) Online peer discussion aimed at reaching consensus is core feature of design about response (principle 4) Focus on learning not just marks, sense of control/challenge enhanced motivation (principle5) Repeated cycle of topics and tasks provide opportunities to act on feedback (principle 6) VLE captures all interactions allowing course leader to monitor progress and adapt teaching (principle 7)

Benefits Students worked exceptionally hard Written responses of exceedingly high standard Students took responsibility for learning High levels of motivation: atmosphere in class improved Online interactions showed powerful ‘scaffolding’ and community building Feedback with 560 students through peer and self-feedback (model answers) Easy for tutors to monitor participation Improved mean exam performance (up from 51-59%, p<0.01) weaker students benefit most

Has it worked? 23

24

25

Guidelines for Implementation A single principle or many? Tight-loose – maintain fidelity to the principles (tight) but encourage disciplines to develop their own techniques of implementation (loose) Balance teacher feedback with peer and self-generated feedback Focus on developing students’ own ability for critical evaluation Create opportunities for ‘learning communities’ to emerge The more actively engaged students are, the better the course design Interest in FA Summative A can undermine the value of fa e.g.

Developments since REAP Principles of Assessment and Feedback approved by University Senate and embedded in policy (2008) Use of principles to inform curriculum renewal and Quality Assurance processes ‘Feedback as Dialogue’ campaign to gain commitment of students PEER Project (Peer Evaluation in Education Review) Work on the way we document our courses and programmes

29

Peer Review in Education Evaluation [PEER] The aims of the PEER project are to: Review evidence base for peer review Develop educational designs for peer review (and self-review) Identify software support for peer review Pilot implementations of peer review with large student numbers Produce guidelines for higher education – why do it, how to do it, pitfalls and solutions and software possibilities. see http://www.reap.ac.uk/peer.aspx 30

PEER Project: Core Ideas All graduate development in higher education requires that: Students learn to evaluate critically the quality and impact of their own work (e.g. academic texts, problem solutions, designs) Students learn to evaluate critically the work of others (e.g. peer review and peer feedback) Ref: Nicol, D (2010) The foundation for graduate attributes: developing self-regulation through self and peer assessment. www.reap.ac.uk/peer.aspx

The research on peer feedback Mainly about peers marking each others work Where peer feedback is the focus it is usually about peers augmenting teacher feedback by increasing the quantity and variety of feedback

The focus of PEER project Scenarios where students make evaluative judgements about the work of peers and provide a feedback commentary, usually written Not talking about .....collaborative tasks where students give each other informal feedback .....scenarios where the focus is on students evaluating each other’s performance in group working .....scenarios where the focus is on students grading/marking each other’s work although some rating of performance might be part of the peer design

Benefits of feedback construction (1) Constructivist rather than transmission perspective Producing is cognitively more demanding than reading feedback from others: students cannot be passive Students actively exercise assessment criteria (process/reprocess) from many different perspectives See how others tackle assignments and learn that quality can be produced in different ways Develop writing skills through commentaries Deepens critical thinking in the discipline

Benefits of feedback construction (2) Learn to assess own work, to monitor and evaluate their own productions - same skills involved Develops self-regulation – for life beyond university. Changes the power relationship in class (shares responsibility across students). Can develop learning communities

Giving and receiving feedback Giving and receiving feedback within same domain enriches the whole process. Students are able to compare the feedback they receive with the feedback they produce And learn how different reviewers perceive their work – a single source of feedback is never satisfactory Can develop a sense of shared responsibility across students for learning

Example: peer feedback Students write essay on one topic from three Each student provides feedback on three essays in another topic anonymously using rubric The rubric: write a short summary of the essay, comment on and rate (four point scale) the structure, arguments, evidence, writing, suggest ways of improving the essay. Students receive peer reviews of their own essays Then review, comment on and rate their own essay using same rubric. Graded: for participating in the task, for their own essay and for their review of it. Finally students rate 3 reviews (on others’ work) and comment on how useful they think they would be to author. Software used to support administration

Six learning opportunities in peer review Producing the target assignment Reviewing and producing peer feedback Receiving feedback from peers Responding to peer reviews (on own reviews of those given to others) Self-reviewing one’s own work (e.g. after peer review) Receiving feedback from the teacher – e.g. on the assignment, peer review or self-review 1-3 comprise the essentials of peer review while 4-6 are further options. 38

Principles of effective peer review Peer review should Engage students in active use of criteria and standards Involve students in constructing commentaries in relation to peer judgements, not just marks Lead to practice in both analytic (componential) and holistic (configurational) judgements about quality Facilitate dialogue around the object and quality of the review Ensure an atmosphere of mutual trust and respect Integrate self-review activities within peer review designs Ensure that making peer judgements is a regular activity and not a one-off event www.reap.ac.uk/PEER.aspx

Some of my Publications Nicol, D (2010) The foundation for graduate attributes: developing self-regulation thorugh self and peer assessment, QAA Enhancement Themes, Scotland Nicol, D (2010) From monologue to dialogue: Improving written feedback in mass higher education. Assessment and Evaluation in Higher Education. 35(5), 501 -517 Nicol, D and Draper, S (2010) A blueprint for transformational organisational change in HE: REAP as a case study, Published by the Higher Education Academy, UK (see reap.ac.uk website) Nicol, D (2009) Transforming assessment and feedback: Enhancing integration and empowerment in the first year, Published by Quality Assurance Agency, Scotland (http://www.enhancementthemes.ac.uk/documents/firstyear/FirstYear_TransformingAssess.pdf Nicol, D (2009), Assessment for learner self-regulation: Enhancing achievement in the first year using learning technologies, Assessment and Evaluation in Higher Education, 34(3), 335-352 Nicol, D (2007), Laying the foundation for lifelong learning: cases studies of technology supported assessment processes in large first year classes, British Journal of Educational Technology, 38(4), 668-678 Nicol, D (2007) E-assessment by design: using multiple-choice tests to good effect, Journal of Further and Higher Education.31(1), 53-64. Nicol, D. & Milligan, C. (2006), Rethinking technology-supported assessment in relation to the seven principles of good feedback practice. In C. Bryan and K. Clegg, Innovations in Assessment, Routledge. Nicol, D, J. & Macfarlane-Dick (2006), Formative assessment and self-regulated learning: A model and seven principles of good feedback practice, Studies in Higher Education, 31(2), 199-218. See www.reap.ac.uk/resources.aspx for copies.