Re-engineering Assessment Practices in Scottish Higher Education [REAP] Dr David Nicol, Project Director Centre for Academic Practice and Learning Enhancement University of Strathclyde (CAPLE) University of Strathclyde, 25 th April 2006

Scottish Funding Council: Invitation £6m for HE/FE – competitive bidding E-learning transformational projects Embedding and substitution New approaches to teaching/ learning Measurable benefits to institution/sector Partnerships with other HEIs/FEIs

Rationale for the REAP proposal  Assessment – key driver of student learning  Assessment is a major cost in HE: economies of scale limited  Assessment influences a wide range of organisational, pedagogical and business processes in HE

Aims of REAP Project Re-engineer assessment practices in 3 HE institutions using supportive technologies. Enhanced approaches - self, peer, tutor, to enhance motivation & learner self-regulation Integrate technologies – online tests, simulations, e-voting, e-portfolios, VLEs, admin systems and online-offline interactions Focus is on large 1 st year classes Improve learning quality & show efficiencies Embed new processes in institution

Partners and roles  Strathclyde University: 5 departments – one per faculty with large 1 st year classes ( Mechanical Engineering, Marketing, Pharmacy, Psychology, Primary Education)  Glasgow Caledonian Business School (one faculty with range of implementations)  Glasgow University (developing electronic voting technologies)  Allows synergies and comparisons across disciplines /faculty/institution.

Research on Assessment in HE (1) Teaching/learning paradigm Transmission Assessment paradigm Transmission [teacher-centred] Constructivist [student-centred]

Some key research findings Students are always engaged in self- assessment/self-regulation of their own learning (Winne, 2005; Black & Wiliam, 2005). Logically entailed by constructivist ideas. The act of using teacher feedback implies that self-assessment must be present (Sadler, 1983, Nicol and Macfarlane-Dick, 2006) Feedback in HE is being significantly reduced so how are students still learning? The question is how we can scaffold students learning so they become better at self-regulation of learning (Lajoie, 2005)

References Nicol, D. & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 34 (1), Nicol, D. & Milligan, C. (2006), Rethinking technology-supported assessment in relation to the seven principles of good feedback practice. In C. Bryan and K. Clegg, Innovations in Assessment, Routledge.

Scaffolding self regulation: the seven principles of good feedback practice 1. Clarify what good performance is (goals, criteria, standards). 2. Facilitate reflection and self-assessment in learning 3. Deliver high quality feedback to students 4. Encourage peer and tutor dialogue around learning 5. Encourage positive motivational beliefs & self esteem through assessment 6. Provide opportunities to close the feedback loop 7. Use feedback information to shape teaching Source: Nicol and Macfarlane-Dick (2006) Formative assessment and self-regulated learning: A model and seven principles of good feedback practice, Studies in Higher Education

Black and Wiliam (2005) As well as assessing and marking (through discussion and clear guidance) their own work they also assess and mark the work of others. They do this in a very mature and sensible way.…. They take pride in clear and well presented work that one of their peers may be asked to mark. Any disagreement about the answer is thoroughly and openly discussed until agreement is reached. Alice, Teacher at Watford School, UK

Questions to ask about assessment Do students actively engage with assessment criteria and standards? Are there formal/informal opportunities for self and peer assessment processes? What kind of feedback is provided: does it help students to self-assess, self-correct? Does the feedback focus on learning goals rather than on marks? Is feedback acted upon? How is feedback used to shape teaching?

Educational basis of REAP  Enhanced formative assessment resulted in learning gains that were ‘among the largest ever reported’ (Black & Wiliam, 1998)  Assessment acts should establish the basis for students to undertake their own assessment activities in the future (Boud, 2000).  Students are already self-regulating their own performance; teachers must build on this capacity (Yorke, 2003; Nicol & Macfarlane-Dick, 2004/05)  Gibbs & Simpson (2004). FAST project

Definition (1) What do we mean by assessment? Assessment of learning (summative) – certification and progression. Assessment for learning (formative) – to support learning (developmental) ‘Assessment involves identifying appropriate standards and criteria and making judgements about quality’. (Boud, 2000)

Definitions (2) Who is involved in formative assessment and feedback? Tutor Peer External (e.g. placement supervisor) Computer generated Self

Sadler (1989) For feedback to benefit learning students must know: 1. What good performance is (goals, criteria) 2. How current performance relates to good performance (compare) 3. How to act to close the gap No 2 means that students ‘must already possess some of the same evaluative skills as the teacher’ (Royce Sadler, 1989).

The context of use of online tests (MCQs) 1.Students repeating MCQs (closing loop) 2.Case study with MCQs as open book (self- correcting) 3.Unique MCQs per student and sharing (peer dialogue) 4.Confidence ratings with MCQs (reflection/ self-assessment) 5.Students construct MCQs (creating criteria) 6.MCQs before lecture (just in time teaching) – testing shaping teaching

Questions and discussion