Oxford Centre for Staff and Learning Development Developing a variety of assessment methods Chris Rust Oxford Centre for Staff and Learning Development Oxford Brookes University

Oxford Centre for Staff and Learning Development Constructive alignment - issues of validity “We continue to assess student learning - and to graduate and certify students much as we did in 1986, 1966, or 1946, without meaningful reference to what students should demonstrably know and be able to do” (Angelo, 1996) “Assessment systems dominate what students are oriented towards in their learning. Even when lecturers say that they want students to be creative and thoughtful, students often recognise that what is really necessary, or at least what is sufficient, is to memorise” (Gibbs, 1992)

Oxford Centre for Staff and Learning Development Constructive alignment - what is it? “The fundamental principle of constructive alignment is that a good teaching system aligns teaching method and assessment to the learning activities stated in the objectives so that all aspects of this system are in accord in supporting appropriate student learning” (Biggs, 1999)

Oxford Centre for Staff and Learning Development Constructive alignment: 3-stage course design What are “desired” outcomes? What teaching methods require students to behave in ways that are likely to achieve those outcomes? What assessment tasks will tell us if the actual outcomes match those that are intended or desired? NB Importance of consistent use of the same appropriate verb in all three stages This is the essence of ‘constructive alignment’ (Biggs, 1999)

Oxford Centre for Staff and Learning Development Why variety? Constructive alignment and validity Traditional assessment only assesses a narrow range of skills Fairness Logic of outcomes-based course design Choice and variety can interest & motivate (& aid flexible learning) Be strategic Economical & efficient Student learning & skills development (& retention)

Oxford Centre for Staff and Learning Development Student learning and strategic assessment “Assessment defines what students regard as important, how they spend their time and how they come to see themselves as students and then as graduates If you want to change student learning then change the methods of assessment” (Brown, G et al, 1997)

Oxford Centre for Staff and Learning Development Maximising student potential 1 - Clarify expectations/keep them busy “Setting students assignments as soon as soon as they arrive at university could help cut dropout rates………integrate students into university life as quickly as possible. This involves making them aware of the quality and quantity of work expected from them…..The freedom…is too much for some and they probably need more structure in the first year.” Higher, 24/1/03 Reporting Student Transition and Retention (STAR) project (in 5 universities)

Oxford Centre for Staff and Learning Development Maximising student potential 2 - Pace learning Multiple-staged assignments Self-assessment Peer-assessment CAA

Oxford Centre for Staff and Learning Development Maximising student potential 3 - Skills development information retrieval ICT skills academic writing skills time & project management self-assessment marking exercises/peer assess reflection group skills etc……

Oxford Centre for Staff and Learning Development Maximising student potential 4 - Positively reinforce/allow for early failure (a) Low self-belief can adversely affect achievement (leading to drop-out): Believe failure due to lack of intelligence Leads to ‘learned helplessness’ Difficult tasks prompt giving up Overly concerned with ‘saving face’ Mantz Yorke, based on study of six institutions. ILT symposium on widening participation and promoting student retention, 27th September, 2001

Oxford Centre for Staff and Learning Development Maximising student potential 4 - Positively reinforce/allow for early failure (b) Possible assessment solutions: Adjust the curriculum to foster development Set sub-goals for longer assignments Allow for ‘slow learning’ in the programme’s assessment strategy Make first semester assessments primarily formative Mantz Yorke, based on study of six institutions. ILT symposium on widening participation and promoting student retention, 27th September, 2001

Oxford Centre for Staff and Learning Development Maximising student potential 5 - Feedback 11 conditions under which assessment supports learning (Gibbs and Simpson, 2002) 5.Sufficient feedback is provided, both often enough and in enough detail 6.The feedback focuses on students’ performance, on their learning and on actions under the students’ control, rather than on the students themselves and on their characteristics 7.The feedback is timely in that it is received by students while it still matters to them and in time for them to pay attention to further learning or receive further assistance 8.Feedback is appropriate to the purpose of the assignment and to its criteria for success. 9.Feedback is appropriate, in relation to students’ understanding of what they are supposed to be doing. 10.Feedback is received and attended to. 11.Feedback is acted upon by the student

Oxford Centre for Staff and Learning Development 5 Strategies for diversifying assessment Change the criteria Change the task Mechanise assessment Assess groups Involve the students See: Brown, S., Rust, C. & Gibbs, G (1994)) Strategies for diversifying assessment in higher education, Oxford, Oxford Centre for Staff Development Rust, C. (2001)A briefing on assessment of large groups: LTSN Generic Centre Assessment Series No. 12, York, LTSN, available at: full_record&section=generic&id=12

Oxford Centre for Staff and Learning Development Involve the students - 1 marking exercise Immediate resultsparticipants av. mknon participants av. mk. Cohort 1 (99/00) Cohort 2 (00/01) Cohort 3 (01/ Results 1 year later Cohort Cohort Rust, C., Price, M & O’Donovan, B.(2003) "Improving students’ learning by developing their understanding of assessment criteria and processes” Assessment and Evaluation in Higher Education, Vol. 28, No. 2

Oxford Centre for Staff and Learning Development Involve the students - 2 self assessment Strengths of this piece of work Weaknesses in this piece of work How this work could be improved The grade it deserves is….. What I would like your comments on

Oxford Centre for Staff and Learning Development Involve the students - 3 peer marking using model answers Scenario: Engineering students had weekly maths problem sheets marked and problem classes Increased student numbers meant marking impossible and problem classes big enough to hide in Students stopped doing problems Exam marks declined (Average 55%>45%) Solution: Course requirement to complete 50 problem sheets Peer assessed at six lecture sessions but marks do not count.Exams and teaching unchanged Outcome: Exam marks increased (Av. 45%>80%)

Oxford Centre for Staff and Learning Development Involve the students - 4 peer feedback Scenario Geography students did two essays but no apparent improvement from one to the other despite lots of tutor time writing feedback Increased student numbers made tutor workload impossible Solution: Only one essay but first draft required part way through course Students read and give each other feedback on their draft essays Students rewrite the essay in the light of the feedback In addition to the final draft, students also submit a summary of how the 2nd draft has been altered from the1st in the light of the feedback Outcome: Much better essays

Oxford Centre for Staff and Learning Development So why don’t we? Not convinced (personally) Institution/colleagues/students not convinced Tradition/inertia/no incentive to change Time and/or other costs? Too difficult (need help/training) Not allowed (regulations/procedures) Other