Enhancing staff-student dialogue about assessment in HE Prof. Kay Sambell, Northumbria University.

Students benefit from being explicitly supported to learn to make judgements about the quality of work rather than being dependent on others to perform that function for them (Sadler, 1989; Boud, 1995; Sambell & McDowell, 2006; Nicol, 2009; Carless, 2006). Self-regulation (Sadler, 1989) – The student comes to hold a concept of quality roughly similar to that held by the teacher – Is able to monitor continuously the quality of what is being produced during the act of production itself. – Has a repertoire of alternative moves or strategies from which to draw 2

How, in practice, do students on our courses know what is required in an assignment? How do they develop a ‘feel’ for standards? How do they ‘see’ /know, during the act of production, how they might improve their work? 3

ESCalate Pedagogy and Practice Development Grant: Exploring the Rules of Engagement via Exemplars. Collaborative project involving Joint Hons team at Northumbria and PGCE/Early Childhood team at Roehampton. Explored use of exemplars to improve students’ appreciations of ‘effective’ and ‘ineffective’ approaches to assignment writing in Education. 4

The case for using exemplars Offer students active, participative, dialogic experiences of assessment process (Rust, 2005) Build in opportunities for students to rehearse making and discussing judgments about the relative merits of different examples of student work. Make tacit, implicit or ‘invisible’ rules and expectations surrounding what lecturers require for success both visible and accessible to students. Lecturers’ notions of quality are often tacitly held, and require disciplinary and contextual interpretation if they are to be adequately understood (Bloxham & West, 2007) 5

Exemplars are not model answers – Exemplars are ‘key examples chosen so as to be typical of designated levels of quality or competence. The exemplars are not standards themselves but are indicative of them...they specify standards implicitly.’ (Sadler, 1989 cited in Handley et al, 2008, p44). 6

One case study in large-group teaching: Childhood Studies – Lecturers chose 4 x 500 word exemplars carefully to ‘Engineer in’ the essence of the discipline and illuminate tutors’ perceptions of effective and ineffective approaches. Represent ‘common mistakes novices make’, to bring them to visibility, trying to help students notice conceptual mistakes in time to change approaches, if necessary. 7

The Lecturers’ Perspectives – Exemplars 1-3 represented adequate responses of varying standard. – Exemplar 4 was chosen because it ‘misunderstood the key concept entirely.’ – All exemplars displayed errors of convention (spelling, citation, grammar). The lecturers saw these features as of less significance than deeper level features and the ability to think about and communicate ‘ideas that matter.’ 8

How were exemplars used in the case study? Preparation before workshop Students prepared short piece of writing (>1 side A4). 500 word response to the task ‘Explain the social construction of childhood to a 1 st year student.’ Phase 1 Brought this to session, where they were given 4 exemplars to read and discuss. Students engaged in criteria- generating activities Then asked to work in small groups, using criteria to place exemplars in rank order. Phase 2 Tutors revealed and discussed rationale for their rankings. Phase 3 Students asked to generate feedback for each exemplar& reflect on how, in the light of the tutor dialogue, they might change own work. 9

Few close-up studies of how students interact with exemplars – Most (evaluative) data gathered retrospectively. Our project used classroom observation to illuminate – What illustrative qualities, strengths and weaknesses did students originally notice/discuss in exemplars? – What impact did dialogue with lecturers and peers appear to have on students’ views of the exemplars on their views of assessment practice at university level? on their own approaches to assessment? 10

Research methods 2 researchers observed the whole lecture group. The dialogue of one student-group (4 students) working on the classroom tasks was recorded verbatim, another (5 students) was observed, with field notes taken. Further 6 interviews, with students from various group, conducted after the session. 11

Findings: Student perspectives. In interview, all students reported exemplars useful activity “I think seeing it just makes you understand it more. Like, someone can stand there and say, 'You shouldn't do this and that' but until you've actually seen it then you don't know what that looks like.” “I was just writing how I thought it should be done, just in my own head. But now I know what they are looking for and what I should be, how I should be writing, so yes, I think it's been helpful.” 12

Intention to alter their approaches ‘We saw what other people had done, and that gave you more ideas’. ‘I used rather informal language, and I saw I have to change that’. ‘I would look more for theory, in future, read a lot more to look for different views.’

Observation of phase 1: Students only paid attention to surface technical features... Students initially looked for ‘strategic-related features’: by focusing on the form (e.g. the essay) with no reference to broader learning and understanding in the discipline. – (Harrington et al, 2006)

– “I thought some of them immediately looked a bit like they weren't going to be quite right. The one that had bullet points in it. I was a bit, 'Well that's a bit strange for an essay…” – “There were a couple where a lot of it started with 'I' and 'My' and that's just immediately, when you look at, well, when we looked at those two they kind of jumped out as, 'Oh-oh, this might not be great!’” – “References was something that we noticed really quickly. One just had it in the text not at the bottom and some didn't put them, like, the people in brackets in the text either. I don't think any of them got them all correct.’ 15

Students’ inability to rank exemplars in same order as teachers. Classroom observation revealed that high proportion of students either – ‘muddled’ the two ‘best’ exemplars. – preferred the ‘worst’ exemplar- 4- and disliked the ‘best’ (exemplar 1). – ‘I had the best as worst! To me, it looked really bare!’ 16

When tutors discussed their ranking students began to realise that tutors were privileging.... ‘deep-related features’: developing subject understanding, insight into subject; appreciating engagement with different viewpoints. – (Harrington et al, 2006)

How did students respond to their lack of alignment with teachers’ views? Looking again: “When I first read number 4, I thought it was really good, I liked that she said what she thought. But then like, I went back and read it again and it totally doesn't follow the question or anything.” Trying to ‘see’ at a deeper level. Those who could ‘see’ tried to explain to others. 18

But for most ‘seeing afresh’ entailed lengthy discussion... Peer dialogue immediately after tutor discussion. C. I want to know why they think this one's the best one. B. It's good, but in my eyes it's a bit like, it doesn't flow. A. Yes. B. It's a bit 'bitty'… C. Should have a full stop after 'culture'… B. That's grammar! A. I'm sure you could get over a little grammar mistake like that. B. No, no… Look, 'She's got a full stop after 'culture' but it needs to be there. Peer dialogue 25 minutes after tutor discussion. A: See, all of these examples in here talk about different things. Like, she's talking about nature / nurture and how children are influenced by the birth order. And she's talking about race, gender, culture, class and time. And she's just got a general sort of… B …how do you know what's supposed to be right? A: [Pause]. 4 talks about development. It's not wrong. It's just not right. 19

Thanks for listening! For further information about the Rules of Engagement project please contact the project director or see the project pages on the ESCalate website: or read the case study in full in the ESCalate Discussion Paper ‘Rethinking Feedback: an Assessment for Learning Perspective.’ 20

Lesson learned? One thing of importance/value in this kind of ESCalate-supported work, which will continue to be important in the future. 21