1. Enhancing students’ appreciation of academic literacy via exemplars. Prof. Kay Sambell, HCES. Northumbria Conference, September 2010

2. Overview Will outline a development project, recently funded by the Higher Education Academy’s Subject Centre, ESCalate, which explores ways of enabling students to make effective transitions to assignment writing in higher education. 2

3. Aims of project To involve students actively and explicitly in assessment processes – So students no longer passive recipients or helpless victims of university assessment practices. Help share lecturers’ understandings of the goals envisaged and relate these to the goals in the mind of the student (Nicol and Milligan, 2006) and help students internalize a discipline’s expectations/notions of quality (Gibbs, 2009) 3

4. Theoretical premises Useful to induct students into means of establishing levels of performance in given domain (Boud & Assocs, 2010). Active student engagement in assessing own learning essential to self-regulation (Sadler, 1989; Earl, 2003) – The student comes to hold a concept of quality roughly similar to that held by the teacher. – Is able to monitor continuously the quality of what is being produced during the act of production itself. – Has a repertoire of alternative moves or strategies from which to draw. 4

5. Helping students understand university assessment processes: it’s harder than it looks! Involves more than simply ‘publishing’ criteria. ‘Transparency’ seems to have become muddled with the idea of ‘writing things down’ (Orr, 2007: 646).  active, participative, dialogic experiences ‘Students need to learn about assessment in the same way they do about anything else’ (Rust et al, 2005). Important to make tacit, implicit or ‘invisible’ rules and expectations surrounding what lecturers require for success both visible and accessible to students. – Lecturers’ notions of standards require disciplinary and contextual interpretation if they are to be adequately understood (Bloxham & West, 2007) 5

6. More than ‘just’ a study skills issue Academic literacy = capacity to operate with fluency, confidence and competence within the identified domain. 6

7. Exemplars: a way forward? Exemplars are ‘key examples chosen so as to be typical of designated levels of quality or competence. The exemplars are not standards themselves but are indicative of them...they specify standards implicitly.’ (Sadler, 1989 cited in Handley et al, 2008, p44). 7

8. Constructing effective exemplars Handley et al (2008: 44) suggest that exemplars – may be complete assignments or excerpts. – may be authentic pieces of student work, or may be (re)constructed by staff (so as to illustrate specific pedagogic points in as transparent manner as possible). 8

9. In our case study Exemplars of previous student work chosen by lecturers to represent effective and ineffective work. Based on 500 word response to the task ‘Explain the social construction of childhood to a 1 st year student.’ – Exemplar 1 was chosen because it represented a ‘sound response,’ with a ‘clear thesis statement’ based on ‘an effective summary of challenging and appropriate literature.’ – Exemplar 2 represented ‘reasonable, but more limited work’, ‘based largely on repeating chunks of a basic introductory text.’ – Exemplar 3 was ‘a fair attempt, but tended to include detail, as illustrative material, which needed clearer introduction and explanation.’ – Exemplar 4 was chosen because it ‘misunderstood the key concept entirely.’ 9

10. Exemplars carefully chosen by lecturers to… – ‘Engineer in’ the essence of the discipline. Represent common mistakes novices make, to bring them to visibility, trying to help students notice conceptual mistakes in time to change approaches, if necessary. – All exemplars displayed errors of convention (spelling, citation, grammar). The lecturers saw these features as of less significance than deeper level features and the ability to think about and communicate ‘ideas that matter.’ 10

11. How were exemplars used? Preparation before workshop Students prepared short piece of writing (>1 side A4). 500 word response to the task ‘Explain the social construction of childhood to a 1 st year student.’ Phase 1 Brought this to session, where they were given 4 exemplars to read and discuss. Students asked to work in small groups and place exemplars in rank order. Phase 2 Tutors revealed and discussed rationale for their rankings. Phase 3 Students asked to generate feedback for each exemplar& reflect on how, in the light of the tutor dialogue, they would change own work. 11

12. Exemplars integrated, not delivered as a 'one off’ workshop. Lecturers’ overall aim on module 1.Move students from absolute  contextual knowing, appreciating that knowledge re childhood is relative/provisional Engage students actively in activities to challenge students’ previously uncritical assumptions, beliefs (via EBL, collaborative learning, student research). 2.Help develop students’ academic literacy (understanding of the expectations and requirements of the discipline: involves ‘seeing’ and thinking as a disciplinary specialist does). 12

13. Research focus Few close-up studies of how students interact with exemplars – Most (evaluative) data gathered retrospectively. Our study used classroom observation to explore – What illustrative qualities, strengths and weaknesses did students originally notice/discuss in exemplars? – What impact did dialogue with lecturers and peers appear to have on students’ views of the exemplars and on their views of assessment practice at university level? 13

14. Research methods 2 researchers observed the whole lecture group. The dialogue of one student-group (4 students) working on the classroom tasks was recorded verbatim, another (5 students) was observed, with field notes taken. Further 6 interviews, with students from various group, conducted after the session. 14

15. Findings: Student perspectives. All students reported exemplars useful activity “I think seeing it just makes you understand it more. Like, someone can stand there and say, 'You shouldn't do this and that' but until you've actually seen it then you don't know what that looks like.” “I was just writing how I thought it should be done, just in my own head. But now I know what they are looking for and what I should be, how I should be writing, so yes, I think it's been helpful.” 15

16. Students’ responses: Observation Phase 1 Focused on ‘surface’ features and ‘how to write.’ – “I thought some of them immediately looked a bit like they weren't going to be quite right. The one that had bullet points in it. I was a bit, 'Well that's a bit strange for an essay…” – “There were a couple where a lot of it started with 'I' and 'My' and that's just immediately, when you look at, well, when we looked at those two they kind of jumped out as, 'Oh-oh, this might not be great!’” Ignored issues pertaining to subject domain. 16

17. Preoccupied with external conventions: conformity and compliance. Referencing “References was something that we noticed really quickly. One just had it in the text not at the bottom and some didn't put them, like, the people in brackets in the text either. I don't think any of them got them all correct.” Unsurprising, given the ‘frustrations’ academics often voice, which focus on surface features? (cf Greasley & Cassidy, 2010). 17

18. Phase 2: Inability to rank exemplars in same order as teachers. Classroom observation revealed that high proportion of students either – ‘muddled’ the two ‘best’ exemplars. – preferred the ‘worst’ exemplar- 4- and disliked the ‘best’ (exemplar 1). ‘I had the best as worst! To me, it looked really bare!’ 18

19. Student responses to their lack of alignment with teachers’ views Disbelief/embarrassment: “What we thought was best or worst was different to what they thought!” Looking again: “When I first read number 4, I thought it was really good, I liked that she said what she thought. But then like, I went back and read it again and it totally doesn't follow the question or anything.” Trying to look at a deeper level. 19

20. A few started to appreciate teachers’ requirements and explain to peers... “The first time I read it I thought it was pretty good but then … with number 4, once I had read it again, I said something to my friend and she was still, like, 'No, it's really good.' And I went, 'Yeah, but if you read it again it's got nothing to do with the thing. Like, the nature / nurture: that's got nothing to do with social construction. That's more like socialisation.' So she did read it again and she was like, 'Oh yeah!… I get it.’” 20

21. But for most appreciating tutors’ viewpoint was a huge struggle... Peer dialogue immediately after tutor discussion. C. I want to know why they think this one's the best one. B. It's good, but in my eyes it's a bit like, it doesn't flow. A. Yes. B. It's a bit 'bitty'… C. Should have a full stop after 'culture'… B. That's grammar! A. I'm sure you could get over a little grammar mistake like that. B. No, no… Look, 'She's got a full stop after 'culture' but it needs to be there. Peer dialogue 25 minutes after tutor discussion. A: See, all of these examples in here talk about different things. Like, she's talking about nature / nurture and how children are influenced by the birth order. And she's talking about race, gender, culture, class and time. And she's just got a general sort of… B …how do you know what's supposed to be right? A: [Pause]. 4 talks about development. It's not wrong. It's just not right. 21

22. Varied response at individual level Natalie- crossing Rubicon Before university there wasn’t really much chance to take choices when you were writing, you just did essays the way they expected you to and they all pretty much looked the same. But now I’m at university I can see I have to make a choice about what I focus on, and that no assignments will look the same. Because [here] you are reading your own stuff and you are deciding yourself, I think you kind of have to just take that initiative. I think they want us to draw on a lot of different viewpoints and not just think, 'Right, this is what I think, this is what I'm going to write about' Daisy-toeing the line I liked number 3. When they said that they liked number 1 that just made me even more confused….I can’t see how to get mine like that. Number 4 went off topic because she chose her own book. So you've got to really stick to the reading list or any suggestions that they say to you. 22

23. In other recent assessment-related research studies, similar theme emerges: – Whilst most students find assessment induction helpful, views of the nature of the ‘help’ vary: Some students valued interventions for ‘deep-related features’: developing subject understanding, insight into subject; appreciating engagement with different viewpoints. Some valued for ‘strategic-related features’: focusing on the form (e.g. the essay) with no reference to broader learning and understanding in the discipline. » (Harrington et al, 2006) – Students using criteria ‘mechanistically’: developing strategies designed to earn marks, rather than express understanding of the topic.’ » Bloxham & West (2007) 23

24. Individual students’ views of knowledge influence the way they ‘see’ learning and assessment. Those students who view the exemplars via absolutist positions of knowing will inevitably find it difficult to see subject knowledge as contested and relative, so struggle to appreciate/see lecturers’ more sophisticated implicit expectations of ‘good’ and ineffective work within the domain. 24

25. Students’ prior experiences of assessment and learning often run counter to lecturers’ expectations in HE. ‘At school you've got told what you needed to put in your essays. You'd get criteria… Well not like these criteria…more than that: exactly what you need and what you don't need to put in.’ At college, you’re given all the criteria, but it’s more explicit. I mean, really, you can’t go wrong – you know all you have to do is put in the things they tell you. If that goes in, then really, you’re going to pass. But here, it’s like, well, you’ve got your criteria, but you’ve got to get an angle. You’ve got to look into it more, much more. By yourself. Dominant theme: for many students ‘doing’ assessment seen differently from ‘doing’ the subject. 25

26. Thanks for listening! For further information about the Rules of Engagement project please contact the project director kay.sambell@northumbria.ac.uk or see the project pages under development on the ESCalate website: http://escalate.ac.uk/6488 http://escalate.ac.uk/6488 26