Ipsative assessment: comparison with past performance Dr Gwyneth Hughes

Summary Defines ipsative assessment and it potential benefits Builds on findings of research on ipsative feedback for distance learners funded by CDE I am senior lecturer at IoE where I do research on learning and teaching in HE and I lead a large JISC funded project Assessment careers Technology enhanced assessment. An initial project looking at types of feedback learners received on a DL MA also CDE funded. This presentation is about a second phase of implementing ipsative feedback on that MA. First a bit more about ipsative assessment

Ipsative assessment as an alternative to criteria-referenced assessment

Personal best Assessment by comparison with yourself rather than in comparison to others or external criteria and standards,–like a personal best in athletics. Assessing progress rather than outcomes.

What is ipsative assessment ? An ipsative assessment compares current performance with a previous performance. Ipsative feedback on progress Ipsative grades awarded for progress Self-referential assessment not necessarily a self-assessment Hughes, G. 2011. Aiming for Personal Best: a Case for Introducing Ipsative Assessment in Higher Education Studies in Higher Education 36 (3): 353 - 367 Ipsative assessment is not necessarily a self-assessment-it could be –but ipsative feedback and/or grades for progress could be provided by the assessor or peers or self. I term it a self-referential assessment rather than necessarily a self-assessment.

Possible benefits of ipsative assessment Improves self-esteem and confidence by rewarding progress (particularly for those who do not achieve high grades) It might motivate learners to act on feedback Gives tutor and students a longitudinal view of assessment Assessing progress makes more sense if a long-term view is taken in multiple linked assessments over time rather than individual modular assessments Particularly for distance learners who are very reliant on written feedback and do not have an equivalent for feedback in class

MA in Applied Educational Leadership and Management Phase 1 of study in 2009-2010 showed that ipsative feedback was rare However, both students and tutors felt that ipsative feedback would be beneficial Ipsative grades were perceived to be problematic and so not developed for the next phase of the study-focus on ipsative feedback. http://cdelondon.wordpress.com/2010/07/28/use-of-ipsative-assessment-in-distance-learning/ Funded study by CDE for a distance learning programme-Action research – two phases. This distance programme which is delivered jointly by IOE and London University International programmes.

Ipsative cumulative scheme for 3 sequential assessments Students in the previous study find feedback that indicates what to do to improve or feedforward very helpful but how do they know if they have successfully acted on feedforward (apart from a better mark perhaps?) In this ipsative scheme Feed forward 2 can build on feed forward 1 if remedial action has not been taken.  

Excerpt from the new form Please indicate what feedback you were given for your draft of this assignment, in terms of how you could improve (if applicable): Please indicate what feedback you were given, for your last assignment, in terms of how you could improve: Please comment on the extent to which you feel you have responded to feedback:  

MA in Applied Educational Leadership and Management Students submit a draft pieces of work for feedback Students fill in a new Assignment Submission form when they submit their work for assessment identifying actions arising from previous feedback (self assessment) Students received ipsative feedback from tutors on how well they have addressed the feedback (self-referential assessment) Students’ previous assignments and feedback were made available in the VLE for consultation. This programme was used to try out ipsative feedback. A meeting with the tutors and programme leader resulted in an agreed intervention. 3. Means that tutors do not necessarily have to compare the new piece of work with the previous one, they can use the student self-assessment as a basis for ipsative feedback. However, previous assignments and feedback needed to be easily accessible (if students did not fill in the self-assessment) VLE.

Evaluation methods The written feedback given to 28 students was analysed 13 students were interviewed about the usefulness of feedback including the ipsative feedback The 3 main tutors were interviewed about the new approach to giving feedback (2 others were part-time and declined to be interviewed) 28 students international about half non native English 17 female and 11 male.

Excerpt from the new form Please indicate what feedback you were given, for your draft of this assignment, in terms of how you could improve (if applicable): - Methodology:clearer statement on exact nature of research To identify and use a clearer narrative thread (student 22) Please comment on the extent to which you feel you have responded to feedback: -In the methodology section I have added more detail and have given further thought to sampling and validity of data -tried to use more signposts to direct the reader through the narrative more clearly (student 22)  

Analysis of use of the form and written feedback Not all students used the form – some were unable to access it Some who filled in the form were at the box-ticking end of the spectrum e.g. I have tried to act on all the feedback received (student 11) This is to be expected for an introduction of a new procedure and students will need time to get used it and also to realise that there are benefits to completing the form in that they will get further feedback.

Ipsative Feedback to students from tutors ‘It is clear that you took careful note of the advice you received on the draft and the previous assignment. You have worked hard to improve the narrative flow and have taken steps to develop your critical engagement with the sources.’ (Tutor A) ‘It is a pity you did not have the chance to revise your assignment on the basis of the feedback provided earlier, as this would probably result in a better grade.’ (Tutor C) There was evidence that tutors provided ipsative feedback –in contrast to the previous study where this was non-existant.

Analysis of use of the form and written feedback Some tutors used the form to provide ipsative feedback One tutor gave ipsative feedback when students did not use the form The form is not essential for comparing with previous performances as some tutors appeared to have used the records in the VLE to look back to previous comments they had written. Was the form useful? This suggests that having a process such as this helps tutors get a sense of previous feedback and performance but easy access to previous feedback and assessments is needed also to supplement the form –in the VLE.

Reaction to the new approach to feedback – the students The form helped students to reflect on how and to what extent they had responded to tutor’s comments ‘It [the form] allowed me to show how I had attempted to improve on my last assignment and also encouraged me to reflect more deeply on my submission.’ (Student 22) Contributing to motivation and confidence building ‘[the comment] gave me confidence and made me feel proud of my work …. It also made me aware of the changes that I made that improved my work so I will know what to do next time.’ (Student 16)

Reaction to the new approach to feedback – the students ‘My tutor gave particularly good feedback after my draft assignment. It was easy to understand and follow and was very specific so I could see exactly what I needed to improve and how. I think her feedback made a big difference to my final mark.’ (Student 16) ‘…the tutor reminded me of not having addressed an issue raised about my first draft. I did email the tutor and highlighted an extract from the final essay where I had covered the suggestion made. I believe the tutor had missed that point and the comment was then removed from my final feedback. I was told, however, that this would not result in a revised mark.’ (Student 13) Focus on short-term grade improvement

Reaction to the new approach to feedback – the tutors The form helped tutors write feedback ‘I was already on the journey to that approach. When providing feedback I tended to approach it in my normal way and then made sure that I adapted things to meet the requirements of the new front sheet.’ (Tutor A) Concern about raising expectations ‘I could see from the comments that some students expected their revised draft to lead to an A (grade), and it was subsequently disappointing for these students to gain a sense of progress without an A.’ (Tutor C) Tutors agreed that the form helped them and students and tutors A and C used the form but tutor B less so. Concerned that ipsative feedback might raise expectations.

The benefits of ipsative feedback The form helped some students reflect on feedback A structured approach such as including a submission form might help both students and tutors take a longer-term view of assessment Ipsative feedback can be motivational for distance learners

The challenges for implementing ipsative feedback Ipsative feedback is new and needs time to be accepted Students may expect instant higher grades Scaling up needs the practice to be embedded across a programme and tutors may need development time Tutors are not accustomed to viewing each others feedback and records of previous feedback are not routinely stored in VLEs Plans to scale it up and continue on the programme

Scaling up Funding for a institution-wide project on assessment of £200,000

References Broadfoot, P. 1996. Education, Assessment and Society. Buckingham: Open University Press. Carless, D. 2006. Differing perceptions in the feedback process. Studies in Higher Education. 3, no. 2: 219-233. Hughes, G. (2011) Aiming for Personal Best: a Case for Introducing Ipsative Assessment in Higher Education Studies in Higher Education 36 (3): 353 – 367 Hughes, G. Wood, E. & Okumoto, K. 2009 Use of ipsative assessment in distance learning Centre for Distance Education Report, University of London. http://cdelondon.wordpress.com/2010/07/28/use-of-ipsative-assessment-in-distance-learning/  Nicol, D. & Macfarlane,-Dick, D. 2006. Formative assessment and self regulated learning: a model and seven principles of good feedback practice. Studies in Higher Education 31 no. 2: 199-218. Institute of Education University of London 20 Bedford Way London WC1H 0AL Tel +44 (0)20 7612 6000 Fax +44 (0)20 7612 6126 Email info@ioe.ac.uk Web www.ioe.ac.uk