Frances Chetwynd Chris Dobbyn The Open University. UK Consistency v Autonomy: effective feedback to a very large cohort 13/04/2012HEA Stem Conference 2012

Introduction Project: Investigating major changes to assessment and feedback on a new module Discussion Why make changes? What changes were made? Views on changes? 13/04/2012HEA Stem Conference 2012

TU100: My Digital Life 60 credits with extremely varied teaching material 4500 students and 200 tutors Poor retention and progression levels in predecessor courses 13/04/2012HEA Stem Conference 2012

4,500 different combinations 13/04/2012HEA Stem Conference 2012 Image: Ralph Greenwell Sense Board with a specially-designed language, Sense

Challenges Build skills for future, self- regulated learning Create joined-up assessment Improve retention and progression Change a tutor culture deeply wedded to marks Preserve consistency at the same time as promoting tutor autonomy 13/04/2012HEA Stem Conference 2012

Earlier research findings Chetwynd & Dobbyn (2011) reported problem in the areas of: Assignment questions Marking guidance Tutor feedback practices Motivation of Marking Guides was consistency of marking over feedback Effective feedback essential for building independent, self-regulated learners (Nicol 2008) 13/04/2012HEA Stem Conference 2012

New feedback taxonomy 13/04/2012HEA Stem Conference 2012 Chetwynd & Dobbyn (2011)

A new project Research project based on TU100 My Digital Life New assessment materials and assessment style New approach to tutor advice Evaluation Identify choke points in the course as potential targets for remedial material 13/04/2012HEA Stem Conference 2012

New assessment materials Explicit testing of skills e.g. List the instructions your program will step through in order to parse the RSS feed… e.g. To help you structure your answer, sample reports are given as answers to Block 2 Part 3 Activity 29 and Block 2 Part 5 Activity /04/2012HEA Stem Conference 2012

New assessment materials Explicit concentration on skills 80% Assignment questions: 20% skills mark English language usage Programming skills Structure/layout Numeracy (scientific notation; units) Referencing 13/04/2012HEA Stem Conference 2012

New Tutor Guides Emphasis on feedback rather than marks New content Explicit feedback advice Separation of retrospective and future- altering feedback advice New structure Separation of skills and content feedback advice Repetitive guidance removed Granularity of marks reduced 13/04/2012HEA Stem Conference 2012

Evaluation 1.Selected group of tutors; semi-structured interviews Before first assignment Half way through module After EMA marking 2.Monitoring reports 3.End of presentation survey 13/04/2012HEA Stem Conference 2012

Preliminary results from interviews Varying degrees of enthusiasm! All liked – focus on skills; future-altering advice Some liked – marks granularity; new structure Some disliked – greater freedom; separation of skills 13/04/2012HEA Stem Conference 2012

Preliminary results from interviews Has made me think about things differently… …Not completely confident about how to tackle skills aspects… I could see the flow of questions and where they were going … …Sometimes difficult to make retrospective feedback more positive 13/04/2012HEA Stem Conference 2012

13/04/2012HEA Stem Conference 2012 References Brown, E. and Glover, C. (2006) Evaluating written feedback. In Innovative Assessment in Higher Education C. Bryan & K. Clegg Abingdon: Routledge. Crisp, B. (2007). Is it worth the effort? How feedback influences students subsequent submission of assessable work. Assessment and Evaluation in Higher Education 32(5), pp Chetwynd, F. D. & Dobbyn, C. (2011): Assessment, feedback and marking guides in distance education, Open Learning: The Journal of Open, Distance and e-Learning, 26(1), pp Gibbs, G. (2010) Does assessment in open learning support students? Open Learning: The Journal of Open and Distance Learning, 25(2), pp Gibbs, G. & Simpson, C. (2004) Conditions under which assessment supports students learning, Learning and Teaching in Higher Education, 1, pp Hounsell, D. (2007) Towards more sustainable feedback. In Rethinking Assessment in Higher Education D. Boud & N. Falchikov (Eds) Abingdon: Routledge. Lea, M. R. & Stierer, B. V. eds. (2000) Student writing in higher education, new contexts, Maidenhead: Open University Press. Lea, M. R. & Street, B. V. (1998) Student writing in higher education: An academic literacies approach, Studies in Higher Education, 23(2), pp Nicol, D. (2008) Transforming assessment and feedback: Enhancing integration and empowerment in the first year. Quality Assurance Agency, Scotland. Nicol, D. & Macfarlane-Dick, D. (2006) Formative assessment and self-regulated learning: a model and seven principles of good feedback practice, Studies in Higher Education, 31(2), pp Price, M., Carroll, J., ODonovan, B. & Rust, C. (2011) If I was going there I wouldnt start from here: a critical commentary on current assessment practice, Assessment & Evaluation in Higher Education, 36(4), pp Sadler, D. R. (1989) Formative assessment and the design of instructional systems, Instructional Science, 18(1), pp Walker, M. (2009) An investigation into written comments on assignments: do students find them usable? Assessment & Evaluation in Higher Education, 34(1), pp Weaver, R. (2006) Do students value feedback? Student perceptions of tutors written comments. Assessment and Evaluation in Higher Education, 31(3), pp