Diversity in Level 1 GEES Assessment: moving from less of more to more of less Tim Stott Professor of Physical Geography & Outdoor Education Faculty of Education, Community & Leisure, Liverpool John Moores University, Liverpool, UK. Prof Tim Stott Stott : Diversity in Level 1 Assessment at Liverpool JMU

Prof Tim Stott Stott : Diversity in Level 1 Assessment at Liverpool JMU Assessment: Background For several decades it has been known that what influences students learning most is not the teaching but the assessment (e.g. Miller and Parlett, 1974). Students have been described as strategic learners who are assessment-led or driven by assessment. Increased pressure on students time may be one reason why students become assessment focussed (Gibbs, 1992). Derek Rowntree stated that if we wish to discover the truth about an educational system, we must first look to its assessment procedures. (Rowntree, 1987, p1).

Prof Tim Stott Stott : Diversity in Level 1 Assessment at Liverpool JMU

Prof Tim Stott Stott : Diversity in Level 1 Assessment at Liverpool JMU Assessment: a changing landscape ? Widening participation in HE Increased numbers of students on modules Wider range of student backgrounds/entry qualifications Issues of retention at Level 1 (Yorke, 2001). Front loading teaching (more teaching hours at Level 1)

Prof Tim Stott Stott : Diversity in Level 1 Assessment at Liverpool JMU Assessment: the context Level 1 24 c (double) module delivered by me since mid- 1990s Introductory module in Earth Science and Climatology which is a core module in the BSc (Hons) Outdoor & Environmental Education 20 topics addressed (eg. plate tectonics, weather systems, micro-climatology etc) 45 hours of contact (35 class based, 10 field trips) 195 independent study Marks, assessment, attendance data available from 2002 Increase in number of students from 29 to 52 this year

Prof Tim Stott Stott : Diversity in Level 1 Assessment at Liverpool JMU Study Aims to assess the effect of increasing the assessment diversity on students performance. to examine whether certain groups of students perform better in certain types of assessment to reflect on these changes to inform future planning

Prof Tim Stott Stott : Diversity in Level 1 Assessment at Liverpool JMU History of Assessment in this module 50%: 3-hr exam A.MCQ B.Short answers questions C.2 x 30 min essay (choice from 6) 50%: 2 x 1500 word Coursework A.Library based essay on individual topic B.Field day project 1.5 days

Prof Tim Stott Stott : Diversity in Level 1 Assessment at Liverpool JMU Data collection Students individual component marks and module totals for Coursework submission Y/N Attendance lists

Prof Tim Stott Stott : Diversity in Level 1 Assessment at Liverpool JMU 2008/2009

Prof Tim Stott Stott : Diversity in Level 1 Assessment at Liverpool JMU

Prof Tim Stott Stott : Diversity in Level 1 Assessment at Liverpool JMU Methods: Regression

Prof Tim Stott Stott : Diversity in Level 1 Assessment at Liverpool JMU Weighted or un-weighted assessment items ?

Prof Tim Stott Stott : Diversity in Level 1 Assessment at Liverpool JMU Results: Pearson correlation coefficients for assessment item v module total

Prof Tim Stott Stott : Diversity in Level 1 Assessment at Liverpool JMU Analyses on Students Ranks

Prof Tim Stott Stott : Diversity in Level 1 Assessment at Liverpool JMU Results: Quartile groups and assessment type 2008

Prof Tim Stott Stott : Diversity in Level 1 Assessment at Liverpool JMU Results: Quartile groups and assessment type 2008

Prof Tim Stott Stott : Diversity in Level 1 Assessment at Liverpool JMU Results: Quartile groups and assessment type 2008

Prof Tim Stott Stott : Diversity in Level 1 Assessment at Liverpool JMU Results: Quartile groups and assessment type 2009

Prof Tim Stott Stott : Diversity in Level 1 Assessment at Liverpool JMU Results: Quartile groups and assessment type 2009

Prof Tim Stott Stott : Diversity in Level 1 Assessment at Liverpool JMU Results: Quartile groups and assessment type 2009

Prof Tim Stott Stott : Diversity in Level 1 Assessment at Liverpool JMU Results: Overall module score v attendance 2009

Prof Tim Stott Stott : Diversity in Level 1 Assessment at Liverpool JMU Conclusions Adding extra 2-3 new assessment items worth 5-10% reduces the weighting of existing items so spreading the influence of individual assessment items on the final module total: more of less The new practical worksheet assessment seemed to work; how best to deploy wikis needs further thought (ongoing LJMU project) Examination of students ranks by quartiles showed the expected pattern for: exam; wiki; field report; weather project BUT the 3 rd quartile of students PERFORMED BETTER than the 2 nd quartile in on-line test; worksheets; rocks teaching resource in 2008 BUT in weather report; field report and wiki in 2009 ? Are assessment items which disrupt the expected pattern good ?

Prof Tim Stott Stott : Diversity in Level 1 Assessment at Liverpool JMU Discussion points arising The sequence of submission dates is important: some students start off eagerly, and fade away; others peak last minute Feedback: good practice to give formative feedback early in the module, but if summative provisional marks are given students can work out that they dont need to attempt later assessments to PASS the module Are items worth of 5%, 10% taken seriously ? High number of non- submissions in 5% wiki submitted near end of module Does sample marking make students submit all items ? Do students feel cheated if their work is not marked ? University regulations: Level 1 just need to pass (40%) BUT if students are athletes training for a marathon (with the race in Level 3) should they be allowed to opt out of training for year 1 ?

Prof Tim Stott Stott : Diversity in Level 1 Assessment at Liverpool JMU References Gibbs, G. (1992) Assessing More Students. Oxford: Oxford Centre for Staff Development. Oxford: Oxford Centre for Staff Development. Gibbs, G. (1999). Using assessment strategically to change the way students learn, in Brown, S. and Glasner, A. (eds) Assessment Matters in Higher Education, Buckingham: SRHE: Open University Press. Miller, C.M.I. & Parlett, M. (1974) Up to the mark: A study of the examination game. Guildford: Society for Research into Higher Education. Stott, T.A., Meers, PM (2002) Using BlackBoard VLE to Support Referral Students at JMU, LJMU Internal Report to Learning Development Unit. Stott, T. A. (2006) Evaluation of the Use of Supporting Diagrams and Video Clips in Blackboards on-line assessment tests. Poster presented at 1st Pedagogical Research in Higher Education (PRHE) conference, Pedagogical Research: enhancing student success, 2-3 May 2006, Liverpool Hope University Taras, M. (2001) The use of tutor feedback and student self-assessment in summative assessment: towards transparency for students and for tutors. Assessment and Evaluation in Higher Education, 26, 6, pp Yorke, M. (2001). Formative assessment and its relevance to retention. Higher Education Research and Development 20 (2),

Prof Tim Stott Stott : Diversity in Level 1 Assessment at Liverpool JMU Thank You for Listening