Dr Claire Kotecki (STEM) & Dr Prithvi Shrestha (WELS) Academic literacy and communicating assessment to students on L1 Science modules: student perceptions Dr Claire Kotecki (STEM) & Dr Prithvi Shrestha (WELS) STEM Futures – Supporting Students to Succeed, 6th eSTEeM conference, The Open University, 25-26 April 2017

Academic Literacy (AL) – a key to success in Science? Widely considered as central to academic knowledge building and success (Snow, 2010) Academic language may pose challenges to many students who are at risk of underachievement (Cummins, 2014) Disciplinary variation and associated academic language practices (Haneda, 2014) means AL in science requires distinct understanding and approach Some studies have shown that students with lower academic literacy skills are unlikely to succeed in science (Kirby & Dempster, 2015) Implications for overall student retention and progression in STEM subjects. Linked to implications for how assessment is designed and communicated to students

Academic Literacy Challenges in Science at Level 1 Students begin their first L1 Science module with large disparity in AL levels For many reasons, students don’t necessarily go through Access first even if recommended AL has to ‘share space’ with other key areas where students have a skills gap – scientific concepts & literacy, maths skills, DIL etc. There is no single unified approach to AL development across Science courses Little work has been done to assess effectiveness of approaches as it is often overlooked Increasing AL levels at L1 can have a positive effect on student retention and progression.

Level 1 Science Modules This project focuses on S104: Exploring Science At the time of this study, this was functioning as an entry-level module although it was not originally designed as such It is a module that has had problems with retention and progression This module has now been replaced by S111 (first presentation 16J) and S112 (first presentation 17J) as a pair of modules. The expected cohort for S111 is similar to that of S104, within the boundaries of know changes in overall student cohort for the OU There is a STEM access module, Y033, however students often choose not to study this despite recommendations and so entry-level L1 modules have some students who need the same AL skills developed but in a cohort environment where more able students may find this a patronising approach

Project Aims To assess student understanding of key terminology and perceptions of assessment Longitudinal student script analysis via a combination of quantitative tools to characterise linguistic complexity, use of key terminology and structure Analysis of how assessment tasks are communicated to students to provide an holistic view of assessment communication, student expectation, cohort performance and value-based tutor feedback Potentially shed light on which kind of students make best AL progress – the most or least adept – and link to feedback communication

Design of research methodology Survey of Level 1 science module assessment materials Analysis of student assignments Pre-module student survey Post-module student survey Post-module follow-up student interviews

Survey of L1 science module assessment materials and student assignment analysis Assignment tasks analysis: instruction or process words used, task types, text types (genres) Assignment Guidance analysis Sample of high and low achievers from a previous cohort (8 X 6 = 48 assignments)

Pre-module student survey One month prior to module start (S104 2016B) Key terminology (instruction/ process words) used – open definitions/ descriptions. e.g., “Plants generate energy using the process of photosynthesis. Define the term ‘photosynthesis’.” Types of assessment (including response types and text types) – usefulness rating scale (1-5); no context given. (e.g., long written answer, practical activity) Student attitudes towards assessment and learning – open comments (e.g., What do you feel the purpose of tutor marked assignments is?)

Process words and task types in S104 TMAs Define and explain Outline, describe, identify, state Definitions and explanations Short answers TMA02 Explain, describe, reflect Explanation Personal reflection Short answer TMA03 Complete, list, summarise, explain, describe, reflect List, summary, explanation, reflection TMA04 State, describe Description, report Short & long answers TMA05 Explain, write formula, calculate, state, sketch, label, reflect Explanation, calculation, reflection Long answer TMA06 Identify, explain, discuss, indicate, name, state, draw, assign, suggest, calculate Explanation, discussion, recommendation, account of a topic Short and long answers

Student Survey Analysis: Phase 1 Students were asked to rate their understanding of key process words Students were then given the opportunity to demonstrate their understanding of these words in context

Student Survey Analysis: Preliminary findings Student perceptions of comprehension of process words differs from demonstrated comprehension

Student Survey Analysis: Perceptions of assessment Students were asked about their perceptions of the purpose and usefulness of the range of assessment types on S104

Student Survey Analysis: Preliminary findings Student perceptions of purpose of assessment differs according to the assessment type

Post-module student survey After module completion (Oct – Nov 2016) Based on S104 experience Key terminology (instruction/ process words) used in S104 – clarity rating scale (1-5). (e.g., Clearly state the difference (or differences) that distinguish the correct definition from the incorrect one.) Assignment guidance – clarity rating scale (1-5) Types of assessment (including response types and text types) – usefulness rating scale (1-5); context given. e.g., Long written answers Example: Describe how the higher order structure of the two classes of proteins, globular and fibrous, depends on the primary structure and relate higher order structure to function, giving two examples for each class of protein. (Guideline: up to 420 words.) Attitudes towards assessment and learning – open comment. (E.g., What do you feel the purpose of the tutor marked assignments in S104 was and how useful did you find them in your study?)

Where is the project heading? Complete all project phases and progress to analysis and dissemination Look towards S111 17J and S112 18B presentations to study the approach now in use on the Science pathway Look backwards to Access level to compare progress measures in a module with a much more defined AL structure. Write up at least one journal article for a peer reviewed journal

References Cummins, J. (2014). Beyond language: Academic communication and student success. Linguistics and Education, 26, 145-154. doi:http://dx.doi.org/10.1016/j.linged.2014.01.006 Haneda, M. (2014). Why should we care about academic language? Linguistics and Education, 26, 88-91. doi:http://dx.doi.org/10.1016/j.linged.2014.01.007 Kirby, N. F., & Dempster, E. R. (2015). Accommodating those Most at Risk. Responding to a Mismatch in Programme Selection Criteria and Foundation Biology Performance. International Journal of Science Education, 37(18), 3093-3117. doi:10.1080/09500693.2015.1121550 Snow, C. E. (2010). Academic Language and the Challenge of Reading for Learning About Science. Science, 328(5977), 450.