1. Dr Claire Kotecki (STEM) & Dr Prithvi Shrestha (WELS)
Academic literacy and communicating assessment to students on L1 Science modules: student perceptions
Dr Claire Kotecki (STEM) & Dr Prithvi Shrestha (WELS)
STEM Futures – Supporting Students to Succeed, 6th eSTEeM conference, The Open University, April 2017

2. Academic Literacy (AL) – a key to success in Science?
Widely considered as central to academic knowledge building and success (Snow, 2010)
Academic language may pose challenges to many students who are at risk of underachievement (Cummins, 2014)
Disciplinary variation and associated academic language practices (Haneda, 2014) means AL in science requires distinct understanding and approach
Some studies have shown that students with lower academic literacy skills are unlikely to succeed in science (Kirby & Dempster, 2015)
Implications for overall student retention and progression in STEM subjects.
Linked to implications for how assessment is designed and communicated to students

3. Academic Literacy Challenges in Science at Level 1
Students begin their first L1 Science module with large disparity in AL levels
For many reasons, students don’t necessarily go through Access first even if recommended
AL has to ‘share space’ with other key areas where students have a skills gap – scientific concepts & literacy, maths skills, DIL etc.
There is no single unified approach to AL development across Science courses
Little work has been done to assess effectiveness of approaches as it is often overlooked
Increasing AL levels at L1 can have a positive effect on student retention and progression.

4. Level 1 Science Modules
This project focuses on S104: Exploring Science
At the time of this study, this was functioning as an entry-level module although it was not originally designed as such
It is a module that has had problems with retention and progression
This module has now been replaced by S111 (first presentation 16J) and S112 (first presentation 17J) as a pair of modules.
The expected cohort for S111 is similar to that of S104, within the boundaries of know changes in overall student cohort for the OU
There is a STEM access module, Y033, however students often choose not to study this despite recommendations and so entry-level L1 modules have some students who need the same AL skills developed but in a cohort environment where more able students may find this a patronising approach

5. Project Aims
To assess student understanding of key terminology and perceptions of assessment
Longitudinal student script analysis via a combination of quantitative tools to characterise linguistic complexity, use of key terminology and structure
Analysis of how assessment tasks are communicated to students to provide an holistic view of assessment communication, student expectation, cohort performance and value-based tutor feedback
Potentially shed light on which kind of students make best AL progress – the most or least adept – and link to feedback communication

6. Design of research methodology
Survey of Level 1 science module assessment materials
Analysis of student assignments
Pre-module student survey
Post-module student survey
Post-module follow-up student interviews

7. Survey of L1 science module assessment materials and student assignment analysis
Assignment tasks analysis: instruction or process words used, task types, text types (genres)
Assignment Guidance analysis
Sample of high and low achievers from a previous cohort (8 X 6 = 48 assignments)

8. Pre-module student survey
One month prior to module start (S B)
Key terminology (instruction/ process words) used – open definitions/ descriptions. e.g., “Plants generate energy using the process of photosynthesis. Define the term ‘photosynthesis’.”
Types of assessment (including response types and text types) – usefulness rating scale (1-5); no context given. (e.g., long written answer, practical activity)
Student attitudes towards assessment and learning – open comments (e.g., What do you feel the purpose of tutor marked assignments is?)

9. Process words and task types in S104 TMAs
Define and explain
Outline, describe, identify, state
Definitions and explanations
Short answers
TMA02
Explain, describe, reflect
Explanation
Personal reflection
Short answer
TMA03
Complete, list, summarise, explain, describe, reflect
List, summary, explanation, reflection
TMA04
State, describe
Description, report
Short & long answers
TMA05
Explain, write formula, calculate, state, sketch, label, reflect
Explanation, calculation, reflection
Long answer
TMA06
Identify, explain, discuss, indicate, name, state, draw, assign, suggest, calculate
Explanation, discussion, recommendation, account of a topic
Short and long answers

10. Student Survey Analysis: Phase 1
Students were asked to rate their understanding of key process words
Students were then given the opportunity to demonstrate their understanding of these words in context

11. Student Survey Analysis: Preliminary findings
Student perceptions of comprehension of process words differs from demonstrated comprehension

12. Student Survey Analysis: Perceptions of assessment
Students were asked about their perceptions of the purpose and usefulness of the range of assessment types on S104

13. Student Survey Analysis: Preliminary findings
Student perceptions of purpose of assessment differs according to the assessment type

14. Post-module student survey
After module completion (Oct – Nov 2016)
Based on S104 experience
Key terminology (instruction/ process words) used in S104 – clarity rating scale (1-5). (e.g., Clearly state the difference (or differences) that distinguish the correct definition from the incorrect one.)
Assignment guidance – clarity rating scale (1-5)
Types of assessment (including response types and text types) – usefulness rating scale (1-5); context given. e.g., Long written answers
Example: Describe how the higher order structure of the two classes of proteins, globular and fibrous, depends on the primary structure and relate higher order structure to function, giving two examples for each class of protein. (Guideline: up to 420 words.)
Attitudes towards assessment and learning – open comment. (E.g., What do you feel the purpose of the tutor marked assignments in S104 was and how useful did you find them in your study?)

15. Where is the project heading?
Complete all project phases and progress to analysis and dissemination
Look towards S111 17J and S112 18B presentations to study the approach now in use on the Science pathway
Look backwards to Access level to compare progress measures in a module with a much more defined AL structure.
Write up at least one journal article for a peer reviewed journal

16. References
Cummins, J. (2014). Beyond language: Academic communication and student success. Linguistics and Education, 26, doi:
Haneda, M. (2014). Why should we care about academic language? Linguistics and Education, 26, doi:
Kirby, N. F., & Dempster, E. R. (2015). Accommodating those Most at Risk. Responding to a Mismatch in Programme Selection Criteria and Foundation Biology Performance. International Journal of Science Education, 37(18), doi: /
Snow, C. E. (2010). Academic Language and the Challenge of Reading for Learning About Science. Science, 328(5977), 450.