1. Layering methods to analyse the relationship between language use and attainment among OU undergraduate students Maria Leedham, Elizabeth J. Erling, Lina Adinolfi Contact: m.e.leedham@open.ac.uk Faculty of Education and Language Studies The Open University

2. This paper will report on a research project at the OU into the relationship between use of academic language and attainment among a small cohort of Health and Social Care undergraduate students. Layers 1 & 2 – MASUS and contextual research explore further research which ‘layers’ methods to investigate the texts Layers 3 & 4 - intuitive identification of lexical chunks, corpus linguistic extraction of chunks

3. Context: a Health and Social Care course (HSC) K204: Working with Children and Families “This course is designed to meet the educational and training needs of those who work (or will work) with children and their families across social care, childcare, health, education, and leisure settings.” Level 2, 60 points, distance-learning course, 6 essays + 1 exam 26 students 72 HSC texts were analysed 7 students interviewed

4. Layers 1 & 2: THE ORIGINAL STUDY

5. Layer 1: MASUS Measuring Academic Skills of University Students Categories A Use of source material B Structure and development of text C Control of academic writing style D Grammatical correctness E Qualities of presentation Text analysis: Two language specialists and one subject specialist Rating 4 Excellent/no problems/ accurate/very appropriate 3 Good/minor problems/mainly accurate/largely appropriate 2 Only fair/some problems/often inaccurate/often inappropriate 1 Poor/major problems/inaccurate/inappropriate 1.0-2.4 = low score (Bonanno and Jones 1997)

6. Layer 1: MASUS Findings: Attainment and language

7. Layer 1: MASUS Findings: MASUS scores for cohort The number of students in sample with low language scores (1.0 – 2.4) (n=26)Assign. 1(n=25)Assign. 3(n=24)Assign. 5 AUse of source material727%520%28% BStructure and development1558%1456%1563% CAcademic writing style831%728%417% DGrammatical correctness415%728%625%

8. Layer 2: Contextual investigation Analysis of course materials  insight into important key concepts and language of the field Tutorial visits  a contact point for researcher and students Interviews with students  insights into students’ views of their own writing Collaboration between language and discipline specialists  ensured that both language and discipline expertise were represented

9. Layers 3 & 4: SEARCHING FOR LEXICAL CHUNKS

10. Lexical chunks Conventionalised word combinations Contribute to ease of language production and reception Demonstrate membership of academic community Layer 3: corpus linguistic research on chunks E.g. work from Douglas Biber, Ken Hyland Layer 4: intuitive identification of chunks E.g. work from Alison Wray, Pauline Foster

11. Layer 3: Corpus linguistic extraction of lexical chunks 28,000 words 15 HSC texts Assignments 1, 3, 5 Used WordSmith Tools v.5 Extracted 3, 4, 5 word chunks Compared no. of types of chunks over time Ongoing… could extend to cover keyness

12. Layer 3: Corpus linguistic extraction of chunks Assignment 1 : Why should we listen to children? quality of life(31, 4/4 texts) listening to children (17, 4) children and families (10, 4) right to be heard (6, 4) Assignment 3: Child development theories can have both a positive and a negative influence on how practitioners respond to children. Discuss. in the reader (27, 3/6 texts) negative influence on (7, 4) as well as (5, 4) I will discuss (5, 3) Assignment 5: Critically discuss how the quality of life of children living away from home can be improved by those working with children and their families. living away from home (49, 4/5) quality of life (37, 5) it is important (8, 3) if we are to (4, 2)

13. Layer 3: Corpus linguistic extraction of chunks Assignment 5 if we are to (4, 2) = Example of students taking on the role of HSC worker?

14. Layer 4: Intuitive identification of lexical chunks Advantage of using intuition Can identify single chunks Simulates the tutor’s experience Difficulties with intuition Subjective Difficult to justify Problem with boundaries Partial solution is to adopy Wray and Namba’s 11 criteria (2003) “By my judgement…” Two types of chunks focused upon: Linkers B Structure and development of text Discipline-specific C Control of academic writing style

15. Layer 4: Intuitive identification of chunks Case study A: Laura - Assignment 1: Extract Key: Red = ill-formed or inappropriate chunks Blue =well-formed and appropriate chunks

16. Layer 4: Intuitive identification of chunks Case study A: Laura - Assignment 5: Extract

17. Layer 4: Intuitive identification of chunks Case study B: Sally - Assignment 1: Extract

18. Layer 4: Intuitive identification of chunks Case study B: Sally - Assignment 5: Extract Key: Red = ill-formed or inappropriate chunks Blue =well-formed and appropriate chunks

19. Layer 4: Intuitive identification of chunks Findings There is a correlation between the grades given to students and the intuitive assessment of the nature of chunks they use in their writing (ie the number and range of discipline-specific chunks used appropriately) Low achieving students appear to use fewer discipline- specific chunks than higher achieving ones. There appears to be an improvement in the range and number of chunks produced accurately by individual learners over time, whether low achieving or higher achieving.

20. Conclusions Students need more direct language development Investigation into the features of highly valued academic language can help in developing language support MASUS – offers a way of gathering judgements on aspects of texts from multiple raters, including a discipline-based perspective Contextual data – provides explanations for changes in students’ texts and differences in students’ abilities/desires to acquire and use the ‘highly valued’ features of academic language Corpus linguistics – offers a way of quantifying the number and range of chunks across texts & gives insights into disciplinary differences Intuitive extraction of chunks – provides a language specialist’s view of which chunks are most marked – and how this markedness might affect student attainment

21. References BONNANO, H & JONES, J. (1997) Measuring the Academic Skills of University Students: The MASUS procedure. University of Sydney. ERLING, E. J. (2009) An investigation into the relationship between the use of academic language and attainment – with a focus on students from ethnic minorities. Open University. WIKTORSSON, M. (2003) Learning idiomaticity: a corpus-based study of idiomatic expressions in learners' written production. Lund, Lund University WRAY, A. & NAMBA, K. (2003) Formulaic language in a Japanese-English bilingual child: a practical approach to data analysis. Japan Journal for Multilingualism and Multiculturalism, 9, 24-51.

22. Acknowledgements We would like to thank Student Services at the Open university, who funded the research project The relationship between academic language use and attainment – with a focus on students from ethnic minorities in 2008-2009. We would also like the thank the researchers involved in the project: Elizabeth Erling, David Hann, John Kearsey, Christina Healey, Christine Buller, Kerry Bannister, Chris Lee, Zoe Doye, Harish Mehra; and the project supervisor, Jim Donohue.

23. Maria Leedham Centre for Language and Communication Faculty of Education and Language Studies The Open University Walton Hall Milton Keynes MK7 6AA m.e.leedham@open.ac.uk m.e.leedham@open.ac.uk Dr. Elizabeth J. Erling Open English Language Teaching Dept. of Languages Faculty of Education &Language Studies The Open University Walton Hall Milton Keynes MK7 6AA e.j.erling@open.ac.uk e.j.erling@open.ac.uk Lina Adinolfi Open English Language Teaching Dept. of Languages Faculty of Education & Language Studies The Open University Walton Hall Milton Keynes MK7 6AA l.adinolfi@open.ac.uk