Oracy Assessments Research What is out there and what does it do?

We have looked into four assessments of spoken English: These are collectively referred to as ‘the assessments’. GCSE Speaking and Listening exams (GCSE S&L) The Trinity College Graded Exams for Spoken English (Gese [lower-case to distinguish from GCSE]) The American Common Core States Standards (CCSS) The exams used by Oracy Australia (OA). GCSE S&L CCSS Gese OA Output Level; contributes to overall English grade (e.g. A*, C, etc.) Assessments are still currently being developed. Numbered Grade, depending on difficulty of exam set Satisfactory High Very High Outstanding Format One person or a group speaking, usually in front of the whole class One-to-one throughout Components Presenting, discussing, role-playing Comprehension and Collaboration; Presenting Knowledge and Ideas Conversation, Topic Presentation/Discussion, Interactive Task, Listening Task Personal Involvement*, Prepared Reading Aloud, Memorised Interpretation, Discussion Stratified by Level Age Non-age-related grades Opportunity to prepare N/A Examiner’s discretion Low Medium

The assessments all define context-specific skills based on certain scenarios Presenting gives rise to ‘Communicating and adapting language’ E.g. AQA’s GCSE S&L: Discussion & listening gives rise to ‘Interacting and responding’ Role-play gives rise to ‘Creating and sustaining roles’

Weight is given to three kinds of context. GCSE S&L Gese CCSS OA Presenting Discussing Role-play Reading aloud ‘Interactive task’ Listening The assessments agree that presenting and discussing (and to an extent role-play) are important indicators of oracy-skills. Is this true? Candidates listen to a spoken ‘text’ and are then asked reasonably complicated questions about it to which they answer ‘yes’ or ‘no’. Listening Candidates are given a ‘prompt’ statement upon which they are expected to found a conversation. Marks are awarded for how well they lead and develop the discussion. Interaction

Oracy Australia (Year 8) Assessment criteria describe what to look for in performances of a given quality Oracy Australia (Year 8) GCSE (Level 3) CCSS (7th grade) Gese (Grade 6) Communicating and Adapting Language Candidates • effectively communicate information, ideas and feelings, promote issues and points of view • adapt talk to a variety of situations and audiences, using non-verbal features to add to impact • use a range of well-judged vocabulary and sentence structures to achieve different purposes, including competent and appropriate use of standard English Presenting CCSS.ELA-Literacy.SL.7.4 Present claims and findings, emphasizing salient points in a focused, coherent manner with pertinent descriptions, facts, details, and examples; use appropriate eye contact, adequate volume, and clear pronunciation. CCSS.ELA-Literacy.SL.7.5 Include multimedia components and visual displays in presentations to clarify claims and findings and emphasize salient points. CCSS.ELA-Literacy.SL.7.6 Adapt speech to a variety of contexts and tasks, demonstrating command of formal English when indicated or appropriate. Conversation Phase Show understanding of the examiner by responding appropriately to questions Give information about the prepared Make the sequence of events clear by referring back to previous events and forward to forthcoming events Answer questions on the prepared topic, Ask the examiner at least one question about the topic area Take the opportunity provided to include samples of the language functions and language items listed opposite, where appropriate Personal Involvement You must show that you can structure your talk clearly and logically within the given time limit include your own ideas and opinions as well as the facts communicate spontaneously apply appropriate visuals to aid and illustrate the spoken word speak out audibly and clearly communicate with the listeners enthusiastically and sincerely NB: These strata were chosen to be roughly comparable, not exact matches Can these criteria be described as skills sets? They aspire to comprehsiveness. They are extremely detailed and numerous, especially into the higher levels.

Marks depend on how many of these things are demonstrated (‘covered’) in a performance. Two types of assessment use ‘coverage’ in different ways: Music Grade Style Figure-skating Style Candidates sit an appropriately difficult exam and pass or fail based on their coverage of its criteria. Everybody sits the same exam. Their skills are given a level based on their coverage of each level’s criteria. In the Gese, the level of coverage determines whether or not a grade is passed. In the OA, there are four extents of coverage; these are given as a grade At GCSE, a level is awarded based on coverage of what that level specifies. Level 4 X Y Z n Level 5 = Level 4 At Grade X you should be able to… Satisfactory Good High Outstanding Criterion Was it demonstrated? Pass = x y Z n > 50%

We need a different way of assessing, which can be taken seriously and implemented widely. e.g. Someone assessing grade 8 of the Gese is looking for coverage of 37 assessment criteria in an exam which lasts 15 minutes Pros Easy to implement Pros Useful to students & government Cons Subjective Not taken seriously Cons Difficult to implement Increasing complexity This can be remedied by Standardising the assessor: same person training Making the test more rigorous / complex It is worth noting that all of the assessments looked at are summative.

Describing skills with reference to context is in contrast to our approach of defining skills which can be used in any context Usually the assessments are very specific and cannot be generalised; extreme examples include: ‘Ask at least one question’ ‘Greet the examiner’ ‘Use multimedia’ The assessments’ skills are described using verbs; ours are categorised (initially) using adjectives (e.g. ‘Cognitive’) Oracy Physical Voice Body Cognitive Words Content Reasoning Emotional Adjectives Nouns

Back-up

Skilled assessors could reduce the significance of both complexity and subjectivity, as is currently done in the moderation of GCSE Drama exams. This would make the whole business of oracy-assessment quite complex and centralised It can be thought of as ‘standardising the assessor’

It should be noted that all of these assessments are summative. A formative, diagnostic assessment could be more easily simple and useful at the same time, as subjectivity is not so undesirable and standardisation less important.