Oracy Assessment Possibilities

We want an assessment which will diagnose gaps in students’ oracy skills Do we want to assess skills-groups separately, or together? How do we assess or describe progression? 1 2

1 We want to assess all skills with the same assessment but gain separate outputs Of limited value; However, an overall mark may be a useful additional output (which could be summative) Too time consuming; Results may be misleading due to interdependence of oracy skills – a result may be affected by something you aren’t looking at Not diagnostic: too blunt an output; tells you there may (or may not) be a problem but not of what kind or what to do about it. Tells you where the problem may lie; Efficient use of time and resources Overall mark given Best option Discrete mark given Assessed separately Assessed together

How do we describe or assess progression? 2 How do we describe or assess progression? There are two things which provide a scale or hierarchy which can be used to measure proficiency: Skills can be deployed to a greater or lesser extent The situation in which skills are deployed can be made more challenging Poor skills Good skills Excellent skills How this is done is explained in the next few slides.

Situations can be disaggregated into the following elements Purpose (persuading, describing, etc.) Audience (many, few, one-to-one, etc.) Format (face-to-face, telephone, etc.) Situations Rules (scripts, conventions, etc.) Relationship (formal, informal, intimate, etc.) Level of shared understanding (about subject, about attitudes, etc.) Context-hierarchy explanation slide 1/5

We have split these elements into ‘contexts’ which can be described on a scale of difficulty, and ‘settings’ which simply provide a range, rather than a scale, of challenges Contexts Settings Provide a scale: Easier Harder Should be covered Purpose Explain Discuss Audience Large, small, one-to-one, etc. Relationship Friends Strangers Format Face-to-face, telephone Level of shared understanding High Low Rules Scripted, conventional, open Changing the element we have marked ‘setting’ would exercise oracy skills in a different way. Some students would find different settings more challenging than others. But it is not possible to predict the way in which this would happen, as is broadly possible with ‘context’ elements. Context-hierarchy explanation slide 2/5

Example 1: Daniel Shindler’s lesson on Subtexts Students were placed in a group of three. They were asked to give a short presentation in a manner of their choice that would answer the questions: ‘What is subtext?’ and ‘What kind of question is that?’ Context Setting Purpose To inform/explain/entertain Size of Audience Medium (class) Relationship Peers Format Small group presentation Level of shared understanding High but variable Rules Freestyle presentation but still a discrete task; all members of the group had to speak Context-hierarchy explanation slide 3/5

Example 2: Changing the ‘context’ elements A more difficult version of this task would change the elements we have labeled context. The group of students must now explain what they did in lesson to a group of visiting academics. Context Setting Purpose To explain Size of Audience Medium (class + academics) Relationship Peers & older strangers Format Small group presentation Level of shared understanding The academics do not know what the lesson was about Rules Discrete task; freestyle presentation Context-hierarchy explanation slide 4/5

Level of shared understanding Changing the elements we have marked ‘settings’ ensures a broader coverage of situations and the use of skills Context Purpose To explain/inform/entertain Relationship Peers Level of shared understanding High but variable Working title for situation Setting 1 Size of audience Format Rules One-to-one ‘Face-to-face’ Talking in pairs procedure ‘Tell your friend about subtext’ Setting 2 Size of audience Format Rules Whole class ‘Face-to-faces’ Free style Daniel’s lesson Setting 3 Size of audience Format Rules Assembly ‘Podium’ Bullet-point style ‘Tell the school about subjects and practice communication strategies’ Context-hierarchy explanation slide 5/5

Context is kept constant 2 Given that context-elements and skills-deployment both provide hierarchies, one must be kept constant while the other is assessed. So either… Skills are kept constant …or… Context is kept constant Certain baseline level of skills-deployment expected in every context, e.g. ‘good skills’ The difficulty of context is set appropriately for, say, an age-group The level of skills-deployment is assessed within that context A series of increasingly difficult contexts are devised (stage 1, stage 2, etc.) within which students perform. For both types of assessment, settings are kept constant. It would be necessary, however, to run the assessments across a variety of settings to get a full picture of a student’s competence. A context is passed or failed with reference to the expected level

An oracy assessment in which skills are kept constant would look like this: Setting also kept constant Emotional Cognitive Physical Level 1 Context Pass/fail Pass/fail Pass/fail Level 2 Context Pass/fail Pass/fail Pass/fail Level 3 Context Pass/fail Pass/fail Pass/fail Level 4 Context Pass/fail Pass/fail Pass/fail Level 5 Context Pass/fail Pass/fail Pass/fail

An oracy assessment in which context is kept constant would look like this: Emotional Cognitive Physical Level 3 Context Level 1-5 Setting 1 Emotional Cognitive Physical Level 2 Context Level 1-5 Setting 1 Emotional Cognitive Physical Level 1 Context Level 1-5 Setting 1 Difficulty increases

Each style has good and bad points Skills are kept constant Context is kept constant Skills needed for a specific context can be specified in detail. Marking against this is quite objective. Greater opportunity for assessing ‘natural’ situations. Pros Each student would need to ‘sit’ several contexts as part of one assessment. This would take up an inordinate amount of time. Marking relies on judgement of the observer which may be too subjective. Cons How many of a context’s skill-requirements would have to be met to progress? How many in order for test not to be a blunt instrument? This style of test is too time-consuming to be used.

It is possible to make a hybrid assessment It could look like this: Emotional Cognitive Physical Appropriate in context Skills scale Level 1 Level 1 Level 1 Yes / no Level 2 Level 2 Level 2 Yes / no Level 3 Level 3 Level 3 Yes / no Level 4 Level 4 Level 4 Yes / no Level 5 Level 5 Level 5 Yes / no Etc...

All assessments need to meet three criteria for viability: They must be practicable in a classroom environment They must give outputs which are valuable to students They must be teachable to other schools

Back-up

This will be a context-based skills-level An alternative hybrid would develop context-appropriate expectations of skill-levels This could be developed from a series of more and more difficult real-life contexts using this process What is a more difficult version of that context? Answer codified What constitutes the successful exercise of a skill in the more difficult context? What constitutes the successful exercise of a skill in a given context? Reiterate until you have a set of context based skills-levels, say from 1-5 This will be a context-based skills-level

DRAFT Skills are assessed on their appropriateness to context. Indicators for this are worked out beforehand. Context 1 Emotional Cognitive Physical Level 1 ‘Did you do… [less]’ ‘Did you do… [more]’ ‘Did you do… [less]’ ‘Did you do… [more]’ ‘Did you do… [less]’ ‘Did you do… [more]’ Level 2 Level 3 Level 4 Level 5 These boxes would contain detailed lists of what kind of skills deployment is expected This would still need too many different performances

DRAFT It may be possible to develop assessment foci (AF) in the same way Physical Cognitive Emotional Skills framework generates context-specific AF (probably more than four for each skills group) Voice Body Words Reasoning Content Self-awareness Awareness of others CONTEXT 1 AF Level to which AF is met is graded Setting 1 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 Setting 2 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 Setting 3 This model would be far too complex to assess