1. Oracy Assessment Possibilities

2. We want an assessment which will diagnose gaps in students’ oracy skills
Do we want to assess skills-groups separately, or together?
How do we assess or describe progression? 1 2

3. 1
We want to assess all skills with the same assessment but gain separate outputs
Of limited value;
However, an overall mark may be a useful additional output (which could be summative)
Too time consuming;
Results may be misleading due to interdependence of oracy skills – a result may be affected by something you aren’t looking at
Not diagnostic: too blunt an output; tells you there may (or may not) be a problem but not of what kind or what to do about it.
Tells you where the problem may lie;
Efficient use of time and resources
Overall mark given
Best option
Discrete mark given
Assessed separately
Assessed together

4. How do we describe or assess progression?2
How do we describe or assess progression?
There are two things which provide a scale or hierarchy which can be used to measure proficiency:
Skills can be deployed to a greater or lesser extent
The situation in which skills are deployed can be made more challenging
Poor skills
Good skills
Excellent skills
How this is done is explained in the next few slides.

5. Situations can be disaggregated into the following elements
Purpose (persuading, describing, etc.)
Audience (many, few, one-to-one, etc.)
Format (face-to-face, telephone, etc.)
Situations
Rules (scripts, conventions, etc.)
Relationship (formal, informal, intimate, etc.)
Level of shared understanding (about subject, about attitudes, etc.)
Context-hierarchy explanation slide 1/5

6. We have split these elements into ‘contexts’ which can be described on a scale of difficulty, and ‘settings’ which simply provide a range, rather than a scale, of challenges
Contexts Settings
Provide a scale: Easier Harder
Should be covered
Purpose
Explain Discuss
Audience
Large, small, one-to-one, etc.
Relationship
Friends Strangers
Format
Face-to-face, telephone
Level of shared understanding
High Low
Rules
Scripted, conventional, open
Changing the element we have marked ‘setting’ would exercise oracy skills in a different way. Some students would find different settings more challenging than others. But it is not possible to predict the way in which this would happen, as is broadly possible with ‘context’ elements.
Context-hierarchy explanation slide 2/5

7. Example 1: Daniel Shindler’s lesson on Subtexts
Students were placed in a group of three. They were asked to give a short presentation in a manner of their choice that would answer the questions: ‘What is subtext?’ and ‘What kind of question is that?’
Context
Setting
Purpose
To inform/explain/entertain
Size of Audience
Medium (class)
Relationship
Peers
Format
Small group presentation
Level of shared understanding
High but variable
Rules
Freestyle presentation but still a discrete task; all members of the group had to speak
Context-hierarchy explanation slide 3/5

8. Example 2: Changing the ‘context’ elements
A more difficult version of this task would change the elements we have labeled context. The group of students must now explain what they did in lesson to a group of visiting academics.
Context
Setting
Purpose
To explain
Size of Audience
Medium (class + academics)
Relationship
Peers & older strangers
Format
Small group presentation
Level of shared understanding
The academics do not know what the lesson was about
Rules
Discrete task; freestyle presentation
Context-hierarchy explanation slide 4/5

9. Level of shared understanding
Changing the elements we have marked ‘settings’ ensures a broader coverage of situations and the use of skills
Context
Purpose
To explain/inform/entertain
Relationship
Peers
Level of shared understanding
High but variable
Working title for situation
Setting 1
Size of audience
Format
Rules
One-to-one
‘Face-to-face’
Talking in pairs procedure
‘Tell your friend about subtext’
Setting 2
Size of audience
Format
Rules
Whole class
‘Face-to-faces’
Free style
Daniel’s lesson
Setting 3
Size of audience
Format
Rules
Assembly
‘Podium’
Bullet-point style
‘Tell the school about subjects and practice communication strategies’
Context-hierarchy explanation slide 5/5

10. Context is kept constant2
Given that context-elements and skills-deployment both provide hierarchies, one must be kept constant while the other is assessed.
So either…
Skills are kept constant
…or…
Context is kept constant
Certain baseline level of skills-deployment expected in every context, e.g. ‘good skills’
The difficulty of context is set appropriately for, say, an age-group
The level of skills-deployment is assessed within that context
A series of increasingly difficult contexts are devised (stage 1, stage 2, etc.) within which students perform.
For both types of assessment, settings are kept constant. It would be necessary, however, to run the assessments across a variety of settings to get a full picture of a student’s competence.
A context is passed or failed with reference to the expected level

11. An oracy assessment in which skills are kept constant would look like this:
Setting also kept constant
Emotional
Cognitive
Physical
Level 1 Context
Pass/fail
Pass/fail
Pass/fail
Level 2 Context
Pass/fail
Pass/fail
Pass/fail
Level 3 Context
Pass/fail
Pass/fail
Pass/fail
Level 4 Context
Pass/fail
Pass/fail
Pass/fail
Level 5 Context
Pass/fail
Pass/fail
Pass/fail

12. An oracy assessment in which context is kept constant would look like this:
Emotional
Cognitive
Physical
Level 3 Context
Level 1-5
Setting 1
Emotional
Cognitive
Physical
Level 2 Context
Level 1-5
Setting 1
Emotional
Cognitive
Physical
Level 1 Context
Level 1-5
Setting 1
Difficulty increases

13. Each style has good and bad points
Skills are kept constant
Context is kept constant
Skills needed for a specific context can be specified in detail. Marking against this is quite objective.
Greater opportunity for assessing ‘natural’ situations.
Pros
Each student would need to ‘sit’ several contexts as part of one assessment. This would take up an inordinate amount of time.
Marking relies on judgement of the observer which may be too subjective.
Cons
How many of a context’s skill-requirements would have to be met to progress? How many in order for test not to be a blunt instrument?
This style of test is too time-consuming to be used.

14. It is possible to make a hybrid assessment
It could look like this:
Emotional
Cognitive
Physical
Appropriate in context
Skills scale
Level 1
Level 1
Level 1
Yes / no
Level 2
Level 2
Level 2
Yes / no
Level 3
Level 3
Level 3
Yes / no
Level 4
Level 4
Level 4
Yes / no
Level 5
Level 5
Level 5
Yes / no
Etc...

15. All assessments need to meet three criteria for viability:
They must be practicable in a classroom environment
They must give outputs which are valuable to students
They must be teachable to other schools

16. Back-up

17. This will be a context-based skills-level
An alternative hybrid would develop context-appropriate expectations of skill-levels
This could be developed from a series of more and more difficult real-life contexts using this process
What is a more difficult version of that context?
Answer codified
What constitutes the successful exercise of a skill in the more difficult context?
What constitutes the successful exercise of a skill in a given context?
Reiterate until you have a set of context based skills-levels, say from 1-5
This will be a context-based skills-level

18. DRAFT
Skills are assessed on their appropriateness to context. Indicators for this are worked out beforehand.
Context 1
Emotional
Cognitive
Physical
Level 1
‘Did you do… [less]’
‘Did you do… [more]’
‘Did you do… [less]’
‘Did you do… [more]’
‘Did you do… [less]’
‘Did you do… [more]’
Level 2
Level 3
Level 4
Level 5
These boxes would contain detailed lists of what kind of skills deployment is expected
This would still need too many different performances

19. DRAFT
It may be possible to develop assessment foci (AF) in the same way
Physical
Cognitive
Emotional
Skills framework generates context-specific AF (probably more than four for each skills group)
Voice
Body
Words
Reasoning
Content
Self-awareness
Awareness of others
CONTEXT 1 AF
Level to which AF is met is graded
Setting 1 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7
Setting 2 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7 3 4 5 6 7
Setting 3
This model would be far too complex to assess