3. Creating an assessment Reflections and Evaluation
Key changes?
Assessment
Creating an assessment
The scoring rubric
The exemplars
Scope
Prompts & Purpose
Rubric
Exemplars
Scale
Reporting
What are R’s
By category
The layout
Notes
Struc & Lang. notes
R3 is not L3!
Have a read
Your thoughts
What
How
Purpose
Considerations
Making own prompt?
Prompts
Other prompts
Prompt pack
Administration
Overview
Generic exemplars
Marking a script
Scoring sheets
Your thoughts
Entering the data
Viewing reports
Interpreting the Overall Score
Imprecision
Individual Elements
R3 does not equal R3
The table
The ILP
What you need Considerations
Suggested process
Have a go
Splashing in the Waves
Depressing Dogs
Third time lucky
Reflections
Teaching and Learning
Progress
Reflections and Evaluation
Other prompts
An assessment tool
Suggestion for analysis

4. E-asTTle Writing – why more changes?
Both Mathematics and reading have been updated and recalibrated in the last two years. The writing was ‘out of sync’ with the others.
The original scoring rubric uses curriculum levels. Progressions in writing do not necessarily fit neatly into curriculum levels.
The previous rubrics did go down to Level 1, but these could not be recorded in e-asTTle and Year 1-3 student scores could not be inputted into e-asTTle.
The scoring rubrics were not well enough supported with exemplars and were difficult to use accurately even with moderation processes in place.

5. What are the key changes? – Scope of the Tool
The tool can now be used to assess writing from students in Year 1 through to Year 10. The previous version assessed from year 4 to Year 10.
In order to be assessed accurately by the new tool, students should be able to write at least one or two simple ideas.
(Students who score in the lowest category for every element assessed are not well targeted by the assessment, as it may not accurately indicate the students’ skills and next learning steps.)

6. What are the key changes? - Prompts and purposes
The tool emphasises ‘prompts’ rather than ‘tasks’.
20 open-ended prompts encourage students to choose subject matter relevant to their experience.
There is a range of prompts for younger or older students.
There are now five writing purposes, important to writing in general.

7. What are the key changes? - Rubric
There is now one rubric for all writing purposes.
There are seven elements of writing, similar to those in the previous version but with changes that require a shift in thinking by teachers.
There is a range of achievement categories on a continuum from R1 to R6/R7 for each element.
The tool is aligned to the NZ curriculum:
it reflects the intentions of the literacy learning progressions and national standards
it is not specific to a particular learning area
However mainly developed through an analysis of student responses to writing prompts.

8. What are the key changes? - Rubric
Original Audience Awareness and Purpose, Content/Ideas, Structure/Organisation, Language Resources, Grammar, Punctuation, Spelling
Revised
Ideas, Structure and Language, Organisation, Vocabulary, Sentence Structure, Punctuation, Spelling

9. What are the key changes? - Annotated Exemplars
There 76 annotated exemplars include a set of prompt specific exemplars and generic exemplars to accompany each writing prompt.
Prompt specific exemplars are examples of writing for the prompt being used. They show a range of abilities.
The generic exemplars are not necessarily for the same purpose or prompt.
The exemplars are actual samples of student writing, and are representative rather than ideal examples.
The annotated exemplars are essential tools for ensuring consistent scoring decisions.

10. What are the key changes? – Measurement scale
The scale provides a measure of writing proficiency and is linked to curriculum performance levels, allowing students’ scores to be reported within curriculum bands.
The new scale is not directly comparable to the previous e-asTTle Writing scale.
Note: A conversion table is being developed to provide some comparison between scores in the new and previous versions.

11. What are the key changes? – Report formats
There are some minor changes to the report formats to accommodate the new rubric.
The introduction of + to clearly show error of measurement in scores.

12. Assessment – What
E-asTTle writing assesses a student’s ability to independently write continuous texts across a variety of communicative purposes.
These are:
Describe
Explain
Recount
Narrate
Persuade

13. Assessment – How
Up to 40 minutes
Scored using rubric and annotated exemplars (both generic exemplars and prompt specific exemplars).
Scoring rubric linked to literacy learning progressions but developed through analysis of student samples
Curriculum level is not part of the marking
When analysed a curriculum level for each element and overall score will be available

14. Creating a writing assessment - Purpose
Create Test – Choose Writing/Tuhituhi prompt
Choose the purpose first
Describe
Explain
Recount
Narrate
Persuade
Consider the students being assessed, the type of response that the purpose will deliver and the information you are wanting to acquire.
Model this process using e-asTTle

15. Creating a writing assessment – Can you make your own prompt?
Yes and No
Yes the rubric can be used to assess teacher’s own prompts.
However, DO NOT put the scored results for your own prompts into e-asTTle as the difficulty of the ‘made up’ prompt is unknown.
‘Teacher made’ prompts should assess continuous texts using the five purposes as this is what the rubric was developed for. A conversion table is available so that you do not put this data into e-asTTle.
Choosing a variety of prompts according to the needs and interests of students will increase student
engagement. Doing so should not significantly increase time spent on marking if the prompts have the same
purpose. However, introducing each prompt and discussing the topic will make test administration more
complicated. Teachers who choose to use more than one prompt might want to consider assessing different
prompts at different times.
e-asTTle training area

16. Creating a writing assessment - Considerations
When choosing a prompt consider possible sources of difficulty
Level of abstraction
Complexity of the structure eg. Narrative harder than description or recount
Topics better suited to some age levels – 5 in slightly simpler language as most likely to be used by younger students (the recounts and 3 describes)
Can use more than one prompt to cater for and engage students, but probably wise to do so at different times
Choosing a variety of prompts according to the needs and interests of students will increase student
engagement. Doing so should not significantly increase time spent on marking if the prompts have the same
purpose. However, introducing each prompt and discussing the topic will make test administration more
complicated. Teachers who choose to use more than one prompt might want to consider assessing different
prompts at different times.

17. Creating a writing assessment - Prompts
Model this process using e-asTTle
Supporting material show the different bits of information available including
See other prompts here

18. Creating a writing assessment - Prompts
Model this process using e-asTTle
Supporting material show the different bits of information available including

19. Creating a writing assessment – Test creation
Once a prompt has been chosen the prompt pack is created. It includes:
Admin instructions
Student booklet
Glossary
Rubric
Annotated exemplars
Choosing a variety of prompts according to the needs and interests of students will increase student
engagement. Doing so should not significantly increase time spent on marking if the prompts have the same
purpose. However, introducing each prompt and discussing the topic will make test administration more
complicated. Teachers who choose to use more than one prompt might want to consider assessing different
prompts at different times.

20. Creating a writing assessment – Test administration
Don’t forget to assign tests to students – cannot put results and generate results until this is done.
Your turn – please spend time reading the administration guidelines
Choosing a variety of prompts according to the needs and interests of students will increase student
engagement. Doing so should not significantly increase time spent on marking if the prompts have the same
purpose. However, introducing each prompt and discussing the topic will make test administration more
complicated. Teachers who choose to use more than one prompt might want to consider assessing different
prompts at different times.
Important point – it is independent writing so for those junior students who are use to flash card support etc. you can say something like “We can have a go at seeing what you can do all by yourself today.”

21. The scoring rubric – What are R’s
The progressions for each element within the scoring rubric is by category - not curriculum level
R means Rubric
Do not use the rubric in isolation – use it in conjunction with the specific exemplars, the generic exemplars and the Structure and language notes.

22. The scoring rubric – by category
Using student samples and the literacy learning progressions the rubric has been able to capture stages of skill development in writing.
A little like children’s art?

23. The scoring rubric – like categorising children’s art
Random scribble
Controlled scribble
Pre symbolism
Symbolism
Realism
3 dimensional space

24. The scoring rubric – the layout
The focus
Definition
Category
Descriptor
Notes
Generic exemplars – for this element only

25. The scoring rubric – R3 is not Curriculum Level 3!1 2 3 4 5 6
Ideas R1 R2 R3 R4 R5 R6
Structure & Language R1 R2 R3 R4 R5 R6
Organisation R1 R2 R3 R4 R5 R6 R7
Vocabulary R1 R2 R3 R4 R5 R6
Sentence Structure R1 R2 R3 R4 R5 R6
Punctuation R1 R2 R3 R4 R5 R6 R7
Spelling R1 R2 R3 R4 R5 R6
*** Representative diagram only

26. The scoring rubric – Notes
THEY ARE NOT REQUIREMENTS – for example, in Punctuation there are notes for ‘Other Punctuation” at R2 and R3. Other punctuation is not required in the descriptor until R4.
The intention of the notes is to describe ‘what else’ you might see in scripts.

27. The scoring rubric – Structure and Language notes
These notes are prompt specific i.e. The notes vary depending on which purpose is being written
Check carefully that these notes relate to the prompt/s being used.
For example the prompts ‘Adult and child’ ‘Girl’ ‘The Market’ ‘Stick Insect’ and ‘Dogs at the beach’ use the same structure and language notes

28. The exemplars
76 exemplars in total. There is a set of generic exemplars as well as exemplars specific to each prompt
The exemplars are not exemplary items. They represent actual samples of student writing
The annotated exemplars provide excellent comparative reference points for ensuring consistent marking.
Some exemplars are not truly written to the purpose etc.

29. The specific exemplars
These are generated when creating a prompt pack in e-asTTle.
They directly relate to the prompt you have chosen.
The show a range of scores for the prompt you are using.
DIRECT comparisons can be made
Use these before going to the generic exemplars.

30. The generic exemplars
They are referenced at the bottom of the marking rubrics.
They do not necessarily directly relate to the prompt you are using.
Use these for further reference if the specific exemplars have not enabled a definitive decision.
There are only one set of generic exemplars.

31. The scoring rubric – Have a read
Your turn – please spend time becoming familiar with the scoring rubric using the glossary and definitions as reference material.
Make sure you look at the Structure and Language notes for ‘Dogs at the Beach’ which is ‘Describe a moment in time’.
Highlight bits which are important or you will need to remember.

32. What we found out about the scoring rubrics - Key points from the group

33. Marking a script – What you need
Prompt Specific Exemplars
Marking Rubric
Structure & Language Notes
The
Script
Glossary & Definitions
Generic Exemplars

34. Marking a script – Considerations
Age is irrelevant! Mark it for what it is.
Mark the script in front of you, not what you know about the student who wrote it.
You can only use the evidence in front of you. e.g. If technical words are not there, there is a limit to how far you can score in spelling.
There is no scoring rubric for neatness of handwriting!
Beware of ‘proximal marking’. i.e. Mark each element independent and separate from the other elements.
It is possible to be at R6 for spelling and R2 for punctuation.

35. Marking a script – Suggested process
Read the text right through
Go to one of the seven elements and read the descriptors and notes. Get a feel for which category it may fit.
Check against the prompt specific exemplars for comparative qualities.
Still not sure? – use the relevant generic exemplars as another check.

36. Marking a script – Have a go
The script being given to you is an annotated exemplar.
Have a go at marking it and then we will discuss the difficult decisions
We will then compare to the annotated scores.

37. Splashing in the Waves – How close were we?
Element R1 R2 R3 R4 R5 R6 R7
Ideas
Structure and Lang.
Organisation
Vocabulary
Sentence Structure
Punctuation
Spelling

38. Have another go – same prompt, different sample!
Depressing Dogs
Have a go at marking and try and come to agreement over scores for each element with a partner.
Remember to use the rubric, structure and language notes, generic exemplars and specific exemplars to help with your decisions

39. Depressing Dogs - How did we go this time?
Element R1 R2 R3 R4 R5 R6 R7
Ideas
Structure and Lang.
Organisation
Vocabulary
Sentence Structure
Punctuation
Spelling

40. Third Script – Your choice
Choose one of the recount or narrative scripts and mark by yourself or with a partner. Once complete we will get into ‘script groups’ and analyse and scores and debate.
Don’t forget to use the group scoring grid.

41. Third time lucky – your choice
Element R1 R2 R3 R4 R5 R6 R7
Ideas
Structure and Lang.
Organisation
Vocabulary
Sentence Structure
Punctuation
Spelling

42. How was it? – Your thoughts and reflections

43. Checkpoint No.1 – have you assigned it?
Entering the data
Checkpoint No.1 – have you assigned it?
Change the date to represent when the actual sample was taken rather than the date the data was entered
Put the attitude and seven element scores onto e-asTTle
Click save after each student
Reports are ready!
Preferably model this

44. Viewing reports
Links to each one to go here
Need to wait to complete

45. Viewing reports
Links to each one to go here
Need to wait to complete

46. Viewing reports – Interpreting the Overall Score
The curriculum levels reported for e-asTTle Writing are based on a standard-setting exercise undertaken to link performance on an e-asTTle assessment with the descriptions of writing competence provided in the Literacy Learning Progressions.
A curriculum level of 4A for example means that
“given 40 minutes to write to a particular prompt under test conditions, the student has produced a text of sufficient quality to indicate they have the writing skills and competencies described as appropriate for students working at an advanced stage for Level 4 of the curriculum.”
Links to each one to go here
Need to wait to complete

47. Viewing reports – Interpreting the Overall Score
e-asTTle curriculum level attempts to identify the student’s performance in the context of an e-asTTle writing assessment.
A student who has been assessed by e-asTTle Writing at a particular curriculum level will not necessarily have produced a piece of writing that is of the same standard as a National Standards illustration.
National Standards illustrations show examples completed by students in class “largely by themselves”.
An e-asTTle assessment is completed in 40 minutes under test conditions without any teacher or peer feedback, or access to writing aides such as dictionaries.
Links to each one to go here
Need to wait to complete

48. Viewing reports – Taking imprecision into account
No educational assessment is perfectly precise. e-asTTle Writing provides a margin of error by presenting scale scores within a “plus or minus” range, for instance 1250 ± 40.
If the student was to repeat the assessment we could expect him/her to score somewhere between 1210 and 1290 scale units about 70% of the time.
Imprecision also needs to be taken into account when considering the curriculum level descriptor. A student who scores 4A is probably somewhere in the range 4P to 5B.
Links to each one to go here
Need to wait to complete

49. Viewing reports
Links to each one to go here
Need to wait to complete

50. Viewing reports – R3 does not equal R3 or Level 3!
Element
R score
Explain - Caring for Planet Earth
Narrate - I heard a whisper
Recount – Time with friends
Overall score 2A 3B
Ideas R3
Structure and Lang. 3P
Organisation 2P
Vocabulary
Sentence Structure
Punctuation
Spelling
Need to look at curriculum levels scores in the ILP rather than just the scores on paper

51. Viewing reports – Individual Elements
In the revised e-asTTle writing each element has only 6 or 7 scoring categories. Moving from say a R2 to a R3 will involve a large shift in the associated curriculum level.
It is unwise to treat the element curriculum scores as precise measures and comparisons with the old element scores will be very tenuous.
It is best for teachers to point to the rubric scores given for each element when discussing a report with students. .
Links to each one to go here
Need to wait to complete

52. Viewing reports – Individual Elements
The reporting system in the e-asTTle tool is built on a system that uses many more bands than this and which are linked to curriculum levels.
These curriculum bands can impress a sense of precision that isn't there and mean we have to be very aware of the margin of error in an e-asTTle score when interpreting or comparing them. R1 R2 R3 R4 R5 R6
Links to each one to go here
Need to wait to complete 1B 1P 1A 2B 2P 2A 3B 3P 3A 4B 4P 4A 5B 5P 5A 6B 6P 6A

53. Use to improve teaching and learning
This is an assessment tool.
“The primary purpose of assessment is to improve students’ learning and teachers’ teaching as both student and teacher respond to the information that it provides.”
Therefore we take the information from these samples to identify areas for improvement and deliberate teaching to support the student in his/her learning.
Would like to do this one together

54. Use to improve teaching and learning
Suggestion – use the curriculum level scores to indicate needs and areas for future teaching because R scores by themselves will be misleading.
i.e. R3 ≠ R3
Then go back to the rubric to identify the progression needed. Use documents such as the literacy learning progressions to support you in the teaching of this element.
Example: Sentence structure R3 – 2A
Use the rubric to see the demands of R4 to identify next steps for this element.
Would like to do this one together

55. Viewing reports – showing progress between samples
Links to each one to go here
Need to wait to complete

56. What now for you and your school?
Is this a tool that can add to the improvement in teaching and assessment of writing in your school?
Is it useable and manageable?
How might you use it?
When?
How?

57. Evaluation
Please spend a minute or two reflecting on this seminar and how it has helped you come to terms with the revised writing tool.
Your feedback helps us to evaluate our own performance and provides us with information on how to improve the experience for others in the future.
Thank you

58. Other prompts

59. Other prompts

60. Other prompts

61. Other prompts

62. Other prompts

63. Other prompts

64. Other prompts

65. Other prompts

66. Other prompts

67. Other prompts

68. Other prompts

69. Other prompts

70. Other prompts

71. Other prompts

72. Other prompts

73. Other prompts

74. Other prompts

75. Other prompts

76. Other prompts

77. Home