ACC Kahui Ako 7 June Moderation-developing a shared understanding of Levels 1-5 in Writing and Maths

Whakatauki E kore e taea e te whenua kotahi  Ki te raranga i te whariki Kia mohio tatou ki a tatou Ma te mahi tahi o nga whenu,ma te mahi tahi o nga kairaranga, ka oti tenei whariki The tapestry of understanding cannot be woven by one strand alone. Only by the working together of strands, and the working together of weavers, will such a tapestry be completed.

The Purpose of today’s workshop To build our knowledge to make dependable overall teacher judgments. By: Establishing clear understandings about the judgements and the reference material Building an understanding about the progressions of learning Knowing what to notice and recognise when making an OTJ Challenging you to think about possible next steps In making overall teacher judgments, there are two key elements which contribute to their dependability. Dependability- Firstly, is the judgment valid? – are we assessing what we set out to assess? Secondly, is the judgement reliable – if the assessment were repeated, would we come to the same conclusion?

The learning pathway - our focus on progress and growth for school, work and life Our education system is designed to support students to build the knowledge, skills and capabilities they need to be successful. The expectations for learning and progress are outlined in our curriculum documents and through the standards that we have set. Discuss overall vision of the curriculum and what we all want for our children.

Components of Moderation 1. Shared understanding of the reference materials used to make decisions, including definition of terms. 2. Shared understanding of progressions in the reference materials. 3. Shared understanding of what to notice and recognise when making decisions. 4. Processes to ensure judgments in relation to reference materials are made in a similar way. Reference material could be NZC, NS, LLP, Numeracy framework, Learning Progressions Framework illustrations etc. This is about understanding the step up in skills and behaviours that students exhibit as they move through the progressions of a specific area or aspect. This is developing the knowledge of how students acquire skills and knowing what this looks like in practice. What are the key elements, big ideas, components that need to be noticed in order to be able to make dependable judgments. These are the things we do to ensure consistency is developed and maintained. Mitchell, K., & Poskitt, J. (2010)

The value of common reference material, progressions & frameworks Builds consistency of what to notice and recognise Highlights gaps in learning opportunities Gives a consistent framework within and across schools Progress through learning steps is made visible These are central to strong moderation in and across schools By common frameworks, we are referring to the frameworks that support us to implement the NZC They include LLP, NS illustrations, NS posters for Writing Reading and Maths, Number Framework. We now have the Learning Progression Framework illustrations to Level 5 All these contribute to the above bullet points. The frameworks have been developed to refine/ unpack what is in the National Standards and give more detail to what to notice when making decisions and to notice what we might have missed teaching up to end Year 10

Moderation Moderation is the process of sharing expectations and understanding of standards in order to improve the consistency of decisions (reaching agreement). (TKI Assessment) Year 6 student AT the standard means… Year 6 student Above the standard means… Year 6 student Below the standard means… The term moderation often brings to mind the process of meeting with others, following teacher assessments, to compare evidence and see if there is agreement. However, moderation ‘acts’ occur a long way before this type of meeting and in various ways, as this definition in the slide indicates. Sometimes moderation occurs naturally and informally (e.g. teacher discussions) and other times you as a leader have to be very deliberate. It will not happen by accident. Today we need to moderate our understanding of the terminology of the standards and the judgements so we have a shared understanding across the schools/Kahui Ako so we can trust the decisions. Share with a buddy your definition of AT for a Year 6 student. Now if they were Above. Now if they were Below

Evidence: Overall Teacher Judgments The Overall Teacher Judgment (OTJ), as a judgment of each student's achievement in relation to the National Standards, is central to the implementation of the standards initiative overall. Research suggests: Evidence suggests OTJs lack dependability 51% indicated that school-developed resources such as descriptions of performance or annotated work samples were used Principals’ perception is that their own schools’ OTJs are more reliable (~90%) Increasingly proportions using National Standards data to inform the provision of tailored teaching interventions and to target the acceleration of specific groups of students NSSSME - https://www.educationcounts.govt.nz/publications/series/National_Standards The extent to which school developed resources accurately describe student capability? - This can contribute to inconsistencies in judgments The targets were increasingly differentiated to accelerate progress for specific groups of students (57% of National Standards targets were differentiated in 2011 and an average of 78% of targets were differentiated across the three areas in 2013). The level of challenge inherent in schools' targets appears to be an ongoing cause for concern In an attempt to increase reliability some schools can place too much emphasis on test results leading to an impact on the validity of the judgment. Nationally developed tools with use of multiple sources of evidence supports dependability.

Definitions – What is At, Above, Below, Well below? For a student to be judged at the standard, evidence of achievement should be, on balance, closest to the standard for their year level or years at school. For a student to be judged above the standard, evidence of achievement should be, on balance, closest to the standard above their year level or years at school. For a student to be judged below the standard, evidence of achievement should be, on balance, closest to the standard below their current year level or years at school. For a student to be judged well below the standard, evidence of achievement should be, on balance, closest to the standard two or more years below their current year level or years at school. http://assessment.tki.org.nz/Overall-teacher-judgment/Definitions-of-achievement Handout will be given out later

Tasks Assessment Tools Conversations and Observations Student self assessment Student peer assessment Group work Discussions Group teaching – guided reading and writing Explanations Conferencing Questioning by student Justifying the choices they make in their work Observations – running records Tasks Completed tasks Draft writing books ARBs tasks Work completed away from the teacher that is not highly scaffolded Assessment Tools PAT, STAR, Observational Survey E-asTTle reading, writing & Maths Range of evidence required across the curriculum- Students need multiple opportunities to demonstrate their understanding/learning which in turn means multiple opportunities for teachers to notice. Role of Standardised tools Standardised test information is an important form of evidence and has a function in moderation. For example, correlation between standardised tool and OTJ can provide a degree of confidence. Non-correlation does not necessarily mean an OTJ is incorrect. Continually Periodically

Years 9 and 10 ERO Literacy and Mathematics in Years 9/10 (2012) …urgent need to improve practice in using literacy and mathematics assessment information for planning, implementing and reviewing the curriculum for all Years 9 and 10 students It would be useful for the MoE to clarify expectations for monitoring student achievement and progress at Years 9 and 10, and the use of students’ achievement information to plan programmes. Literacy and Language Pedagogy within Subject Areas, years 7-11, (Woolf Fisher 2013) Teachers effectively teach content, and content vocabulary … but minimise the literacy challenge for students, do it with best interests of students in mind. This evidence provides for rationale for providing illustrations that exemplify high quality examples of the type of learning that can occur at Level 4/5 and years 9 and 10 of schooling. ERO publication referred to can be found here – Quote reference to Page 32 http://www.ero.govt.nz/publications/literacy-and-mathematics-in-years-9-and-10-using-achievement-information-to-promote-success/ Woolf fisher document can be found here – Statement related to text on Page 7 and 8 https://www.educationcounts.govt.nz/publications/series/Secondary_Literacy/Literacy_and_Language_Pedagogy …do it with the best interests of students in mind. This links to this text on Page 7. “We suggest that the teachers in this study do not want students who might struggle with the complex literacy and language demands to be excluded from opportunities to engage with valued scientific and mathematical knowledge because of these literacy and language demands. Thus, one explanation for the relative low frequency of literacy and language teaching in these classrooms is that the teachers have responded to challenging aspects of reading and writing by minimising the amount and challenge of reading and writing that the students do.”

NZ Curriculum Indicators Outcomes Sign posts Talking points: 2007: the New Zealand curriculum was released. Provides the big picture of what we want students to learn, how we want them to learn, and the principles and values that underpin these. Sets the broad direction or outcomes for student learning. (eg in reading we want students to be able to decode, make meaning and think critically) The literacy progressions and revised numeracy framework followed. Help us identify the knowledge and skills students need to meet the demands of the curriculum as they progress through school. The progressions provide more detailed indicators. (e.g. in reading, as students read we want them to recognise 100-200 high frequency words and so on.) Skills in literacy and numeracy enable students to access other curriculum areas. 2010: the National Standards were published to support assessment for learning in the classroom. The standards provide reference points or sign posts at each year level (e.g. in reading after one year we want students to be able to decode, make meaning and think critically about texts at green level). 2014: PaCT frameworks developed to support teachers to make consistent, reliable teacher judgements. They also provide a deeper insight into the full breadth of the reading, writing and maths curriculum for years 1-8 2016: The Learning Progressions Frameworks launched; an online tool that illustrates the significant steps that students take as they develop their expertise in reading, writing and mathematics from Years 1-10, spanning levels 1-5 of the New Zealand Curriculum (NZC). The LPF includes the PaCT content and places it in context.

Refining not redefining The Learning Progression Framework comes from the NZC. The principles of literacy learning in NZ is based on the literacy acquisition framework (diagram in the middle) These theoretical understandings underpin all of our key literacy documents. Refer to Pg 8 Reading and Writing Standards. Of course, this slide is looks particularly at literacy acquisition. For Maths, the Numeracy framework plays a similar role. The frameworks for literacy acquisition provides common language and understandings across all the resources shown above, including the framework. Aspects have been mapped against the framework for literacy acquisition from Effective Literacy Practice Pg 26 y5-8 and Pg 25 yr 1-4.

The overview of the curriculum, NCEA framework, the Learning Progression Framework, the National Standards and the PaCT. Shows on one page where everything fits. Things you may wish to note The black lines around the NZC curriculum levels skid graph – clarifying that there are overlaps but the levels are aligned to year levels Learning progressions not aligned to curriculum levels or NS. They indicate the significant steps they take as they progress in reading, writing and maths. The standards describe reference points or signposts of achievement at each year level e.g. End of Year 4. Assessing progress and achievement in relation to the standards is an integral part of teaching and learning across the New Zealand Curriculum. The tracking students section shows how PaCT can use the LPF to synthesise this information and provide recommended OTJs from teacher judgments using classroom observations etc. It also allows for finer grain reporting of progress (rather than At to At progress information for example). Note how the PaCT scale goes beyond the End of Year 8 NS. The Learning Progressions are up to the end of Year 10, NS up to end of Year 8 and PaCT to end of Year 9. The addition of Science, English and Social Science allows the LPF to go to End of year 10. It is necessary for PaCT to go to end of Year 9 so the performance of students above for ‘End of year 8’ can be captured.

The Learning Progressions Frameworks READING Making sense of text: processing system Making sense of text: text structure Making sense of text: vocabulary knowledge Making sense of text: reading critically Reading to organise ideas and information for learning. Acquiring and using information and ideas in informational texts. Reading for literary experience. WRITING Writing meaningful text: encoding; Writing meaningful text: text features. Writing meaningful text: vocabulary knowledge. Using writing to think and organise for learning. Creating texts to communicate knowledge and understanding Creating texts for literary purposes Creating texts to influence others. MATHEMATICS Additive thinking Multiplicative thinking Patterns and relationships Using symbols and expressions to think mathematically Geometric thinking Measurement sense Statistical investigations Interpreting statistical and chance situations. Just as The New Zealand Curriculum provides the framework for the National Standards, the PaCT encourages teachers to dive deep into the Curriculum when making judgments about their students. The PaCT frameworks break down mathematics, reading and writing into aspects. The frameworks were developed by New Zealand curriculum experts using the New Zealand Curriculum, the National Standards, the Literacy Learning Progressions and the Numeracy Framework. The standards in maths are directly aligned to the Mathematics and Statistics learning area in the NZC and so in developing the PaCT, eight ‘aspects’ reflecting each of the core learning streams of the mathematics learning area of the NZC have been created. The aspects that have been created for the reading and writing frameworks reflect what students need to know and be able to do to meet the reading and writing demands of subjects across the Curriculum. The aspects on the framework are not ‘new’ or changed. They provide a greater degree of explicitness around… “read, respond to and think critically” (Reading) in order to support greater understanding of content knowledge and progression. “think mathematically and statistically, solve problems and model situations” (Maths) “think about, record, and communicate” (Writing) Maths Notes about the aspects for each learning area: Eight aspects (4 Number and Algebra, 2 measurement and geometry, 2 statistics and probability) Focus on solving problems and modelling situations in a context. Not intended to ignore the importance of basic facts, as this knowledge is required in the application of problem solving. The Reading and Writing aspects deliberately use illustrations from across the curriculum (e.g. Writing in a science context), as these are cross-curricular competencies. Reading There are seven aspects for Reading The first four about reading behaviours and processing text We’ve been told that the progression is quite close together for a start (learn a lot about reading quite quickly) and then spread out more Five to seven are about the wider, cross-curricular purposes of reading Writing There are seven aspects for Writing First three aspects are about ‘encoding’ – the act of writing it down. Fourth aspect- being able to write to help organise their thinking and learning Five to seven – writing for a particular purpose or audience To help teachers make judgments that are valid and dependable, all aspects should be considered as prompts when making interim and Overall Teacher Judgments. The PaCT has been designed to help teachers: Capture a comprehensive view of a learners expertise in reading, writing and mathematics; Track a learners developing expertise in reading, writing and mathematics over time; and Relate a learners developing expertise to the standards If needed you can go into more details about each aspect but this may be better left until implementation sessions. Aspects descriptions can be found here https://www.google.co.nz/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&ved=0ahUKEwj-q9DWk-7SAhWFgLwKHX0NDS4QFggYMAA&url=http%3A%2F%2Fcpl.org.nz%2Fcontent%2Fdownload%2F1291%2F5050%2Ffile%2FHandout%25202%2520The%2520aspect%2520framework%2520and%2520descriptions%2520final.pdf&usg=AFQjCNGJCCpD_KAwRW_IFuxZOCIl49ELZw&sig2=eS7kvV_EROY0KEXSHSQyoQ&bvm=bv.150475504,d.dGc

Aspect progression A set of illustrations Aspects of Writing Writing meaningful text: encoding Writing meaningful text: using knowledge of text structure & features Writing meaningful text: Vocabulary Using writing to think and organise for learning Creating texts to communicate knowledge and understanding Creating texts for literary purposes Creating texts to influence others A set of illustrations Aspects of Writing The significant stages of development within an aspect are described by illustrations which shows the increases in the sophistication of student response in relation to the text (reading), task (writing) or problem (maths) Each aspect a separate but important aspect of reading, writing or maths Aspects have a different numbers of sets depending on the number of clearly observable, significant steps that have been identified in typical student progression through the aspect. This image is a generic model of a framework for writing The framework shows progression – this is why its not shown as ‘years’ or ‘levels’ Each aspect is broken into clearly observable and distinct learning steps represented by the circles The steps within the aspects are conceptually distinct. There is a clear difference in the level of performance/sophistication between one stage and the next. Each dot represents a different set of illustrations. Up to 8 set of illustrations for each aspect. Some 5, some 8. Some aspects have less steps as they do not all have the same degree of complexity. There may be less steps but each step may take longer to get through. They do not map directly to the levels of the curriculum. The PaCT does not describe expertise at each year or curriculum level. Instead, it has been developed to clearly describe the breadth of developing expertise. This was a deliberate design decision (see the notes at the bottom of this page for more detail to support this idea) This also means the gap between learning steps will vary (as reflected by the different gaps between bubbles). The learning steps reflect the cumulative nature of learning, The illustrations are designed to prompt teachers to think, “Can my student do things like this?” Further information to share if required The PaCT has been designed to support teachers to make dependable Overall Teacher judgments by identifying what students know and can do at a point in time. The reading, writing and mathematics frameworks describe student expertise from school entry to beyond the end of level 4 of the New Zealand Curriculum. They are structured around the progression of significant development steps in each aspect, describing the expertise of both the most highly achieving students, and the least highly achieving students within this period.    Not curriculum levels - The frameworks focus on the progression of significant development steps. The frameworks are structured around the progression of significant development steps in each aspect. This is in contrast to the National Standards, which were developed to correspond with the year-level structure of the school system, and the New Zealand Curriculum, which has four roughly equivalent levels spanning years 1 to 8. In essence, the development did not set out to describe expertise at each year or curriculum level. Instead, it set out to clearly describe the breadth of developing expertise, and National Standards expectations were superimposed on the frameworks once they were complete. If they had been linked to curriculum levels they would have somewhat been normed for each set of illustrations and aspect when it is actually about the bigger picture across all aspects.

The Aspect Framework Refined the Literacy Acquisition Framework Made more explicit what is meant by: Learning the code Making meaning Thinking critically To support teachers to know what to notice and recognise when looking at evidence the MOE has developed what is known as the aspects framework. With each of these 7 aspects for writing there are sets of illustrations that illustrate each discrete step of learning across the curriculum levels. The frameworks are what sits behind the PaCT tool. There is value in the framework regardless of whether a school decides to use PaCT. They are too useful to ignore. Lets take one and see how it might help us to be really clear what to notice and recognise in the evidence we collect. So in making and OTJ in writing all these aspects need to be considered.

Set of illustrations Each set describes the breadth of expertise at each step on the progression and is conceptually distinct from both the set below and the set above. Illustrations within a set will show different students, different contexts but as a whole provide a picture of the significant things to notice and recognise at this step in the progression. Each observable step is represented by a set of illustrations, not a single illustration. Each set has been checked for authenticity and relevance by 100s of teachers. The illustrations provide detailed examples of NZ students in NZ schools, often using texts commonly found in schools, but also using online texts. The sets have been carefully designed and extensively trialled to ensure they are conceptually distinct from the sets that precede or follow.

Illustrations Illustrations are NOT assessment tasks to see if students can do them. They should prompt questions such as: What evidence do I have? Is this the type of level that ______ could demonstrate? Is this the ‘kind of way’ that ______ would solve problems? Is this the ‘kind of way’ that ______ would write ideas, experiences or information Is this the ‘kind of way’ ______ would respond to a text Important point - Teachers need to provide opportunities for students to show what they can do. Are there gaps in the learning opportunities we are providing our students? An ordering activity such as we’ve just done, whether using the frameworks or other reference materials such as the standards or the LLP, may highlight areas in which students have not been given the opportunity to show what they know and can do. Teachers need to inquire into how well they know their learners by using these prompts or similar ones. Possible discussion point: How would you use these kinds of questions to support moderation and Teaching and Learning

Task (1) Writing Years 1-6 groups: Get into groups of 3 so each year level within your group is represented (Yr 1,2 3) (Yr 4,5,6) (Yr 7/8/9/10) Order the illustrations Read the grey box on each A3 in the A3 books. Discuss the step up between the sets. What do you notice? Using your evidence pack decide which set is the best fit. Justify to your group Important to note each page is a set of illustrations for one learning step – it is not an illustration of one child’s work. The purpose of this activity is to become familiar with the progressions of learning in writing from Level 1to Level 5

Task (2) Maths Years 1-6 groups: Get into groups of 3 so each year level within your group is represented (Yr 1,2 3) (Yr 4,5,6) (Yr 7/8/9/10) Order the illustrations Read the grey box on each A3 in the A3 books. Discuss the step up between sets. What do you notice? Using your evidence pack decide which set is the best fit. Justify to your group Important to note each page is a set of illustrations for one learning step – it is not an illustration of one child’s work. The purpose of this activity is to become familiar with the progressions of learning in writing at Level 4 and 5

Key Messages