Graeme Cosslett NZCER 2010 Unpacking PAT Tests from the NZCER Marking Website

Building a greater understanding of the online reporting Reporting at the: Individual Student level Class level Schoolwide level Understanding: Scale scores, Stanines, Box plots, Measurement error, Interpreting reports, Generating the reports

How are they doing? Class of 2010

Test scores Alex22Juliet31 Allen25Kelly18 Amy40Michael36 Andrew7Nichole30 Bean6Patrick35 Becca19Paul10 Bill11Peter31 Catherine7Rose18 Cathy23Sara24 Christina9Simon1 Daniel18Spencer26 Dianna41Steve14 Gary24Tessa15 Janine14Tom28 Jessie14Tony16 Joy6Tracy25

Grouped test scores

Dianna41Daniel18 Amy40Kelly18 Michael36Rose18 Patrick35Tony16 Juliet31Tessa15 Peter31Janine14 Nichole30Jessie14 Tom28Steve14 Spencer26Bill11 Allen25Paul10 Tracy25Christina9 Gary24Andrew7 Sara24Catherine7 Cathy23Bean6 Alex22Joy6 Becca19Simon1 Average20.1 St.Dev10.5 Median18.5 Ordered test scores

How well are these students doing? It depends!

Questions left unanswered What kind of knowledge and skill do students have at different achievement levels? How much better is a student doing who gets 35 compared to 25? How do scores on this test compare with scores on a harder or easier test? What score should I expect the students to get in a year’s time?

We can start with a measurement scale. How do we answer these kinds of questions?

Measurement scales Allow you to convert a test score to a location on an equal interval scale. You can then see how one score compares with another, no matter which PAT test was used. You compare a student’s location on the scale with national norms for different year levels.

Measurement scales You can locate the difficulties of each test question on the same scale. We can then compare a student’s location on the scale with the location of different questions.

Measuring progress More comprehension skills Less comprehension skills By Year 6, 50% of students have reached this level Questions involve longer passages containing abstract ideas and more complex vocabulary

Create your own reading comprehension and mathematics scales that cater for achievement from Year 4 to 10. Describe the characteristics of tasks at different points on each scale. Tasks that can be done using counting strategies.

How difficult? Put the tasks in order from least difficult to most difficult. Locate each task somewhere on your scale Which tasks are the most difficult? Why? How would a typical Year 7 do on each of the tasks? On each of your scales locate where you think the average Year 7 student would be

20 patm 34 patm 66 patm 75 patm Year 7 Student Year 10 Student

Comparing students and items Students will typically answer correctly: 50% of the test items that are at the same location on the scale as they are more than 50% of the test items that are below their own location on the scale less than 50% of the test items that are above their own location on the scale.

Increasing skill as a reader As students improve in comprehension they demonstrate increasing skill in: –using abstract information –using separated information –using multiple pieces of information –using implied information –rejecting competing information –using vocabulary –using grammatical structures.

Vocabulary demands Word ListDog & Pukeko (%)Square Eyes (%) other Percentage of words in word list one to 10

The items in PAT Reading Comprehension according to text

Unpacking grammatical structures

Year 6 Year 7 Average progression in reading vocabulary for New Zealand students Year 5 Year 4 Year 8 Year 9 Year 10

Year 6 Year 7 Average progression in mathematics for New Zealand students Year 5 Year 4 Year 8 Year 9 Year 10 PAT:Mathematics Scale Year 3

The PAT:Mathematics Scale

Stanines

Where on the scale? Two students score 50% on PAT Mathematics tests. Student one has sat Test 5 and student two has sat Test 6. Test 6 is harder than Test 5. Who will be located higher on the scale?

What Tom got right

Tom six ± 3.6 seven six

Toms’ Maths Report Which national year group is Tom being compared with? How good is stanine 7? How many Geometry and Measurement questions did Tom get wrong? Which was the easiest Geometry and measurement question? Was Tom expected to get question 30 correct?

Sam Year 6 March, Year 6

Question types Retrieval Comprehension without needing to infer Local inference Comprehension of implied information from parts of the text Global inference Comprehension of implied information from across the text as a whole

Local inference What does the poet compare herself with in the last verse? (Inferring the meaning of a simile from 1 line.) Why did Glen shrug? (Inferring a character’s intention from implied information across 6 sentences.) What was Adam imagining when he was playing with the water cannon? (Inferring a character’s thoughts from information in 4 paragraphs.)

Global inference What is the main purpose of this text? Which sentence would the author most likely agree with? What impression of the brain does this text create? What will probably happen next in the story?

PAT:Vocabulary items

90 patv 70 patv 50 patv 28patv 16 patv Year 7 Student Year 10 Student