2. PERSPECTIVES ON QUALITY AND EQUITY FROM LARGE-SCALE ASSESSMENT STUDIES John Ainley and Eveline Gebhardt

3. Outline Properties of large-scale assessments Perspectives from NAPLAN: 2008 to 2013 – National changes in reading achievement – Improvements in reading for Indigenous students – Differences in changes across jurisdictions Messages from PISA – Changes in reading and mathematics achievement – Student background – Jurisdictional changes in reading and mathematics – Variation between and within schools Summary

4. Perspectives from NAPLAN

5. National mean reading scores: 2008-2013

6. National mean numeracy scores: 2008-2013

7. The story so far: Improvement in reading in years 3 and 5 No change in reading in years 7 and 9 No significant national change in numeracy at Years 3, 5 (note exception of Queensland), 7 or 9

8. Changes in mean reading scores: Indigenous and non-Indigenous students

9. Jurisdictional mean Year 3 reading scores: 2008-2013

10. Jurisdictional mean Year 5 reading scores: 2008-2013

11. The story extended: Improvement in reading in years 3 and 5 No change in reading in years 7 and 9 No significant national change in numeracy at Years 3, 5, 7 or 9 Larger improvement in reading for Indigenous than non-Indigenous students (Years 3 & 5) Differences among jurisdictions with large increases in: – Queensland – Northern Territory

12. Interpretation Emphasis on the early years (school and pre-school) – Programs and resources – Focus on reading in the early school and pre-school years Programs intended to improve outcomes for Indigenous students Structural change in Queensland – Introduction of a foundation/preparatory year – Accompanied by a strong school improvement strategy

13. Messages from PISA

14. Change in reading literacy: 2000 - 2012

15. Change in mathematical literacy: 2003 - 2012

16. Changes in achievement and student background Reading 2000-2009Mathematics 2003-2012

17. Changes in achievement & socioeconomic background (ESCS): overall gradient Reading 2000-2009Mathematics 2003-2012 20032012 Australia42 Canada3431 USA4535 UKna41 Finland33 Germany4743 20002009 Australia4746 Canada3832 USA5242 UKna44 Finland2531 Germany5244

18. The story so far Modest declines in reading and mathematics among 15 year-olds Extent of decline is similar for reading and mathematics Does not appear to be associated with differences in student background.

19. Jurisdictional change in reading literacy: 2000 -2012

20. Jurisdictional change in mathematical literacy: 2003 -2012

21. Summary of jurisdictional changes in PISA reading and mathematics Reading change (00-12) JurisdictionChange VIC1 QLD-13 WA-19 NT-23 NSW-26 ACT-27 TAS-30 SA-37 Mathematics change (03-12) JurisdictionChange VIC-10 QLD-16 NSW-17 TAS-30 ACT-30 WA-32 NT-45 SA-46

22. The story extended Modest declines in reading and mathematics among 15 year-olds Extent of decline is similar for reading and mathematics Does not appear to be associated with differences in student background Differences among jurisdictions in the extent of decline Changes in reading and mathematics achievement appear to run parallel

23. Variation between and within schools Variance – dispersion calculated as the mean of the squared deviations of observed values from a mean – total variance made up of: variance within schools and variance between schools’ mean scores Percentages of total variance Within / total Between / total % variance between schools is an indicator of the extent to which schools differ

24. Between-school variance in reading and mathematics in selected countries

25. Change in mean mathematics score (2003-12) and change in % between-school variance

26. The story extended a little further Modest declines in reading and mathematics among 15 year- olds Extent of decline is similar for reading and mathematics Does not appear to be associated with differences in student background Differences among jurisdictions in the extent of decline Changes in reading and mathematics achievement appear to run parallel Declines in Australia associated with larger differences among schools In general for mathematics larger changes in between school variance are associated with larger declines

27. Speculation Differentiation appears to be associated with lower overall achievement (Willms, 2010) Differentiation among schools may be associated with differentiation in teaching expertise among schools Possibly a reciprocal relationship arising from attractiveness of schools to teachers Caution about policies that exacerbate differences.

28. Thank you Questions Discussion