1 OECD country practices on reporting and implications for Australia Brisbane, Australia 2-3 June 2005 Barry McGaw Director for Education Organisation for Economic Co-operation and Development 2005 Curriculum Corporation Conference Curriculum and assessment: Closing the gap

2 National reflections on PISA results.

3 Germany seeking more specific comparisons Social equity (OECD regression slope – country regression slope) Reading literacy Source: OECD (2001) Knowledge and skills for life, Table 2.3a, p.253. High quality Low equity High quality High equity Low quality Low equity Low quality High equity

4 Denmark’s concern about efficiency Source: OECD (2001) Knowledge and skills for life, Fig. 3.7a, p.91.. Cumulative expenditure per student to age 15 ($US equivalent PPP) Reading literacy

5 Place of national assessments r Recognition that more could be known domestically l Denmark judged to have no ‘culture of evaluation’ l Some countries report that international comparisons have stimulated domestic evaluations r Why bother? l Weighing a pig won’t make it fatter. l Without weighing the pig, how can you know how well feeding regime is working? r Purposes of assessment l Accountability – summative l Improvement – diagnostic, formative r Scope l Sample or census? l Depends on whether focus on system, schools, students

6 Driving system reform may be helped by having disaggregated data Improving systems by monitoring and improving units: - education - health

7 A UK health example: Reducing wait time in Accident and Emergency Units

8 % of A&E patients waiting no more than 4 hrs Target first introduced in National Health Service Plan, June 2000 Public Service Agreement announced, June 2001 No improvement occurring Source: Barber, M. (2005) Presentation to Informal Meeting of OECD Education Ministers, St Gallen, Switzerland.

9 % of A&E patients waiting more than 4 hrs Dept of Health taskforce begins Source: Barber, M. (2005) Presentation to Informal Meeting of OECD Education Ministers, St Gallen, Switzerland.

10 % of A&E patients waiting no more than 4 hrs Dept of Health taskforce begins Accident & Emergency included in hospital star ratings Source: Barber, M. (2005) Presentation to Informal Meeting of OECD Education Ministers, St Gallen, Switzerland.

11 % of A&E patients waiting no more than 4 hrs Incentive scheme introduced Dept of Health taskforce begins Accident & Emergency included in hospital star ratings Tailored support for specific problems Performance management Target revised to take account of clinical exceptions Source: Barber, M. (2005) Presentation to Informal Meeting of OECD Education Ministers, St Gallen, Switzerland.

12 % of A&E patients waiting no more than 4 hrs Source: Barber, M. (2005) Presentation to Informal Meeting of OECD Education Ministers, St Gallen, Switzerland.

13 Driving system reform may be helped by having disaggregated data Improving systems by monitoring and improving units: - schools? - teachers?

14 Data form and data use r Breadth of data l What we measure signals what we value. l Does what we don’t measure signal what we don’t value? l Watch for unintended consequences. r Type of data l student performances –measurements of current performance –estimates of value added by school –comparisons with ‘like’ schools l other data on schools – input, process, outcomes? r Uses of data l school (and system) only l public use –results or rank orders –website accessibility (UK, Norway, Just4kids, Standard & Poors)

15 Do assessment programmes make a difference? r Research evidence l a little but it is positive – programmes improve systems –Hanushek, E.A. & Raymond, M.E. (2004) The effect of school accountability systems on the level and distribution of student achievement, Journal of the European Economic Association, 2(2- 3, p r System evidence l England reports improvement among poorest performing schools l Less improvement of next 15% r Importance of monitoring the monitoring l Evaluating trends in performance –overall –for subgroups (as in US No Child Left Behind Act requirements l Evaluating interventions intended to improve performance

16 Thank you.