Improving both quality and equity OECD Programme for International Student Assessment (PISA) Improving both quality and equity Insights from PISA 2000 EI Round Table, Florence, 17 November 2003 Andreas Schleicher Head, Indicators and Analysis Division OECD

Improving both quality and equity 1. Why we need to worry The significance of educational outcomes for individuals and societies 2. Where we are today What PISA shows about student performance and the social distribution of learning opportunities in OECD countries 3. Where we can be What the best performing countries show can be achieved 4. How we can get there Levers for policy that emerge from PISA

The significance of educational outcomes. Changing contexts for education systems.

Changing contexts for education Living with an ageing population In some OECD countries, the share of over-65s in the population is growing twice has fast as the share of under 35s is shrinking Unwanted but needed School systems in most OECD countries will need to integrate unprecedented numbers of immigrants ...but only few are prepared

Changing contexts for education Knowledge workers the only fast growing share of the workforce By 2002 - about the time when school reforms put in place today will begin to show effects in labour markets... ...Manufacturing output in OECD area is likely to double… …Manufacturing employment is likely to shrink to 10% of the workforce …And this is affects all jobs…

Total number of pages of repair manuals for Opel cars Traditional jobs are changing too… Literacy skills in the manufacturing sector Pages Total number of pages of repair manuals for Opel cars from 1933 to 1998 Source : Georg Spöttl, 2002 Year 1933 1951 1966 1983 1998

Changing contexts for education Knowledge workers - the new “capitalists“ Knowledge is becoming the key economic and social resource … and the only scarce one Collectively, knowledge workers own the modern means of production ... and through their stakes in pension and mutual funds, they have become majority shareholders and owners of key businesses in the knowledge society The knowledge society is a society of juniors and seniors ... rather than of bosses and subordinates

Unique opportunities Unlimited upward mobility… Knowledge changes rapidly Everybody starts from ignorance Knowledge differs from traditional means of production in that it cannot be inherited or bequeathed Knowledge is “public” Knowledge has to be put in a form in which it can be taught and is therefore universally accessible Every impediment to mobility… …is perceived as a form of discrimination … If our education systems deliver on their promises

Where we are today. What PISA shows about student performance and the social distribution of learning opportunities in OECD countries.

PISA - The OECD Programme for International Student Assessment A regular assessment of the yield of education (2000, 2003, 2006, 2009,…) including and beyond the curriculum Comparable skill measures that can guide policy decisions Insights into the mix of factors which contribute to the development of knowledge and skills and how these factors operate similarly or differently across countries A strong substantive and cross-cultural core for defining performance targets

PISA - The OECD Programme for International Student Assessment The most comprehensive international assessment to date Geographic and economic coverage 340,000 students randomly sampled All 30 OECD countries plus a growing number of non-OECD countries Subject matter coverage Reading, Mathematics, Science Cross-curricular competencies Variety of task formats Depths A total of 7 hours of assessment material

Three broad categories of key competencies Using “tools” interactively to engage with the world To analyse, compare, contrast, and evaluate e.g. Using language, symbols and texts Interacting with information Capitalising on the potential of technologies To think imaginatively PISA 2000: A new concept of literacy Interacting in diverse groups Acting autonomously Accessing, managing, integrating and evaluating written information in order to develop ones knowledge and potential, and to participate in, and contribute to, society e.g. Relating well to others Co-operating, working in teams Managing and resolving conflicts e.g. Acting within the bigger picture Forming and conducting life plans Taking responsibility and understanding rights and limits To apply knowledge in real-life situations To communicate thoughts and ideas effectively

Using “tools” interactively to engage with the world To analyse, compare, contrast, and evaluate e.g. Using language, symbols and texts Interacting with information Capitalising on the potential of technologies To think imaginatively Reading literacy Interacting in diverse groups Acting autonomously Using, interpreting and reflecting on written material e.g. Relating well to others Co-operating, working in teams Managing and resolving conflicts e.g. Acting within the bigger picture Forming and conducting life plans Taking responsibility and understanding rights and limits To apply knowledge in real-life situations To communicate thoughts and ideas effectively

Mathematical literacy Using “tools” interactively to engage with the world To analyse, compare, contrast, and evaluate e.g. Using language, symbols and texts Interacting with information Capitalising on the potential of technologies To think imaginatively Mathematical literacy Interacting in diverse groups Acting autonomously Emphasis is on mathematical knowledge put into functional use in a multitude of different situations in varied, reflective and insight-based ways e.g. Relating well to others Co-operating, working in teams Managing and resolving conflicts e.g. Acting within the bigger picture Forming and conducting life plans Taking responsibility and understanding rights and limits To apply knowledge in real-life situations To communicate thoughts and ideas effectively

Using “tools” interactively to engage with the world To analyse, compare, contrast, and evaluate e.g. Using language, symbols and texts Interacting with information Capitalising on the potential of technologies To think imaginatively Scientific literacy Interacting in diverse groups Acting autonomously Using scientific knowledge, identifying scientific questions, and drawing evidence-based conclusions to understand and make decisions about the natural world e.g. Relating well to others Co-operating, working in teams Managing and resolving conflicts e.g. Acting within the bigger picture Forming and conducting life plans Taking responsibility and understanding rights and limits To apply knowledge in real-life situations To communicate thoughts and ideas effectively

Under development: PISA assessment of PISA self-reports on: Using “tools” interactively to engage with the world Under development: PISA assessment of Problem-solving skills PISA self-reports on: Dispositions to learning Learning strategies Engagement with school Interacting in diverse groups Acting autonomously

Not yet developed Using “tools” interactively to engage with the world Interacting in diverse groups Acting autonomously Not yet developed

What the best performing countries show can be achieved. Where we can be. What the best performing countries show can be achieved.

Strong impact of social background on performance High Performance High performance Low social equity High performance High social equity Strong impact of social background on performance Moderate impact of social background on performance Low performance Low social equity Low performance High social equity Low Performance

Quality and equity can be achieved together High Performance Quality and equity can be achieved together ‘dumbing down’ is not an inevitable consequence of the pursuit of equity ‘levelling up’ is achievable (e.g. Finland, Korea, Canada) High performance Low social equity High performance High social equity Strong impact of social background on performance Moderate impact of social background on performance Low performance Low social equity Low performance High social equity Low Performance .

Is it all innate ability? Variation in student performance 22 21-25 21 19-24 8 3-10 16 10-20 7 5-9 6 4-9 5 3-9 19 17-21 1 10 9-11

Is it all innate ability? Variation in student performance 22 21-25 21 19-24 8 3-10 16 10-20 7 5-9 6 4-9 5 3-9 19 17-21 1 10 9-11

Is it all innate ability? Variation in student performance Variation of performance within schools Variation of performance between schools 22 21-25 21 19-24 8 3-10 16 10-20 7 5-9 6 4-9 5 3-9 19 17-21 1 10 9-11

Policy levers. Overall findings

Policy Levers Performance in reading Students from advantaged backgrounds… … have a greater chance of coming to school more engaged in reading and entering into a virtuous circle of increasing reading interest and improved reading performance … but not all engaged students come from privileged homes… … and those from more modest backgrounds who read regularly and feel positive about it are better readers than students with home advantages and weaker reading engagement Schools can make a significant difference to bring students into the virtuous circle Seeking mutual reinforcement of cognitive skills and motivation, particularly for boys

Policy Levers Student approaches to learning The ability to manage one’s learning is both an important outcome of education and a contributor to student literacy skills at school Learning strategies, motivation, self-related beliefs, preferred learning styles Different aspects of students’ learning approaches are closely related Well-motivated and self-confident students tend to invest in effective learning strategies and this contributes to their literacy skills Immigrant students tend to be weaker performers … but they do not have weaker characteristics as learners Boys and girls each have distinctive strengths and weaknesses as learners Girls stronger in relation to motivation and self-confidence in reading Boys believing more than girls in their own efficacy as learners and in their mathematical abilities

Policy Levers Student engagement at school An important outcome in itself Disaffection at age 15 can potentially be a precursor to the onset of more serious problems among vulnerable young people Engagement at age 15 is likely to influence students’ choices and educational pathways The prevalence of disaffected students varies significantly across schools in each country Only weak link to student’s social background – there is thus scope for school policy/practice to engage students But strong link to school’s social background Students in schools with strong average engagement tend to perform well Engagement and performance seem to work complementary The school climate seems to make more of a difference than resources For individual students, strong performance does not necessarily ensure strong engagement at school Relationship complex

Policy levers. Some characteristics shared by some strongly performing countries Combining the empirical results obtained through PISA with qualitative information on the socio-cultural conditions and education policy strategies.

. High Performance High performance Low social equity High performance High social equity plus Netherlands Strong impact of social background on performance Moderate impact of social background on performance Low performance Low social equity Low performance High social equity Low Performance .

Sympathy doesn’t raise standards – aspiration does In the countries studied National research teams report a strong “culture of performance” Which drives students, parents, teachers and the educational administration to high performance standards PISA shows that students perceived a high degree of teacher support Which should not be simply equated with “achievement press”

Governance of the school system In the countries studied… Decentralised decision-making is combined with devices to ensure a fair distribution of substantive educational opportunities The provision of standards and curricula at national/subnational levels is combined with advanced evaluation systems That are implemented by professional agencies Process-oriented assessments and/or centralised final examinations are complimented with individual reports and feed-back mechanisms on student learning progress

. High degree of school-level autonomy High Performance E.g. Learning environment and course offering High degree of school-level autonomy Low degree of school-level autonomy % Variance between schools 11% 20% 9% 76% 7% Strong impact of social background on performance Moderate impact of social background on performance 75% 71% r=.51 Low Performance .

Organisation of instruction In the countries studied… Schools and teachers have explicit strategies and approaches for teaching heterogeneous groups of learners A high degree of individualised learning processes Disparities related to socio-economic factors and migration are recognised as major challenges Students are offered a variety of extra-curricular activities Schools offer differentiated support structures for students E.g. school psychologists or career counsellors Institutional differentiation is introduced, if at all, at later stages Integrated approaches also contributed to reducing the impact of students socio-economic background on outcomes

. High degree of integration Early selection and stratification High Performance Early selection and institutional stratification High degree of integration Early selection and stratification Strong impact of social background on performance Moderate impact of social background on performance Low Performance .

Support systems and professional teacher development In the countries studied… Effective support systems are located at individual school level or in specialised support institutions Teacher training schemes are selective The training of pre-school personnel is closely integrated with the professional development of teachers Continuing professional development is a constitutive part of the system Special attention is paid to the professional development of school management personnel

. Students perceived teacher support High degree of support High Performance Students perceived teacher support High degree of support Low degree of support Strong impact of social background on performance Moderate impact of social background on performance Low Performance .

Summary of common characteristics Uniformity Diversity “hit and miss” Universal high standards “Inputs” Outcomes Bureaucratic Devolved responsibility Look up Look outwards Received wisdom Data and best practice Motivating feedback and incentivising success and innovation Evaluation to control Prescription Informed profession

One challenge – different approaches The future of education systems needs to be “knowledge rich” Informed professional judgement, the teacher as a “knowledge worker” Informed prescription National prescription Professional judgement Uninformed prescription, teachers implement curricula Uninformed professional judgement The tradition of education systems has been “knowledge poor”

Further information www.oecd.org www.pisa.oecd.org email: pisa@oecd.org Andreas.Schleicher@OECD.org … and remember: Without data, you are just another person with an opinion